• Charecteristics and Types
• Internal and External Validity
• True experimental designs
• Quasi experimental designs
• Evaluative and other designs
• References

QUASI EXPERIMENTAL DESIGNS

Introduction

• this design is distinguished from the true experiment by the lack of random assignment of participants to groups.
• This lack of control limits confidence in the internal validity of the study.
• Otherwise the design is exactly same as the experimental design.
• Data are predominantly quantitative.
• Many field experiments actually use a quasi experimental design when the assignment of participants to the various experimental and control groups cannot be controlled by the researcher.
• When the researcher is not allowed control of the treatment variable and cannot achieve randomization because of ethical considerations, institutional policies or other situational factors, the researcher chooses quasi experimental studies.
• Quasi experiments do not have equivalence created by random assignment or that do not have control groups for comparison, rather comparisons are made with nonequivalent groups or with periodic measurement of the same group that may be different due to a number of variables extraneous to the causal variable of interest. Refer to Appendix B

Types of Quasi experimental designs

1. NON EQUIVALANT GROUP DESIGNS

• These designs are those in which dependent variable measures are obtained for an experimental and a comparison group (nonrandomly assigned) before and after introduction of the independent variable to the experimental group.
• When non equivalent group designs are employed, the control group may also receive a treatment, usually traditional.
• The threat to validity are selection-maturation,, selection-history and regression.

i. Basic nonequivalent group design/Untreated control group design with pretest and posttest

• One group of individuals is designed to receive a program, service or treatment and a second group of individuals is designated as a control group.
• Similar to classical experimental before-after design except that participants are not randomly assigned to either groups.
• The  design is used in field research in which variable relationships characterizing human subjects are tested.

  O1    X    O2      (  X: Treatment,  O: Observation)

  O1          O2

• Threats to validity are:
  • selection bias
  • bias related to measurement of the outcome variable like, instrumentation.

ii. Untreated control group design with pretest measures at more than one time interval

• There are advantages of administering pretests more than one time that strengthen the basic nonequivalent groups design.
• Multiple pretests permit assessment of whether the groups are becoming disparate at different rates from O1 to O2 when the effect could not be due to the treatment and also allow assessment of spurious treatment effects due to statistical regression.
• However it is often  problematic to obtain pretest measures when the field researcher is unable to delay implementation of treatment moreover funding sources are often unwilling to expend monies for pretest, however researcher should strive to achieve at least one more pretest measure to the basic nonequivalent groups design.

O1     O2     X     O3      (X:Treatment, O: Observation)

O1       O2           O3

iii. Post test only designs

a. Non equivalent post test only design:

Here no pretest is held

X     O2     (X: Treatment, O: Observation)

   O2    

b. Multiple control groups: Use of multiple control groups for comparison

X    O1       (X: Treatment, O: Observation)

       O2

       O3

       O4                  

iv. Other designs

• These designs are useful when a nonequivalent control group cannot be obtained and are applicable when the researcher must create circumstances that are conceptually similar to the no-treatment control group or in other words when only a single group of participants is available for the research.

a. Removed treatment design with pretest and posttest

• A single pretest-posttest is fundamental.
• A third post test is added which is followed by removal of the treatment and one more post test.

O1   X   O2   O3   X-    O4 (X: Treatment, O: Observation, X-: Treatment removed)

• Subjects often remove themselves from the intervention or are removed by factors that cannot be controlled.
• The O3-O4 data collections serve as the no treatment control for the O1-O2 data collections.
• One group participates throughout the course of the study
• When the treatment is effective there are expected differences between O1 and O2 and between O3 and O4.
• These differences must be in opposite directions with a clear change following removal of the treatment variable.
• This latter effect is needed to distinguish differences between the experimental and control conditions that are due to removal of the treatment rather than to the treatment simply having no long term effect.

b. Repeated treatment design

• Participants are exposed to the treatment, removed from treatment and then exposed again to the treatment.
• The design is best employed when the treatment effects are expected to dissipate rapidly or when the effects are expected to be cumulative.

O1       X         O2       X         O3       X         O4   

 (X: Treatment,  O: Observation)

To interpret the effects of treatment the O3, O4 difference must be in the same direction as the O1, O2 difference.

c. The reversed Treatment Non equivalent control Group design with pretest and posttest

O1   X+   O2    (X+, X-: Reversed treatment,  O: Observation)

O1   X-     O2

X+ represents an expected effect in one direction and X- represents an expected effect in the opposite direction. The design is resistant to selection-maturation threats because maturation rarely occurs at the same time in opposite directions in the two groups. It is stronger design in terms of construct validity because it necessitates a carefully specified causal theory to hypothesize an effect in one direction in one group and an opposite effect in the other group. Cook and Campbell emphasize that to be maximally interpretable the reversed treatment design requires a placebo control group that receives a treatment not expected to influence the outcome measure except through a reactive effect of experimental procedures and a control group that receives no treatment for a baseline of no cause.  The design is often difficult to implement because treatments that produce negative effects are considered unethical and deliberate designing of these experimental conditions is questionable.

d. The cohort design with cyclical turnover: 

This design can be useful for studies in which some participants are exposed to a treatment because they cycle through an organization but others are not; cohorts differ minimally; archival records can be used to compare cohorts on specific characteristics.

O1

X         O1

2. INTERRUPTED TIME SERIES DESIGNS

These designs are those in which the independent variable effects are inferred by comparing measures of the dependent variable obtained at multiple time intervals with multiple dependent variable measures obtained after introduction of the independent variable.

i. Basic interrupted time series design:

It uses a single group of participants and can use different but similar units of analysis. The design is relevant for quality assurance studies and also can be convenient for conducting pure research. It can supplement information gathered using nonequivalent group designs. When the interrupted time series design is used, the researcher must know exactly when the participants were exposed to the treatment variable in order to infer its effects. If a treatment effect occurs the outcome measures taken after to the treatment should be different from those taken before exposure to the treatment. Hence the time series measures are interrupted.

At least 30 time periods should be available when the researcher considers using interrupted time series design without a comparison group. This allows sufficient data to be gathered for an evaluation of a trend over time following the intervention. The design which has an adequate number of time period observations has a distinct advantage for generalization of results over a variety of time periods. The advantage is the opportunity to examine results within subgroups of participants, thus external validity is strengthened. The study participants can be stratified according to individual and setting characteristics to determine if effects hold across subgroup. Multiple observations after the treatment decrease the potential for measurement error.

O1   O2    O3    O4    O5   X     O6   O7    O8    O9   O10

Cook and Campbell describe several types of interruption effects that can occur. The first is sharp change in mean scores when the interruption occurs, indicating different levels or intercepts of the slopes of pre and post treatment scores. The second interruption effect that can be observed is a change in slope of the mean scores before and after treatment. The interruption effect can also be characterized as continuous or discontinuous and as instantaneous or delayed. Different effects can occur simultaneously in various combinations. The major strength of the basic time series design is that it allows the assessment of outcome effects that are due to maturation prior to the introduction of the treatment.

Multiple observations also make it possible to examine the trend of pretest scores for an effect that is due to statistical regression. Furthermore with a large number of time period observations, extraneous effects due to history or cyclic influences can also be evaluated.  The major threat is history thus the researcher must collect data frequently and record carefully all possible events or circumstances that could reasonably influence outcomes. The other threats are instrumentation, selection bias and reactive effects of experimental procedures.

 ii. Interrupted time series designs with a nonequivalent no treatment group time series:

O1    O2   O3  O4    O5   X    O6   O7   O8   O9    O10

O1    O2    O3   O4    O5      O6    O7   O8   O9   O10

The addition of a no treatment control group time series to the basic time series design makes it strong in terms of threat of history because the effects of history can be tested. The design is still   be subject to the threat of selection-history interaction if one group is influenced by an event at the time that it is also exposed to the treatment and the threat be enhanced by the addition of factors that increase the nonequivalence of the groups.

iii. The interrupted time series design with nonequivalent dependent variables

Oa1   Oa2    Oa3   Oa4   Oa5   X    Oa6   Oa7   Oa8    Oa9   Oa10

Ob1   Ob2  Ob3    Ob4   Ob5         Ob6   Ob7    Ob8    Ob9   Ob10

The subscripts refers to different measures 

This design has enhanced construct validity. The effects of history can be assessed and the construct validity strengthened by collecting data for some variables that should show treatment effects and for others that should not. Sufficient theory should exist to establish the conceptual relationship of the dependent variables with the treatment

iv. Interrupted time series design with removed treatment:

O1  O2  O3   O4  X   O5  O6 O7 O8 O9  X- O10  O11  O12 O13

Here two interrupted time series are joined together i.e. O1 to O9  represents one time series that allows assessment of a treatment effect; O5 to O13 allows assessment of the removal of the treatment.  For maximum inference researcher would expect one form of change between O4 and O5 and an opposing change between O9 and O10.  However the second series adds protection from the threat of history to internal validity. Regardless of the effects of resentful demoralization, the design is quite strong because two alternative explanations are required to negate results that occur in a different direction after removal of a treatment.

v. Interrupted time series design with switching replications:

O1    O2    O3         O4   O5  O6  O7   O8   X   O9   O10   O11

O1    O2    O3   X    O4    O5   O6   O7   O8      O9   O10  O11

In this group two nonequivalent groups are exposed to a treatment and serve as control groups for each other. There are two important reasons for including this design: firstly the design has a strong internal and external validity and secondly it is a practical design that nurse researchers should consider whenever the nonequivalent control group is available. The replication built into the design makes it possible to extend the findings beyond a single population.

vi. Interrupted time series design with multiple replications:

O1  O2  X  O3    O4  X-  O5  O6  X  O7  O8  X-  O9  O10  X O11  O12  X-  O13  O14

The design is powerful in testing the causal hypothesis and has the added advantages of modifications that allow the testing of more than one treatment in single study as well as testing interaction effects of treatments. Two treatments can be tested by substituting X1 for X and X2 for X-. The design is however vulnerable to the extraneous effects of cyclical maturation, so random scheduling of treatments and their removal is suggested, while preserving the alternating treatment and removal unless cyclic effects can be ruled out by a strong theoretical base. The weakness of the design is that it is possible only when the effects of a treatment are expected to extinguish rapidly. Also the design ordinarily requires that the researcher have the ability to control the circumstances of the experiment, which is usually not possible with the nursing phenomena. It may be a useful design for testing forms of behavior modification interventions

vii. Interrupted time series design with strengthening the treatment over time:

O1   O2   X    O3    O4    X+1   O5     O6     X+2    O7    O8

 3. REGRESSION DISCONTINUITY DESIGNS

This design is rarely used in nursing, involves systematic assignment of subjects to groups based on cut-off scores on a pre intervention measure, is considered attractive from an ethical point of view and merits consideration.

 4. DOSE RESPONSE DESIGN

The outcomes of those receiving the different doses of a treatment, not as a result of randomization are compared. For example in complex and lengthy interventions, some people attend more sessions or get more intensive treatment than others. The rationale for a quasi experimental dose response analysis is that if a larger dose corresponds to better outcomes, this provides supporting evidence for inferring that the treatment caused the outcome. The difficulty however is that people tend to get different doses of the treatment because of differences in motivation, physical function or other characteristics that could be driving the differences in outcome and not the different doses themselves. When a quasi experimental dose response analysis is conducted within an experiment, the dose response evidence may well be able to enhance causal inferences.

WITHIN SUBJECTS DESIGNS

A. Overview of within subject experimental design

Research designs whether true experiments, quasi experiments or descriptive, each category has unique features as well as some have shared. They have in common within-subject and between subjects design. Is a neglected area in nursing research. There are at times particularly in the clinical studies, when the best control only can be achieved by having participants randomly assigned to order of treatment rather than to groups.  A basic assumption in these studies is that the study participants themselves provide the best control group. It is then possible to use very small, unique clinical populations for whom there would otherwise be no adequate control group.

B. Difference between within and between subjects designs

i. Between subjects design: or parallel design, is commonly used in RCT

• Subjects are assigned to groups randomly or because they possess a particular attribute or experience.

• Subjects may be enrolled in only one group

• Each group is exposed to or represents different values of the independent variable

• Responses of all members of one group are compared with those of another group. The groups are expected to differ.

• Example is posttest only control group experimental design in which subjects are randomly assigned to either a treatment or a control group.

ii. Within subject design or crossover design :wherein the same subjects are crossed over to the new intervention after receiving their usual treatment and vice versa.

a.  They are handled in one of the following ways.

1. Subjects are observed or assessed repeatedly. Subjects are exposed to all interventions associated with the investigation.

2. Related subjects, those who have been matched on important attributes or experiences are delegated to different groups or are taken to be representatives of different groups. Matching approaches include, matching by mutual selection. Those who are matched in this way include people who are paired off such as husbands and wives. A pair may be considered as matched by mutual selection when it is reasonable to presume that the pair is more similar on the outcome measure than those who have not been matched.

b. Responses of individual subjects are expected to differ. The researcher begins by comparing the responses of the subject that is by making within subject comparisons. These comparisons are aggregated for all subjects.

The examples of these designs are: repeated strategies, using subjects as their own control, crossover, correlated groups, split pilots, randomized block design.

PRROBLEMS WITH QUASI EXPERIMENTAL DESIGNS

• Few treatments can’t be introduced quickly rather their impact on participant, evolve over time

• Treatment effects may have a variety of periods of delay that differ among specific populations and from one time to another.

• Although an abundance of archival data exists that theoretically could be used for time series analysis, they are often difficult to locate and researcher may have problems obtaining their release.

• It is often problematic to obtain 30 or more observations recommended for data analysis

• because of the advantages of these designs for nursing research and the problems that have just been listed nurse administrators, educators, researchers, and clinicians are encouraged to consider their potential applications and to anticipate their use when data are routinely gathered and stored.

PATCHED UP DESIGNS AND STRUCTURAL MODELLING

The notion of patched up designs allows the researchers to mix and match strategies to best achieve practicality, feasibility and research rigor. Structural modeling depends on strong theory to specify the relevant variables influencing specific hypothesized outcomes. The focus is on comprehensive identification and measurement of all input, process and outcome variables. It entails specifying the true pretreatment assignment processes, theory based specification of outcomes and modeling of extraneous factors that influence the outcomes beyond the intervention, including treatment and mediating subject and treatment systems variables.

 C. Choice of the designs: is based on the pragmatic issues:

Evaluate the merits and demerits, it is prudent to choose the within subject approach to achieve  a powerful test of the hypothesis