|
INTERNAL AND EXTERNAL VALIDITY
A. Internal Validity:
Cook and Campbell identified 12 types of extraneous variables that if left uncontrolled my produce effects that the researcher could mistake for the effect of the independent variable. Thretas to internal validity are
1. History: refers to the events or circumstances other than the introduction of the treatment variable that occur coincident with the time interval between the pretest and posttest measurements. E.g. An event that received much media coverage and attention and that may have influenced responses in the target population. Researchers must be attuned to changes at one study site that could make comparison of outcome variables across several hospitals biased.
2. Maturation: refers to changes within the study participants themselves that occur overtime and that are not related to any specific event such as tiring, gaining weight, becoming more knowledgeable. It becomes difficult for the researcher to determine whether changes observed over time can be attributed to the independent variable, to maturational changes in the participants or perhaps to an interaction effect between the treatment and maturational changes.
3. Testing: refers to the learning that results from being tested at time 1 that affects responses to the test at time 2, regardless of the introduction of the treatment variable. Thus the process of measuring itself can introduce a threat to internal validity, especially when reactive measures are used. Participants may recall information on the pretest or be sensitized to aspects of the experiment, especially when tests are unusual or memorable.
4. Experimental mortality: refers to nonequivalent attrition of study participants from the experimental and control groups that renders meaningful comparisons between the groups difficult. Random attrition is a far less serious threat than when participants are systematically dropping out of one treatment condition more than another. When the latter occurs researcher should attempt to evaluate a small random sample of dropouts to determine why they left the study.
5. Instrumentation: refers to changes that occur in the measurement instruments, observers or raters that potentially produce changes in the dependent variable measurements. This threat of validity is most pronounced in studies using repeated measures designs
6. Statistical regression: refers to movement of mean scores from Time 1 to time 2 that most often results when study participants are selected on the basis of scores that are at the extremes of the distribution.
7. Selection bias: refers to the selection of participants on a nonrandom basis that may produce differences in the experimental and comparison group participants with regard to the criterion measurement irrespective of the differential exposure to the treatment.
8. Interactions with selection: means that a number of the previously described threats to validity can interact with selection causing spurious treatment effects e.g. selection-maturation, selection-history, selection-instrumentation.
9. Diffusion or imitation of treatments: refers to the introduction of a treatment that involves information when the experimental and control group participants may be able to interact with one another, directly or indirectly and learn about information intended for others.
10. Compensatory equalization of treatments: refers to the use of an experimental treatment that has actual or potential value to participants in cases in which authorities or participants may be unwilling to tolerate an imposed inequity in the distribution of the treatment.
11. Compensatory rivalry by respondents receiving less desirable treatments: refers to the assignments of study participants to the experimental and control groups in which control group participants are disadvantaged by the absence of the treatment in contrast to experimental participants and thus are motivated to compete for equity. Compensatory equalization is mainly a response by administrators and compensatory rivalry is a response by participants.
B. External Validity:
Cook and Campbell identified six main factors, which if controlled the researcher can achieve generalizability by replicating the study with different participants, in different settings and at different times. Treats to external validity are:
1. Interaction of selection and treatment: refers to the effects obtained that are applicable only to the specific individuals who participated in the study.
2. Interaction of setting and treatment: refers to the effects obtained that are applicable only to the specific setting in which the experiment is conducted. The milieus of the settings may vary widely, with some being more innovative more pleasant and more competitive. The question is whether or not results obtained in one setting can be generalized to other settings that because of their particular environments would be different from the original settings.
3. Interaction of history and treatment: refers to the effects obtained that are applicable only to the specific time period within which the study is conducted. Unusual occurrences that coincide with a study period can make the extrapolation of results to other periods of time questionable. Although the researcher can attempt to plan in a way that avoids obvious unusual occurrences, it is often impossible to avoid happenings that could make the findings unique to the study time period. Replication of study at different times is the logical approach to counteracting the interaction effect of history.
4. Reaction or interaction effect of pretesting: means that following exposure to pretest, the participants no longer remain representative of the target population which has not been pretested. Thus the findings cannot be generalized to the target population This effect occurs because the nature of the pretest makes participants aware of certain issues or events of which they would not otherwise be aware, causing them to respond to the treatment in a unique way.
5. Reactive effect of experimental procedures: is the effect produced by the procedures of the experiment that make the participants who are exposed to these procedures no longer representative of the target population - “Howthorne effect”.
6. Multiple treatment interference: refers to effects produced by multiple exposures of participants to a treatment so that the results may be generalizable only to individuals who also receive the same multiple exposures to the treatment in the same sequence.
|
