|
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Home l Research l Reviews l Theories l Updates l Abstract l News l Specialties |
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Outline
-----------------------------------------------------------------------------------
II. Essential characteristics of Experimental designs To test a relationship and most confidently infer this result, experimental designs must be characterized by three essential elements: randomization, manipulation and control. Randomization refers to participants being assigned by chance to either receive or not receive the treatment condition or intervention. A number of procedures exist for assigning individuals to groups such as coin toss, a random numbers table or computerized random number generators. The key characteristic of all these procedures is that each participant has an equal and known probability of being assigned to either the control or the experimental group. Randomization helps eliminate bias by spreading variability due to extraneous variables equally across the groups under study. The advantage of assigning participants to groups in a random manner is that this should result in the group’s initially being similar to one another prior to the intervention. Random assignment to condition does not guarantee that the two groups will be similar to one another. Based on sampling theory, significant difference(p<0.05) between the two groups will occur in 1:20 cases of assigning participants to groups. The sample size should be large enough or else researcher may wish to consider some additional methods of distributing important variables such as matching or use of more homogeneous population. In some research, it is necessary to randomly assign treatment conditions, to units other than the individual participants. Randomization may be cluster, stratified, fixed or by random assignment. Manipulation is the process of maneuvering the independent variable so that its effect on the dependent variable can be observed. The causative variable must be amenable to manipulation by the investigator, i.e. the researcher does something to subjects in the experimental condition. It is essential that researchers conduct manipulation checks to see whether the manipulation had its intended effect or perhaps resulted in an unintended effect that could compromise the validity of an experiment. In working to avoid unintended effects researchers should be cautious of the use of reactive measures which can influence participants’ responses to the dependent variable. Even though researcher does not actively manipulate the control group, it is important that he or she be aware of what may be happening to them Control group should experience all the same things as participants in the experimental group, except the independent variable. Ethical considerations, organizational policy or some variables like attitudes, age, disease etc which cannot be manipulated may disallow manipulation. The ability of the researcher to manipulate the independent variable is a major source of control in experimental studies. Cook and Campbell identify three uses of the term control in relation to research designs, all of which involve elimination of threats to valid inference namely the researchers control over the research environment; control over the experimental variable; the ability to identify and rule out threats to internal validity. The last type of control is typically achieved through the use of a comparison or control group and through attention to sources of variance. In many nursing studies control groups receive the usual or traditional methods of care rather than no treatment against which effects of the experimental intervention are measured. Kerlinger discusses the merits of experimental designs in terms of their ability to control variance or to take into account factors that may contribute to differences in the dependent variable. To provide valid answers to research questions, three kinds of variance must be considered: systematic or experimental variance which refers to the systematic effect of independent variable on the dependent variable and is a type of variance which should be enhanced; extraneous variance which refers to the effects of extraneous variables on the dependent variables, which is to be controlled by the design by building the extraneous variable into the design as an independent variable, eliminating or holding constant an extraneous variable by selecting participants as homogeneous as possible on that variable, matching participants and using statistical control; error variance which refers to the variability of measures due to random fluctuations including errors of measurement, which can be controlled by standardizing the instrument/ measurement conditions (keeping the time of the day, place instructions, personnel constant) or using sensitive, reliable instruments. Internal validity is the primary objective of experimental methodology III. Threats to internal and external validity: Cook and Campbell identified 12 types of extraneous variables that if left uncontrolled my produce effects that the researcher could mistake for the effect of the independent variable. These are
Cook and Campbell identified six main factors, which if controlled the researcher can achieve generalizability by replicating the study with different participants, in different settings and at different times. They are:
IV. Types of experimental designs: There are of two major types of experimental designs, true experiments and quasi experiments. True experiments include the random assignment of units to comparison groups for inferring a change that has been caused by treatment. Random assignment is an essential component of true experiments that is designed to achieve comparability of comparison groups. Quasi experiments have a treatment, outcomes and units to be analyzed, but random assignment of units to comparison groups is not included for determining the groups of units to be compared. An important assumption underlying all true experimental designs is that equivalence of groups is maintained throughout the course of the experiment and is not compromised by things such as differential attritions – i.e. variable dropout rates may make experimental and control subjects different on critical factors at the time of the analysis. If experimental and control groups become nonequivalent, the design then becomes quasi experimental. Refer to Appendix A V. True experimental Design/ Randomized clinical trials/ controlled trials: A. Overview of true experimental design The experimental level of research is designed to test theory in laboratory settings or in controlled clinical trials, thus the purpose is to test theory. The greater the degree of control in the research setting, the greater the confidence, that the research findings are accurate. The experimental design the most controlled of all research designs entails manipulation of the independent variables and requires control of all intervening variables. On the basis of theory developed from previous research, each step in the experiment requires a predictive hypothesis regarding the effect of the independent on the dependent variables. Cause and effect is always predicted based on theory, thus predictive hypotheses are written at the beginning of the study stating the precise nature of both the manipulation of the independent variable and its effects on the independent variables. All assumptions are spelled out or are either verified by previous research or tested by the current research. All logical steps between cause and effect are specified. Although causal laws can never be proved researchers in experimental designs should be aware of three criteria for causality: a temporal relationship – cause must precede the effect in time; an empirical relationship - there must be an evidence that the independent variable and the dependent variable are associated; a spurious relationship – the relationship cannot be explained by the influence of a third variable. The key issue of the design is internal validity or the assumptions that changes in the dependent variable are actually due to the independent variable. Experimental and control groups are created by random assignment. Ethical concerns for the protection of human and animal subjects are the most restrictive. Experimenters more than any kind of researchers, are required to establish that all subjects rights are protected to the greatest degree possible and potential harmful effects are counterbalanced by potential benefits. Data collection is quantitative and prospective. Data analysis is designed to discriminate between and among experimental and control groups. Although experimental studies are the most directly applicable to nursing practice because of their controlled samples, they are the least widely generalizable in and of themselves. This is simply due to the controlled narrowness of the study and sample. Experiments are deigned to be repeated on many samples with small variations in the independent variable over time. A single experiment on a single small sample adds to the test of a part of the theory but not of the entire theory. Many experiments with different samples may be required to increase generalizability to the point where findings can be widely applied in practice. Thus the true experiment is regarded as the cornerstone of scientific research, the most powerful strategy for testing causal hypothesis and achieving the four criteria namely: establishing causal relationships; manipulating independent variable; measuring the impact of independent variable on the dependent variable; minimizing or accounting for the effects of factors other than the independent variable on the dependent variable. B. Types of true experimental designs: A number of different experimental designs meet the criteria for true experiments. These include but are not limited to the classical experimental designs, the factorial design, the multiple treatment groups-repeated measures design and the Solomon four group design. 1. The classical experimental design: Kothari describes them as formal designs either with or without a control group namely before and after without control design, before and after with control design. Similarly post test only design with control grou.p
R X O2 R O2 (R: Random assignment, X: Treatment, O: Observation)
2. Factorial designs: Classical experimental designs allow the manipulation of a single variable at a time, holding all other conditions constant. Factorial designs permit the manipulation of more than one independent variable at a time. Furthermore, interaction effects between variables are revealed and the simultaneous testing of multiple hypotheses is thus allowed. The researcher can study the effects of each of the variables (known as main effects) and the interactions between them or their joint effects on the independent variable. Though commonly used design is 2 x 2 factorial design, experiments can be conducted with any number of categories ad independent variables. Each factor must have two or more levels thus when new factors are added the analysis becomes more complex. E.g. 4 X 5 design means that one of the factors has four categories and the other has five. These designs can also be extended to have more than two factors as in 3 x 2 x 4 factorial design, which means that there are three factors consisting of three, two and four categories respectively. Factorial designs are also referred to as “levels of treatment” designs. When one of the factors cannot be manipulated, such as sex or race it is referred to as blocking variable. The design that incorporates the blocking variable is known as the randomized block design. Stratified randomization is often used in this case to ensure that approximately equal numbers of participants within categories are randomly assigned to the various groups. 3. Multiple Treatment Groups- repeated measures design: This design uses several experimental groups each of which receives a different treatment. Control is achieved through comparison among groups and from measures taken on the dependent variable for all groups at time 1 and time 2. It is always not true that a control group will be used that has no treatment at all. There can also be more than one pretest and posttest measure for the dependent variable for all groups. 4. Solomon Four Group Design: This is a rigorous design by controlling effects on the dependent variable that may be due to factors other than the independent variable. It is essentially a combination of pretest-posttest and posttest only designs. This design requires a large group of homogeneous subjects to make up the four groups, thus is used less often than other designs. It is difficult to introduce the treatment simultaneously for all groups, in order to avoid extraneous temporal effects and statistical analysis complex. The design consists of four groups of which two are control and two are experimental. Pretest and posttest are carried out with one control and one experimental group only and post tests only for the other control and experimental group.
C. Strengths and weaknesses of the design: 1. Strengths: Ability to diminish bias; permit the researcher to maximize systematic variance and to control the extraneous and error variance; control the threats to validity; degree of confidence they allow the researcher in inferring causal relationships, because of high internal validity 2. Limitations: the number of potentially interesting research variables that are not within researcher’s purview to manipulate; ethical concerns; organizational prohibitions; human characteristics often interfere with the researcher’s ability to control a number of variables experimentally; sample representativeness/ homogeneous sample; many cases population is unknown; requires many replications to achieve generalizations; artificiality in laboratory testing; Hawthorne effect; many of the phenomena to be tested lack theoretical background which could be either exploratory or descriptive studies than experimental; issues related to the outcome measure like relevant outcome variables, validity, reliability and systematic bias. VI. Quasi experimental designs: A. Overview of Quasi experimental designs: this design is distinguished from the true experiment by the lack of random assignment of participants to groups. This lack of control limits confidence in the internal validity of the study. Otherwise the design is exactly same as the experimental design. Data are predominantly quantitative, data analysis distinguishes between and among treatments and among treatment groups. Many field experiments actually use a quasi experimental design when the assignment of participants to the various experimental and control groups cannot be controlled by the researcher. E.g. The nurse may want to use an inpatient hospital population in an experiment but physicians control who is admitted to the hospital. A nurse therefore may not have control over extraneous or intervening variables in the available population. The researcher is often not allowed control of the treatment variable and cannot achieve randomization because of ethical considerations, institutional policies or other situational factors. In such circumstances, the researcher chooses quasi experimental studies. Quasi experiments do not have equivalence created by random assignment or that do not have control groups for comparison, rather comparisons are made with nonequivalent groups or with periodic measurement of the same group that may be different due to a number of variables extraneous to the causal variable of interest. Refer to Appendix B B. Types of Quasi experimental designs: 1. Non equivalent group designs: These designs are those in which dependent variable measures are obtained for an experimental and a comparison group (nonrandomly assigned) before and after introduction of the independent variable to the experimental group. When non equivalent group designs are employed, the control group may also receive a treatment, usually traditional. The threat to validity are selection-maturation,, selection-history and regression.
O1 X O2 ( X: Treatment, O: Observation) O1 O2
O1 O2 X O3 (X: Treatment, O: Observation) O1 O2 O3
O1 X+ O2 (X+, X-: Reversed treatment, O: Observation) O1 X- O2
O1 X O1 2. Interrupted time series designs: These designs are those in which the independent variable effects are inferred by comparing measures of the dependent variable obtained at multiple time intervals with multiple dependent variable measures obtained after introduction of the independent variable.
O1 O2 O3 O4 O5 X O6 O7 O8 O9 O10
O1 O2 O3 O4 O5 X O6 O7 O8 O9 O10 O1 O2 O3 O4 O5 O6 O7 O8 O9 O10
Oa1 Oa2 Oa3 Oa4 Oa5 X Oa6 Oa7 Oa8 Oa9 Oa10 Ob1 Ob2 Ob3 Ob4 Ob5 Ob6 Ob7 Ob8 Ob9 Ob10 The subscripts refers to different measures
O1 O2 O3 O4 X O5 O6 O7 O8 O9 X- O10 O11 O12 O13
O1 O2 O3 O4 O5 O6 O7 O8 X O9 O10 O11 O1 O2 O3 X O4 O5 O6 O7 O8 O9 O10 O11
O1 O2 X O3 O4 X- O5 O6 X O7 O8 X- O9 O10 X O11 O12 X- O13 O14
O1 O2 X O3 O4 X+1 O5 O6 X+2 O7 O8 3. Regression discontinuity design: This design is rarely used in nursing, involves systematic assignment of subjects to groups based on cut-off scores on a pre intervention measure, is considered attractive from an ethical point of view and merits consideration. The outcomes of those receiving the different doses of a treatment, not as a result of randomization are compared. For example in complex and lengthy interventions, some people attend more sessions or get more intensive treatment than others. The rationale for a quasi experimental dose response analysis is that if a larger dose corresponds to better outcomes, this provides supporting evidence for inferring that the treatment caused the outcome. The difficulty however is that people tend to get different doses of the treatment because of differences in motivation, physical function or other characteristics that could be driving the differences in outcome and not the different doses themselves. When a quasi experimental dose response analysis is conducted within an experiment, the dose response evidence may well be able to enhance causal inferences. C. Problems with Quasi experimental designs:
D. Patched up designs and structural modeling: The notion of patched up designs allows the researchers to mix and match strategies to best achieve practicality, feasibility and research rigor. Structural modeling depends on strong theory to specify the relevant variables influencing specific hypothesized outcomes. The focus is on comprehensive identification and measurement of all input, process and outcome variables. It entails specifying the true pretreatment assignment processes, theory based specification of outcomes and modeling of extraneous factors that influence the outcomes beyond the intervention, including treatment and mediating subject and treatment systems variables. A. Overview of within subject experimental design: Research designs whether true experiments, quasi experiments or descriptive, each category has unique features as well as some have shared. They have in common within-subject and between subjects design. Is a neglected area in nursing research. There are at times particularly in the clinical studies, when the best control only can be achieved by having participants randomly assigned to order of treatment rather than to groups. A basic assumption in these studies is that the study participants themselves provide the best control group. It is then possible to use very small, unique clinical populations for whom there would otherwise be no adequate control group. B. Difference between within and between subjects designs:
C. Choice of the designs: is based on the pragmatic issues:
A. Overview of evaluative design: Evaluative studies are intended to provide data on the success of action programs.. These research designs are based on the objectives of the program and therefore several designs are based may be combined into one. Many clinical and educational programs have attempted to change something in the current system, improve some practice or enhance problem solving mechanism. These studies are both descriptive and experimental, testing the effect of some manipulation or change. Data may be either qualitative or quantitative and usually both are included. To conduct evaluation research we need to be aware of the differences between using research principles for evaluative purposes and using them for projects that are strictly research endeavors. B. Assumptions of the design:
C. Limitations: The ideal evaluative design has the major ingredients of an experimental design, but there is one more important difference. Because the study is evaluating a specific program, the program participants usually constitute the experimental participants rather than a sample of individuals chosen specifically for a research project. This is an important distinction because, the selection of these participants is usually determined by the program protocol, not the researcher. Thus they are not randomly assigned and there is no guarantee of equivalence between experimental and control participants. Kothari describes 4 formal experimental designs as follows a. Completely randomized design: Involves only two principles namely replication and randomization. The replications may be equal or unequal. It provides maximum number of degrees of freedom and is used when experimental areas happen to be homogeneous. There are two types. First one is two group randomized design in which sample is selected randomly, the experimental and control groups are given different treatments. The second is the random replication design in which a number of repetitions of the treatment. b. The randomized block design: The randomized block design may offset the weakness in conventional randomization by grouping participants who share same characteristics so that like can be compared with like. Participants are matched and put into groups. The groups are then randomly selected to be either experimental or control. The number of groups depend upon, the purpose of the study. One of the limitations of this design is the difficulties involved in managing a large number of groups and in getting adequate sample sizes for each. c. Latin squares design: the treatments in this design are so allocated among the plots that no treatment occurs more than once in any one row or any one column. The two blocking factors may be represented through rows and columns. The merit is that it enables differences in a variable to be eliminated in comparison to the effects of different varieties of the same variable on the dependent variable. In this design one must assume that there is no interaction between treatments and blocking factors. The number in each treatment, row and column must be equal. d. PRPP (Partially Randomized Patient Preference) design: This design has the clear advantage in terms of persuading potential subjects to participate in a study because those with a strong preference get to choose their treatment condition. Those without strong preference are randomized but those with a preference are given the condition they prefer and are followed up as part of the study. The two randomized groups are part of the true experiment and the two groups who get their preference are part of quasi experiment. This type of design can yield valuable information about the kind of people who prefer one condition over another but the evidence of effectiveness of the treatment is weak, because who elected a certain treatment likely differ from those who elected the other alternative. These pre intervention differences rather than the alternative treatment could account for any observed difference in outcomes at the end of the study. e. The Zelen design: In the conventional RCT, participants who meet the inclusion criteria are randomized after they consent to take part. The process of seeking consent in this design depends on whether the single or double consent approach is used. In single version, participants in the control group are not asked consent, not made aware of the trial but included in the analysis. Only from those in experimental groups, consent is obtained. In the double version, all are asked consent, if some in experimental group refuse then they are offered the usual treatment/available alternatives. controlled experiments are considered as the ideal methods in science. True experimental designs are the most powerful method of testing cause-and-effect hypothesis between variables. Although, experimental methods have many limitations. In health sciences, there are many constraints to do experiments and testing the results. Experiments are sometimes criticized for their artificiality. 1. Brink P J, Wood M J. Advanced design in nursing research. 2nd ed. New Delhi: Sage publications; 1998. 2. Parahoo K. Nursing research. 2nd ed. London: Palgrave macmillan; 2006. 3. Knapp T R. Quantitative nursing research. New Delhi: Sage publications; 1998. 4. Grady K A, Wallston B S. Research in health care setting: applied social research methods series. Vol. 14. New Delhi: Sage publications; 1988. 5. Thomas B S. Nursing research. St. Louis: C V Mosby Co.; 1990. 6. Mateo M A, Kirchoff K T. Using and conducting nursing research in the clinical setting. 2nd ed. Philadelphia: W. B. Saunders CO.; 1999. 7. Senn S. Crossover trials in clinical research. New York: John Wiley and Sons; 1993. 8. Couchman W, Dawson J. Nursing and health care research, a practical guide. London: Bailliere Tindall; 1996. 9. Boniface D R. Experimental design and statistical methods. London: Chapman and Hall; 1995. 10. Woods N F, Catanzaro M. Nursing research theory and practice. St. Louis: C V Mosby Co.; 1988. 11. Lo Biondo-Wood G, Haber J. Nursing research. 3rd ed. St. Louis: Mosby; 1990. 12. Notter L E, Hott J R. Essentials of nursing research. 4th ed. New York: Springer publishing co.; 1988. 13. Kothari C R. Research methodology. New Delhi: Wishwa prakashan; 1990. 14. Polit D, Beck C T. Nursing research. 8th ed. New Delhi: Lippincott Williams & wilkins; 2008. 15. Burns N, Grove S K. Understanding nursing research. 4th ed. St. Louis: Saunders; 2007. |
|||||||||||||||||||||||||||||||||||||||||||||||||||
|
copyright @2008 Hosted with support from AIPPG |
||||||||||||||||||||||||||||||||||||||||||||||||||||
