|
EVALUATING AND CRITIQUING
NURSING RESEARCH
Date of last revision :
08-10-08
-------------------------------------------------------------------------
Outline
-----------------------------------------------------------------------------------------------------
I. Introduction
BACK TO TOP
Evidence based practice, is the use of the best clinical evidence in
making patient care decisions and such evidence typically comes from
research conducted by nurses and other health care professionals. Research
is systematic inquiry that uses disciplined methods to answer questions or
solve problems. Nursing research has experienced remarkable growth
in the past three decades, providing nurses with an increasingly sound
evidence base from which to practice. Yet many questions endure and much
remains to be done to incorporate research based evidence into nursing
practice. The authenticity of the research findings, need to be assessed
by careful critical analysis as to broaden the understanding, determine
evidence for use in practice and provide a background for conducting
further study.
II. Definition of critique
BACK TO TOP
1. A research critique is a careful appraisal of the
strengths and weaknesses of the study1
2. An intellectual research critique is a careful, complete
examination of a study to judge its strengths, weaknesses, logical links,
meaning and significance.2
3.
The process of
objectivity and critically evaluating a research report’s content for
scientific merit and application to practice, theory or education.3
III. Steps in conducting
research critique2
BACK TO TOP
1. Read and critique the entire study. A research critique
involves examining the quality of all steps of the research process
2. Examine the organization and presentation of the research
report. A well prepared report is complete, concise, clearly presented and
logically organized. It does not involve excessive jargon that is
difficult for students and practicing nurses to read. The reference need
to be complete and presented in a consistent manner.
3. Examine the significance of the problem studied for nursing
practice. The focus of nursing studies needs to be on the significant
practice problems if a sound knowledge base is to be developed for the
profession.
4. Identify strengths and weakness of a study. All studies
have strengths and weaknesses, so attention must be given to all aspects
of the study.
5. Be objective and realistic in identifying the study’s
strength and weaknesses. Be balanced in the critique. Try not to be overly
critical in identifying a study’s weaknesses or overly flattering in
identifying strengths
6. Provide specific examples of the strengths and weaknesses
of a study. Examples provide evidence for your critique of the strengths
and weaknesses of a study.
7. Provide a rationale for your critique. Include
justifications for the critique and document ideas with sources from the
current literature. This strengthens the quality of the critique and
documents the use of critical thinking skills.
8. Suggest modifications for future studies. Modifications in
future studies will increase the strengths and decrease the weaknesses
identified in the present study.
9. Discuss the feasibility of replication of the study. Is the
study presented in enough detail to be replicated?
10. Discuss the usefulness of the findings for practice. The
findings from the study need to be linked to the findings of previous
studies. All those findings need to be examined for use in clinical
practice.
IV. Phases of research
critique
BACK TO TOP
Phases of research critique are
described briefly here:
A. QUALITATIVE RESEARCH
BACK TO TOP
When critiquing them, one must examine
differences between the qualitative approaches like grounded
theory, phenomenology, or ethnography. Five standards have been developed
to evaluate qualitative studies:
1. Descriptive
vividness:
The study purpose, significance and
interpretations must be articulated in such detail and richness that the
reader has the sense of personally experiencing the event and clearly
understand the significance of the findings. The threats to descriptive
vividness include:
-
Failure to
include essential descriptive information
-
Lack of
clarity in description
-
Inadequate
interpretative/analytic skill (what is most essential, characteristic and
defining about a given phenomenon).
Guidelines:
i. Was the significance of the study adequately described?
ii. Was the purpose of the study clearly described?
iii. Were the interpretations presented in a descriptive way
that illuminated more than the quotes did?
2. Methodological
Congruence:
It requires knowledge of the methodological
approach the researchers used and whether that approach was consistent
with the philosophical basis of the study. Methodological excellence has
four dimensions
a. Adequate documentation of the participants:
Requires a
detailed description of the study participants, rationale for why and how
the participants were selected and a description of the context and
location where the study was conducted. Threats include:
-
Failure to
describe the participants in detail
-
Failure to
provide a rationale for selecting the participants
-
Failure to
describe the context or location of the study so that others can determine
if the findings are applicable to their setting.
b. Careful attention to the procedural approach: How careful
the researcher is in applying selected procedures for the study? To the
extent possible the researcher must clearly state the steps that were
taken to ensure that data were accurately recorded and the data obtained
are representative of the data as whole. Examine the description of
assumptions, the data collection process, role of researcher for threats
to the procedural approach. Threats include
-
Failure to
articulate the assumptions associated with the research
-
Failure to
establish trust with the participants, open dialogue and a conversational
approach to data collection.
-
Failure to
ask appropriate questions that address the participant’s beliefs,
experiences, values or perceptions.
-
Failure to
adequately describe the data collection process
-
Failure to
spend adequate time gathering data or to conduct multiple interviews
-
Failure to
describe the data collection procedures used by multiple data collectors
-
Failure to
use appropriate process for selecting and gaining access to participants
-
Failure to
detail the role of the researcher during the interview process
-
Failure to
describe the qualitative expertise of the researchers
Guidelines:
i. Did the researcher
identify the philosophical or theoretical base of the study?
ii. Were the assumptions
underlying the study articulated? Were the assumptions and data collection
procedures congruent?
iii. Was adequate trust
established with the participants? Was there an open dialogue with a
conversational approach to data collection?
iv. Were research
questions articulated? Did the researcher ask questions that explore
participant’s experiences, beliefs, values or perceptions?
v. Was the data collection
process adequately described?
vi. Did the researcher
spend sufficient time with participants gathering data? Did the researcher
conduct multiple interviews?
vii. Was the approach of
multiple data collectors similar?
viii. Was the method of
selecting and gaining access to the study participants reasonable?
ix. Was the role of the
researcher during the interview process described? Were the researcher’s
qualitative credentials and expertise described?
c. Adherence to ethical standards: requires recognition and
discussion by the researcher of the ethical implications related to the
study. The report must indicate that the researcher took action to ensure
that the rights of the participants were protected during the study.
Examine the data gathering process and identify potential threats which
include:
-
Failure to
inform participants of their rights
-
Failure to
obtain informed consent from the participants
-
Failure to
protect participant rights
d. Auditability:
The research report needs to be
sufficiently detailed to allow a second researcher with a similar
background and philosophical approach, using the original data and the
decision trail, to arrive at conclusions similar to those of the original
researcher. Threats:
-
Failure of
the researcher to record the nature of the decisions made, the data on
which they are based and the decision trail, rules for arriving at
conclusions. Other researchers with a similar background and philosophical
background are not able to arrive at similar conclusions after applying
the decision rules to the data.
-
Failure to
include enough participant quotes to support the findings. The
interpretative statements developed do not correspond with the findings.
-
Failure to
provide quotes that are sufficiently rich or detailed to allow judgments
to be made. This flaw also has been described as not achieving saturation
or redundancy in the data
-
Failure to
provide specific examples of the phenomenon being investigated.
Guidelines:
i. Was the decision trial used in arriving at conclusions
described in adequate detail? Can the findings be linked with the data?
ii. Were enough participant quotes included to support the
findings?
iii. Were the data sufficiently rich to support the
conclusions? Were the findings validated by data? Did the participants
describe specific examples of the phenomenon being investigated?
3. Analytical and interpretative preciseness:
The analytical
process involves a series of interpretations and transformations during
which concrete data are transformed across several levels of abstraction.
The outcome imparts meaning to the phenomenon under study. It requires
that the researcher involve others in the interpretative process and
present a meaningful picture of the phenomenon under study. Threats
include:
-
Failure to
present the findings in a way that yields a meaningful picture of the
phenomenon under study.
-
Failure to
return the findings to participants or experts in the area or to readers
who determine if the results are consistent with common meanings and
understandings.
-
Failure to
involve two or more researchers in data analysis or to describe how
disagreements about data analysis were handled.
4. Philosophical
or theoretical connectedness:
Requires that the findings developed from
the study be clearly expressed, logically consistent and compatible with
the knowledge base of nursing. Study assumptions, methodological
procedures and interpretative/analytic approach must be consistent with
the philosophical or theoretical basis of the study. Threats are:
-
Failure to
link data to nursing practice
-
Failure to
identify a philosophical or theoretical basis for the study
-
Failure to
cite references for the philosophical or theoretical approach used
-
Failure to
link the philosophical or theoretical basis of the study with the study
assumptions, data collection procedures and analytical and interpretative
approach.
Guidelines:
i. Was a clear connection
made between the data and nursing practice?
ii. Did the researcher
identify the philosophical or theoretical basis for the study? Were
citations provided for the philosophical or theoretical approach used?
iii. Was the theoretical
and philosophical basis of the study consistent with the study
assumptions, data collection process and analysis and interpretative
methods used? Were citations provided for the philosophical or theoretical
approach used?
5. Heuristic relevance:
It is reflected in the reader’s ability to
recognize the phenomenon described in the study, its applicability to
nursing practice and its influence on future research. The dimensions
include:
a. Intuitive recognition:
Readers immediately recognize the
phenomenon, its connection to their personal experience and its
relationship to nursing practice. Threat includes failure to present the
findings in a way in which the reader can recognize them as being
consistent with common meanings and experiences. Guidelines are:
b. Relationship to
the existing body of knowledge:
Similarities between the current knowledge base and the study findings add
strength to the findings. The researcher needs to explore reasons for
differences. Examine the degree to which the authors compared and
contrasted the study findings with the results of other researcher’s work.
Threats include:
-
Failure to
examine existing body of knowledge
-
Failure to
compare and contrast the study findings with those of other studies.
-
Failure to
describe the lacunae or omissions in current understandings that would
account for unique findings.
c.
Applicability to nursing practice, research and education:
In the discussion section examine
implications of study findings and suggestions for future research.
Threats include:
-
Failure to
link study findings to nursing practice, research or education
-
Failure to
emphasize how the findings extended what was previously reported in the
literature
-
Failure to
identify implications of the study for related cases
-
Failure to
summarize suggestions for future research
B. QUANTITATIVE
RESEARCH
BACK TO TOP
There
are four critical thinking phases namely comprehension, comparison,
analysis and evaluation. Comparison and analysis are done simultaneously.
1. Comprehension:
-
understanding the terms and concepts in the
report, identifying the elements or steps of the research process such as
problem, purpose, framework and design. It grasps the meaning, nature and
significance of these steps.
2. Comparison:
3. Analysis:
4. Evaluation:
As the critique is on a
quantitative study for the assignment, the guidelines are presented in
detail below.
V. Guidelines for a
quantitative study critique:
BACK TO TOP
I. Title:
II. Abstract:
III. Introduction
BACK TO TOP
I. Problem statement:
a.
What is the
study problem? Is it easy to locate?
b.
Is the
problem stated clearly and unambiguously? Is it easy to identify?
c.
Does the
problem statement build a cogent and persuasive argument for the new
study? Has the research problem been substantiated with adequate
experiential and scientific background material?
d.
Does the problem stated, expresses a
relationship between two or more variables or at least between dependent
and independent variable, implying empirical testability?
e.
Does the
problem specify the nature of the population being studied?
f.
Does the
problem have significance and relevance to nursing? Is the quantitative/
qualitative approach appropriate?
g.
Is there a
match between the research problem and paradigm and methods used?
h.
Is the
problem sufficiently narrow in scope without being trivial?
i.
Was this
study feasible to conduct in terms of money commitment; the researcher’s
expertise; availability of subjects, facility, equipment; ethical
considerations?
j.
Has the
research problem been placed within the context of an appropriate
theoretical framework?
k.
Does the
statement or purpose specify the nature of the population being studied?
ii. Purpose:
a.
What is the
study purpose?
b.
Does the
purpose narrow and clarify the focus or aim of the study and identify the
research variables, population and setting?
c.
Is it worded
appropriately? Are verbs used appropriately to suggest the nature of the
inquiry and or the research tradition?
iii. Objectives:
a.
Formally
stated? Clearly and concisely stated?
b.
Logically
linked to purpose?
c.
Linked to
concepts and relationships from the framework?
d.
Measurable
or potentially measurable and achievable?
e.
Do they
clearly identify the variables and population studied?
iv. Hypotheses:
Are they
a.
Properly
worded?
b.
Stated
objectively without value laden words?
c.
Stating a
predictive relationship between variables?
d.
Stated in
such a way that they are testable?
e.
Directional
or non directional/ research or statistical? Is the direction clearly
stated? Are they causal, associative or simplex versus complex?
f.
Is there a
rationale for how they were stated?
g.
Clearly and
concisely expressed with variables and study population?
h.
Logically
linked to the research problem and purpose?
i. Linked to
concepts, relationships from the framework and literature review?
j.
Used to
direct the conduct of the study?
k.
Absent? If
so is their absence justified? Are statistical tests used in analyzing the
data despite the absence of stated hypothesis?
l. Derived from
a theory or previous research? Is there a justifiable basis for the
predictions?
m. Specific to
one relationship so that each hypothesis can be either supported or not
supported?
v. Conceptual framework:
a. Is the study
framework identified? Is a particular theory or model identified as a
framework for the study?
b. Is the
framework explicitly expressed or must be extracted from the literature
review?
c. Does the
absence of a framework detract from the usefulness or significance of the
research?
d. Does the
framework describe and define the concepts of interest or major features
of the theory/ model so that readers can understand the conceptual basis
of the study?
e. Does the
framework present the relationships among the concepts?
f. Is a map or
model of the framework provided for clarity? If a map or model is not
provided, develop one that presents the study’s framework and describe it.
g. If there was
an intervention, was there a cogent theoretical basis or rationale for the
intervention?
| 