INSTRUCTIONS
- State the key study research question(s) or hypotheses.
- If a theory(s) is examined or tested, state this theory and its major concepts. If no theory per se is examined, identify the overarching ideas that guide the study.
- Summarize the reviewed literature. Is the literature grounded in social work (and human services) practice?
- Classify the research design. Major choices among quantitative methodologies include: randomized trial (same as true experiment), quasi-experiment, pre-experiment, single-subject/single-case, and time series. If there is an intervention (aka treatment), briefly describe it.
- How did the researcher(s) describe the participants in this quantitative study? How were the participants selected for the study (convenience, purposive, snowball, etc.)? Describe some key characteristics of the study sample (age, gender, ethnicity, key identifying features not mentioned in replying to prior question, etc.).
- Describe the specific criteria and rules used to select the sample. This includes identifying the method of sampling as probability or non-probability. Also, indicate what particular type of probability (simple random, systematic, stratified, cluster, PPS, etc.) or non-probability (convenience/availability, purposive, snowball, etc.) is utilized. (Be sure not to mix up the process of sampling – how persons are chosen to participate in study with that of assignment – how (if at all) participants are assigned to groups/treatment conditions.
- Identify the study’s major variables, classifying these as independent or dependent. If there is an intervention (treatment/stimulus), identify this. You may classify some independent variables as control variables. A control variable is an independent variable that is extraneous to the key research question(s) but which, if not controlled for, could bias study results.
- State the operational definition of two variables in the study. This definition is one and the same as the specific way in which the variable is measured. Please select a study that includes the measurement of at least one “latent” variable.
- When a variable is “hard data” (e.g., birth weight, height), it is self-evident that its reliability is extremely high, so such variables are presumed to be measured “without” error. While you may have such variables in your study, please be sure to include at least one latent variable (i.e. where measurement error is important to assess).
- The quality of measurement of variables is essential to conducting any good research study. For your chosen quantitative study, please assess the reliability of the measures used to capture latent variables found in your study. Assess the reliability of the measures (high/good versus “medium/marginal” versus low/poor) that capture the latent constructs. Note that the reliability of a multi-item scale is often indicated by coefficient data.
- In addition to assessing the reliability of the measurement of any latent variables, please also assess the validity of the measures. To do this, it will be important to understand the difference between reliability and validity. Point out any problems with the validity of the studies’ measures. Also, pay particular attention to social desirability and to other factors that could lead to uncertainty about the quality of the measurement process. (Note: the focus here is on the validity of the study’s measures not on the validity – internal or external – of the research design).
- What are the key study results? (Don’t interpret results here; instead describe them in straightforward, reasonably detailed fashion).
- Given that most quantitative research explores the relationships between variables, please assess the limitations of and/or strengths of the study design with a specific focus on the drawing of causal conclusions. If the study is an experiment, discuss the threats to internal validity that are most relevant. For instance, which key threats cause problems? Which do not? What aspects of research design (i.e., random assignment to groups) prevent and/or facilitate the drawing of causal conclusions? If the study is not an experiment then, rather than discussing the threats to internal validity, use more general language, that is, speak in terms of possible confounding variables and related issues that may affect the drawing of causal conclusions.
- People often speak of “bias” in research. However, “bias” in a research context has a specific meaning that differs from the colloquial usage of the term. In research, bias relates to the quality of inferences drawn from what is found in a sample to the “parent” population from which the sample is drawn. As such, please identify key populations, groups, and/or settings in which you think results from this study would “generalize.” (Hint: the greater the similarity of a population/group/setting to the same characteristics of the study sample, the greater the expected similarity of results).
- Identify key practice or policy recommendations made by the author and/or that you would make based on the study.
- Identify recommendations for future research made by author and/or that you think flow from the study.
- Your comments: What caught your attention? Did the study results turn out as you expected? How could the study be improved? Is the study an important one?
