Research Methodology: Quantitative - Dissertation Writer's Retreat

A true experiment, also called a classical experiment, refers to the random assignment of participants to a control group and a treatment group in order to determine causality. The researcher administers a test or set of tests to both groups and evaluates whether there are observed differences in the outcomes between the two groups.

The researcher can use this design to assess causality because participants are assigned to the groups randomly. Each person in the sampling frame has an equal opportunity of being assigned to either group, so differences in outcomes must therefore be a result of chance, or a result of the different ways each group were treated in the experiment.

Why choose true experiment?

This design is commonly used when the researcher is concerned with the effectiveness of some sort of treatment, and manipulation of variables is possible.

What type of question can be answered using this approach?

One example would be, “Does tutoring have an effect on student performance?” Students would be randomly assigned to one of two groups; one that receives standard instruction, and one that receives tutoring in addition. Both groups would be tested to measure performance outcomes. True experiment researchers typically analyze data obtained from experiments through analysis-of-variance (ANOVA ) techniques.

Tips for true experiment researchers

• Determine your research question from the literature before selecting your research design.
• In some disciplines, the true experiment is the “standard” approach.
• Use the true experiment only if your body of science considers it the preferred approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• You must explain to your reader your rationale for selecting the true experiment design.

The quasi-experiment shares characteristics with the true experiment, but it does not use random assignment. The researcher compares control and treatment groups and manipulates independent variables, but the groups themselves occur naturally.

Why choose quasi-experiment?

This design is commonly used when the researcher is concerned with the effectiveness of some sort of treatment and manipulation of variables is possible, but random assignment is not possible.

What type of question can be answered using this approach?

The same example of “Does tutoring have an effect on student performance?” could also be used for the quasi-experiment method in a situation in which the researcher was unable to select which students would be assigned to either group, but was still able to assign one group to receive standard instruction and one to receive additional tutoring. Quasi-experiment researchers typically analyze data obtained from this type of experiment through analysis-ofvariance (ANOVA ) techniques.

Tips for quasi-experiment researchers

• Determine your research question from the literature before selecting your research design.
• Researchers tend to consider the quasiexperiment a compromise to the true experiment.
• Use the quasi-experiment as an alternative to the true experiment only if your body of science considers it an acceptable approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• You must specifically explain to your reader your rationale for selecting the quasi-experiment design.

A natural experiment refers to naturally occurring contrasts that resemble an experiment. In this type of experiment, the researcher does not manipulate the independent variable; the manipulation occurs naturally. The level or presence of the independent variable occurs outside of the experiment.

Why choose natural experiment?

This design is used when the researcher is concerned with an issue that involves a change in independent variable beyond the researcher’s control.

What type of question can be answered using this approach?

One might ask, “Are recidivism rates affected by legislative changes in sentencing requirements?” In this example, the researcher compares recidivism rates prior to the legislative change with recidivism rates after the change. Researchers typically analyze data obtained from this type of experiment through time series or multiple regression techniques.

Tips for natural experiment researchers

• Determine your research question from the literature before selecting your research design.
• Use the natural experiment only if your body of science considers it an acceptable approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• You must specifically explain to your reader your rationale for selecting the natural experiment design.

In an ex post facto experiment, also called a causal-comparative experiment, the researcher attempts to identify already existing causal factors between groups that are not manipulated by the researcher. In this type of design, the researcher does not administer a treatment; rather the researcher seeks to identify a cause that has already occurred. This method infers the possibility of causality by comparing participants where the characteristic is present with participants where the characteristic is absent.

Why choose ex post facto?

This design is used when manipulation is not possible.

What type of question can be answered using this approach?

This design includes studying participants who differ on an independent variable to see how they differ on a dependent variable, or studying participants who differ on a dependent variable and study how they differ on a set of independent variables. An example of ex post facto research includes examining the records of students who have dropped out of school, compared to those who stayed in school, in order to find possible dropout causes. Researchers typically analyze data obtained from this type of experiment through descriptive techniques as well as analysis-of-variance (ANOVA ) techniques.

Tips for ex post facto experiment researchers

• Determine your research question from the literature before selecting your research design.
• Use the ex post facto experiment as an alternative to the true experiment only if your body of science considers it an acceptable approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• Some researchers consider the ex post facto design exploratory research. Such researchers often use the results of ex post facto experiments to determine causal hypotheses as the basis for designing true experiments.
• An ex post facto research design can look similar to a correlational design, depending on how the data are analyzed. A major difference is that ex post facto research is used in disciplines that value the true experiment, whereas correlational research is used in some disciplines as the primary approach. Further, correlational research tends to use multiple regression techniques rather than analysis-of-variance (ANOVA ) techniques.
• You must specifically explain to your reader your rationale for selecting the ex post facto design.

In this method, participants are recruited to complete a questionnaire used to evaluate, explain, or predict attitudes or behaviors. Survey research comprises both questionnaire and interview approaches, and it may involve cross-sectional or longitudinal designs. Because doctoral research must move beyond simple description, survey research at the dissertation level incorporates the correlational design after the data has been collected.

Why choose survey?

This design is used when the researcher seeks to describe a relationship amongst multiple variables capable of being measured and collected in a survey format.

What type of question can be answered using this approach?

One example is, “How do demographic factors affect citizens’ attitudes toward the use of force by police?” The researcher would collect demographic information from survey respondents, such as race, gender, and age, along with circumstances in which they believe it is appropriate for a police officer to use force. Survey questions would be obtained from the scholarly literature. Researchers typically analyze data obtained from this type of study through multiple regression techniques.

Tips for survey researchers

• Determine your research question from the literature before selecting your research design.
• In some disciplines, survey design is the “standard” approach.
• In some disciplines, survey design is commonly paired with correlational design.
• Use a survey design only if your body of science considers it the most preferred approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• You must specifically explain to your reader your rationale for selecting the survey design.

Correlational research is an assessment of predictive relationships among multiple variables. This type of design is intended to assess whether variables relate to one another, and if so, to what extent.

Why choose correlational?

This design is used when the researcher seeks to describe a relationship among multiple variables. The variables can be collected in a survey format, but other types of variables are also used in correlational research.

What type of question can be answered using this approach?

A researcher might wonder, “What is the relationship between social structural influences and crime?” The researcher could collect crime rate data as a dependent variable, along with independent variables such as concentrated poverty, residential mobility, and family structure. Researchers typically analyze data obtained from this type of study through multiple regression techniques.

Tips for correlational researchers

• Determine your research question from the literature before selecting your research design.
• In some disciplines, correlational design is the “standard” approach.
• Use a correlational design only if your body of science considers it the most preferred approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• A correlational research design can look similar to an ex post facto design, depending on how the data are analyzed. A major difference is that correlational research is used in some disciplines as the primary approach, whereas an ex post facto research design is used in disciplines that value the true experiment.
• You must specifically explain to your reader your rationale for selecting the correlational design.

Developmental research seeks to assess change over time. One approach to this design is through a longitudinal study that looks at the same participants at different points in the lifespan. An alternative approach is a cross-sectional study of subjects who are at different ages.

Why choose developmental?

This design is used when the researcher is concerned with human lifespan development.

What type of question can be answered using this approach?

An example of this type of design is research aimed at understanding the psychological functioning of people from backgrounds of domestic violence at different points in their lifespan. In this example, the researcher might compare measurements of psychological functioning taken in early childhood, adolescence, and young adulthood. Researchers typically analyze data obtained from this type of experiment through time series, multiple regression, and analysis-of-variance (ANOVA ) techniques.

Tips for developmental researchers

• Determine your research question from the literature before selecting your research design.
• The developmental research design is a specialized approach.
• The developmental research design may comprise elements of the correlational design.
• Use a developmental design only if your body of science considers it the most preferred approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• You must specifically explain to your reader your rationale for selecting the developmental design.

The Delphi process is an iterative technique designed to solicit expert opinion regarding a variable that is not fully understood. In this design, the researcher administers multiple rounds of a survey to a panel of experts who are anonymous to each other. The responses from each round are shared with the other experts in order to build consensus, as the experts are expected to revise their answers during each round.

Why choose Delphi process?

This design is used when the researcher is investigating little understood phenomena.

What type of question can be answered using this approach?

An example of this type of design is research aimed at understanding long-term trends in technology development. In this example, the researcher is not able to determine long-term success trends from the scholarly literature, so experts in the field are asked to render their opinions. By sharing the answers with the experts in multiple rounds of questioning, the researcher seeks to build a consensus of what factors contribute to long-term success in technology development. Researchers typically analyze data obtained from this type of study through computing means and reporting quartile ranges.

Tips for Delphi process researchers

• Determine your research question from the literature before selecting your research design.
• The Delphi process research design is a specialized approach that tends to be discipline-specific. It is not a common approach in the literature.
• Use a Delphi process design only if your body of science considers it the most preferred approach for investigating your research question, or if you are specifically suggesting that it is a better way to approach your question than previous approaches in the literature.
• You must specifically explain to your reader your rationale for selecting the Delphi process design.

Stephen W. Verrill, PhD

Dr. Stephen W. Verrill is a faculty member in Capella University’s School of Public Service Leadership. He teaches public safety and research courses, chairs and serves on dissertation committees, serves as a comprehensive examination reader, and is a subject matter expert for course revisions. He is also a quantitative research methodologist for the Dissertation Writer’s Retreat.

Dr. Verrill’s scholarly interests are criminological and criminal justice theory, quantitative methodology, police behavior, and criminal justice education issues. He is the author of a theoretical and methodological monograph, as well as several scholarly articles.

Dr. Verrill earned his PhD in criminology from the University of South Florida. He also holds an MS in criminology, as well as undergraduate degrees in criminal justice and business administration. Prior to academia, Dr. Verrill worked in various capacities of private and public law enforcement.

Achen, C. H. (1982). Interpreting and using regression. Newbury Park, CA : Sage.

Berry, W.D., & Feldman, S. (1985). Multiple regression in practice. Newbury Park, CA : Sage.

Bray, J.H., & Maxwell, S.E. (1985). Multivariate analysis of variance. Newbury Park, CA : Sage.

Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally College Publishing.

Green, S.B., & Salkind, N.J. (2007). Using SPSS for windows: Analyzing and understanding data (5th ed.). Upper Saddle River, NJ: Prentice Hall.

Grimm, L.G., & Yarnold, P.R. (Eds.). (1995). Reading and understanding multivariate statistics. Washington, DC: American Psychological Association.

Grimm, L.G., & Yarnold, P.R. (Eds.). (2000). Reading and understanding more multivariate statistics. Washington, DC : American Psychological Association.

Huck, S.W. (2000). Reading statistics and research (3rd ed.). New York: Longman.

Iversen, G.R., & Norpoth, H. (1976). Analysis of variance. Newbury Park, CA : Sage.

Lewis-Beck, M.S. (1980). Applied regression: An introduction. Newbury Park, CA : Sage.

Miller, D.C., & Salkind, N.J. (2002). Handbook of research design & social measurement (6th ed.). Thousand Oaks, CA : Sage.

Pampel, F.C. (2000). Logistic regression: A primer. Newbury Park, CA : Sage.

Ostrom, C.W., Jr., Time series analysis: Regression techniques (2nd ed.). Newbury Park, CA : Sage.

Schroeder, L.D., Sjoquist, D.L., & Stephan, P.E. (1986). Understanding regression analysis: An introductory guide. Newbury Park, CA : Sage.

Tabachnick, B.G., & Fidell, L.S. (2001). Using multivariate statistics (4th ed.). Boston: Allyn and Bacon.