The type of research design that you need depends on the type of research question that you have. Descriptive and relational questions can be answered with non-experimental designs, and causal questions must be answered by experimental designs. Note: these design categories are independent from pre-test and post-test designs, so you can have a pre-post non-experimental design or a pre-post experimental design.
In non-experimental designs, researchers are measuring phenomena as they exist in the world, and they are not systematically manipulating anything, meaning there is no intervention. Because no systematic manipulation occurs, these designs can answer only descriptive or relational questions. Interactions between researchers and the participants in the study should be limited to what is necessary for collecting data. To collect data, researchers might ask participants to fill out surveys or another type of measure. If direct interaction with participants is impossible or might invalidate the data by biasing participants, an observational approach might be appropriate. In observational research, researchers do not directly interact with participants, but they collect data by carefully observing participant behaviors. An example of observational research would be counting the number of contributions from each student in an in-class discussion.
In experimental designs, researchers systematically manipulate a variable to measure how the intervention affects another variable. For education, commonly the way a topic is taught is manipulated and learning outcomes are measured. This manipulation allows researchers to answer causal questions – it allows researchers to say that they systematically changed one variable, and therefore, differences in the measured variable are likely due to that change (the reason “likely” is used will be explained in a future post about inferential statistics).
If you manipulate some variables and not others, then you have a quasi-experimental design. Because some of the independent variables are manipulated (e.g., instructional style) and some are not (e.g., gender), it cannot be a true experiment. Quasi-experimental designs use many of the same methods and analyses as experimental designs. The conclusions drawn from these analyses, though, are different from experimental designs. Causal relationships can only be concluded if the variable is manipulated. Otherwise, we can only discuss the relationship between variables.
|Non-experimental design||Researchers do not manipulate anything about the learning experience||Observing interactions on a discussion board||Appropriate for descriptive and relational questions|
|Experimental design||Researchers manipulate a variable to determine whether it affects the outcome||Teaching different sections of a course with different styles and measuring test scores||Appropriate for causal questions|
|Quasi-experimental design||Researchers manipulate some variables and not others||Exploring the interaction of teaching style and gender on test scores||Be careful about drawing causal conclusions about non-manipulated variables|
To view more posts about research design, see a list of topics on the Research Design: Series Introduction.