The type of research design that you need depends on the type of research question that you have. Descriptive and relational questions can be answered with non-experimental designs, and causal questions must be answered by experimental designs. Note: these design categories are independent from pre-test and post-test designs, so you can have a pre-post non-experimental design or a pre-post experimental design.

Non-Experimental Design (descriptive and relational questions)

In non-experimental designs, researchers are measuring phenomena as they exist in the world, and they are not systematically manipulating anything, meaning there is no intervention. Because no systematic manipulation occurs, these designs can answer only descriptive or relational questions. Interactions between researchers and the participants in the study should be limited to what is necessary for collecting data. To collect data, researchers might ask participants to fill out surveys or another type of measure. If direct interaction with participants is impossible or might invalidate the data by biasing participants, an observational approach might be appropriate. In observational research, researchers do not directly interact with participants, but they collect data by carefully observing participant behaviors. An example of observational research would be counting the number of contributions from each student in an in-class discussion.

Experimental Design (causal questions)

In experimental designs, researchers systematically manipulate a variable to measure how the intervention affects another variable. For education, commonly the way a topic is taught is manipulated and learning outcomes are measured. This manipulation allows researchers to answer causal questions – it allows researchers to say that they systematically changed one variable, and therefore, differences in the measured variable are likely due to that change (the reason “likely” is used will be explained in a future post about inferential statistics).

If you manipulate some variables and not others, then you have a quasi-experimental design. Because some of the independent variables are manipulated (e.g., instructional style) and some are not (e.g., gender), it cannot be a true experiment. Quasi-experimental designs use many of the same methods and analyses as experimental designs. The conclusions drawn from these analyses, though, are different from experimental designs. Causal relationships can only be concluded if the variable is manipulated. Otherwise, we can only discuss the relationship between variables.

If you design your experimental intervention for a specific learning environment, then you have a design experiment. For example, if you work with a science classroom at your local high school to design an integrated computing activity, then you are designing the interventions (i.e., the activity) for a specific learning environment (i.e., a local classroom). This approach has high ecological validity because the intervention is designed to work in an authentic learning environment, but this design process must be repeated multiple times with various learning environments to gain experimental validity and, ultimately, generalizability.

Design	Definition	Example	Research Question
Non-experimental design	Researchers do not manipulate anything about the learning experience	Observing interactions on a discussion board	Appropriate for descriptive and relational questions
Design experiment	Researchers design manipulations within the context of a learning experience	Creating a CS unit for a math class at a local school	Be careful about drawing causal conclusions for a generalized audience
Quasi-experimental design	Researchers manipulate some variables and not others	Exploring the interaction of teaching style and gender on test scores	Be careful about drawing causal conclusions about non-manipulated variables
Experimental design	Researchers manipulate a variable to determine whether it affects the outcome	Teaching different sections of a course with different styles and measuring test scores	Appropriate for causal questions

To view more posts about research design, see a list of topics on the Research Design: Series Introduction.