Research Design: Non-Experimental and Experimental Designs

The type of research design that you need depends on the type of research question that you have. Descriptive and relational questions can be answered with non-experimental designs, and causal questions must be answered by experimental designs. Note: these design categories are independent from pre-test and post-test designs, so you can have a pre-post non-experimental design or a pre-post experimental design.

Non-Experimental Design (descriptive and relational questions)

In non-experimental designs, researchers are measuring phenomena as they exist in the world, and they are not systematically manipulating anything, meaning there is no intervention. Because no systematic manipulation occurs, these designs can answer only descriptive or relational questions. Interactions between researchers and the participants in the study should be limited to what is necessary for collecting data. To collect data, researchers might ask participants to fill out surveys or another type of measure. If direct interaction with participants is impossible or might invalidate the data by biasing participants, an observational approach might be appropriate. In observational research, researchers do not directly interact with participants, but they collect data by carefully observing participant behaviors. An example of observational research would be counting the number of contributions from each student in an in-class discussion.

Experimental Design (causal questions)

In experimental designs, researchers systematically manipulate a variable to measure how the intervention affects another variable. For education, commonly the way a topic is taught is manipulated and learning outcomes are measured. This manipulation allows researchers to answer causal questions – it allows researchers to say that they systematically changed one variable, and therefore, differences in the measured variable are likely due to that change (the reason “likely” is used will be explained in a future post about inferential statistics).

If you manipulate some variables and not others, then you have a quasi-experimental design. Because some of the independent variables are manipulated (e.g., instructional style) and some are not (e.g., gender), it cannot be a true experiment. Quasi-experimental designs use many of the same methods and analyses as experimental designs. The conclusions drawn from these analyses, though, are different from experimental designs. Causal relationships can only be concluded if the variable is manipulated. Otherwise, we can only discuss the relationship between variables.

If you design your experimental intervention for a specific learning environment, then you have a design experiment. For example, if you work with a science classroom at your local high school to design an integrated computing activity, then you are designing the interventions (i.e., the activity) for a specific learning environment (i.e., a local classroom). This approach has high ecological validity because the intervention is designed to work in an authentic learning environment, but this design process must be repeated multiple times with various learning environments to gain experimental validity and, ultimately, generalizability.

DesignDefinitionExampleResearch Question
Non-experimental designResearchers do not manipulate anything about the learning experienceObserving interactions on a discussion boardAppropriate for descriptive and relational questions
Design experimentResearchers design manipulations within the context of a learning experienceCreating a CS unit for a math class at a local schoolBe careful about drawing causal conclusions for a generalized audience
Quasi-experimental designResearchers manipulate some variables and not othersExploring the interaction of teaching style and gender on test scoresBe careful about drawing causal conclusions about non-manipulated variables
Experimental designResearchers manipulate a variable to determine whether it affects the outcomeTeaching different sections of a course with different styles and measuring test scoresAppropriate for causal questions

To view more posts about research design, see a list of topics on the Research Design: Series Introduction.

3 thoughts on “Research Design: Non-Experimental and Experimental Designs

  1. Pingback: Research Design: Series Introduction | Lauren Margulieux

  2. Pingback: Research Design: Dependent and Independent Variables | Lauren Margulieux

  3. Pingback: Research Design: Inferential Statistics for Causal Questions | Lauren Margulieux

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s