Education research aims to understand how people learn and the effects of learning environments, including sociocultural factors. The ultimate goal is typically to improve learning and enable learners to achieve their goals. How researchers build this understanding and achieve this goal is called research design, which is critical to the quality and validity of the knowledge produced. Research design includes several aspects:

• Crafting research questions that are interesting and answerable
• Selecting research methods that are appropriate and thorough
• Identifying or designing measurements that provide reliable and valid data
• Conducting appropriate analysis of data based on the type of data and the research questions

Because my Ph.D. is in a social science, psychology, about half of my graduate coursework was about research design. In the computer science education community, the most requested talk in my repertoire is about research design. This interest is likely due to many people in our field not having formal training in this critical aspect of education research. Instead, they learn these skills primarily through apprenticeship. This series is designed to help those learning research design through an apprenticeship model.

I originally created this material as a primer for the CS education group I worked with while I was a graduate student at Georgia Tech. I like to think I’ve learned a lot since then, so I’m revamping the content for this series. It will focus on CS education topics in the examples, but the concepts apply to all social sciences. If you are learning about education research design or just need a refresher, this series will help you select the appropriate questions, methods, measures, and analyses for conducting education research. To do this, I’ll cover the following topics.

• Types of Research Questions
• Pre-tests and Post-tests
• Non-experimental and Experimental Design
• Independent and Dependent Variables
• Types of Data: Quantitative, Qualitative, and Mixed
• Levels of Measurement
• Survey Design, Demographics, Validity, and Reliability
• Preparing Data for Quantitative Analysis
• What Statistical Significance Means
• Descriptive Statistics
• Inferential Statistics for Relational Questions
• Inferential Statistics for Causal Questions
• Interpreting and Calculating Effect Sizes
• Additional Analyses: Interrater Reliability and Demographics
• Improving your Design before Collecting Data

To guide the application of concepts to projects – Educational Research Design Worksheet

For additional, and free, information, I use the Research Methods Knowledge Base as a textbook when I research methods. It goes into depth about specific scenarios and types of validity and reliability.

I should disclaim that in my formal psychology training, qualitative and observational research was not discussed. As an education researcher, I have engaged in professional development to learn these relevant skills. However, my tendencies lean toward quantitative and experimental research. I hope to convince more skilled qualitative researchers to add their expertise to resources for the community.