When using inferential statistics, we commonly use two paradigms to understand our results: null hypothesis significance testing (NHST, aka statistical significance) and effect sizes (aka the quantitative difference between groups). NHST determines whether there is a difference between groups, and effect sizes describe the difference between groups. While there is a movement away from NHST towards effect sizes, the current best practice typically includes both. This post explains how to interpret and calculate effect sizes.

The last post discussed inferential statistics for relational questions. This post will discuss inferential statistics for causal questions. Most commonly when asking a causal research question, an experimental design is appropriate. In this case, the purpose of inferential statistics is to determine whether the difference between groups is greater than what would be expected due to chance. For example, if you tested a new teaching method and found that students who received this new teaching method scored 5 points higher on a test than students who didn’t, it would matter if the standard deviation were 2 points or 20 points. If the standard deviation were 2 points, then the mean of the intervention group is 2.5 standard deviations higher than the other group, and the average student in the intervention group scored better than 98% of the other group. However, if the standard deviation were 20 points, then the difference in means likely isn’t greater than what would be expected due to chance.

The purpose of inferential statistics is to determine if the features of a sample (e.g., participants in a study) are representative of a population (i.e., all people belonging to a group). For example, the question “Do first-generation college students study more than other students?” must be answered with inferential statistics unless you can collect data from all college students. Thus, inferential statistics allow you to make some generalizations about your findings beyond your sample, and they are used to answer relational and causal research questions. The posts on inferential stats will cover many topics (at least two semesters of stats classes), so to break it up, this post will focus on only relational questions.

Descriptive statistics provide summaries of quantitative data, which can include quantified qualitative data. They are intended to describe the data as they are rather than draw conclusions beyond your sample of participants. Thus, they are used for descriptive research questions. Inferential statistics are needed to address relational or causal research questions.

In the scientific method, we collect data to support or refute hypotheses, not to prove or disprove them. We frame scientific research in this way because there might be factors that we are unaware of that affect the results. In human-subjects research, one reason we cannot prove or disprove hypotheses is that we use samples to represent populations rather than whole populations.

When handling quantitative data, there are a number of steps that need to be completed before you can run your first test. This post describes a basic protocol for data cleaning and tools that you can use for analysis.

Creating a Data File

When you create your data file, analytic software expects that individual participants are represented on the rows and variables are represented on the columns like in the example table below.

Surveys are prominent in quantitative educational research for measuring students’ opinions or self-reported data, such as study habits. Data collected from survey research can be any level of measurement. For example, if you ask the question “How much time did you study per week?” you could collect ordinal data (e.g., “none” “some,” or “a lot”) or ratio data (e.g., number of hours). This example also illustrates the most common criticism of survey data, which is that it can be unreliable. Even if students don’t skew their answers to be socially desirable based on what they think the researcher wants to hear, they might not be able to give an accurate response from memory. For validity, it is better to have direct measurements, but in education, this is often impractical or invasive. Evaluate the trade-offs in your work.