Article Summary: Margulieux et al. (2019) Review of Measurements in Computing Education Research

50 free eprints are available at this link. If the free eprints run out, please contact me, lmargulieux@gsu.edu.

Motivation 

To review the variables that computing education researchers measure and how they measure them. The particular aim of this review was to highlight areas for improving standardization in the field so that we can more easily make comparisons among projects when appropriate. The review favors quantitative data analysis (as standardization is antithetical to the goals of much qualitative data analysis) but considers the important contribution that qualitative data makes.

Measurement versus data 

The first section of the paper is a short primer of often misunderstood concepts in measurement. It is intended for only readers who never had formalized measurement training or who want to check their understanding. The section explains common mistakes or questionable data analysis methods the authors have seen while reviewing, like using the split-mean method or difference/gain scores. For the purpose of a summary, I’ll focus on only the most fundamental point–measurement is not always the same as data. A researcher can use a qualitative measurement to create quantitative data, e.g., by asking a students to write programs (qual measurement) and giving them numeric grades (quant data). Similarly, a researcher can measure continuous data, such as a numeric grade from 0-100, and record ordinal-level data, such as a letter grade from A to F. This difference is important because researchers need to consider the data transformations that occur after measurement to use the correct analysis tools/tests and draw valid conclusions.

Continue reading

Article Summary: Wood et al. (1976) Role of Tutoring in Problem Solving

Motivation

Explore the nature of the scaffolding process in which someone with more experience helps someone with less experience to complete a task that the less experienced person would not be able to complete on their own.

Key Elements of Scaffolding

Wood et al. argue that scaffolding is an essential part of human learning for a large range of tasks, such as communication and movement, but focuses on problem solving and higher-order skill acquisition. Within this focus, scaffolding

  • is provided in a social context by an expert, instructor, or person with more experience to a novice or person with less experience
  • is individualized to the learner yet likely follows a common structure among learners
  • enables the novice to “solve a problem, carry out a task, or achieve a goal which would be beyond his unassisted effort” (p.90)
  • allows the learners to complete as much of the task themselves as possible (i.e., the expert does not model the complete task for the novice to observe and imitate)
  • controls the parts of a task or reduces a problem solving space that would inhibit the novice’s success so that the novice can focus on components within his capability.

Wood et al. hypothesize that scaffolding helps learners to master a task more quickly.
Continue reading

Article Summary: Stürmer et al. (2018) Relationship Building for Higher Distance Education

Motivation

To evaluate the effects of the fast-friendship procedure on social integration and retention in an online college course.

Combating Challenges of Online Education with Social Integration

Though online education has flexibility and accessibility benefits, it also has significant challenges. Online learners must be more self-organized and self-regulated than face-to-face learners, and they face increased social isolation. Moreover, online learners must have high digital literacy and manage several technical tools that may not be integrated, either technologically or curricularly. These challenges are often harder to overcome for non-traditional students, who likely have significant professional and familial obligations, and for students from underrepresented groups, who are often first-generation college students and/or from low socioeconomic status families (implying less access to tech growing up, poorer preparation for college during K-12 education, and higher likelihood of holding a job during college)—the very students who many hope online education would help the most. These challenges can become impassable barriers, leading to low academic performance and high dropout rates. Continue reading

Article Summary: Kolluri (2018) Advanced Placement: The Dual Challenge of Equal Access and Effectiveness

Motivation

To review the literature on the Advanced Placement (AP) program to determine whether it is reaching both its historical goal of equipping advanced students with college-level skills and credit and its more recent goal to serve students from marginalized backgrounds.

The History and Evolution of AP

Over 50 years ago, the AP program began as a collaboration between elite private schools and universities to provide advanced high school students will opportunities to engage in college-level curriculum and, thus, develop college-level knowledge skills and earn college credit. As college degrees became more common, AP expanded its audience, especially with the goal of serving students who are underrepresented on college campuses. Therefore, AP aims to provide both college-level curriculum for advanced students and equal access to under-served students. While advanced students and under-served students are by no means mutually exclusive, the school and community systems in which they tend to learn are often different. Continue reading

Article Summary: Parkinson & Cutts (2018) Relationship between Spatial Skills and Computer Science

*this paper won the Chair’s Best Paper Award at the 2018 International Computer Education Research conference

Motivation

To propose a theoretical model of the relationship between spatial skills and computer science (mostly programming) performance and explore the cognitive processes that contribute to both.

Spatial Skill and STEM Performance

Spatial skill is a person’s accuracy and time to complete spatial reasoning tasks, which represent subskills and include

  • spatial visualization – mental rotation or transformation of objects
  • spatial relations – understanding relationships to landmarks or orientations of objects, such as using a map
  • closure speed – completing a partially obscured or incomplete pattern
  • closure flexibility – identifying a pattern that is partially obscured or incomplete
  • perceptual speed – identifying an unobscured pattern
  • visual imagery – translating text or verbal representations to graphic or symbolic representations

Continue reading

Article Summary: Glass & Kang (2018) Laptops/Phones in the Classroom

*this summary is for an article featured in a Voice of America article that I did an interview for based on my Case for Laptops in the Classroom blog post.

Motivation

To measure the effect of access to laptops, cell phones, and tablets on student performance in lecture-based classes. Glass and Kang predicted that access to devices would allow students to divide their attention during lecture and hurt their performance on assessments. (For those of you who don’t care about lecture-based classes, stick with it, I’ll connect it back to you at the end).

Study design

Students in sections of a cognitive psychology, lecture-based course were allowed to bring their devices to some class periods but not to others. This is a great design because students are being compared to themselves, not to other students who might have other technology habits. In addition, the researchers weren’t forcing students to use devices, they were allowing them to use devices as they normally would. The other clever aspect of the study design was that they measured performance at two time points, during class on just-presented information and a few weeks later as an exam. Continue reading

Article Summary: Margulieux et al. (2018) Varying effects of subgoal labels in 3 STEM fields

Motivation

My original intention was to replicate the effects of subgoal labeled worked examples and expository text across different disciplines, but it didn’t really work out like that.

Subgoal learning in expository text

The subgoal learning framework is typically used to break down worked examples into functional pieces that are small enough for novices to grasp (and so small that experts often have a hard time verbalizing because the have become so automatic, further explained here). Subgoals have been used in many fields that focus on procedural problem solving since the 1970s, and most of my work has been applying the framework to programming education. In this work, I explored adding subgoal labels to expository text (i.e., the text that abstractly describe the problem solving procedure) in addition to worked examples (i.e., a concrete problem with the worked out solution that learners can use as a model). I found the combination of both subgoal labeled text and examples to further improve performance in programming over subgoal labeled worked examples alone (Margulieux & Catrambone, 2016). I argued that because students tend to struggle to translate between abstract descriptions of procedures and concrete examples of procedures, having the same subgoal labels in both types of instruction helps them to make connections between the two.

Continue reading