Motivation

My original intention was to replicate the effects of subgoal labeled worked examples and expository text across different disciplines, but it didn’t really work out like that.

Subgoal learning in expository text

The subgoal learning framework is typically used to break down worked examples into functional pieces that are small enough for novices to grasp (and so small that experts often have a hard time verbalizing because the have become so automatic, further explained here). Subgoals have been used in many fields that focus on procedural problem solving since the 1970s, and most of my work has been applying the framework to programming education. In this work, I explored adding subgoal labels to expository text (i.e., the text that abstractly describe the problem solving procedure) in addition to worked examples (i.e., a concrete problem with the worked out solution that learners can use as a model). I found the combination of both subgoal labeled text and examples to further improve performance in programming over subgoal labeled worked examples alone (Margulieux & Catrambone, 2016). I argued that because students tend to struggle to translate between abstract descriptions of procedures and concrete examples of procedures, having the same subgoal labels in both types of instruction helps them to make connections between the two.

Students have this issue in every STEM field, so I figured these results would replicate for procedures in other STEM fields (statistics and chemistry) using the same framework. Well…

Results

There was a different pattern of results for each discipline. In the original programming study, giving students subgoal labeled text alone did not improve performance. It was only the interaction of subgoal labeled text AND examples that made the subgoal labeled text effective and caused the group who received both to perform best. In statistics, though, the subgoal labeled text had no effect at all, whether by itself or combined with subgoal labeled examples. My post hoc explanation for this was that the procedure I used (solving problems with the Poisson distribution) was too straightforward, meaning students did need help to understand it.

To test this hypothesis, I tried solving stoichiometry problems in chemistry to add a level of complexity. Students had to figure out how to plug numbers into an equation, like for the stats procedure, while balancing the equation for number of molecules needed. The result was that subgoal labeled text improved performance, whether the students had the subgoal labeled examples or not. If students received both subgoal labeled text and examples, they performed best, but the effect of the subgoals was additive (i.e., subgoal labeled text improved performance and subgoal labeled examples improved performance, so having both double improved performance) rather than interactive like it was for programming (i.e., subgoal labeled text improved performance ONLY when paired with subgoal labeled examples).

This pattern of results led me to believe that complexity of the problem matters for whether subgoal labeled text is effective. Subgoal labeled worked examples were consistently effective regardless of discipline. I also argued as an idea for future research that students’ familiarity with tools might effect whether subgoal labeled text is effective by itself or only when paired with subgoal labeled examples. Expository text is valuable in instruction because it provides an abstract explanation of a procedure that will work for all problems within a class. However, if students are unfamiliar with the tools, such as a programming language, used to solve problems, then they will find it impossible to translate that abstract knowledge into the concrete steps needed to solve a specific problem. Using subgoal labels in both text and examples can show student the connections between abstract and concrete that they might otherwise miss.

Why this is important

Replication studies are always important but are either not conducted or published enough (thanks Instructional Science for publishing this one!). Even though this replication study didn’t work as expected, by conducting the same study in different STEM disciplines, we are able to see differences among disciplines. Understanding the differences allows us to explain why student learn and perform differently and how instruction should adapt to the demands of each discipline. While there will always be more similarities in how people learn than differences, identifying differences is important for effective instruction and learning.

This article is also discussed in a post on Mark Guzdial’s blog, Phys.org, and ScienceBulletin.org.

Margulieux, L. E., Catrambone, R., & Schaeffer, L. M. (2018). Varying effects of subgoal labeled expository text in programming, chemistry, and statistics. Instructional Science, 46(5), 707-722. doi: 10.1007/s11251-018-9451-7

Margulieux, L. E., & Catrambone, R. (2016). Improving problem solving with subgoal labels in expository text and worked examples. Learning and Instruction, 42, 58-71. doi: 10.1016/j.learninstruc.2015.12.002

For more information about the article summary series or more article summary posts, visit the article summary series introduction.