Motivation 

To examine the factors that make computer supported collaborative learning (CSCL) environments effective. I’ll admit that the authors refer to this paper as a single meta-analysis, but I’d argue they’ve really done three meta-analyses with subsets of the same (large) set of papers. At the very least, I hope the amount of work that the authors put in isn’t the new standard for completing a meta-analysis.

Three research questions for CSCL

The authors chose to examine the following three research questions simultaneously based on the same set of papers because the efficacy of CSCL environments involves multiple, interrelated factors and can be compared to multiple, valid control groups. CSCL researchers manipulate only a subset of these factors in each study to determine the efficacy of specific interventions. While this controlled approach is scientifically sound, it means that a single study cannot compare CSCL environments to a range of possible alternatives. Therefore, the authors simultaneously considered 356 papers that included 425 studies to determine which features of CSCL are more effective compared to which alternatives.

“RQ1: What are the effects of [collaborative learning] versus [non-collaborative learning] (i.e., individual learning) in computer-supported settings on students’ knowledge gain, skill acquisition, and perceptions?

RQ2: What are the effect of computer use versus no computer use in [collaborative learning] settings on students’ knowledge gain, skill acquisition, perceptions, group task performance, and social interaction?

RQ3: What are the effects of the use of extra technology-mediated learning environments or tools, or supporting strategies in CSCL on students’ knowledge gain, skill acquisition, perceptions, group task performance, and social interaction compared with not adopting them?” (p. 804)

The Effect of Collaborative Learning vs. Non-Collaborative Learning

The first research question used a subset of 84 studies, which included 11,684 participants. For the three components of the comparison, effect size (Hedges’ g), 95% confidence intervals (CI), and number of studies (in parentheses) are

• Knowledge achievement (73), g = 0.42, CI = 0.32 – 0.53
• Skill acquisition (17), g = 0.64, CI = 0.34 – 0.94
• Perceptions (26), g = 0.38, CI = 0.22 – 0.55

For each aspect of the comparison, collaborative learning improved outcomes over non-collaborative learning by a medium effect size. These results suggest, in computer-supported learning environments, using collaborative learning strategies and activities can improve outcomes. Moderator analysis found no variability in different education levels or different subject areas, but it found that quasi-experimental studies tended to have higher gains than randomized experimental studies, studies with sample sizes less than 250 tended to have higher gains than those with sample sizes over 250, and studies that lasted 1 month-1 semester tended to have higher gains than those that were longer or shorter.

The Effect of Computer-Supported Collaborative Learning vs. Non-Computer-Supported Collaborative Learning

The second research question used a subset of 71 studies, which included 11,286 participants. For the five components of the comparison, effect size (Hedges’ g), 95% confidence intervals (CI), and number of studies (in parentheses) are

• Knowledge achievement (64), g = 0.45, CI = 0.33 – 0.56
• Skill acquisition (9), g = 0.53, CI = 0.34 – 0.72
• Perceptions (23), g = 0.51, CI = 0.21 – 0.81
• Group task performance (14), g = 0.89, CI = 0.42 – 1.36
• Social interaction (5), g = 0.57, CI = 0.28 – 0.86

For most aspects of the comparison, computer-supported environments improved outcomes over non-computer-supported environments by a medium effect size, and group task performance improved by a large effect size. These results suggest, in collaborative learning, using computers to support activities and communication can improve outcomes. Moderator analysis found the same effect of research design as for the first research question and that sample sizes less than 100 tended to have higher gains that those over 100.

The Effects of Extra Environments/Tools and Strategies

For the third research question, the analyses for environments/tools and strategies were conducted separately. Environments/tools used a subset of 77 studies, which included 7190 participants. The environments and tools considered were basic online discussion, enhanced online discussion, visual representation tool, group awareness tool, graph or multimedia, adaptive or intelligent tutoring system, and virtual environment. For the five components of the comparison, effect size (Hedges’ g), 95% confidence intervals (CI), and number of studies (in parentheses) are

• Knowledge achievement (61), g = 0.55, CI = 0.39 – 0.71; Basic and Enhanced Online Discussion tools were not significant, and Group Awareness and Graphs or Multimedia tools had larger effect sizes.
• Skill acquisition (10), g = 0.79, CI = 0.42 – 1.15
• Perceptions (32), g = 0.32, CI = 0.18 – 0.46; Basic Online Discussion, Visual Representation, and Adaptive/Intelligent Tutoring tools were not significant.
• Group task performance (31), g = 0.66, CI = 0.42 – 0.90; Enhanced Online Discussion tools were not significant, and Visual Representation and Group Awareness tools had larger effect sizes.
• Social interaction (27), g = 0.40, CI = 0.25 – 0.55

Strategies used a subset of 65 studies, which included 6269 participants. The strategies considered were teacher’s facilitation, peer feedback or assessment, role assignment, and instruction and guidance. For the five components of the comparison, effect size (Hedges’ g), 95% confidence intervals (CI), and number of studies (in parentheses) are

• Knowledge achievement (42), g = 0.38, CI = 0.26 – 0.51; Teacher’s Facilitation was not significant.
• Skill acquisition (15), g = 0.65, CI = 0.28 – 1.03
• Perceptions (26), g = 0.23, CI = 0.07 – 0.38; though the overall gain is significant, none of the gains for the four individual components were significant.
• Group task performance (21), g = 0.45, CI = 0.21 – 0.69; Peer Feedback and Assessment and Role Assignment were not significant.
• Social interaction (28), g = 0.58, CI = 0.41 – 0.74

Why this is important

This comprehensive review makes the case that both the “computer supported” and “collaborative learning” components of CSCL are critical to the success of CSCL. Furthermore, the results of the third research question help to pinpoint the the learning environments, tools, and strategies that make CSCL effective. Given the prevalence of computer-supported learning and the multitude of interrelated features than came make a learning environment effective or not, this paper synthesizes 17 years of work in CSCL to help the community determine where to devote effort to make learning environments most successful.

Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The Role of Collaboration, Computer Use, Learning Environments, and Supporting Strategies in CSCL: A Meta-Analysis. Review of Educational Research, 88(6), 799-843.

For more information about the article summary series or more article summary posts, visit the article summary series introduction.