Provide a framework for more consistent, qualitative evaluation of student responses to open-ended questions that can be used in many disciplines to determine the degree to which a student has mastered a learning objective.
Part 1: Structure of the Observed Learning Outcome (SOLO) Taxonomy
The SOLO taxonomy was designed based on student responses to open-ended questions in many disciplines. The taxonomy has 3 dimensions:
- Capacity: the pieces of information required to produce the response, ranging from low (i.e., only the information in the question and one relevant piece of information) to high (i.e., the question, multiple pieces of relevant information, interrelations among information, and abstract principles are included in response)
- Relating operations: the relationship between the question and response, ranging from illogical (e.g., tautologies), to question-specific information only (i.e., answers the question without relating to principles or concepts), to information that generalizes beyond the specific question (i.e., relating response to abstract principles and concepts).
- Consistency and closure: the consistency between information provided and the conclusion that the student comes to, ranging from not answering the question, providing inconsistent evidence or jumping to conclusions, to consistent evidence and multiple conclusions based on relevant possible alternatives.
100 free eprints are available at this link. If the free eprints run out, please contact me, email@example.com.
To review the variables that computing education researchers measure and how they measure them. The particular aim of this review was to highlight areas for improving standardization in the field so that we can more easily make comparisons among projects when appropriate. The review favors quantitative data analysis (as standardization is antithetical to the goals of much qualitative data analysis) but considers the important contribution that qualitative data makes.
Measurement versus data
The first section of the paper is a short primer of often misunderstood concepts in measurement. It is intended for only readers who never had formalized measurement training or who want to check their understanding. The section explains common mistakes or questionable data analysis methods the authors have seen while reviewing, like using the split-mean method or difference/gain scores. For the purpose of a summary, I’ll focus on only the most fundamental point–measurement is not always the same as data. A researcher can use a qualitative measurement to create quantitative data, e.g., by asking a students to write programs (qual measurement) and giving them numeric grades (quant data). Similarly, a researcher can measure continuous data, such as a numeric grade from 0-100, and record ordinal-level data, such as a letter grade from A to F. This difference is important because researchers need to consider the data transformations that occur after measurement to use the correct analysis tools/tests and draw valid conclusions.
Explore the nature of the scaffolding process in which someone with more experience helps someone with less experience to complete a task that the less experienced person would not be able to complete on their own.
Key Elements of Scaffolding
Wood et al. argue that scaffolding is an essential part of human learning for a large range of tasks, such as communication and movement, but focuses on problem solving and higher-order skill acquisition. Within this focus, scaffolding
- is provided in a social context by an expert, instructor, or person with more experience to a novice or person with less experience
- is individualized to the learner yet likely follows a common structure among learners
- enables the novice to “solve a problem, carry out a task, or achieve a goal which would be beyond his unassisted effort” (p.90)
- allows the learners to complete as much of the task themselves as possible (i.e., the expert does not model the complete task for the novice to observe and imitate)
- controls the parts of a task or reduces a problem solving space that would inhibit the novice’s success so that the novice can focus on components within his capability.
Wood et al. hypothesize that scaffolding helps learners to master a task more quickly.
To evaluate the effects of the fast-friendship procedure on social integration and retention in an online college course.
Combating Challenges of Online Education with Social Integration
Though online education has flexibility and accessibility benefits, it also has significant challenges. Online learners must be more self-organized and self-regulated than face-to-face learners, and they face increased social isolation. Moreover, online learners must have high digital literacy and manage several technical tools that may not be integrated, either technologically or curricularly. These challenges are often harder to overcome for non-traditional students, who likely have significant professional and familial obligations, and for students from underrepresented groups, who are often first-generation college students and/or from low socioeconomic status families (implying less access to tech growing up, poorer preparation for college during K-12 education, and higher likelihood of holding a job during college)—the very students who many hope online education would help the most. These challenges can become impassable barriers, leading to low academic performance and high dropout rates. Continue reading
To review the literature on the Advanced Placement (AP) program to determine whether it is reaching both its historical goal of equipping advanced students with college-level skills and credit and its more recent goal to serve students from marginalized backgrounds.
The History and Evolution of AP
Over 50 years ago, the AP program began as a collaboration between elite private schools and universities to provide advanced high school students will opportunities to engage in college-level curriculum and, thus, develop college-level knowledge skills and earn college credit. As college degrees became more common, AP expanded its audience, especially with the goal of serving students who are underrepresented on college campuses. Therefore, AP aims to provide both college-level curriculum for advanced students and equal access to under-served students. While advanced students and under-served students are by no means mutually exclusive, the school and community systems in which they tend to learn are often different. Continue reading
*this paper won the Chair’s Best Paper Award at the 2018 International Computer Education Research conference
To propose a theoretical model of the relationship between spatial skills and computer science (mostly programming) performance and explore the cognitive processes that contribute to both.
Spatial Skill and STEM Performance
Spatial skill is a person’s accuracy and time to complete spatial reasoning tasks, which represent subskills and include
- spatial visualization – mental rotation or transformation of objects
- spatial relations – understanding relationships to landmarks or orientations of objects, such as using a map
- closure speed – completing a partially obscured or incomplete pattern
- closure flexibility – identifying a pattern that is partially obscured or incomplete
- perceptual speed – identifying an unobscured pattern
- visual imagery – translating text or verbal representations to graphic or symbolic representations
*this summary is for an article featured in a Voice of America article that I did an interview for based on my Case for Laptops in the Classroom blog post.
To measure the effect of access to laptops, cell phones, and tablets on student performance in lecture-based classes. Glass and Kang predicted that access to devices would allow students to divide their attention during lecture and hurt their performance on assessments. (For those of you who don’t care about lecture-based classes, stick with it, I’ll connect it back to you at the end).
Students in sections of a cognitive psychology, lecture-based course were allowed to bring their devices to some class periods but not to others. This is a great design because students are being compared to themselves, not to other students who might have other technology habits. In addition, the researchers weren’t forcing students to use devices, they were allowing them to use devices as they normally would. The other clever aspect of the study design was that they measured performance at two time points, during class on just-presented information and a few weeks later as an exam. Continue reading