Improving reading interventions through learning science

Overview

About the project

Team EvolvEd is working on a capstone project and is comprised of five students in the Masters of Educational Technology and Applied Learning Sciences (METALS) program at Carnegie Mellon University. Together with a leading producer of educational content and technology, EvolvEd is working to inform features for the next iteration of a reading intervention program. The areas of focus for this project are: student motivation, student accountability and personalization of the student experience.

Hunt Statement

Through analyzing student interactions with the program and its ecosystem, we will design learner-centered features aimed at improving student outcomes, engagement, and accountability in phonics, personalized for various types of struggling readers.

Research

  • Heuristic Analysis

    Systematic evaluation of the user interface

    Considering that “surprise happens to the prepared mind,” we conducted a heuristic analysis to evaluate the current setup of the program in terms of user experience and eLearning design principles.

  • Adaptivity Analysis

    A systematic evaluation of how the program personalizes the student experience at different levels.

    This evaluation helps us reveal potential areas where the program could improve its personalization techniques.

  • Contexual Inquiry

    Observing and questioning users while they work in their own environments.

    This interview technique explores the motivations and thought processes of students as they use the program.

  • Log Data Analysis

    Detecting useful patterns in student performance on instructional activities

    Analyzing the students’ interactions with the software gives us empirical evidence about the types of activities they struggle with the most (or find too easy) and in what contexts.

  • Competitive Analysis

    We analyzed the software to see how other products handle the challenges that the program faces.

Insights

1. In order for students to maintain confidence in their ability to complete certain instructional activities, it is essential for the program to respond appropriately when a task is too difficult.

2. Even if feedback in the form of direct instruction is well-aligned to a student’s errors, it is less effective if it is repeated several times after the same mistakes and can be detrimental to engagement.

3. Students need practice applying isolated reading skills in the context of full sentences/paragraphs.

4. Students value being able to track their progress and see evidence of their own learning. Progress tracking screens/meters should reflect their interests.

5. Generic feedback is less helpful than feedback that addresses individual students’ errors more directly.

6. Students are more engaged with instructional activities are contextualized such that the learning goals for the task are clear.

7. Teachers can quickly become overwhelmed if a large amount of supplementary materials are available to them.

8. Teachers tend to have good intuitions about the root cause of their students’ issues, so they should have some way of directly manipulating their students’ experience in instructional software.

Summer

  • Design

    We are developing an interactive storytelling prototype that motivates students to practice skills they struggle with the most, enabling them to become competent and confident in reading long passages.

  • Implementation

    We are integrating Twine into a web app in order to present a “choose-your-own adventure” style narrative game and record students’ choices as they make their way through the story and complete instructional activities.

Meet the Team

Zachary Mineroff
Coordinator
Roger Strang
Researcher
Julia Ridley
Content & Creative Director
Lu Yang
Designer
Chenxin Wang
Developer