Team EvolvEd is working on a capstone project and is comprised of five students in the Masters of Educational Technology and Applied Learning Sciences (METALS) program at Carnegie Mellon University. Together with a leading producer of educational content and technology, EvolvEd is working to inform features for the next iteration of a reading intervention program. The areas of focus for this project are: student motivation, student accountability and personalization of the student experience.
Through analyzing student interactions with the program and its ecosystem, we will design learner-centered features aimed at improving student outcomes, engagement, and accountability in phonics, personalized for various types of struggling readers.
Systematic evaluation of the user interface
A systematic evaluation of how the program personalizes the student experience at different levels.
This evaluation helps us reveal potential areas where the program could improve its personalization techniques.
Observing and questioning users while they work in their own environments.
This interview technique explores the motivations and thought processes of students as they use the program.
Detecting useful patterns in student performance on instructional activities
Analyzing the students’ interactions with the software gives us empirical evidence about the types of activities they struggle with the most (or find too easy) and in what contexts.
We analyzed the software to see how other products handle the challenges that the program faces.
1. In order for students to maintain confidence in their ability to complete certain instructional activities, it is essential for the program to respond appropriately when a task is too difficult.
2. Even if feedback in the form of direct instruction is well-aligned to a student’s errors, it is less effective if it is repeated several times after the same mistakes and can be detrimental to engagement.
3. Students need practice applying isolated reading skills in the context of full sentences/paragraphs.
4. Students value being able to track their progress and see evidence of their own learning. Progress tracking screens/meters should reflect their interests.
5. Generic feedback is less helpful than feedback that addresses individual students’ errors more directly.
6. Students are more engaged with instructional activities are contextualized such that the learning goals for the task are clear.
7. Teachers can quickly become overwhelmed if a large amount of supplementary materials are available to them.
8. Teachers tend to have good intuitions about the root cause of their students’ issues, so they should have some way of directly manipulating their students’ experience in instructional software.
We are developing an interactive storytelling prototype that motivates students to practice skills they struggle with the most, enabling them to become competent and confident in reading long passages.
We are integrating Twine into a web app in order to present a “choose-your-own adventure” style narrative game and record students’ choices as they make their way through the story and complete instructional activities.