Case Study
Integrating an A.I. Tutor to Improve Student Outcomes
Product & Project Overview
Company/Client: HMH/NWEA
Product: MAP Reading Fluency
Project Goal: Integrate 3rd Party A.I. reading tutor into reading assessment and reporting so students can improve scores and educators can measure performance.
My Role and Team: Lead Product Designer plus 2 Jr. Designers
Key Stakeholders: Sr. Product Manager, Technical Product Manager, Product Owner, Sr. UX Researcher, Lead Developer, Psychometrician, Lead Assessment Designer
Product Environment: Desktop and iOS
Duration: February 2023 - July 2025
Status: On Production
Product Description: Instead of teachers listening to each student read orally one at a time, this product allows teachers to assess students’ reading abilities through varying measures all at one time. Students read aloud and the system scores their performance. Educators use the results to align curriculum with the student’s reading level.
The product is divided into two separate experiences: a student experience and an educator experience.
Example of Student Interface
Example of 3rd Party A.I. Reading Tutor Student Experience
Example of Educator Interface
Select any image to view full-screen
Example of 3rd Party A.I. Reading Tutor Teacher Experience
The Challenge
The MAP Reading Fluency product has historically only been an assessment in which the student takes the assessment and the teacher reviews the results.
However, teachers were asking for help acting on assessments results data. They wanted to know how they could help struggling students and challenge students already exceeding expectations.
Students were at all different levels of fluency and had different needs. Also, teachers just do not have the time to tutor each student individually.
In response, HMH acquired an existing A.I. reading tutor that is adaptive to the student’s performance. My job was to integrate it into the assessment product.
My Approach
As the Lead Designer, I owned the end-to-end product integration experience from discovery through implementation for both students and educators. This included:
Scoping, defining and scheduling the work to be done over 2 years
Surfacing design concerns in large technical meetings
Converting technical requirements to UI needs
Customizing the 3rd party A.I. tutor to match the product’s brand
Facilitating several stakeholder workshops using methods such as discovery, empathy, and prioritization
Conducting low-fidelity concept validation with users
Utilizing and building upon the design system to accommodate new components
Designing an immaculate high fidelity prototype that was well received in user testing and passed rigorous reviews with stakeholders
Working closely with Deque to ensure accessibility met all WCAG 2.2 criteria
Handing off the prototype to the development team fully annotated for interactions and accessibility
Conducting a comprehensive design QA on staging and production
Stakeholder Workshops
These workshops helped everyone understand the problems, what needed to be achieved and when.
User-flows for Technical Stakeholders to Understand Design Intent
The backend aspect of this project was an enormous project due to the complexity of connecting multiple databases, security concerns, and overlapping user permissions. I constantly created flow charts to help product and technical teams understand the design implications of their technical decisions.
Low-fidelity concept validation with users
I believe in getting feedback early and often. This was critical to the success of this project.
High-Fidelity Prototype
I felt confident moving forward with the high-fidelity prototype because it reflected the value users expected.
I used Figma components from the design system created by my design team. The design system I was working from incorporated Highcharts data visualization library. I also worked with a representative from Deque to ensure the design was accessible to all users.
Results
Feedback from testing with real users has been overwhelmingly positive. Teachers now have a way to intervene with struggling students and get them the practice they need to improve their reading fluency
Teachers and School Administrators can assess how each district, school, grade, class, and student are performing.
The product’s growth & retention rates increased after implementation
.
Key Learnings
Having solid research and a great team was critical to the success of this project. I was thrilled to see it all come together.
Reading literacy has worsened in the United States since Covid-19. So many kids are behind. I hope my work on this project helps students get back on track.
This project also helped me take a significant jump in my design leadership. Before HMH/NWEA, I had only worked as a designer for agencies and direct clients. So this was my first in-house position as a product designer. I learned a great deal about working across diverse work groups to collaborate and gain support for my work. I also learned how to lead my team through ambiguity, tight timelines, and guiding the design through established stagegates such as accessibility reviews, content equity & inclusion reviews, technical reviews and design system reviews.