Knewton Alta

Bringing Adaptive Courseware to Market

Director of User Experience Research & Design

My Role

In 2016 I was hired as Director of User Experience Research & Design to lead and grow the UX team in building out Knewton Alta, the forthcoming direct-to-consumer courseware offering leveraging Knewton's adaptive technology and knowledge graph for STEM disciplines.

My Product Portfolio

I see UX as an end-to-end concern, and quickly got involved in multiple key aspects of the customer experience:

Knewton Alta (Responsive Web App)
Knewton.com (WordPress Marketing)
Feedback & Support (Intercom + Jira)

My Responsibilities

Establish Product Design thinking and process in an engineering-lead culture that likes to ship, then fix

Lead Product Design and Strategy

Lead Product Research - know our users

Partner with Product, Support, Sales

Mentor and Grow the UX Team

3

UX Designers*

NYC

1

UX Researchers*

NYC

7

Product Managers

NYC

50

Engineers

NYC, Mauritius

Bringing Adaptive Courseware to Market

Knewton Alta is an adaptive learning courseware product for higher education.

The technology can measure a student’s aptitude against specific learning objectives within the knowledge graph and assess a student's real-time mastery. In addition, Alta gives Instructors analytics and insights to intervene with struggling students as well as automated just-in-time remediation.

Starting in 2016 I directed or personally designed nearly every customer-facing asset from the marketing site, to the responsive web app, to the support experience and internal tools.

One of many custom illustrations and animations commissioned for Alta for a more delightful user experience

A quick introduction to the Knewton mission and ethos.

Building a Culture of Design Thinking

I put a high value on an open exchange of ideas and information from various sources to be a better-informed UX Team and a design-thinking organization.

I would frequently share inspiration and links with my team and to a community board accessible to the entire company, and refer back to these often to build awareness and evangelize for UX as a core discipline vital to our mission.

Balancing Quality with Speed

The Knewton Product and Engineering team mantra like many in the industry was 'ship and learn', so moving fast and being scrappy was a requirement.

In order to validate our UX work we had to rapidly get from problem set to solutions for testing, and often iterate on features to improve them over time. This is a push-pull that can be uncomfortable for a perfectionist designer but contributed to our growth as a team.

Collaborative whiteboarding sessions were an almost daily technique to build consensus and share knowledge in a visual way.

Mapping the End-to-End Customer Experience: Rapidly building a shared understanding of the end-to-end customer journey, from onboarding to in-product communications to support, with a focus on optimizing customer communication. Ideation started with internal stakeholders, then was vetted with potential customers and users in Higher Ed to inform the UX design.

Dashboards: Concept sketches for data roll-ups and core functionality for administrators of higher education institutions to manage multiple courses across many domains and instructors, and even multiple campuses.

Notifications: Ideating on in-product notifications UI and features to consolidate user comms into a single flexible space.

The Alta Design System

Familiar & Accessible

With speed and quality being two core principles for the UX Design team, implementing a design system within Figma was an obvious choice. By building on top of Material UI we could get to market faster while leveraging a visual language already familiar to students and instructors alike.

Introducing Design Sprints

Visualizing Progress and Learning Objective Mastery

I ran modified design sprints and other collaborative whiteboarding exercises with key stakeholders to quickly align on the problem set and ideate on solutions for exploration by the UX research and design team.

Student Progress Design Sprint

A summary presentation of the goals, process and outputs of a design print focused on visualizing progress and mastery for students.

Synthesis of Ideas from the Group

Capturing the design sprint output into sketches that ultimately informed the rapid paper prototype testing with 50+ students from Marshall University.

Paper Prototype Research with Students

Through their instructors in partnership with our sales and customer success teams, a relatively large sample of active students from Marshall University were asked to circle the concept that most clearly communicates assignment progress and discuss why/why not.

student feedback.pdf

Product Design

Alta Student Experience: Adaptive Assignments and Just-in-Time Remediation

Below are a mix of design variations we tested as well as designs that shipped as a result of that design sprint, backed by research, ideation, iteration and continual user feedback.

Students ultimately expressed feelings of anxiety and stress when the progress bar sometimes went backwards during an adaptive assignment experience (consistent with the true nature of adaptivity). We later scaled back the progress visualization in size and narrowed it's focus to individual learning objectives.

We also later discussed implementing a high water mark so students would never experience long assignments and a backslide from their current level of mastery.

Alta Instructor Experience: Course Dashboard

Views of the Instructor dashboard experience with a focus on simple analytics to help a busy instructor track and manage student performance. We provide clear metrics filterable by progress against mastery of a given learning objective, assignment status, individual student and individual question performance granularity and more to help instructors intervene and even plan lectures to target trouble areas.

An Intercom onboarding feature tour for Instructors

Modern, clean login page. Pretty standard stuff.

Registration page with a novel, if naive means of identifying instructors from students through self-selection. Instructors were verified by our team using multiple vectors and given access to course and assignment building and metrics.

My Courses list, filterable by active state

Course outlines designed by administrators and course designers at universities can serve as the basis for many custom variations for course sections taught by other instructors and even TA's, each with their own metrics.

Adaptive assignment mastery and summative quiz and test analytics, with average grade rollup per course section with ability to drill down to individual activities.

Student Metrics with data visualizations that also serve to filter the data shown in the student table below.

Consistent, Verifiable Positive Results

Measured 3x/semester via a short, in-app survey that enjoyed high levels of engagement (35%), our customer satisfaction surveys measured consistent, high levels of satisfaction for instructors (86%) and even for students (67%) which is somewhat remarkable for what is effectively a homework and assessment experience.

Student Pilot Feedback, Canada 2020

“It was useful to have as it broke down the steps really well so that it was easy to identify one's mistakes“

“I found that I learned the concepts from Knewton not just how to do the questions”

“It was very easy to use. It gave good instructions and taught me a lot of topics that I struggled with”

“I personally got stuck on geometric series and it provided a great explanation that I couldn't find in external sources”

“I think it provided a valuable experience for my learning and it challenged me to go beyond my original thinking and made it into a critical challenge question to test my knowledge”

“[Alta] gave multiple examples of similar questions so you could really understand how to solve a type of question/apply a method of solving before proceeding”

“I used Knewton Alta as my main study method for the midterm and I had an increase of 15% in my midterm mark"

“Really enjoyed using the system. Appreciated that it was aware of where I was struggling and could help me through those aspects of the course“

Instructor Satisfaction Survey (in-app), End of Semester, Fall 2019

I LOVE KNEWTON ALTA! I wish all of my classes had software this awesome. Most importantly, the support I receive is far beyond the support I get from other publishers I use for my other classes.

Flexible, immediate feedback and instruction to students, transfer of responsibility to student, actually learning from homework mistakes.

I have experienced homework and test managers from the perspective of Lumen Learning, MyMathLab, and Moebius (Maple TA).

But Knewton's interface has been those most elegant and streamlined.

About

Report abuse