WileyPLUS

Transforming Digital Education @Wiley

Director of User Experience, Digital Education

Responsibilities

Lead UX Design and Strategy

Lead Product Research

Implement a design system

Build & Mentor the UX Team

Establish process with Product & Engineering

Manage UX Agency

Products

WileyPLUS

Higher Ed Courseware, 10MM users


Knewton Alta

Adaptive Higher Ed Courseware (STEM)


Wiley Efficient Learning

Professional Certification Test Prep

6

UX Designers

NYC, Hoboken, Cincinnati

3

UX Reseachers

Cincinnati

8

Product Managers

Hoboken

200

Engineers

 Hoboken, Moscow, Sri Lanka

The Problem:

"We are facing negative user feedback, rock-bottom customer satisfaction, and tons of design and technical debt."

Behold the Before Times

'New WileyPLUS' users complained of a long list of pain points that disrupter instructor workflows and prevented students from completing some assignments. Internally, key adoptions were difficult to win, while existing customer relationships were at risk. As a result, relationships between Wiley Sales, Support, and Product were strained, making it difficult to focus on new features.

The Student Assignment Experience: 👏 So. 👏 Many. 👏 Navs. The cluttered assignment UI is embedded within an iFrame of a custom front-end of the Canvas LMS.
The Student Resource Hub: A confusing to navigate list of supplemental materials for instructors to assign or for students to browse to support completion of the course.
The Instructor Assignment Builder: A multistep process built into an overlay with large amounts of un-scannable metadata and no clear sense of the current state or next steps.
The Instructor Policy Review Step: Policies, or settings that govern the experience, grading, or availability of content and features for students were complex, wordy, and difficult to parse.

A Mountain of Negative Feedback

What are the actual problems we need to solve for our users?

Part of the problem in the years before I arrived was that all customer feedback went directly, unfiltered, into the product. ALL of it. 

Shortly after joining Wiley, I set out to conduct a series of focus groups with instructors and student users in higher education, the most vocal of which was supplied by our frustrated sales colleagues. Each focus group was either focused on the pain points of a single discipline or on specific tasks, such as assignment building.

Additionally, we interviewed numerous internal stakeholders in Sales, Marketing, Customer Support, Content, and even SME's to better understand the problem space. In many cases, what we learned confirmed what we already knew, but the research helped us to formulate a plan to address customer pain points in priority order.

In the process, we collected some choice feedback from disgruntled users. There was a common, if broad, refrain.

If you think that's bad, imagine what the Sales Team was saying!

User Experience Goals

Alleviate Urgent User Pain Points

Reduce friction for assignment building and editing in New WileyPLUS

Improve Learning Management System (LMS) integration (80% of our user base)

Simplify policies and settings complexity

Solve numerous miscellaneous bugs and issues

Increase poor user sentiment

Increase sales and adoptions

Create a More Intuitive User Experience

Develop patterns and repeatable mental models across learning platforms

Enable self-service with user onboarding and in-product support

What does the UI want me to do next? Clarify prominence and priority of calls to action

Provide context clues and number of steps in a given process

Reduce need for training and documentation

Converge Products onto a Design System

Modernize the WileyPLUS UI to better compete in the marketplace


Base system on existing React-based MUI.com to move faster


Wiley products to share patterns, aesthetics and mental models

Reduce design and development effort for repeatable elements

Address accessibility concerns and VPAT issues with verified and tested components 

Discovery: Process & Tools

Kick-Off, Value Proposition Design Exercise

Starting with our kick-off in October of 2019, what follows is many months of information gathering, user research, and design exploration.

At our kick-off in Oct 2019 we conducted a Value Proposition Design exercise and gathered jobs, pains and gains, pictured.

A snippet of the output of the VPD exercise, we used these as the basis for research as well as to inform our approach for feature design.

We conducted focus groups and 1:1 interviews to refresh our personas.

It was important to understand the product portfolio and high level architecture to begin planning what would lead to an inititive to converge and re-platform our products on a micro-services architecture (ongoing)

Early ideation for organisms (groups of molecules) that form information-rich headers in the courseware experience.

UX Research Dashboard & Interactive Convergence Roadmap

I asked the team to create a UX Research Dashboard to capture our research activities with a number of different data points and rollups, such as sheer engagement metrics by initiative, touch points by discipline, Institution and region, and more. Clicking through the items on the upper-left lead to in-depth data per initiative and linked to readouts with a summary of findings and insights from each research initiative. This proved to be very valuable.
The Student and Instructor Journey interactive roadmap (built in Figma) highlights features and enhancements across Wiley's 3 main courseware products converging on a shared design system. Zoom views for individual feature details and captured pain points and opportunities.

Shipped Design

Instructor Tools with Actionable Insights

Responsive Dashboard for Adaptive Assignments

Analytics for Instructors to track class and individual student progress and performance against learning objectives, empowering teachers to focus on improving student outcomes. These tools aid Instructors in the triage of issues in real time, and to plan for their next lecture or exam prep.

Students tab view of an in-progress adaptive assignment, with class progress, struggling student callout, and individual performance metrics for instructors to observe and correct issues as they arise
Mobile class and student performance data for busy instructors on the move

The Assignment Builder

Mitigating the labor and complexity of browsing for content on a granular, question-by-question level with the immediacy of assignment building and organizing details like question order, type, difficulty, and point value in a concise and scalable side-by-side experience.

Browse and preview question content on the left, shopping cart on the right for Instructors to add, view, arrange, preview and set values for gathered questions. Setting clearer user expectations with a persistent stepper.
Drag and drop reordering of questions in the assignment builder tool.
We wove custom illustration into the entire end-to-end experience, with a clean and simple aesthetic throughout to delight our users with a more personal experience.
Empty state illustrations add personality to the process of creating a Question Set, the basis of a summative assignment.

Better Onboarding

A combination of illustration, animation and clear UX writing was added to new onboarding dialogs to both introduce and contextualize new and redesigned experiences to help our customers as well as our sales and support colleagues get on and stay on the same page.

Surprisingly, this type of user onboarding was virtually non-existent on the product before we hit the scene.

Adaptivity measures a student's aptitude for a learning objective based on their answers to questions served from the knowledge graph
Mastery of the learning objective is visualized in an assignment, with a high water mark that prevent struggling students from backsliding and losing any forward achievement
At the preset threshold which can be less than 100%, the student sees a celebration of their achievment, promoting growth mindset
The onboarding was deployed into WileyPLUS for all students experiencing an adaptive assignment for the first-time

The Student Adaptive Assignment Experience

We integrated Knewton's Adaptive technology into New WileyPLUS to enhance the summative offering with adaptive assignment leveraging Wiley content. The result followed and extended existing UX/UI patterns for a seamless and consistent experience.

Assignment launch point iFrame from within the Canvas Learning Management System (LMS) UI, using a custom illustration from a library created in-house
Cover page, assignment not started
In-assignment experience with progress details in drawer and empty-state illustration
In-assignment experience with 50% completion and high water mark indicator
100% progress achievement celebration with confetti animation to add light gamification and promote growth-mindset
Cover page showing 100% progress
A student's question history for granular review of performance by learning objective

Results, Quantified

20% SUS increase, dramatic UEQ improvements after release of the new experience

Wiley contracted a 3rd party to conduct research with WileyPLUS Instructors in 2019, the year I joined as Director of UX, and again in 2021 to measure the impact of more than two years of our work.

Using both a System Usability Scale (SUS), which is a subjective assessment of usability that measures effectiveness, efficiency and user satisfaction, as well as a User Experience Questionnaire (UEQ)to measure things like attractiveness and ability to meet a user's goals, researchers measured a significant increase in overall scores for New WileyPLUS but more than 20% in this timeframe.

System Usability Scale (SUS) Increased by 20%

Usability Improved Overall

The System Usability Scale (SUS) score significantly improved amongst STEM instructors from 2019 - 2021, while it held steady or slightly improved for Accounting Instructors, an area requiring further investment to improve experiences with large and complex question experiences.

20% increase in Usability 

WileyPLUS received an average of 66, which translates to a 41-59 percentile rank. While this suggests room for improvement, it is also worth noting that this SUS score has increased by 19.6 % compared to the SUS score for the 2019 WileyPLUS design, which was 55.2.

Excellent User Experience Questionaire (UEQ) Results 

In 2019, benchmarking was done with instructors across a diverse range of STEM disciplines. Then, two years later after numerous improvements and new flows were shipped, we measured again with instructors.

Instructor CSAT & Future Intent 📈

Further evidence from the same period generated by Wiley's internal "voice of the customer" survey of Instructors at US-based higher Education institutions who use New WileyPLUS clearly shows sharp improvement in both overall satisfaction and future intent ratings over previous terms.