WileyPLUS
Transforming
Digital Education
Welcome to Wiley, Mancuso
In May 2019, Wiley, the 215-year-old Publisher based in Hoboken, NJ, completed its acquisition of Knewton. I was asked to stay aboard to lead UX for Wiley’s digital education products. I saw this as a unique opportunity to improve multiple products with a much larger user base than Knewton. I wasn't as sure about the reverse commute.
The ferry from Manhattan to Hoboken, NJ.
After a brief period of onboarding at Wiley, the UX Design strategy came into view. We would build on the overwhelmingly positive user sentiment for Knewton Alta based on the UX work I had led for the past 3 years at Knewton, and leverage the same design system for New WileyPLUS, for the first time creating a common visual language for Wiley's Education products.
To be successful, we would need a high degree of communication and collaboration among the Product-UX-Engineering triumvirate. So, I was sent to Moscow with the VP of Engineering to build a (new, better) relationship with the 150 person Russian engineering team - all Wiley employees of nearly 20 years, and evangelize about the value of using a design system and our new focus on UX research and design to validate our work. The relationship with UX and Product thereafter dramatically improved, and UX gradually earned the respect for our roles and expertise, and we began to row in the same direction again.
Red Square: My trip to Moscow to meet with the Russian engineering team in the Fall of 2019. Note: Wiley has since reorganized its global engineering resources in light of geopolitical events.
Director of User Experience, Digital Education
My Role
After years of underinvestment in UX, this UX leadership role was created for me in the Digital Education Division as part of the Academic and Professional Learning Group at Wiley. It's been a unique opportunity to grow as a leader and make a real impact in a vitally-important industry.
My Product Portfolio
WileyPLUS ($56M in FY21)
(Wiley's flagship courseware product)
Knewton Alta
(Lower-cost adaptive courseware)
Wiley Efficient Learning
(Test prep for professional certification)
My Responsibilities
Lead UX Design and Strategy
Lead Product Research
Mentor and Grow the UX Team
Build process with Product & Engineering
Manage UX Agency
My Direct Reports* & Extended Team
6
UX Designers*
NYC, Hoboken, Cincinnati
3
UX Reseachers*
Cincinnati
8
Product Managers
Hoboken
25
Engineers
Hoboken
150
Engineers
Moscow
25
Engineers
Sri Lanka
My Mission
Driven by a combination of poor user feedback, looming technical and infrastructure limitations, business goals and internal pressures, the executive leadership team challenged my UX Team in partnership with Product and Engineering with measurably improving the overall product experience.
Alleviate Urgent User Pain Points
Reduce friction for assignment building and editing in New WileyPLUS
Improve Learning Management System (LMS) integration (80% of our user base)
Simplify policies and settings complexity
Solve numerous miscellaneous bugs and issues
Increase poor user sentiment
Increase sales and adoptions
Create a More Intuitive User Experience
Develop patterns and repeatable mental models across learning platforms
Enable self-service with user onboarding and in-product support
What does the UI want me to do next? Clarify prominence and priority of calls to action
Provide context clues and number of steps in a given process
Reduce need for training and documentation
Converge Products onto a Design System
Modernize the WileyPLUS UI to better compete in the marketplace
Base system on existing React-based MUI.com to move faster
Wiley products to share patterns, aesthetics and mental models
Reduce design and development effort for repeatable elements
Address accessibility concerns and VPAT issues with verified and tested components
Migrate 850,000+ users to a single platform
Move 850k WileyPLUS (legacy) users over to join 150k users on 'New WileyPLUS'
Support planning and messaging of the migration over multiple semesters with a hard deadline for completion
Alleviate the biggest omissions and pain points in NWP for a smoother migration
Behold the Before Times
'New WileyPLUS' users complained of a long list of pain points. Internally, key adoptions were difficult to win, while existing customer relationships were at risk. As a result, relationships between Wiley Sales, Support and Product were strained, making it difficult to focus on new features.
Until 2019, approximately 850,000 users were still on a legacy version of (Old) WileyPLUS with a codebase and server infrastructure considered too frail to reliably accept further updates and enhancements, and user's too fearful to go through the considerable work of migrating and remaking all of their courses and assignments. Meanwhile about 150,000 users had adopted or been migrated to 'New WileyPLUS' with less features and more problems.
The bar in Higher Education is surprisingly low. I set out to change that.
The Student Assignment Experience: So. Many. Navs. The cluttered assignment UI embedded within an iFrame of a custom front-end of the Canvas LMS.
The Student Resource Hub: A confusing to navigate list of supplemental materials for instructors to assign or for student to browse to support completion of the course.
The Instructor Assignment Builder: A multistep process built into an overlay with large amounts of un-scannable metadata and no clear sense of current state or next steps.
The Instructor Policy Review Step: Policies, or settings that govern the experience, grading or availability of content and features for students were complex, wordy and difficult to parse.
Validating the Problems
What are the actual problems we need to solve for our users?
Part of the problem in the years before I arrived was that all customer feedback went directly, unfiltered, into the product. ALL of it.
Shortly after joining Wiley, I set out to conduct a series of focus groups with instructors and student users in higher education, the most vocal of which supplied by our frustrated sales colleagues. Each focus group was either focused on the pain points of a single discipline or around specific tasks, such as assignment building.
Additionally we interviewed numerous internal stakeholders in Sales, Marketing, Customer Support, Content and even SME's to better understand the problem space. In many cases, what we learned confirmed what we already knew, but the research helped us to formulate a plan to address customer pain points in priority order.
In the process we collected some choice feedback from disgruntled users. There was a common, if broad, refrain:
If you think that's bad, imagine what the Sales Team was saying!
Discovery Process
Kick-Off, Value Proposition Design Exercise
Starting with our kick-off in October of 2019, what follows is many months of information gathering, user research, and design exploration based on the roadmaps we developed together.
At our kick-off in Oct 2019 we conducted a Value Proposition Design exercise and gathered jobs, pains and gains, pictured.
A snippet of the output of the VPD exercise, we used these as the basis for research as well as to inform our approach for feature design.
We conducted focus groups and 1:1 interviews to refresh our personas.
I asked the team to create a UX Research Dashboard to capture our research activities with a number of different data points and rollups, such as sheer engagement metrics by initiative, touch points by discipline, Institution and region, and more.
Clicking through the items on the upper-left lead to in-depth data per initiative and linked to readouts with a summary of findings and insights from each research initiative. This proved to be very valuable.
It was important to understand the product portfolio and high level architecture to begin planning what would lead to an inititive to converge and re-platform our products on a micro-services architecture (ongoing)
Instructor Tools with Actionable Insights
Responsive Dashboard for Adaptive Assignments
Analytics for Instructors to track class and individual student progress and performance against learning objectives, empowering teachers to focus on improving student outcomes. These tools aid Instructors in the triage of issues in real time, and to plan for their next lecture or exam prep.
Students tab view of an in-progress adaptive assignment, with class progress, struggling student callout, and individual performance metrics for instructors to observe and correct issues as soon as they arise
Mobile class and student performance data for busy instructors on the move
The Assignment Builder
Mitigating the labor and complexity of browsing for content on a granular, question-by-question level with the immediacy of assignment building and organizing details like question order, type, difficulty, and point value in a concise and scalable side-by-side experience.
Browse and preview question content on the left, shopping cart on the right for Instructors to add, view, arrange, preview and set values for gathered questions. Setting clearer user expectations with a persistent stepper.
Drag and drop reordering of questions in the assignment builder tool.
We wove custom illustration into the entire end-to-end experience, with a clean and simple aesthetic throughout to delight our users with a more personal experience.
Empty state illustrations add personality to the process of creating a Question Set, the basis of a summative assignment.
Better Onboarding
A combination of illustration, animation and clear UX writing was added to new onboarding dialogs to both introduce and contextualize new and redesigned experiences to help our customers as well as our sales and support colleagues get on and stay on the same page.
Surprisingly, this type of user onboarding was virtually non-existent on the product before we hit the scene.
Adaptivity measures a student's aptitude for a learning objective based on their answers to questions served from the knowledge graph
Mastery of the learning objective is visualized in an assignment, with a high water mark that prevent struggling students from backsliding and losing any forward achievement
At the preset threshold which can be less than 100%, the student sees a celebration of their achievment, promoting growth mindset
The onboarding was deployed into WileyPLUS for all students experiencing an adaptive assignment for the first-time
The Student Adaptive Assignment Experience
We integrated Knewton's Adaptive technology into New WileyPLUS to enhance the summative offering with adaptive assignment leveraging Wiley content. The result followed and extended existing UX/UI patterns for a seamless and consistent experience.
Assignment launch point iFrame from within the Canvas Learning Management System (LMS) UI, using a custom illustration from a library created in-house
Cover page, assignment not started
In-assignment experience with progress details in drawer and empty-state illustration
In-assignment experience with 50% completion and high water mark indicator
100% progress achievement celebration with confetti animation to add light gamification and promote growth-mindset
Cover page showing 100% progress
A student's question history for granular review of performance by learning objective
A Quantified, UnQualified Success
20% SUS increase, major STEM UEQ improvements measured 1 year after release of the new experience
Wiley contracted a 3rd party to conduct research with WileyPLUS Instructors in 2019, the year I joined as Director of UX, and again in 2021 to measure the impact of more than two years of our work.
Using both a System Usability Scale (SUS), which is a subjective assessment of usability that measures effectiveness, efficiency and user satisfaction, as well as a User Experience Questionnaire (UEQ)to measure things like attractiveness and ability to meet a user's goals, researchers measured a significant increase in overall scores for New WileyPLUS but more than 20% in this timeframe.
Potential reasons for these improved scores include the opinion that improvements completed in this period have lightened Instructor's workloads and provided them with extra time to focus on their students. In addition, the new Blackboard integration facilitated the adoption of WileyPLUS amongst part time instructors.
Diverse Feedback in STEM & Accounting
In 2019, benchmarking was done with 30+ instructors in disciplines such as STEM Anatomy and Accounting. In 2021, to follow-up that research after numerous improvements were made, we talked to 3x the number of instructors (~100) across a diverse range of STEM disciplines and in Accounting.
Usability Improved Overall
The System Usability Scale (SUS) score significantly improved amongst STEM instructors from 2019 - 2021, while it held steady or slightly improved for Accounting Instructors, an area requiring further investment to improve experiences with large and complex question experiences.
20% increase in Usability
WileyPLUS received an average of 66, which translates to a 41-59 percentile rank. While this suggests room for improvement, it is also worth noting that this SUS score has increased by 19.6 % compared to the SUS score for the 2019 WileyPLUS design, which was 55.2.
Greatly Improved Usability
WileyPLUS scored “Good” on overall results on the UEQ. Hedonic quality (design, pleasantness) scored good while pragmatic quality (usability) scored above average.
The Voice of the Customer has... spoken!
Further evidence from the same time period generated by Wiley's internal "voice of the customer" recurring Qualtrics survey of Instructors at US-based higher Education institutions who use New WileyPLUS clearly shows sharp improvement in both overall satisfaction and future intent ratings over previous terms.