Project Snapshot:

The Copilot Vision team was shipping new writing features fast but lacked user signal to prioritize the core workflow. I led a usability test combing generative and evaluative questions that surfaced a third interaction pattern users preferred over both options the PM had — one that was adopted into the 2026 roadmap and reshaped the workstream's workflow design direction.

​

Due to NDA constraints, specific product details are not disclosed. This case study focuses on research approach, high-level findings and impact.

​​

Problem Statement: 

The PM needed a confident roadmap call on two competing interaction flows (User Flow A or User Flow B), early signal on the voice dictation feature, and a clear read on usability friction in the prototype. The research had to answer three questions:

​

1. Which interaction flow do users prefer, and why?

2. Do users prefer voice dictation or "voice-in, text-out" for Vision writing tasks?

3. What usability friction and unmet expectations exist in the current prototype?

​

My Role:

I owned this study end-to-end, from recruitment and protocol design, user interview moderation, and data analysis. I collaborated with the Principal UX Researcher on the recruitment and protocol design, then indepedently conducted the user interviews, data analysis, and presented findings directly to the PM, designers, and engineers.

​

Process: 

• Recruitment: The team initially wanted general Copilot users. I recommended recruiting half the participants as active AI writing users, since a general pool would have introduced noise unrelated to the features being tested. The team adopted this approach.

• Study Design: I co-designed a hybrid protocol pairing evaluative methods (to gauge usability issues) with generative methods (to uncover unmet expectations beyond the current feature set). Running both in one study meant I  answered the PM's tactical question ("which flow?") and her strategic one ("what do users actually want?") without needing two separate studies.

• Data Collection: I conducted remote 1:1 sessions via UserTesting.com, observing task completion in real time and probing for reasoning and unmet expectations across both flows.

• Data Analysis: I independently analyzed findings across sessions, identified recurring friction patterns, synthesized insights, and provided design recommendations for the PM, designers, and engineers.

​​

Key Insights: 

1. Neither flow was right, and users pointed to a third flow they desired more. Across sessions, participants gravitated toward an interaction pattern neither flow had accounted for. The signal was consistent enough to bring forward this User Flow C as the concrete alternative.

2. Friction was concentrated, not scattered. Usability issues clustered around specific interaction points, and I provided actionable recommendations that gave designers a prioritized set of fixes they could quickly adopt.

3. Users' preferences for voice dictation v.s. voice-in, text-out features were contextual and based on their individual use cases. This gave the product team a good signal for when each feature would be preferred. 

​

Impact:

• Product Roadmap: User Flow C was adopted into the 2026 feature roadmap and triggered a workstream-wide shift in design direction before further development investment was made. The research share-out influenced not only the  Copilot Vision squad, but also those from other adjacent squads in the same workstream.

• UX Design: Critical design recommendations were adopted by the UX designers to improve usability.

• Research Design: Adding participants with experience using GenAI for writing tasks improved the signal quality for the specialized use case and was adopted as the study's documented methodology.

​

Reflection: 

Due to this being a study I conducted at the beginning of my onboarding to the team, I spent extra time finalizing my report to fit the report style of the MAI team. As I got used to standards, I would be able to speed up my report-writing process and provide insights faster to the stakeholders.