Microsoft Foundry
This partnership was done in collaboration with the course HCDE 517: Usability Testing, taught by Katya Cherukumilli at the University of Washington.
IMPACT
Influenced implementation of 2 high priority recommendations for 80k+ users
Evaluated the effectiveness of Foundry's discover and exploration features
Delivered a series of design recommendations to a team of 30+ people
TEAM
4 Usability Researchers
ROLE
Usability Researcher
TIMELINE
Jan-Mar 2026
10 Weeks
SKILLS
Usability Testing
Qualitative Data Collection
Data Triangulation
COLLABORATORS
1 Product Designer
1 Product Manager
1 Research Ops Manager
1 User Researcher
BACKGROUND
Foundry is a comprehensive platform to build, deploy, and manage generative AI applications and agents.
Our client at Microsoft Core AI was most interested in gauging how easy it was for users to complete their tasks with discovery the tools provided to them, and uncovering whether these tools matched their expectations. They expressed that their "north star metric" was conversion rate.
In other words…
Can users successfully discover AI models that suit their needs, and what do their typical workflows look like?
PARTICIPANT PROFILE
Due to the specialized nature of Foundry as a platform for AI developers, we utilized a questionnaire via User Interviews to screen people based on their AI development experience. To reduce bias, we excluded those with UX research experience from the participant pool.
METHODOLOGY
The foundation of our study plan begins with translating business goals into research goals.
Business Goals
The "north star" metric is
conversion rate
Research Goals
Understand how users:
Navigate the platform
Evaluate & compare model information
Determine the most suitable model
FINDINGS & DESIGN RECOMMENDATIONS
Foundry's discovery tools were misaligned in user intention, clarity, and representation of information.
While our users could technically get around just fine relying on their existing knowledge about current AI models their experience searching for and comparing these models could definitely be improved.
Our job was to highlight points of friction that might prevent the completion of a task, so we organized our issues based on high priority (prevents completion of a task), medium priority (causes significant delay), and low priority (has minor effect on usability).
HIGH PRIORITY
Issue 1: Low discoverability of Foundry's own "Ask AI" tool
While our users could technically get around just fine relying on their existing knowledge about current AI models their experience searching for and comparing these models could definitely be improved.
Our job was to highlight points of friction that might prevent the completion of a task, so we organized our issues based on high priority (prevents completion of a task), medium priority (causes significant delay), and low priority (has minor effect on usability).
Reflections
While I learned a lot this summer, from prototyping techniques in Figma to new user research tools, these are some of my biggest takeaways:
1
Precise wording is everything
When my team was drafting screener questions, we used words such as "familiarity" to ask users to describe their experience with AI development tools. However, we didn't realize that "familiarity" looks different for everyone— and in the realm of usability testing, it's important to make sure that everyone has the same baseline when they're evaluating themselves to prevent bias. In the end, we opted to ask users for years of experience.
2
A question for you! And you!
It was a struggle designing those achievements screens, mostly because it was hard for me to understand what was important to our target user base on my own. But, with some Slack messages here and Zoom interviews there, collaborating with our stakeholders really helped ground my perspective on how to best design for our target user base.
3
Juggling B2B2C needs
While there were a lot of moving parts in this project, grounding design decisions in lived experiences and our core objectives prevented me from losing sight and helped me pinpoint that sweet spot for everyone.
