Fine-tune Problem-Solution Fit with Mixed-Method Concept Generation
problem-solution fit
concept development
maxdiff
concept testing
mixed-method
Problem
Existing homeowners primarily used our My Home dashboard to check home values. We wanted to increase engagement, especially for those looking to sell, by providing more personalized insights beyond just a home value estimate. However, it was unclear what types of insights would be most useful and compelling.
Problem
Existing homeowners primarily used our My Home dashboard to check home values. We wanted to increase engagement, especially for those looking to sell, by providing more personalized insights beyond just a home value estimate. However, it was unclear what types of insights would be most useful and compelling.
Research Questions
- What problems do users aim to solve with the Owner dashboard today?
- How well does the Owner dashboard uniquely solve these problems today?
- How to better improve the solution to better meet the problems identify?
Research Questions
- What problems do users aim to solve with the Owner dashboard today?
- How well does the Owner dashboard uniquely solve these problems today?
- How to better improve the solution to better meet the problems identify?
Process
We began by assessing the current product situation, user satisfaction, and alternatives. Upon defining the problem, we generated a list of potential enhancements. The dashboard's aim is to help homeowners better understand their home value by providing them with comprehensive and unique insights such as buyer demand and price/sq. ft.. Using MaxDiff, we gauged user preferences across varied insights, then develop and assess concepts based on users’ perceived level of improvement from existing solutions.
![]()
Process
We began by assessing the current product situation, user satisfaction, and alternatives. Upon defining the problem, we generated a list of potential enhancements. The dashboard's aim is to help homeowners better understand their home value by providing them with comprehensive and unique insights such as buyer demand and price/sq. ft.. Using MaxDiff, we gauged user preferences across varied insights, then develop and assess concepts based on users’ perceived level of improvement from existing solutions.
![]()
Phase 1
Assess Problem-Solution Fit
The assessment shows that current users use the dashboard to track home value out of curiosity and financial considerations, often performing regular checks. While the home estimate feature addresses initial curiosity, there's a gap for more meaningful insights. Users seek broader real estate insights for decisions like buying, selling, and renting, highlighting the need to address their future aspirations beyond just valuation.
![]()
︎︎︎Problem-solution fit interviews focus on grasping the issues users engage our tool to resolve, the milestones they aim to reach, and their reasons to engage alternative solutions.
Phase 1
Assess Problem-Solution Fit
The assessment shows that current users use the dashboard to track home value out of curiosity and financial considerations, often performing regular checks. While the home estimate feature addresses initial curiosity, there's a gap for more meaningful insights. Users seek broader real estate insights for decisions like buying, selling, and renting, highlighting the need to address their future aspirations beyond just valuation.
︎︎︎Problem-solution fit interviews focus on grasping the issues users engage our tool to resolve, the milestones they aim to reach, and their reasons to engage alternative solutions.
Methodology Notes
︎ Why assess Problem-Solution Fit instead of Product-Market Fit?
We focused on assessing problem-solution fit rather than product-market fit because, at the time of our research, the product was still in an early stage without a clearly defined monetization strategy. The goal then was to increase engagement and loyalty within the larger realtor.com ecosystem of products, rather than directly monetize users. Problem-solution fit keeps the focus on the user needs and making sure they are being adequately addressed.
︎ Why interviews instead of survey to assess PSF?
Since this was an early-stage product, we were still uncertain about the full scope of user problems and potential use cases. Interviews were the preferable method for assessing problem-solution fit because:
- Interviews allow for deeper, nuanced discussions to uncover users' needs, jobs-to-be-done, and pain points. We are interested in unexpected usages or pain points that could shape our product vision.
- Small sample sizes from interviews are more feasible than the larger samples needed for statistically significant surveys.
- Interviews allow direct observation of how users interact with our products and alternatives. This provides contextual insights unavailable in surveys.
Methodology Notes
︎ Why assess Problem-Solution Fit instead of Product-Market Fit?
We focused on assessing problem-solution fit rather than product-market fit because, at the time of our research, the product was still in an early stage without a clearly defined monetization strategy. The goal then was to increase engagement and loyalty within the larger realtor.com ecosystem of products, rather than directly monetize users. Problem-solution fit keeps the focus on the user needs and making sure they are being adequately addressed.
︎ Why interviews instead of survey to assess PSF?
Since this was an early-stage product, we were still uncertain about the full scope of user problems and potential use cases. Interviews were the preferable method for assessing problem-solution fit because:
- Interviews allow for deeper, nuanced discussions to uncover users' needs, jobs-to-be-done, and pain points. We are interested in unexpected usages or pain points that could shape our product vision.
- Small sample sizes from interviews are more feasible than the larger samples needed for statistically significant surveys.
- Interviews allow direct observation of how users interact with our products and alternatives. This provides contextual insights unavailable in surveys.
Phase 2
Identify and Rank Improvements
After conducting user interviews and generating a potential list of insights, we employed a MaxDiff survey (n=500+ homeowners) to efficiently gauge relative preferences for various insights, avoiding issues with linear ratings and enabling trade-offs.
Insights for testing were gathered from user interviews and competitor analysis, including both existing dashboard features and new possibilities, highlighting potential improvements like individual home demand insight.
![]()
︎︎︎Items receive utility scores via statistical analysis, indicating their relative preference. These scores determine a rank-order list, with higher positive averages signifying greater participant preference and lower negative averages indicating lesser preference.
Phase 2
Identify and Rank Improvements
After conducting user interviews and generating a potential list of insights, we employed a MaxDiff survey (n=500+ homeowners) to efficiently gauge relative preferences for various insights, avoiding issues with linear ratings and enabling trade-offs.
Insights for testing were gathered from user interviews and competitor analysis, including both existing dashboard features and new possibilities, highlighting potential improvements like individual home demand insight.
Insights for testing were gathered from user interviews and competitor analysis, including both existing dashboard features and new possibilities, highlighting potential improvements like individual home demand insight.
︎︎︎Items receive utility scores via statistical analysis, indicating their relative preference. These scores determine a rank-order list, with higher positive averages signifying greater participant preference and lower negative averages indicating lesser preference.
Phase 3
Develop and Test Concepts
Following the MaxDiff analysis, we carried out concept testing to determine the most effective approach for implementing insights.
We engaged 600+ homeowners in testing various concepts to illustrate the identified demand insights from MaxDiff. We evaluated these concepts alongside a control to gauge their usefulness, uniqueness, and comprehension. Our goal is to pinpoint a solution that not only enhances overall usefulness but also provides a distinct competitive advantage.
![]()
︎︎︎Sequential monadic concept testing involves participants experiencing two randomly selected stimuli from a set of four, including one benchmark and three test concepts. The benchmark is crafted based on attributes like comprehension, utilities, uniqueness, and likelihood of use, with significant testing to highlight differences.
Phase 3
Develop and Test Concepts
Following the MaxDiff analysis, we carried out concept testing to determine the most effective approach for implementing insights.
We engaged 600+ homeowners in testing various concepts to illustrate the identified demand insights from MaxDiff. We evaluated these concepts alongside a control to gauge their usefulness, uniqueness, and comprehension. Our goal is to pinpoint a solution that not only enhances overall usefulness but also provides a distinct competitive advantage.
We engaged 600+ homeowners in testing various concepts to illustrate the identified demand insights from MaxDiff. We evaluated these concepts alongside a control to gauge their usefulness, uniqueness, and comprehension. Our goal is to pinpoint a solution that not only enhances overall usefulness but also provides a distinct competitive advantage.
︎︎︎Sequential monadic concept testing involves participants experiencing two randomly selected stimuli from a set of four, including one benchmark and three test concepts. The benchmark is crafted based on attributes like comprehension, utilities, uniqueness, and likelihood of use, with significant testing to highlight differences.
Impacts
︎
Shaping Product Strategy:
The process provided a clear direction for developing a personalized demand insight to drive engagement that are not offered by competitors.
︎
Quick Cycle from Diagnosis to Improvement
A mixed-method process enabled effective discovery and delivery of concept, resulting in clear roadmap feature.
Impacts
︎
Shaping Product Strategy:
The process provided a clear direction for developing a personalized demand insight to drive engagement that are not offered by competitors.
Shaping Product Strategy:
The process provided a clear direction for developing a personalized demand insight to drive engagement that are not offered by competitors.
︎
Quick Cycle from Diagnosis to Improvement
A mixed-method process enabled effective discovery and delivery of concept, resulting in clear roadmap feature.
Quick Cycle from Diagnosis to Improvement
A mixed-method process enabled effective discovery and delivery of concept, resulting in clear roadmap feature.
Team
-
Elizabeth Ropp - Product Designer
- Nam Pham - Researcher
- Jeremy Han - Product Manager
Team
- Elizabeth Ropp - Product Designer
- Nam Pham - Researcher
- Jeremy Han - Product Manager