Book Review: “UX Strategy” by Jaime Levy

Do you know the difference between an assumption and a hypothesis? 

An assumption is something that you suppose to be true, such as “Often PMs are engineers.” A hypothesis is also something that you suppose to be true, but it is stated in an unambiguous way so that it can be tested

This is the second edition of the famous book UX Strategy by Jaime Levy. While I wanted to read the first one, that never happened! There are a lot of tactical level details in this book which are omitted, in my view, because of the conciseness of the book, nevertheless it should serve as a good reference point! In short, if you want to give structure to your product development philosophy, you should give this book a try. One important caveat the author makes is that the UX strategy practice she described in the first edition is now synonymous with product strategy. So relax all the product managers out here!

Cover Image of the Book “UX Strategy” by Jaime Levy

I have tried capturing the essence of the book in the following headers

A. On Business Strategy, UX Strategy and Product Vision

A shared product vision means that your team and stakeholders have the same mental model for your future product

A good business strategy is one centered around the customer. This is why you must validate your presumed customer segment and their unmet needs. 

  • User experience (UX) strategy lies at the intersection of UX design and business strategy
  • It’s an empirical process! 
  • Experience strategy is the combination of business strategy and UX strategy. The “user experience,” or “UX,” is how a human experiences a digital product while attempting to accomplish a task or goal
  • A stellar UX strategy is a means to disrupt the marketplace through mental model innovation

As a product matures with a growing user base, it’s crucial to revisit your strategy. Conducting validation experiments to discover new customer segments, marketing channels, and revenue streams is a job that is never done

B. On Components of UX Strategy and Lean Startup

Four tenants of UX strategy are 1. Business strategy 2. Value innovation 3. Validated user research 4. Frictionless UX

The discovery phase is where UX strategy begins. The output of the discovery phase should be based on empirical evidence, such as getting direct input from target users before going straight from an idea to wireframes and development

The business strategy identifies the company’s guiding principles for how it will position itself and still achieve its objectives while beating the competition. For this to happen, the business must continually identify and utilize a competitive advantage

For a more mature company, the strategy is about building on the company’s core value proposition while trying to evolve the company’s infrastructure and internal processes to support that growth, often called digital transformation!

Business Model Canvas—customer segments, channels, value propositions, revenue streams, and customer relationships—are elements that are essential to creating a product’s online and offline experience

Lean startup made conducting validated user research a make-or-break aspect of moving forward on a product’s strategy. Validation is the process of confirming that a specific customer segment finds value in your solution

Lean Startup was proposed by Ash Maurya in 2010. Its components are 

  1. Customer segments 
  2. Problem 
  3. Unique value proposition 
  4. Solution 
  5. Channels 
  6. Revenue streams 
  7. Cost structure 
  8. Key metrics
  9. Unfair advantage

C. On User Research

Research hypotheses are the answer to the question “What are the most important things I need to learn to determine if my solution is desirable and viable?”

Use your research to validate your decisions and ensure that the product vision is aligned with the end user’s needs

  • The purpose of conducting user research is to understand the needs and goals of your target customer in order to inform the product’s value proposition
  • Confronting your target customers is nonnegotiable. We must learn as quickly as possible if the idea we are working on is stupid and worthless
  • Don’t take what your stakeholders or team says at face value. To learn what potential customers want, hunt them down in person
  • It’s contested if you should ask a customer what they would pay for a product. Customers may lowball you or have no idea. however, getting a sense of what customers expect to pay may be helpful for informing marketing and pricing strategies

Customer discovery is about listening and not selling. The customer interview is actually made up of three parts: the introduction, the screener, and the interview. When you ask the money-shot question, just capture the essence of the person’s response and, if appropriate, ask any relevant follow-up questions

Qualitative research relies on the observation and collection of nonnumerical insights such as opinions e.g. focus groups, contextual inquiries, and ethnographic studies.

  • Ethnographic research—the study of people in their natural environment—is all about getting to the deep, dark places, much like the qualitative personas
  • Do not begin interviews with small talk. Be professional!

D. On Value Proposition

A value proposition takes the form of a concise statement that summarizes the unique benefits customers can expect from your product or service i.e. value proposition is what a company promises to deliver to the customer

But value propositions are not valuable if they do not solve a real problem. The followings are the steps to define a value proposition

  • Define your primary customer segment
  • Identify your customer segment’s (biggest) problem
  • Create provisional personas based on your assumptions
  • Conduct customer discovery to validate or invalidate your provisional persona and problem statement
  • Reassess your initial value proposition based on what you have learned!

E. On Customer Segmentation and Value Creation

The customer segment is a group of people with a common need. These segments can be identified by a combination of demographic, psychographic, and behavioral attributes. For B2B products, you should create two personas: one for the person who will be paying for the product (i.e., the CTO) and one for the person who will be using the product

The problem statement should not presuppose a solution until the problem has actually been validated. By having product teams focus on the problem statement, they are more likely to have an open mind when ideating on solutions

Don’t confuse persona archetypes with stereotypes. Because personas provide a precise design target and also serve as a communication tool to the development team, the designers must choose a particular demographic characteristic with care

Because the provisional persona represents a group of people rather than one person, think of a concise, descriptive name to characterize the segment, such as “Gen X Parents in Los Angeles” or “Jewish Expats in Berlin.” It is useful to articulate common demographic denominators. Location is useful because it forces your team to pinpoint where a market for your product might exist instead of targeting the entire world

Motivation and behavior lie at the heart of value creation. Whatever product you are devising, it is crucial to understand what will motivate people to use it

F. On assessing customers’ needs and goals

What are their product-relevant hopes and dreams? What do they need to solve their primary pain point? What specific needs or goals aren’t being satisfied by available solutions or workarounds? What are the limitations they face? What is the job they are trying to get done?

  • This section is particularly important to get right because it will inform your product strategy the most. You want actionable statements that address underlying customer concerns

G. On Competition, its definition and analysis

Direct competitors are companies that offer the same or a very similar value proposition to your current or future customers. Indirect competitors offer a different value proposition, but somehow their solution may satisfy the needs of your target customer

Investigate your competition

  • What are they doing right? What are they doing wrong? Why should customers come to you?
  • Conducting research on the competition is a crucial component of business strategy
  • The most efficient way to do a comprehensive competitive analysis is to collect all of the data in a matrix
  • Scour YouTube and Vimeo for publicly available demonstrations, tutorials, and product reviews. If you need more information than what you can find online, you may need to bring in an outside agency
  • For native mobile apps, you can currently get the last month’s downloads without an account using Sensor Tower
  • One always needs to be on your toes, agile, and ready to scrutinize your competitor’s newest ideas and immediately see how they might affect your product vision
  • By benchmarking the competition, you’ll find opportunities to create value by either innovating or optimizing the best UX and business model practices of other competing products
  • If something looks incomplete or missing, did you or whoever did the research overlook an obvious competitor that needs to be considered?
  • SWOT analysis of a nonexistent product or business moves us into the land of make-believe. This is why it’s more meaningful to use it for evaluating competitors from their perspective

H. On what to build?

Reaching beyond existing demand is a key component of achieving value innovation

To get your idea juices flowing on the key features, ask yourself these questions:

  • What will make your provisional personas (hypothesized customers) love this product?
  • What is the aha moment or part of the user’s journey online or offline that makes this product unique?
  • What is a major pain point that you are trying to solve that is not currently being solved by competitors?
  • What kind of workarounds are your potential customers currently doing to accomplish their goals?
  • What is the core benefit for your customers that is derived from the output or manipulation of either your proprietary algorithm and/or data set?
  • What is the functionality or page/screen layout that needs to be designed from scratch because there is no reference for it in any other digital product?

A product recommendation should answer the following questions

  • Can a specific feature be majorly improved or new technology integrated to help customers do something that currently is too complicated or time-consuming for the existing alternatives?
  • How can you make the product experience more personalized or “smart” to increase adoption and engagement?
  • Is there a new revenue stream or disruptive business model that can be experimented with?
  • How can you achieve a competitive advantage that your competitors can’t easily replicate?

If your analysis reveals that the initial value proposition is facing certain risks, your recommendations may need to suggest a pivot on the targeted customer segment or the specific problem

Your future customers need to want to choose your solution over any other because 

a) it’s significantly more efficient than what’s currently out there, 

b) it solves a pain point they didn’t know they had, and/or 

c) it creates an undeniable desire where none existed before

I. On Prototyping

As we prepare to prototype, we’re finally juggling all four of the tenets at the same time. Don’t burn your time, money, or efforts on a product that has not been tested and validated with target customers

These are the critical questions that we need our prototype to answer:

  1. Does the solution solve the problem or major pain points that the target customer expressed?
  2. Does the target customer find the key features valuable?
  3. Would the target customer pay for the product or use it in a way that can be monetized?

The answers to questions 1 and 2 help us validate our value proposition. The answer to question 3 will help us validate our business model

To keep the long story long, this line from the book has resonated well with me. Sometimes, people have fixed ideas, and no amount of research will change their minds. <Then ask yourself> Will I help this person make their product regardless of the research, or do I walk away?

Tough words, how many of us have the courage to walk the talk?! If you don’t get this part right – costs will have to be borne by the team, company and often careers of the people involved whether or not they realise it at that point. In short, don’t solve wrong problems for your own good!

—–

PS: Pop Quiz for UX Geeks!

Q. Do you know the difference between concierge, Wizard of Oz and mechanical turk? 

A. These are the techniques for conducting value proposition experiments!

  • Unlike concierge, customers don’t know that a human is in the loop in case of Wizard of Oz
  • When someone says they are mechanical-turking a product, it typically means they are building a frontend with a human-powered backend to manually simulate a complex digital product. It’s like a crowdsourced version of Wizard of Oz

Book review: “The making of a Manager” by Julie Zhuo

Finally I got hold of this book “Making of a Manager: What to do when everyone looks to you” by Julie Zhuo, I heard about it from Twitter and got intrigued. I recommend reading this book to all managers, especially the ones newly-minted as well those who want to ensure they are not reinventing learning from wildly successful companies like Facebook where the author has learnt the ropes – rising from an intern to VP of Design. Seeing the recommendations from Silicon Valley stalwarts in the first few pages of this book would probably motivate you to dive into as it did for me!

Cover of the Book: “The making of a Manager” by Julie Zhuo

You would find this book easy to read (can be finished over the weekend), situations relatable and advice quite sensible to put into action immediately. That said, the overall tone of the subject matter is generalist in nature and may not especially cover for the peculiarities that PM managers face. That should however not deter you from imbibing the deep lessons embedded in the short 10 chapters of this book. What follows is my key takeaways in the order in which they appear in the book

A. What is management?

  • A manager’s job is to
    • build a team that works well together,
    • support members in reaching their career goals, and
    • create processes to get work done smoothly and efficiently
  • Your job, as a manager, is to get better outcomes from a group of people working together and therefore if the team’s outcomes are mediocre, you cannot be termed as a great manager
  • Being awesome at the job means playing the long game and building a reputation for excellence
  • Manager’s tasks can be filled into 3 buckets – people, process and purpose (making the team aware of what success looks like and ensure they care about achieving it)
    • Your role as a manager is not to do the work yourself, even if you are the best at it, because that will only take you so far. Role is to improve and process people or the purpose of the team! However if you are in survival mode, do what it takes!
  • How to know if you will be a great manager?
    • Do you find it more motivating to achieve an outcome or play a specific role?
    • Do you like talking to people?
    • Can you provide stability for an emotionally charged situation?
  • While manager is a specific role, leadership is the particular skill of being able to guide and influence other people and therefore in order to be a great manager, one has to be a leader!
    • If you can pinpoint a problem and motivate others to work with you to solve it, then you’re leading
    • Also leadership is not something that can be bestowed. It must be earned. People must want to follow you

B. What to do in the first 3 months as a manager?

The playbook varies depending on whether one is a freshly minted manager (apprentice), founding team member grown into a managerial role, new boss leading the team internally or an external hire or a successor replacing someone

  • To have hard conversations, it is essential to internalise that you own your team’s outcomes
  • To get honest feedback from your reportees, ask how their dream manager looks like
  • You need to proactively invest in building relationships. Being vulnerable sometimes helps in generating trust
  • You will be far more successful aspiring to be the leader you want to be and playing to your strengths than trying to live up to some other ideal

C. How to lead a small team?

  • No matter what work you do or the size of your team, knowing how to diagnose and solve problems with your reports is critical to your shared success
  • Trust is the most important ingredient
  • If the answer to your questions around “How are you” is fine for multiple weeks from a report, take it as a sign to prod further!
  • If you don’t truly respect or care about your team members, you cannot fake it! Managing is caring!
  • Supporting and caring for someone doesn’t mean always agreeing with them. What caring does mean is doing your best to help your report be successful and fulfilled in her work
  • If your report feels that your support and respect are based on her performance, then it will be hard for her to be honest with you when things are not going per expectations
  • If you can remove a barrier, provide a valuable new perspective, or increase their confidence, then you’re enabling them to be more successful
  • Your report should have a clear sense at all times of what your expectations are and where he stands
  • Help people play to their strength. However it does not have to always work since sometimes success is a function of personal and org priorities! 
  • When you decide to let someone go, do it respectfully and directly. Don’t open it up to discussion
    • Change is hard, but trust your instincts. Would you hire this person again if the role were open? If the answer is no, make the move
    • The end goal of management is to get better outcomes. When someone isn’t a great fit for his role, there is a cost
    • The growth mindset has taught us that anyone can get better at anything given the will, hard work, and time. The question is, how long would it take? And how would that affect the team?
  • Don’t tolerate brilliant assholes on your team, it actually is better off when they leave! In general, you should make people moves quickly!

D. How to be a great manager? Art of giving feedback

  • For a leader, giving feedback—both when things are going well and when they aren’t—is one of the most fundamental aspects of the job
  • Four common ways to inspire a change in behavior 
    • Set clear expectations in the beginning
    • Give task specific feedback as quickly as you can
    • Share behavioral feedback thoughtfully and regularly 
    • Collect 360 degrees feedback for maximum objectivity
  • Every major disappointment is failure to set expectations
  • You feedback only counts if it makes things better
  • The best way to give critical feedback is to deliver it directly and dispassionately
  • Own the decision, be firm and don’t open it up for discussion!

E. How to manage yourself better?

  • Being a great manager is a highly personal journey, and if you don’t have a good handle on yourself, you won’t have a good handle on how to best support your team
  • No matter how often imposter syndrome rears its ugly head, it doesn’t have to derail you
  • Be brutally honest with yourself
  • The first part in understanding how you lead is to know your strengths—the things you’re talented at and love to do. This is crucial because great management typically comes from playing to your strengths rather than from fixing your weaknesses
  • Develop a growth mindset i.e. be motivated to seek out the truth and ask for feedback because you know it’s the fastest path to get you where you want to go
  • When a negative story takes hold of you, step back and question 
    • Is your interpretation correct? 
    • Are there alternative views you’re not considering? 
    • What can you do to seek out the truth?
  • To fight self-doubt, visualise success. It is a powerful tool!
  • Maximize on the job learning. Treat your manager as the coach and not as a judge!
  • Take advantage of formal training! Or maybe professional coaching
  • When you invest in your personal learning and growth, you’re not just investing in your own future but also the future of your team

F. How to organise meetings?

  • Analyse efficacy of meetings by their purpose. It could be broadly of 5 types 1. Making a decision 2. Information sharing 3. Providing feedback 4. Generating ideas 5. Strengthening relationships 
  • A great decision making meeting has the following components
    • Gets a decision made
    • Includes the people most directly affected by the decision as well as a clearly designated decision-maker
    • Presents all credible options objectively and with relevant background information, and includes the team’s recommendation if there is one
    • Gives equal airtime to dissenting opinions and makes people feel that they were heard
  • Invite right people to the meeting
  • Give people a chance to come prepared. Float a pre-read if possible 
  • Before concluding the meeting, summarise next steps
  • Make it safe for people to contribute. Sticky notes or having a round-robin around the table are helpful
  • Some meetings don’t need you and be ruthless in culling ineffective meetings
    • If you trust that the right outcomes will happen without you, then you don’t need to be there

G. How to hire well?

  • Hiring doesn’t just matter at scale—even a single great hire can make a big difference in your team’s outcomes.
  • The most important thing to remember about hiring is this: hiring is not a problem to be solved but an opportunity to build the future of your organization
  • Design your team intentionally. Have a thoughtful one-year hiring plan in the beginning. Tweak as you go along
  • Hiring is your responsibility. Align with your recruiter in defining the role, sourcing, the onboarding process and onboarding really closely
  • Describe the role as clearly as possible and deliver an amazing interview experience
  • Examine a candidate’s past experience. They are probably the closest predictor of performance at the job!
  • Hiring is a gamble but do make smart bets! And since it’s a gamble, reject weak hires!
    • If you are going to make a bet, bet on someone who has at least a few passionate advocates in the interviewing panel!
  • References matter the most, contact your common connections towards the same
  • Prepare your interview questions in advance. Take a long term view with top talent!
  • Build a team with diverse perspectives, hire people who are more capable than you!
  • You can’t create great outcomes without consistently attracting talented people and ensuring that they can also hire well
  • Make it clear that building the team is not just one person’s job, it’s everyone’s job!

H. How to make things happen?

  • Start with a concrete vision. An inspiring vision is bold. It doesn’t hedge. You know instantly whether you’ve hit it or not because it’s measurable. And it’s easily repeated, from one person to the next. It doesn’t describe the how, it simply describes what the outcome will be
  • Create a believable game plan i.e. strategy
  • Craft a plan based on your team’s strengths
  • Focus on few things well, prioritise
  • Define who is responsible for what
  • Treat big projects like a series of small projects. Keep in mind the planning fallacy: our natural bias to predict that things will take less time and money than they actually do
  • Choose perfect execution over perfect strategy
  • Good process is ever evolving. A resilient organization isn’t one that never makes mistakes but rather one whose mistakes make it stronger over time

I. How to lead a growing team?

  • Differences between big and small teams
    • People treat you differently. They’re less likely to tell you the ugly truth or challenge you when they think you’re wrong, even if you’d like them to
    • Context switching, everyday
    • You pick and choose your battles i.e. what are the most important topics for you to pay attention to, and where are you going to draw the line
    • At higher levels of management, the job starts to converge regardless of background. Success becomes more and more about mastering a few key skills: hiring exceptional leaders, building self-reliant teams, establishing a clear vision, and communicating well
  • Delegation is an art. It doesn’t mean you walk away!
    • The rule of thumb for delegation goes like this: spend your time and energy on the intersection of 1) what’s most important to the organization and 2) what you’re uniquely able to do better than anyone else
    • Anything your report can do just as well or better than you, you should delegate
  • People trump projects—a great team is a prerequisite for great work.
  • Beyond people, you and your report should be aligned on why you’re doing what you’re doing and what success looks like
  • The act of constantly trying to replace yourself means that you create openings to stretch both your leaders and yourself

J. How to nurture culture?

  • As you manage more and more people, you’ll play a bigger role in shaping culture. Don’t underestimate the influence that you can have
    • Pay attention to your own actions—the little things you say and do—as well as what behaviors you are rewarding or discouraging
  • Make a list of the aspects of culture that you admire about other teams or organizations. Why do you admire them? What downsides does that team tolerate as a result?
  • Never stop talking about what is important
  • If you say something is important to you and you’d like the rest of your team to care about it, be the first person to live that value
  • When a report does something difficult that is in the spirit of your team’s values, recognize them for it
  • Invent traditions. Rituals are powerful

This is a great book, do read and internalise the lessons. Almost all of those are universally applicable across roles!

PS: Additional recommendations from the author for frequent referencing are the book “Crucial Conversations”, articles like High Output Management and How to Win Friends and Influence People

Book Review: ‘Decoding the Why’ by Nate Andorsky

This book “Decoding the Why: How Behavioral Science is Driving the Next Generation of Product Design” by Nate Andorsky can be summed as the one attempting to bridge the gap between what is loosely known as gamifying the product experience and a typical product development in non-games companies. As you would have observed, even before Covid – product companies were focusing on driving user engagement and retention through non-monetary interventions. In a horizontal product world – everyone trying to do almost everything else – user attention is literally a currency!

Image 1: Book Cover of “Decoding the Why” by Nate Andorsky

In author’s own words, the book is intended to give readers a baseline understanding of how behavioral science integrates into the product design. What now follows is summary of the book in Q&A format

What are the limitations of an archetypal user research oriented product development process?

Basic flaw with user research could be that most of the time, one is asking users to provide explanations of behaviors they don’t truly understand themselves! OTOH, mimicking competitors’ products creates an echo-chamber of product design

Our innovations will only live up to their full potential if they are built on an understanding of the human experience. While context plays a significant role in how our behavior manifests, the underlying mechanisms remain constant and this is where behavior oriented process gives an advantage

How does the human brain function?

Tversky and Kahneman’s seminal paper, “Judgment Under Uncertainty: Heuristics and Biases,” proposed that we attack complex problems using a limited number of heuristic principles—basically shortcuts. The brain typically operates in one of two modes: automatic and reflective, often referred to as System 1 and System 2 thinking. System 1 runs on autopilot and it’s possible for these systematic errors to be understood and “predicted”. System 2 is slower, more methodical, and dedicated to solving complex problems

How does one get started on this new approach? 

The author introduces three kinds of data points one should be cognizant of

  1. SAY data: subjective information collected from users
  2. DO data are the analytics—what users are doing on the website or product
  3. The WHY data. If you understand the WHY data behind the DO and SAY data, it unlocks a world of possibilities

How do we fight inertia wrt current user behavior?

We have a bias toward the present, giving stronger weight to present payoffs than those that will happen in the future. Present Bias ties into another theory, Hyperbolic Discounting, which states that we have time-inconsistent preferences

The way we perceive our future selves is similar to the way that we perceive a stranger. The further out into the future the loss or gain happens, the more heavily we discount it. To offset Present Bias, we must close the gap between our current selves and our future selves

In general users who perceive more personal stability over time tend to behave in a more future-oriented fashion—one that aligns better with their future self. However when the reward is too far off in the distant future, one needs a substitution to fill the void

OTOH users need guidance when they make decisions. Without guidance, they can fall into Barry Schwartz’s paradox of choice i.e. when presented with too many options, we have a hard time making a decision. This is why good on-boarding and product tutorials may be sometimes critical to product adoption!

How do we design for a reward cycle?

It is important to first consider the desired action before one integrates the reward

The fundamental building blocks of a reward cycle are: trigger, action, investment and reward. Nir Eyal notes that the reward should be variable, and there should be an investment component where the user puts something into the product such as time, data, effort, social capital, or money

Dopamine is the high you get from experiencing something new and exciting. The nervous system produces dopamine during the experience of reward, but even more interesting is that the production of dopamine also happens in anticipation of a reward. The closer we get to receiving a reward and missing it, the more likely we are to engage in said behavior again. Overall, rewards and incentives don’t necessarily have to be financial as anyone who has watched Tik Tok’s growth can vouch for!

How to choose between intrinsic and extrinsic motivators?

Daniel Pink, NY Times Best Selling author opines that the three elements he believes make up true motivation are autonomy, mastery, and purpose. Intrinsic motivation drives creative tasks that are open to interpretation.

A good rule of thumb is to leverage external rewards to promote interest in an activity that is easy but may not initially interest a user. If it is a specific, relatively easy task with a concrete outcome, easy to do, and with an endpoint, extrinsic motivators would be a good place to start

Then, if possible, trigger intrinsic motivations to reinforce the action over the long term

How do we get people to keep acting on their intentions?

Commitment devices help us follow through on our intentions. A common form of a commitment device involves a person voluntarily giving up something of value that they can only regain access to by following through on their commitment. It’s more effective to structure a commitment contract where the reward money is theirs to lose rather than gain. Think of an example, of incentivising student participation in class by tying higher participation to a picnic the school will organise later in the year

Commitment contracts don’t always have to come in the form of money. These commitments can be to another person or a group of people. Goals should not be so big or far in the distant future that they feel unattainable

One should not underestimate the power of cues in precipitating an action. Others give us cues regarding what we should do. Even a small subset of a larger group making a confident move can influence the rest of the group. Social norms and the pull of the crowd can help your users follow through on their intentions. The way to drive behavioral change predicatively is by changing norms and changing the rules, effectively changing someone’s environment. Important part is to understand how to change the behavior, and the outcome will follow

Why are goals so powerful?

Regulatory Mode Theory studies the development of goal-pursuit as well as motivation. The theory lays out two main approaches from a social cognition perspective regarding the pursuit of goals: one is assessment, and the other is locomotion. Scoring high on assessment means you evaluate your options before making a decision and while scoring high on locomotion means you need to always be doing something

Streaks are a powerful way to incentivize action. They provide a sense of progress that plays into the natural way we see the world, our desire to avoid loss, and our need for growth and accomplishment

For example, the primary focus of Duolingo is to make the long-term benefits of learning a new language more immediately salient. Winning streak that increases its attractiveness as it lengthens, becoming a self-reinforcing system. On Duolingo, the grouping of these streaks is completely arbitrary. However, streak loss can be demotivating for some participants. Duolingo has countered this by allowing learners the ability to restore a broken streak

In short, people who set goals make more progress!

How do we signal trust?

We have a natural inclination to trust people who look and act like us, a bias called similarity bias. Aribnb used this to build trust between two people who have never met before in the absence of multiple reviews by creating similarity between the renter and rentee. However, if users share too little or too much about themselves, acceptance rates go down

So how did they design for this? Airbnb nudged users to write an introduction of the correct length and to include the right details about themselves. Airbnb also discovered that at a certain point, reviews trumped similarity bias. When a listing accumulated more than ten reviews, everything changed, people started trusting ratings more!

How to retain motivation?

Leaderboards spur competition, but they have weaknesses. Leaderboards can backfire. If a user gets too far ahead or too far behind, they disengage. In games, there are techniques employed like Dynamic game difficulty balancing (DGDB), also known as dynamic difficulty adjustment (DDA) or dynamic game balancing (DGB), which is the process of automatically changing parameters, scenarios, and behaviors in a video game in real-time based on the player’s ability, to avoid making the player bored (easy game!) or frustrated (hard game). Overall, a competition between users is a powerful technique to motivate action

How to leverage loss aversion to our advantage? 

Losses psychologically feel twice as great as an equal gain. Only prospective (future) costs are relevant to a rational decision, but we fall prey to the sunk cost fallacy. IKEA effect which describes the increase in the valuation of self-made products. While the participants weren’t experts, they saw their creation as ones created by experts. The more of a hand we have in creating something, the less likely we are to part with it. Fitness app Noom uses this to its advantage in their onboarding flows

Peanut effect as a way to counter big loss aversion!

The peanuts effect is when we fail to consider the consequences of small losses. In this case, it is a good thing. The peanuts effect is part of the reason slot machines steal all of our money. The decision to save is easier when framed in a way that spreads the potential losses into smaller increments. Momentum leverages the default bias and the peanuts effect to help everyday people close their intention-action gap

How to use emotions?

We process information about one identifiable person differently than information about a group of people. Identifiable Victim Effect, which seeks to understand what moves us to offer help. Think back to a powerful marketing campaign, and it would have revolved around stories. To move people emotionally, tap into layer two and convey the emotional experience to your audience. We are visual machines. The brain can identify images seen for as little as thirteen milliseconds. All types of companies can help users achieve their goals by making them the hero of their own journey – the way Pain Squad does for its users

How to integrate behavioral-first approach while building products?

The process is quite simple and can be summed as a series of steps below

  1. What is the problem you are trying to solve?
  2. Collect the SAY and DO data
  3. Compare SAY and DO data
  4. Identify potential behavioral drivers
  5. Identify potential behavioral solutions
  6. Create interventions
  7. Test interventions
  8. Iterate

As one can see, steps 4 and 5 are the most crucial ones and this is where learning and having an opinion about behavioral theories may help

Overall, we are just getting started in this interesting domain of marrying behavioral insights with product design. Academic understanding of what drives behavior precedes the implementation by about ten to fifteen years. Product managers and designers learn by taking theories and putting them into practice. Hopefully one should be able to discern where assumptions break down and where they start to work and then hypothesize why and fine-tune it!

Book Review: Practical User Research: Everything You Need to Know to…” by Dr.Emmanuelle Savarit

“The issue with startups is not that they do not understand what UX is. I think they are just not ready for it”

“Stakeholders may not realize that face-to-face interviews with 12 people are more reliable than a survey with 100 to 200 respondents”

Have you ever wondered when you should involve user research into your product development? Or think that quantitative research is more reliable than qualitative research? Or wonder how user research can influence business strategy? Or confused about the difference between a user profile and user persona?

This book “Practical User Research: Everything You Need to Know to Integrate User Research to Your Product Development” authored by industry veteran Dr.Emmanuelle Savarit offers a systematic approach to incorporate user research into products. This book is a great read, and can be quite easily used in real life product management situations. Let’s dive in

Structure of the Book

The book is divided into 10 chapters. First 2 chapters concentrate on explaining to readers how user research as an expertise has come about and also how traditionally product development happens across industries. In Chapter 3, the author explains how product managers can introduce or evolve user research function into their organisation depending on their maturity. The author argues that the main obstacle to introducing user research into product development is maturity phase of the business. Chapter 4 and 5 explain how to prepare for user research

Chapter 6 explains main methods for quantitative research namely analytics, surveys and card sorting. Chapters 7 and 8 focus on collecting data and analysing findings for a few qualitative methods. Chapter 9 goes into aspects like ethics, compliance and consent in user research – areas which are generally considered as an afterthought. In the last chapter 10, author briefly touches on how user research projects unfold in real life through 10 case studies she has conducted across multiple industries and cultures

A. Introduction to User Research

User research consists of putting an end user’s needs at the center of the researcher’s investigation. Role of the user researcher is to provide evidence-based findings using quantitative and qualitative research methods. In user research, we are not interested in the demographics, job role, etc. We are interested in participant’s motivation or their behavior while using the product

Like many other disciplines, it traces its origins to the military. After WW2, there was a need felt for optimising weapons in order to minimise injuries to soldiers. Many disciplines like ergonomics, design, human-computer interaction, usability and user research emerged consequently

While doing research, we have the choice of taking a quantitative approach or a qualitative one. Often, stakeholders prefer numbers to case studies. However, when the research is properly done, both approaches are reliable and robust

Quantitative Research

  • Takes a top-down approach, starting with the big picture and using deductive reasoning
  • Validates a hypothesis, theory, or preconceived idea
  • Answers the questions how many, how often, and when
  • Doesn’t answer why or how people are using the website

Qualitative Research

  • Takes a bottom-up approach and starts from a specific observation and goes to a generalization and informs theory. It for the same reason leaves room for unexpected avenues
  • Identifies phenomenon, common patterns, and systematic occurrence
  • Useful to answer the questions how and why

Often, businesses have difficulties understanding that user research is not going to provide them with what they want to hear but rather help them identify what the user needs to perform a transaction, to complete a task, or to carry out their daily job

A typical digital product development goes through 5 phases

  1. Discovery
    1. Get a full understanding of the current situation. It helps to decide at the end of this phase whether the project should be moving to alpha
    2. If made correctly, this will limit the risk of failure and create a clear account of what should be done in alpha
  2. Alpha
    1. It is the moment when the team can try different options
    2. At the end of the alpha phase, the team should have a prototype representing the future service
    3. Sometime after completing an alpha phase, the team may realize that it needs to revisit original objectives and needs
  3. Beta
    1. Building stage of product development for the backend and the front end to provide a minimum viable product/service (MVP/MVS)
    2. At the end of the beta phase, the end-to-end minimum viable product should be ready to move to the private beta phase
  4. Private Beta
    1. Way to put live the new service with a small number of users
  5. Live

B. How research can fit into this framework?

Discovery: this phase is an excellent time to do competitive research

Alpha: In this phase, one can conduct pop-up research, also called guerrilla research which is a quick user testing session in which you test screens and prototypes with real users in their natural environment

Beta: One can test prototypes and solutions

Image 1: Integrating User research during PDLC. Credit Dr.Emmanuelle Savarit

The following diagram summaries research methods typically used in developing digital products

Image 2: Methods used in User Research. Credit Dr.Emmanuelle Savarit

How can one arrive at the Research maturity stage for the organisation since it’s so critical for continued success of the function? 

Author derives a version from the famed Nileson Norman corporate maturity model for UX

Image 3: Nileson Norman corporate maturity model for UX. Credit Dr.Emmanuelle Savarit

While rest of the stages are self-explanatory, in Skunkworks user experience the organization realizes that relying on design team intuition is risky, and the organization starts requesting some data

Image 4: Savarit User Research Maturity Model. Credit Dr.Emmanuelle Savarit

The author opines about a new trend in user research org: research ops team! It needs to be put in place once the organization already has a certain degree of UX maturity and a good understanding of the differences and advantages of conducting research in the product development life cycle. Typically sometimes user research leadership dons this role depending on size of the company, number of user researchers and project complexity!

C. How to prepare for User Research?

  1. Identify if you need user research for your product
    1. The first question to ask is “Does my product/service involve users?” f it does not include end users, you will probably not require any user research.
  1. Review the research that was done in the past and identify the gaps
    1. Identifying the gaps is the starting point of conducting user research and finding out what you don’t know
  2. Identify your users
  3. Build a case to get budget for your project
    1. While building the case to stakeholders, one has to make sure that they are not reinventing the wheel. One needs to have a clear account of what they are trying to do and why, what the benefits are of doing it, and what the risks are of not doing it
  4. Get your stakeholders on board to support the project
  5. Put in place the relevant capability to conduct the research

D. Research Preparation

This is the most important part of user research

Image 5: Steps in a Research Plan Document. Credit Dr.Emmanuelle Savarit

Few points to remember here

  1. The methods to collect data are different from the research method that one may be using. The approach to collecting your data is not the way you are going to analyze the data. Following diagram provides a good heuristic

Image 6: Methods to Collect Data, depending on the development phase of the Project. Credit Dr.Emmanuelle Savarit

  1. Every participant has the right to withdraw at any time during the research. This should be respected
  2. Use a screener / questionnaire to recruit people for research. However be wary of panels since sometimes although using a panel is cheaper, but it may provide only those people who spend a lot of time taking part in studies, user research, and marketing just for the incentive
  3. It is easier to get participants if you are giving an incentive
  4. Organizing all documents in a clear format may save the team a lot of time

E. Research Methods // Quantitative Research Methods

Point to note here is that quantitative approaches used in isolation will provide only validation of preconceived ideas

Analytics: few points to note

  1. Business context is important while analysing data. For example, in grocery shopping, it takes more than two minutes to fill up a shopping basket. So time spent metric has to be interpreted carefully vis-a-vis other categories
  2. It’s important to realise that analytics alone is not enough, and does not answer the questions of why, how, and who

Surveys: Generally used to describe a population. A survey is a methodology that uses questionnaires to collect data

  1. The advantage of conducting a survey is that one can get a large sample size that will validate some of the assumptions
  2. Surveys are useful to validate findings from qualitative research or for data validation
  3. One can also pay for a panel; useful if looking for a specific audience that is difficult to reach

Card Sorting: It’s asking intended readers to sort items into groups or categories that make sense to them. Few online tools such as Optimal Workshop’s Optimal Sort or UserZoom’s Card Sorting can be used here. Types of card sorting can be of three types: open, closed, hybrid. Some of the use cases for using card sorting quantitative research techniques are

  1. Designing a new information product or improving one already existing: Open card sorting
  2. Whether different groups of readers use the same approach to finding items: Open
  3. Where new topics fit into an existing website or intranet: Closed
  4. Whether you can reduce the number of categories, by testing whether a smaller set still works well: Closed
  5. Set the tone of the organizational pattern while encouraging participants to generate and name their own categories: Hybrid
  6. Further explore groupings that were unclear in an earlier sort while providing categories for groupings that were clear: Hybrid

Having ~40 cards ensures that one will generate the data needed to make decisions about Information Architecture. Beyond 60 cards, participants will be less likely to complete the card sort

Also one must randomize the order in which participants see the cards and the categories unless there’s a specific order to test. This removes list-primacy bias from the overall result

Two of the techniques to interpret data from card sorting are: dendrogram and similarity matrix

F. Research Methods // Qualitative Research Methods

A qualitative approach requires a small sample however analyzing qualitative data takes more time than the data collection itself. Some of the methods used to collect qualitative data are contextual inquiries/ethnography, interviews, focus groups, user testing, and diary studies

  • To have reliable results with qualitative research, it’s recommended to have a minimum of eight people if you are conducting user testing every two weeks
  • When product is live or in private beta and if one wants to test the end-to-end journey before release, it’s recommended to have between 12 to 24 participants

Contextual inquiries/ethnography

Here the researcher conducts a face-to-face semi-structured/conversational interview within a user’s natural environment. It’s an observational approach. It’s recommended to be used at the start of every project

  1. Understanding their day and which tasks they have to complete at different times helps us understand which information they will require at time t
  2. It enables us to capture not only how they are using the current product but also to identify the other tools/products that they are using
  3. It however can be time-consuming and is more challenging to record

Interviews

User interviews are frequently used to capture general information about users. There are three types of interviews: structured (pre-defined response), semistructured, and unstructured (natural conversation)

Focus Group

It’s a session that lasts between one and two hours and takes place with six to ten participants who discuss a specific topic. It could be useful when you want your stakeholders involved in the project

Usability Testing

Usability testing/user testing is a face-to-face session with real users who are interacting with the product not only to evaluate the functionalities and the efficiency but also to capture the user’s behavior. However Usability is associated with quantifying success rate, task on time or error rates or satisfaction. User testing can be done as soon as there are some screens available. This is also useful for launching new websites and for evaluating end-to-end user journeys

Some techniques are A/B testing, eye tracking (when there is lot of information – tool Tobii Pro), guerilla testing or pop up research and System Usability Scale (SUS)

System Usability Scale

It is a quick usability test that is now a UX standard. It provides a high-level satisfaction score of the usability of a site, application, or any technological item. SUS is a simple, ten-item scale giving a global view of subjective assessments of usability

SUS scores have a range of 0 to 100. It is not a percentage. A score above 68 is above average, and a score below 68 is below average

Image 7: Interpreting SUS: gold standard for usability testing. Created by John Brooke (1986). Credit Dr.Emmanuelle Savarit

Diary Study

This approach permits us to get insights from users while they are experiencing a situation. The study is longitudinal and captures temporal information

  1. Diary studies are used a lot in the field of education to understand the learning process or seeing how streaming services are being watched
  2. It helps us uncover if participants develop any habits over time
  3. It also allows a peak into user impressions as well as their frustrations
  4. This technique is becoming more democratic now with proliferation of smartphones however making sure that participants remain proactive is a challenge for user researchers

How do we analyse Qualitative Data?

Affinity Diagram

A fast way to analyze data is to create an affinity diagram. Jiro Kawakita created this method that simplifies a large amount of data by grouping it by the themes that emerge from the data

  1. However this analysis should be done with as much objectivity as possible
  2. This method is suitable for brainstorming or early-stage analysis to identify the different steps of a user journey

Thematic Analysis vs. Content Analysis

  1. If we identify that the user makes the same comments while booking a flight or conveys the same issues or frustration, that means we have identified a systematic pattern

Difference between User Profile and User Persona

A user profile is a set of characteristics based on demographics, job role, age, the industry the user is working in. A persona is a typical user who has specific characteristics such are interests, goals, behavior, attitude, habits. A persona is not a real user, it is not a case study, and it is not an imaginative character

G. Few ethical questions in User Research

  • How do we make a website or a digital product accessible so that people with a disability can use it in terms of perception, comprehension, navigation, and interaction?
  • Making respondents comfortable, taking their consent for data use, signing up NDAs for protecting company’s information and also understanding that their participation in the research is voluntary, and one cannot force them to respond to our questions

To sum it up, a general process for conducting user research can be summed up as below

  • Get a brief
  • Refine the scope of the research in order to set up stakeholder expectations
  • Work with your stakeholders to draw up the research questions
  • Choose the relevant method to answer the research questions
  • Identify the right participants
  • Collect and analyze the data
  • Extract meaningful and actionable findings
  • Share your findings with your team and stakeholders

And this process mostly remains the same across industries, org maturity levels, engagement model of the user researcher or geographies!