Book Review: ‘Decoding the Why’ by Nate Andorsky

This book “Decoding the Why: How Behavioral Science is Driving the Next Generation of Product Design” by Nate Andorsky can be summed as the one attempting to bridge the gap between what is loosely known as gamifying the product experience and a typical product development in non-games companies. As you would have observed, even before Covid – product companies were focusing on driving user engagement and retention through non-monetary interventions. In a horizontal product world – everyone trying to do almost everything else – user attention is literally a currency!

Image 1: Book Cover of “Decoding the Why” by Nate Andorsky

In author’s own words, the book is intended to give readers a baseline understanding of how behavioral science integrates into the product design. What now follows is summary of the book in Q&A format

What are the limitations of an archetypal user research oriented product development process?

Basic flaw with user research could be that most of the time, one is asking users to provide explanations of behaviors they don’t truly understand themselves! OTOH, mimicking competitors’ products creates an echo-chamber of product design

Our innovations will only live up to their full potential if they are built on an understanding of the human experience. While context plays a significant role in how our behavior manifests, the underlying mechanisms remain constant and this is where behavior oriented process gives an advantage

How does the human brain function?

Tversky and Kahneman’s seminal paper, “Judgment Under Uncertainty: Heuristics and Biases,” proposed that we attack complex problems using a limited number of heuristic principles—basically shortcuts. The brain typically operates in one of two modes: automatic and reflective, often referred to as System 1 and System 2 thinking. System 1 runs on autopilot and it’s possible for these systematic errors to be understood and “predicted”. System 2 is slower, more methodical, and dedicated to solving complex problems

How does one get started on this new approach? 

The author introduces three kinds of data points one should be cognizant of

  1. SAY data: subjective information collected from users
  2. DO data are the analytics—what users are doing on the website or product
  3. The WHY data. If you understand the WHY data behind the DO and SAY data, it unlocks a world of possibilities

How do we fight inertia wrt current user behavior?

We have a bias toward the present, giving stronger weight to present payoffs than those that will happen in the future. Present Bias ties into another theory, Hyperbolic Discounting, which states that we have time-inconsistent preferences

The way we perceive our future selves is similar to the way that we perceive a stranger. The further out into the future the loss or gain happens, the more heavily we discount it. To offset Present Bias, we must close the gap between our current selves and our future selves

In general users who perceive more personal stability over time tend to behave in a more future-oriented fashion—one that aligns better with their future self. However when the reward is too far off in the distant future, one needs a substitution to fill the void

OTOH users need guidance when they make decisions. Without guidance, they can fall into Barry Schwartz’s paradox of choice i.e. when presented with too many options, we have a hard time making a decision. This is why good on-boarding and product tutorials may be sometimes critical to product adoption!

How do we design for a reward cycle?

It is important to first consider the desired action before one integrates the reward

The fundamental building blocks of a reward cycle are: trigger, action, investment and reward. Nir Eyal notes that the reward should be variable, and there should be an investment component where the user puts something into the product such as time, data, effort, social capital, or money

Dopamine is the high you get from experiencing something new and exciting. The nervous system produces dopamine during the experience of reward, but even more interesting is that the production of dopamine also happens in anticipation of a reward. The closer we get to receiving a reward and missing it, the more likely we are to engage in said behavior again. Overall, rewards and incentives don’t necessarily have to be financial as anyone who has watched Tik Tok’s growth can vouch for!

How to choose between intrinsic and extrinsic motivators?

Daniel Pink, NY Times Best Selling author opines that the three elements he believes make up true motivation are autonomy, mastery, and purpose. Intrinsic motivation drives creative tasks that are open to interpretation.

A good rule of thumb is to leverage external rewards to promote interest in an activity that is easy but may not initially interest a user. If it is a specific, relatively easy task with a concrete outcome, easy to do, and with an endpoint, extrinsic motivators would be a good place to start

Then, if possible, trigger intrinsic motivations to reinforce the action over the long term

How do we get people to keep acting on their intentions?

Commitment devices help us follow through on our intentions. A common form of a commitment device involves a person voluntarily giving up something of value that they can only regain access to by following through on their commitment. It’s more effective to structure a commitment contract where the reward money is theirs to lose rather than gain. Think of an example, of incentivising student participation in class by tying higher participation to a picnic the school will organise later in the year

Commitment contracts don’t always have to come in the form of money. These commitments can be to another person or a group of people. Goals should not be so big or far in the distant future that they feel unattainable

One should not underestimate the power of cues in precipitating an action. Others give us cues regarding what we should do. Even a small subset of a larger group making a confident move can influence the rest of the group. Social norms and the pull of the crowd can help your users follow through on their intentions. The way to drive behavioral change predicatively is by changing norms and changing the rules, effectively changing someone’s environment. Important part is to understand how to change the behavior, and the outcome will follow

Why are goals so powerful?

Regulatory Mode Theory studies the development of goal-pursuit as well as motivation. The theory lays out two main approaches from a social cognition perspective regarding the pursuit of goals: one is assessment, and the other is locomotion. Scoring high on assessment means you evaluate your options before making a decision and while scoring high on locomotion means you need to always be doing something

Streaks are a powerful way to incentivize action. They provide a sense of progress that plays into the natural way we see the world, our desire to avoid loss, and our need for growth and accomplishment

For example, the primary focus of Duolingo is to make the long-term benefits of learning a new language more immediately salient. Winning streak that increases its attractiveness as it lengthens, becoming a self-reinforcing system. On Duolingo, the grouping of these streaks is completely arbitrary. However, streak loss can be demotivating for some participants. Duolingo has countered this by allowing learners the ability to restore a broken streak

In short, people who set goals make more progress!

How do we signal trust?

We have a natural inclination to trust people who look and act like us, a bias called similarity bias. Aribnb used this to build trust between two people who have never met before in the absence of multiple reviews by creating similarity between the renter and rentee. However, if users share too little or too much about themselves, acceptance rates go down

So how did they design for this? Airbnb nudged users to write an introduction of the correct length and to include the right details about themselves. Airbnb also discovered that at a certain point, reviews trumped similarity bias. When a listing accumulated more than ten reviews, everything changed, people started trusting ratings more!

How to retain motivation?

Leaderboards spur competition, but they have weaknesses. Leaderboards can backfire. If a user gets too far ahead or too far behind, they disengage. In games, there are techniques employed like Dynamic game difficulty balancing (DGDB), also known as dynamic difficulty adjustment (DDA) or dynamic game balancing (DGB), which is the process of automatically changing parameters, scenarios, and behaviors in a video game in real-time based on the player’s ability, to avoid making the player bored (easy game!) or frustrated (hard game). Overall, a competition between users is a powerful technique to motivate action

How to leverage loss aversion to our advantage? 

Losses psychologically feel twice as great as an equal gain. Only prospective (future) costs are relevant to a rational decision, but we fall prey to the sunk cost fallacy. IKEA effect which describes the increase in the valuation of self-made products. While the participants weren’t experts, they saw their creation as ones created by experts. The more of a hand we have in creating something, the less likely we are to part with it. Fitness app Noom uses this to its advantage in their onboarding flows

Peanut effect as a way to counter big loss aversion!

The peanuts effect is when we fail to consider the consequences of small losses. In this case, it is a good thing. The peanuts effect is part of the reason slot machines steal all of our money. The decision to save is easier when framed in a way that spreads the potential losses into smaller increments. Momentum leverages the default bias and the peanuts effect to help everyday people close their intention-action gap

How to use emotions?

We process information about one identifiable person differently than information about a group of people. Identifiable Victim Effect, which seeks to understand what moves us to offer help. Think back to a powerful marketing campaign, and it would have revolved around stories. To move people emotionally, tap into layer two and convey the emotional experience to your audience. We are visual machines. The brain can identify images seen for as little as thirteen milliseconds. All types of companies can help users achieve their goals by making them the hero of their own journey – the way Pain Squad does for its users

How to integrate behavioral-first approach while building products?

The process is quite simple and can be summed as a series of steps below

  1. What is the problem you are trying to solve?
  2. Collect the SAY and DO data
  3. Compare SAY and DO data
  4. Identify potential behavioral drivers
  5. Identify potential behavioral solutions
  6. Create interventions
  7. Test interventions
  8. Iterate

As one can see, steps 4 and 5 are the most crucial ones and this is where learning and having an opinion about behavioral theories may help

Overall, we are just getting started in this interesting domain of marrying behavioral insights with product design. Academic understanding of what drives behavior precedes the implementation by about ten to fifteen years. Product managers and designers learn by taking theories and putting them into practice. Hopefully one should be able to discern where assumptions break down and where they start to work and then hypothesize why and fine-tune it!

Book Review: “Strategy Excellence for Product Managers: A Guide to Winning Markets…” by Greg Cohen

This book “Strategy Excellence for Product Managers: A Guide to Winning Markets through Product Strategy” combines practical wisdom with theoretical frameworks we are already familiar with. In that sense, it’s a valuable read. The author Greg Cohen has 2 decades’ experience in the industry and the resultant scars are visible in his writing!

The book is divided into 11 chapters, however one can start reading from any chapter. In the beginning, the author tackles the question most business books tend to understate. The distinction between vision and strategy is that although vision creates a shared picture of the future, it does not take into account the fact that markets are dynamic. The author also opines why relative growth in a competitive scenario is validation of a good strategy 

Rest of this post summarises key learning from each chapter

Chapter 1

The vision inspires the team towards a better future. The roadmap shows how the vision will be achieved over time while the strategy is about positioning ourselves where the most value will be in the long-term. There are five inputs to a product strategy exercise namely customer, market, competition, technology and business strategy

For developing customer understanding, voice of customer research helps while when doing market analysis, the objective is to understand which segments are most important for the product vision. OTOH when analysing how technology impacts product strategy, it’s important to understand capabilities, adoption rates and its cost structure 

As a product manager where often the gaps are noticed is: not realising that the choice of strategy has to be evaluated in relation to the company’s capabilities and culture. So if the company is not investing in cutting-edge tech work, product strategy has to reflect that limitation

In short, the product strategy must incorporate the business outcome a product needs to fulfil for the company and the time horizon to meet that objective

Chapter 2

The product vision is any product’s raison d’etre. It is a high-level articulation that communicates the product’s value, often by painting a picture of the future

Unlike the product strategy, which can be complex and involved, the vision is an easy-to-understand articulation of the most important parts of the plan. The vision keeps three crucial factors

  • the target customer
  • the in-scope problems that the product solves; and 
  • the most important dimensions of the product

Chapter 3

A strategic roadmap distills the product strategy into sets of capabilities that one wants to release in a given sequence over a defined time frame. Strategic roadmaps ultimately answers the questions – why and why now

A strategic roadmap helps focus all stakeholders on the pieces of the product that must be developed to achieve the strategy and the roadmap has to be vetted against the team’s capacity to produce a credible plan

The three steps to developing a strategic roadmap are 

  • Laying out the external environment i.e. tech, regulation, market trend, competitors
  • Adding product plans and strategic objectives
  • Vetting the plan with dependencies and resource constraints

Chapter 4

In this chapter, the author introduces a few frameworks for prioritising product backlogs. These are summarised as below

  1. Return on Investment: quickly understood metric for company-wide projects and also easily “sold” to management. However it can sometimes prioritise short-term over long-term sustenance and that not all decisions are ROI led. Sometimes people also forget that ROI calculations are as good as assumptions gone into them
  2. Voice of Customer (VOC): This provides a good picture into customer pain points and of course VOC is essential component of every other prioritisation method. However sometimes market shifts can be missed if one over-indexes on VoC. Customer cannot quite often tell about new segments or markets to go after
  3. Must, Should, Could, Won’t (MoSoCoW): this is often hard to apply correctly and is not quite suited for innovation-led products. However it can be useful for first-level prioritisation in a release 
  4. Kano Model: It looks at how the inclusion or exclusion of a feature contributes to customer satisfaction and how that satisfaction changes based on the level of the implementation. However it does not include product strategy or cost view. Typically any feature in this framework can be modelled along three curves: Must Haves, Delighters and PerformanceHowever one has to consider that customer expectations change over time and today’s delighters could be Must Have very soon. Besides, customers are not often a monolith – so not all features are equally important in these buckets for all 
Kano Model

Image 1: Kano Model

  1. Prioritisation Matrix: This s a lightweight, fast, and effective tool to help stakeholders make their beliefs explicit. It is also likely the best way to link product roadmap to company’s strategy, however it is subjective and doesn’t include costs 
  2. WSJF Method: Weighted Shortest Job First method prioritises those items which deliver the most value soonest. The key factors when applying WSJF are: value; time sensitivity; risk reduction and opportunity enablement; and duration/effort. However it does not take product strategy or customer perspective into account
  3. Feature Life Cycle Based Prioritisation: essentially this method buckets features into proof of concept, validated learning, minimum marketable feature and polish features. This method however considers cost and risk. And based on the company’s strategy, some risky (POC!) features can be front-loaded
  4. Value, Risk, Cost Matrix: It can help us visualize investment allocation across a “portfolio” of new features. This lens is also useful for evaluating investments across projects. However value is not necessarily connected to product strategy and value and risks assessments are highly subjective 
VRC Matrix for Product Backlog Prioritisation

Image 2: Value, Risk, Cost Matrix: Prioritisation Priorities are indicated

One has to frame prioritization in the context of product goals and corporate strategy. Overall objective with prioritization is to develop a winning plan and get stakeholders to understand the prioritization

Chapter 5

As a product manager, one needs to focus on product growth and how the product is doing relative to the market. The Ansoff Growth matrix highlights broad phases per product lifecycle

Ansoff Market Growth Matrix

Image 3: Ansoff Market Growth Matrix. Product development is a depth strategy

Chapter 6

This chapter focuses on product-market fit. PMF typically requires focusing on 3 areas: problem, product and business model. If one takes business model as constant, we have 4 scenarios which product managers find themselves in

Image 4: Problem-product Solution Mix by Greg Cohen

When the business model itself is uncertain the hardest business models to test are those involving channel distribution. It takes time to grow channel relationships, and further time and effort to get those partners active

A nifty trick author proposes when doing customer research is to probe customers around 5Ws. Probe means asking ‘why’ without sounding like a 5-year old – Whys uttered 5 times! One can use phrases like “Tell me more”, “would you describe the last time that happened”, “what is the implication of that”, “how do you deal with that”, “what do you do next”

Chapter 7

This chapter focuses on purposeful learning. The sooner companies, product teams, and product managers are willing to acknowledge that they don’t always have all the answers, nor will they have the complete set of answers upfront, the better they will do at managing the uncertainties and risks of new product development

The most we can ask of any team is to deliver validated learning at all stages of the development process. To do this, you have to do two things: become a purposeful learner and create a learning plan

Few techniques to become a purposeful learner are 

  • PDCA: Plan, Do, Check, Act. Also known as the Deming cycle.55 This model assumes the scientific method can be applied, and that a statistically significant data set can be acquired. It works very well for web analytics and quantitative studies
  • OODA LOOP — Observe, Orient, Decide, Act: OODA explicitly incorporates cultural context. With the continuous application of the OODA loop, you can navigate an ever changing landscape
  • Build, Measure, Learn. Within all these processes, the core is to describe how one believes the world works and test it frequently

OTOH learning plans have a question and a test. The team should first formulate a question and then generate a hypothesis or prediction for each test

Chapter 8

This chapter focuses on basic strategy frameworks. There are two components to strategy – where to compete and how to compete. There are 3 ways in deciding where to compete: scale, depth and innovation. Whereas in choosing how to compete – there are 3 positions: leadership, value and low cost. Leadership companies (Apple) focus on delivering superior capability or performance at a premium price. Value companies (Amazon, Flipkart) focus on delivering a high value to price ratio, and low-cost companies (Dell) look to compete almost purely on price

Niching is when a company selects to serve a narrow segment of the market with a product that matches that segment’s unique needs and preferences. Author opines that the companies choose niching when they will struggle to succeed in the mainstream market. This can be contested, niching is sometimes quite a viable strategy to outcompete a behemoth. Think classifieds as a business model are still thriving – surviving 2 decades of technology disruption!

Chapter 9

This chapter provides introduction to familiar frameworks for doing industry analysis. Porter’s five forces framework is touched viz. threat of rivals, threat of new entrants, threat of substitutes, bargaining power of suppliers, and bargaining power of customers

  • When threat of new entrants is high, profit potential becomes limited
  • Non-consumption, or doing without, is also a substitute
  • Suppliers also have power if they offer highly differentiated offering or where there are no substitutes. Scale strategies and standardizing inputs allow you to improve your bargaining position with suppliers
  • Depth strategies are a way to gain power with customers
  • Innovation strategies, along with leadership or value differentiation strategies, are a way to avoid pure price competition with rivals. Relative market share is calculated by dividing your market share by your largest competitor’s market share

Porter’s Five Forces analysis allows us to understand the pressures on the different players in the industry

Chapter 10

This chapter can be avoided without much loss of information. This focuses on market Maps – powerful visualization tools to compare companies’ strategies for deciding where to compete

Chapter 11

This chapter focuses on pricing. There are three factors that influence the price one can command for the product: value to customer, competitive intensity, and costs. As an example, airlines successfully charge travelers different prices for the exact same product, a technique known as yield management

There are three basic pricing strategies from which to start: Penetration, Skimming, and Maximization

  • Penetration pricing is when a company prices a product low relative to its value. Jio’s earlier plan in Indian telecom industry is one such example. Freemium pricing strategies are a version of this pricing strategy
  • Skim pricing is when a company targets a segment of the market with a high willingness to pay, favoring high margins over volume. Apple’s pricing strategy in India is one such example 
  • Maximization is when a company focuses on optimizing total profit or revenue. Amazon as company seems to follow this
    • Contribution margin is the money left over after all incremental costs have been subtracted from each additional unit sold. Thus, contribution margin is the money that is available to cover the fixed and sunk costs of the organisationAfter subtracting all these costs, any additional money left over is true profit. To maximize profitability, you maximize contribution margin. This decision is independent of fixed and sunk costs

How you charge for your product is a key pricing strategy decision that is as important, if not more important, than what you charge because it is easier to adjust your price than to change your pricing model

You should not miss reading the appendix of this book where the author explains different kinds of product-business fit challenges through the case study format. Overall this book is a good read for a mid-senior level product professional!

Book Review: Practical User Research: Everything You Need to Know to…” by Dr.Emmanuelle Savarit

“The issue with startups is not that they do not understand what UX is. I think they are just not ready for it”

“Stakeholders may not realize that face-to-face interviews with 12 people are more reliable than a survey with 100 to 200 respondents”

Have you ever wondered when you should involve user research into your product development? Or think that quantitative research is more reliable than qualitative research? Or wonder how user research can influence business strategy? Or confused about the difference between a user profile and user persona?

This book “Practical User Research: Everything You Need to Know to Integrate User Research to Your Product Development” authored by industry veteran Dr.Emmanuelle Savarit offers a systematic approach to incorporate user research into products. This book is a great read, and can be quite easily used in real life product management situations. Let’s dive in

Structure of the Book

The book is divided into 10 chapters. First 2 chapters concentrate on explaining to readers how user research as an expertise has come about and also how traditionally product development happens across industries. In Chapter 3, the author explains how product managers can introduce or evolve user research function into their organisation depending on their maturity. The author argues that the main obstacle to introducing user research into product development is maturity phase of the business. Chapter 4 and 5 explain how to prepare for user research

Chapter 6 explains main methods for quantitative research namely analytics, surveys and card sorting. Chapters 7 and 8 focus on collecting data and analysing findings for a few qualitative methods. Chapter 9 goes into aspects like ethics, compliance and consent in user research – areas which are generally considered as an afterthought. In the last chapter 10, author briefly touches on how user research projects unfold in real life through 10 case studies she has conducted across multiple industries and cultures

A. Introduction to User Research

User research consists of putting an end user’s needs at the center of the researcher’s investigation. Role of the user researcher is to provide evidence-based findings using quantitative and qualitative research methods. In user research, we are not interested in the demographics, job role, etc. We are interested in participant’s motivation or their behavior while using the product

Like many other disciplines, it traces its origins to the military. After WW2, there was a need felt for optimising weapons in order to minimise injuries to soldiers. Many disciplines like ergonomics, design, human-computer interaction, usability and user research emerged consequently

While doing research, we have the choice of taking a quantitative approach or a qualitative one. Often, stakeholders prefer numbers to case studies. However, when the research is properly done, both approaches are reliable and robust

Quantitative Research

  • Takes a top-down approach, starting with the big picture and using deductive reasoning
  • Validates a hypothesis, theory, or preconceived idea
  • Answers the questions how many, how often, and when
  • Doesn’t answer why or how people are using the website

Qualitative Research

  • Takes a bottom-up approach and starts from a specific observation and goes to a generalization and informs theory. It for the same reason leaves room for unexpected avenues
  • Identifies phenomenon, common patterns, and systematic occurrence
  • Useful to answer the questions how and why

Often, businesses have difficulties understanding that user research is not going to provide them with what they want to hear but rather help them identify what the user needs to perform a transaction, to complete a task, or to carry out their daily job

A typical digital product development goes through 5 phases

  1. Discovery
    1. Get a full understanding of the current situation. It helps to decide at the end of this phase whether the project should be moving to alpha
    2. If made correctly, this will limit the risk of failure and create a clear account of what should be done in alpha
  2. Alpha
    1. It is the moment when the team can try different options
    2. At the end of the alpha phase, the team should have a prototype representing the future service
    3. Sometime after completing an alpha phase, the team may realize that it needs to revisit original objectives and needs
  3. Beta
    1. Building stage of product development for the backend and the front end to provide a minimum viable product/service (MVP/MVS)
    2. At the end of the beta phase, the end-to-end minimum viable product should be ready to move to the private beta phase
  4. Private Beta
    1. Way to put live the new service with a small number of users
  5. Live

B. How research can fit into this framework?

Discovery: this phase is an excellent time to do competitive research

Alpha: In this phase, one can conduct pop-up research, also called guerrilla research which is a quick user testing session in which you test screens and prototypes with real users in their natural environment

Beta: One can test prototypes and solutions

Image 1: Integrating User research during PDLC. Credit Dr.Emmanuelle Savarit

The following diagram summaries research methods typically used in developing digital products

Image 2: Methods used in User Research. Credit Dr.Emmanuelle Savarit

How can one arrive at the Research maturity stage for the organisation since it’s so critical for continued success of the function? 

Author derives a version from the famed Nileson Norman corporate maturity model for UX

Image 3: Nileson Norman corporate maturity model for UX. Credit Dr.Emmanuelle Savarit

While rest of the stages are self-explanatory, in Skunkworks user experience the organization realizes that relying on design team intuition is risky, and the organization starts requesting some data

Image 4: Savarit User Research Maturity Model. Credit Dr.Emmanuelle Savarit

The author opines about a new trend in user research org: research ops team! It needs to be put in place once the organization already has a certain degree of UX maturity and a good understanding of the differences and advantages of conducting research in the product development life cycle. Typically sometimes user research leadership dons this role depending on size of the company, number of user researchers and project complexity!

C. How to prepare for User Research?

  1. Identify if you need user research for your product
    1. The first question to ask is “Does my product/service involve users?” f it does not include end users, you will probably not require any user research.
  1. Review the research that was done in the past and identify the gaps
    1. Identifying the gaps is the starting point of conducting user research and finding out what you don’t know
  2. Identify your users
  3. Build a case to get budget for your project
    1. While building the case to stakeholders, one has to make sure that they are not reinventing the wheel. One needs to have a clear account of what they are trying to do and why, what the benefits are of doing it, and what the risks are of not doing it
  4. Get your stakeholders on board to support the project
  5. Put in place the relevant capability to conduct the research

D. Research Preparation

This is the most important part of user research

Image 5: Steps in a Research Plan Document. Credit Dr.Emmanuelle Savarit

Few points to remember here

  1. The methods to collect data are different from the research method that one may be using. The approach to collecting your data is not the way you are going to analyze the data. Following diagram provides a good heuristic

Image 6: Methods to Collect Data, depending on the development phase of the Project. Credit Dr.Emmanuelle Savarit

  1. Every participant has the right to withdraw at any time during the research. This should be respected
  2. Use a screener / questionnaire to recruit people for research. However be wary of panels since sometimes although using a panel is cheaper, but it may provide only those people who spend a lot of time taking part in studies, user research, and marketing just for the incentive
  3. It is easier to get participants if you are giving an incentive
  4. Organizing all documents in a clear format may save the team a lot of time

E. Research Methods // Quantitative Research Methods

Point to note here is that quantitative approaches used in isolation will provide only validation of preconceived ideas

Analytics: few points to note

  1. Business context is important while analysing data. For example, in grocery shopping, it takes more than two minutes to fill up a shopping basket. So time spent metric has to be interpreted carefully vis-a-vis other categories
  2. It’s important to realise that analytics alone is not enough, and does not answer the questions of why, how, and who

Surveys: Generally used to describe a population. A survey is a methodology that uses questionnaires to collect data

  1. The advantage of conducting a survey is that one can get a large sample size that will validate some of the assumptions
  2. Surveys are useful to validate findings from qualitative research or for data validation
  3. One can also pay for a panel; useful if looking for a specific audience that is difficult to reach

Card Sorting: It’s asking intended readers to sort items into groups or categories that make sense to them. Few online tools such as Optimal Workshop’s Optimal Sort or UserZoom’s Card Sorting can be used here. Types of card sorting can be of three types: open, closed, hybrid. Some of the use cases for using card sorting quantitative research techniques are

  1. Designing a new information product or improving one already existing: Open card sorting
  2. Whether different groups of readers use the same approach to finding items: Open
  3. Where new topics fit into an existing website or intranet: Closed
  4. Whether you can reduce the number of categories, by testing whether a smaller set still works well: Closed
  5. Set the tone of the organizational pattern while encouraging participants to generate and name their own categories: Hybrid
  6. Further explore groupings that were unclear in an earlier sort while providing categories for groupings that were clear: Hybrid

Having ~40 cards ensures that one will generate the data needed to make decisions about Information Architecture. Beyond 60 cards, participants will be less likely to complete the card sort

Also one must randomize the order in which participants see the cards and the categories unless there’s a specific order to test. This removes list-primacy bias from the overall result

Two of the techniques to interpret data from card sorting are: dendrogram and similarity matrix

F. Research Methods // Qualitative Research Methods

A qualitative approach requires a small sample however analyzing qualitative data takes more time than the data collection itself. Some of the methods used to collect qualitative data are contextual inquiries/ethnography, interviews, focus groups, user testing, and diary studies

  • To have reliable results with qualitative research, it’s recommended to have a minimum of eight people if you are conducting user testing every two weeks
  • When product is live or in private beta and if one wants to test the end-to-end journey before release, it’s recommended to have between 12 to 24 participants

Contextual inquiries/ethnography

Here the researcher conducts a face-to-face semi-structured/conversational interview within a user’s natural environment. It’s an observational approach. It’s recommended to be used at the start of every project

  1. Understanding their day and which tasks they have to complete at different times helps us understand which information they will require at time t
  2. It enables us to capture not only how they are using the current product but also to identify the other tools/products that they are using
  3. It however can be time-consuming and is more challenging to record

Interviews

User interviews are frequently used to capture general information about users. There are three types of interviews: structured (pre-defined response), semistructured, and unstructured (natural conversation)

Focus Group

It’s a session that lasts between one and two hours and takes place with six to ten participants who discuss a specific topic. It could be useful when you want your stakeholders involved in the project

Usability Testing

Usability testing/user testing is a face-to-face session with real users who are interacting with the product not only to evaluate the functionalities and the efficiency but also to capture the user’s behavior. However Usability is associated with quantifying success rate, task on time or error rates or satisfaction. User testing can be done as soon as there are some screens available. This is also useful for launching new websites and for evaluating end-to-end user journeys

Some techniques are A/B testing, eye tracking (when there is lot of information – tool Tobii Pro), guerilla testing or pop up research and System Usability Scale (SUS)

System Usability Scale

It is a quick usability test that is now a UX standard. It provides a high-level satisfaction score of the usability of a site, application, or any technological item. SUS is a simple, ten-item scale giving a global view of subjective assessments of usability

SUS scores have a range of 0 to 100. It is not a percentage. A score above 68 is above average, and a score below 68 is below average

Image 7: Interpreting SUS: gold standard for usability testing. Created by John Brooke (1986). Credit Dr.Emmanuelle Savarit

Diary Study

This approach permits us to get insights from users while they are experiencing a situation. The study is longitudinal and captures temporal information

  1. Diary studies are used a lot in the field of education to understand the learning process or seeing how streaming services are being watched
  2. It helps us uncover if participants develop any habits over time
  3. It also allows a peak into user impressions as well as their frustrations
  4. This technique is becoming more democratic now with proliferation of smartphones however making sure that participants remain proactive is a challenge for user researchers

How do we analyse Qualitative Data?

Affinity Diagram

A fast way to analyze data is to create an affinity diagram. Jiro Kawakita created this method that simplifies a large amount of data by grouping it by the themes that emerge from the data

  1. However this analysis should be done with as much objectivity as possible
  2. This method is suitable for brainstorming or early-stage analysis to identify the different steps of a user journey

Thematic Analysis vs. Content Analysis

  1. If we identify that the user makes the same comments while booking a flight or conveys the same issues or frustration, that means we have identified a systematic pattern

Difference between User Profile and User Persona

A user profile is a set of characteristics based on demographics, job role, age, the industry the user is working in. A persona is a typical user who has specific characteristics such are interests, goals, behavior, attitude, habits. A persona is not a real user, it is not a case study, and it is not an imaginative character

G. Few ethical questions in User Research

  • How do we make a website or a digital product accessible so that people with a disability can use it in terms of perception, comprehension, navigation, and interaction?
  • Making respondents comfortable, taking their consent for data use, signing up NDAs for protecting company’s information and also understanding that their participation in the research is voluntary, and one cannot force them to respond to our questions

To sum it up, a general process for conducting user research can be summed up as below

  • Get a brief
  • Refine the scope of the research in order to set up stakeholder expectations
  • Work with your stakeholders to draw up the research questions
  • Choose the relevant method to answer the research questions
  • Identify the right participants
  • Collect and analyze the data
  • Extract meaningful and actionable findings
  • Share your findings with your team and stakeholders

And this process mostly remains the same across industries, org maturity levels, engagement model of the user researcher or geographies!

Book Review: How to Lead in Product Management by Roman Pichler…

The book “How to Lead in Product Management: Practices to Align Stakeholders, Guide Development Teams, and Create Value Together” by Roman Pichler is a quick read which would resonate well with middle to senior product professionals better. The author relies on his past experiences and literature to press home his points. The chapters are organised fairly independently so one can read it backwards if one wants to!

The author starts the book by outlining key challenges for product leaders. He brings forth an important point that a stakeholder is anybody with an interest in your product!

Key challenges that product leaders face are

    • Lack of transactional powers unlike their engineering counterparts
    • Team can be large and heterogeneous with group dynamics in flux
    • Limited influence in choosing who you want to work with!
    • Dual role in terms of being able to guide as well as contribute to team goals
    • Leadership needs at vision, strategy and tactical level are varied

Thereafter author offers a few tactics to influence people and encourage change namely Behavior Change Stairway Model propagated by Voss

Screenshot 2020-08-12 at 10.09.59 PM
Figure 1: Behavior Change Stairway Model by Voss (2016)

Author opines that empathy is probably the most important leadership quality and in order to do that that, he suggests the following (BTW as the author points out developing empathy is the first step in design thinking, an innovation process originally created by IDEO)

    • Cultivate a genuine caring attitude for the people you want to lead, whether you like them or not
    • Improve your expertise as becoming a competent, well-rounded product person requires a continued learning effort
    • Seek the right management support
    • Choose the right leadership style basis people and org context e.g. visionary, democratice, affiliative, delegative, coaching, directing or autocratic. Ultimately one should be flexible in leadership approach and balance the different leadership styles depending on the needs of the stakeholders and the situation at hand

Essentially effective leadership is centred on three components: leader, followers, and situation

After Introduction, the rest of the book is organised as the following: interactions among stakeholders, shared goals, conversations among teams, conflict and its dynamics, decision making / negotiations and self-leadership. This review is also structured along the same lines

A. Interactions

This chapter is focused on scrum master and her responsibilities although general principles could be applied to product managers as well

For effective interactions, gaining people’s trust is vital to guide and align people and to move forward together. Some tactics to build trust are 1. Coming from a place of curiosity and care 2. Listening with an open mind and 3. Being supportive

    • Organise teams around products since component teams tend to have more interdependencies than feature teams —teams that are organised around features—which makes it harder to quickly validate ideas and to offer new or significantly enhanced functionality
    • Form stable teams since teams with stable membership have healthier dynamics and perform better than those that constantly have to deal with the arrival of new members and the departure of veterans
    • Let the Team own the Solution by including the team members in product discovery and user experience work and allow them to directly observe and interact with users
      • Product discovery refers to the work that determines if and why a new product should be developed and how an existing product can become or stay successful
    • Give the Team Time to Experiment and Learn. PMs should engage in enough discovery and strategy work to see things coming and to make the right choices

Lead the Stakeholders by involving the right people. After identifying stakeholders, author refers to power-interest to suggest how to engage with individuals

Screenshot 2020-08-12 at 10.08.13 PM

Figure 2: Power-Interest Grid Ackerman and Eden (2011)

He offers the following tactics to interact with these stakeholders

    • Subjects can make great allies who can help you secure understanding and buy-in for your product across the business. Keep them involved by aligning product roadmaps. Some examples for subjects are product folks from adjacent teams
    • Typical context setters are powerful execs. Regularly consult them to build and maintain a healthy relationship, but don’t allow the context setters to dictate decisions
    • Stakeholders with high interest and high power are called players . These individuals are important partners for PMs. Consequently, one should establish a trustful relationship with them. Attentively listen to what they have to say and empathise with them, but have the courage to decline their suggestions and requests if they are not helpful to create value for the users and business
    • Everyone else is part of the crowd. As these individuals are not particularly interested in the product and don’t have the power to influence product decisions, it’s usually sufficient to keep them informed

Build a stakeholder community. Instead of interacting with the players on a one-on-one basis, aim to build a stakeholder community whose members work together for an extended period of time and who learn to trust, respect, and support each other. In short, move from stakeholder management to stakeholder collaboration

B. Goals

This chapter introduces a set of product-centric, cascading goals as well as guidelines to help one create the right goals
A goal expresses an aim, something we want to achieve. Some goals are big and may never be fully realised, like a vision; others are SMART—specific, measurable, agreed upon, realistic, and timebound

A chain of goals for PMs typically look like this

Screenshot 2020-08-12 at 10.53.21 PM

Figure 3: A Chain of Goals for PMs (Roman Pichler)

Product Vision: It describes the ultimate reason for creating a product and the positive change it should bring about e.g. healthy eating

    • As the vision is an inspirational and visionary goal, it cannot be measured. In fact, one might never fully realise the vision

User and Business Goals: These are strategic goals that are derived from the vision. Also one must make sure that they are specific and measurable. This allows PMs to select the right key performance indicators (KPIs) and understand if the product is meeting its goals

    • Pay particular attention to the vision, user, and business goals. If these goals are not understood and accepted, then getting people to follow product and sprint goals will be challenging

Sprint Goals: These are a step towards the next product goal, and it covers the next one to four weeks. It is a tactical, short-term goal

Objective is a goal that can be measured. One can therefore select metrics to determine progress towards the goal. If the goal is strategic in nature, like a user or business goal, then the metrics would be typically referred to as key performance indicators or KPIs

OKRs (objectives and key results) are the measures used to determine if an objective has been met

How to make goals great and effective?

    • Shared so that people feel responsible for reaching them
    • Neither pressure individuals to agree with you nor leave it up to others to decide the goals
    • Realistic – achievable and measurable. This does not apply for visionary goals
      Inspirational
    • Alignment creating
    • Holding other people accountable for meeting agreed goals, and don’t let them getting away with ignoring goals they have agreed to

C. Conversations

This chapter helps reflecting on and improving listening and speaking habits so that one becomes even better at understanding and guiding people

  • Listen deeply. Effective listening not only helps you receive what is being said but also allows you to tune into the speaker’s emotions and empathise with the person

Screenshot 2020-08-12 at 11.09.47 PM
Figure 4: Covey’s Listening Levels

PMs have to listen with the intent to understand, not to critique or convince

  • Listen inwardly. When listening, pay attention to feelings and thoughts. If you are getting distracted or overwhelmed by your reaction to what you are hearing, then pause the conversation
    • Give the other person full attention. Pay close but respectful attention to the other person’s body language and look out for inconsistencies, expressions that don’t match the words
    • Also prefer face-to-face meetings over email and telephone conversations, particularly for important conversations
    • The speaker’s choice of words, pitch, volume, and facial expressions, including eye movements, gestures, and other body language elements, often reveal her or his feelings
    • A great way to discover the needs behind people’s words is to ask why questions

Listen with patience. Learn to be comfortable with silence which is often necessary to encourage the other person to continue to talk and to share something that might be uncomfortable or difficult

How to handle difficult conversations?

    • Positive first. Before you say something critical or negative, first share a positive observation. This must be genuine and not flattery
    • Another way to help people receive difficult messages is flipping and framing
      • Name it: What is the problem?
      • Flip it: What is the positive opposite? Example: instead of saying you always come late to the product backlog meeting,” say, “It would be great if you could be on time for the meeting”
      • Frame it: What is the desired outcome of the positive opposite?

How to be kind in speech?

    • Keep your speech free from anger or other unwholesome emotions
    • Be grateful for the other person’s time and interest, even if you disagree with the individual
    • Apply kind speech not only to the people who are present but also to those who are not
    • Postpone continuing the conversation if you feel that no meaningful progress can be made right now, possibly due to the presence of strong, difficult emotions

D. Conflict

This chapter deals with some tactics to deal with conflict which is perfectly normal at work

Some common pitfalls with conflict

    • Win-lose. Treating conflict like a zero sum game gives rise to the following common but unhelpful conflict strategies a. Competitive confrontation b. Passive aggression c. conflict avoidance d. Passivity which is the opposite of competitive confrontation: You give up what you want and agree to the other person’s requests or demands, thereby trying to appease the individual
    • Blame game. Accepting responsibility and moving from a blame frame to a contribution mindset will help the two individuals stop being caught up in a blame narrative and resolve the conflict
    • Artificial harmony. This is germinated in the workplace because of either fear of confrontation, wrong priorities, work culture or lack of trust

How to resolve conflict?
Conflict resolution is not about winning, retaliating, or putting the other person in her or his place. It’s about developing a shared perspective on what happened, agreeing on the changes required, and re-establishing trust

    • Acknowledge any wrongdoing, but do not allow it to define who you are
    • Author offers compassionate communication template which flows along the lines of

“When I see / hear [observation],
I feel [emotion]
because I need / value [need].
Would you be willing to [request]?”

    • Before one starts the conversation to resolve a conflict, one must let go of negative emotions and thoughts. One must be willing to share observations while being mindful of words
    • Uncover needs. In non-violent communication, needs are considered to be at the root of our feelings; they are the real reason why we feel the way we do and why we want what we want
    • A great guide to your needs is your emotions: Becoming aware of them and asking yourself why they are present will usually lead you to your needs
    • Make Your Request Clear, Specific, and Positive
    • When you make a request, ensure that you ask and not demand

What should one do when they cannot resolve a conflict?
Stop the process, talk to your line manager and HR, and consider involving a neutral and skilled mediator who can help resolve the conflict
When you witness conflict between several stakeholders and the individuals don’t show any sign of successfully resolving the disagreement, don’t ignore the situation since a lingering conflict does not only affect the people involved but also impacts the rest of the group by reducing morale and productivity

E. Decision Making and Negotiation

This chapter shares the techniques to help one develop inclusive solutions and reach sustainable agreements

Benefits of collaborative decision making

    • Better decisions since it harnesses power of collective wisdom
    • Stronger alignment
    • Increased motivation

How to be set up for success?

    • The more important a decision is, and the less people know and trust each other, the more beneficial it is to bring everyone together in the same room
    • Employ a dedicated facilitator in situations when people either don’t know about collaborative decision making or don’t trust each other
    • Set ground rules. Always speak from a place of respect for others and assume good intentions on the part of the group members
    • Don’t bargain over positions
    • Delegate a decision if others are better qualified to decide or if your input is not needed
      • Delegation ensures that the best-qualified people decide, and it frees up your time
      • When applied correctly, it also sends a positive signal to the appointed decision makers
      • If you need to be involved in the decision, then do not delegate it to others, but participate in the decision-making process
    • Choose a decision rule. Such a rule clearly states who decides and how you can tell that the decision has been made. Four common decision rules that facilitate group decisions: unanimity, consent, majority and product person decides after discussion
      • Unanimity and consensus are not synonyms. The former means that everyone agrees; the latter refers to reaching some form of agreement
      • Unanimity should be used when the stakes are high but we wary of it getting degenerate into design by committee
      • Consent is the absence of objections. A decision is made when none objects

Taking the right decision making steps

Screenshot 2020-08-13 at 12.07.10 AM

Figure 5: A Collaborative Decision Making Process based on Kaner et al (2014)

    • Gather diverse perspectives. Be wary of groupthink in well functioning teams. Play devil’s advocate, suggest to get out of the building and adapt the group composition
    • Build shared understanding. Help people understand where they are coming from and encourage them to explore the needs and interests behind people’s perspectives
    • Develop an inclusive solution. While it’s great to care about the decisions you make, don’t search for the perfect decision
      • To develop an inclusive solution, start by considering how you can address people’s needs, interest, and concerns while at the same time moving the product in the right direction

Tips for negotiating successfully
Conversation techniques which can be used during negotiation a. Mirroring (repeating the words) b. Labelling. Acknowledge emotions c. Open-ended question to uncover other person’s needs d. Patience for active listening

    • One should not be desperate to strike a deal. Don’t allow the other person to put you under pressure. No deal is better than a bad deal
    • Behavior Change Stairway Model – explained in Chapter 1
    • Principled negotiation method by Fisher and William (2012)
      • People : Separate the people from the problem
      • Interests: Instead of arguing over positions, look for shared interests and needs
      • Options: Invent multiple options, looking for mutual gains, before deciding what to do. Avoid the mistake of prematurely excluding options and opting for one solution
      • Criteria: Use objective criteria or a fair standard to determine the outcome

F. Self-leadership

Self-leadership is about developing yourself, about becoming a happier individual and a better leader

    • Practise mindfulness. Mindfulness means paying attention to the present moment, to what is happening right now. It helps you become more aware of your feelings, thoughts, and moods
      • “The point is not to get carried away by our feelings and thoughts but to relate to them wisely
      • Benefits of developing mindfulness – greater serenity, increased empathy, better decision making, improved communication
    • Hold personal retrospectives. Every week around 30 mins to ask oneself and note A. What did I get done this week? B. how am I feeling right now? C. What changes do I want to make next week? Select one or two things which you can realistically improve

Leverage Failure. Cultivate self-compassion i.e. being kind towards yourself, without ignoring the shortcomings you might have

Do one thing at a time. To focus your efforts, consider setting yourself a daily goal. Spend a few minutes in the morning planning your day, and decide what the main thing is that you want to achieve and what the desired outcome should be

Don’t neglect important but less urgent work like product discovery. Author suggest Eisenhower Matrix towards the same

Screenshot 2020-08-13 at 12.45.13 AM
Figure 6: The Eisenhower Matrix to prioritise time

The book ends with the advice about taking care of one’s health and not prioritizing one’s job over everything else. As the book says, there is more to life than work!

Overall, I feel this book packs a few punches for tackling practical PM situations and can be read quite enjoyably by middle and senior levels of product professionals. I can however identify a few things which could be improvised upon in future editions

    • The book is probably more biased towards Agile ways of working. It also emphasizes role of Scrum Master in first two chapters which seems out of place with overall theme of the book
    • Although chapters are supposed to be independent from one another, a certain background implicitly is assumed which could be bewildering for new PMs
    • Sometimes I feel the content could be sequenced better. Conversion techniques appear under negotiations, whereas stairway case for behavior change appears in two chapters with similar level of detailing
    • For couple times, points seem to be just enumerated and they don’t necessary gel with the structure of the chapter

The book can be ordered from Amazon. Could not find it on Flipkart!