Book Review: Practical User Research: Everything You Need to Know to…” by Dr.Emmanuelle Savarit

“The issue with startups is not that they do not understand what UX is. I think they are just not ready for it”

“Stakeholders may not realize that face-to-face interviews with 12 people are more reliable than a survey with 100 to 200 respondents”

Have you ever wondered when you should involve user research into your product development? Or think that quantitative research is more reliable than qualitative research? Or wonder how user research can influence business strategy? Or confused about the difference between a user profile and user persona?

This book “Practical User Research: Everything You Need to Know to Integrate User Research to Your Product Development” authored by industry veteran Dr.Emmanuelle Savarit offers a systematic approach to incorporate user research into products. This book is a great read, and can be quite easily used in real life product management situations. Let’s dive in

Structure of the Book

The book is divided into 10 chapters. First 2 chapters concentrate on explaining to readers how user research as an expertise has come about and also how traditionally product development happens across industries. In Chapter 3, the author explains how product managers can introduce or evolve user research function into their organisation depending on their maturity. The author argues that the main obstacle to introducing user research into product development is maturity phase of the business. Chapter 4 and 5 explain how to prepare for user research

Chapter 6 explains main methods for quantitative research namely analytics, surveys and card sorting. Chapters 7 and 8 focus on collecting data and analysing findings for a few qualitative methods. Chapter 9 goes into aspects like ethics, compliance and consent in user research – areas which are generally considered as an afterthought. In the last chapter 10, author briefly touches on how user research projects unfold in real life through 10 case studies she has conducted across multiple industries and cultures

A. Introduction to User Research

User research consists of putting an end user’s needs at the center of the researcher’s investigation. Role of the user researcher is to provide evidence-based findings using quantitative and qualitative research methods. In user research, we are not interested in the demographics, job role, etc. We are interested in participant’s motivation or their behavior while using the product

Like many other disciplines, it traces its origins to the military. After WW2, there was a need felt for optimising weapons in order to minimise injuries to soldiers. Many disciplines like ergonomics, design, human-computer interaction, usability and user research emerged consequently

While doing research, we have the choice of taking a quantitative approach or a qualitative one. Often, stakeholders prefer numbers to case studies. However, when the research is properly done, both approaches are reliable and robust

Quantitative Research

  • Takes a top-down approach, starting with the big picture and using deductive reasoning
  • Validates a hypothesis, theory, or preconceived idea
  • Answers the questions how many, how often, and when
  • Doesn’t answer why or how people are using the website

Qualitative Research

  • Takes a bottom-up approach and starts from a specific observation and goes to a generalization and informs theory. It for the same reason leaves room for unexpected avenues
  • Identifies phenomenon, common patterns, and systematic occurrence
  • Useful to answer the questions how and why

Often, businesses have difficulties understanding that user research is not going to provide them with what they want to hear but rather help them identify what the user needs to perform a transaction, to complete a task, or to carry out their daily job

A typical digital product development goes through 5 phases

  1. Discovery
    1. Get a full understanding of the current situation. It helps to decide at the end of this phase whether the project should be moving to alpha
    2. If made correctly, this will limit the risk of failure and create a clear account of what should be done in alpha
  2. Alpha
    1. It is the moment when the team can try different options
    2. At the end of the alpha phase, the team should have a prototype representing the future service
    3. Sometime after completing an alpha phase, the team may realize that it needs to revisit original objectives and needs
  3. Beta
    1. Building stage of product development for the backend and the front end to provide a minimum viable product/service (MVP/MVS)
    2. At the end of the beta phase, the end-to-end minimum viable product should be ready to move to the private beta phase
  4. Private Beta
    1. Way to put live the new service with a small number of users
  5. Live

B. How research can fit into this framework?

Discovery: this phase is an excellent time to do competitive research

Alpha: In this phase, one can conduct pop-up research, also called guerrilla research which is a quick user testing session in which you test screens and prototypes with real users in their natural environment

Beta: One can test prototypes and solutions

Image 1: Integrating User research during PDLC. Credit Dr.Emmanuelle Savarit

The following diagram summaries research methods typically used in developing digital products

Image 2: Methods used in User Research. Credit Dr.Emmanuelle Savarit

How can one arrive at the Research maturity stage for the organisation since it’s so critical for continued success of the function? 

Author derives a version from the famed Nileson Norman corporate maturity model for UX

Image 3: Nileson Norman corporate maturity model for UX. Credit Dr.Emmanuelle Savarit

While rest of the stages are self-explanatory, in Skunkworks user experience the organization realizes that relying on design team intuition is risky, and the organization starts requesting some data

Image 4: Savarit User Research Maturity Model. Credit Dr.Emmanuelle Savarit

The author opines about a new trend in user research org: research ops team! It needs to be put in place once the organization already has a certain degree of UX maturity and a good understanding of the differences and advantages of conducting research in the product development life cycle. Typically sometimes user research leadership dons this role depending on size of the company, number of user researchers and project complexity!

C. How to prepare for User Research?

  1. Identify if you need user research for your product
    1. The first question to ask is “Does my product/service involve users?” f it does not include end users, you will probably not require any user research.
  1. Review the research that was done in the past and identify the gaps
    1. Identifying the gaps is the starting point of conducting user research and finding out what you don’t know
  2. Identify your users
  3. Build a case to get budget for your project
    1. While building the case to stakeholders, one has to make sure that they are not reinventing the wheel. One needs to have a clear account of what they are trying to do and why, what the benefits are of doing it, and what the risks are of not doing it
  4. Get your stakeholders on board to support the project
  5. Put in place the relevant capability to conduct the research

D. Research Preparation

This is the most important part of user research

Image 5: Steps in a Research Plan Document. Credit Dr.Emmanuelle Savarit

Few points to remember here

  1. The methods to collect data are different from the research method that one may be using. The approach to collecting your data is not the way you are going to analyze the data. Following diagram provides a good heuristic

Image 6: Methods to Collect Data, depending on the development phase of the Project. Credit Dr.Emmanuelle Savarit

  1. Every participant has the right to withdraw at any time during the research. This should be respected
  2. Use a screener / questionnaire to recruit people for research. However be wary of panels since sometimes although using a panel is cheaper, but it may provide only those people who spend a lot of time taking part in studies, user research, and marketing just for the incentive
  3. It is easier to get participants if you are giving an incentive
  4. Organizing all documents in a clear format may save the team a lot of time

E. Research Methods // Quantitative Research Methods

Point to note here is that quantitative approaches used in isolation will provide only validation of preconceived ideas

Analytics: few points to note

  1. Business context is important while analysing data. For example, in grocery shopping, it takes more than two minutes to fill up a shopping basket. So time spent metric has to be interpreted carefully vis-a-vis other categories
  2. It’s important to realise that analytics alone is not enough, and does not answer the questions of why, how, and who

Surveys: Generally used to describe a population. A survey is a methodology that uses questionnaires to collect data

  1. The advantage of conducting a survey is that one can get a large sample size that will validate some of the assumptions
  2. Surveys are useful to validate findings from qualitative research or for data validation
  3. One can also pay for a panel; useful if looking for a specific audience that is difficult to reach

Card Sorting: It’s asking intended readers to sort items into groups or categories that make sense to them. Few online tools such as Optimal Workshop’s Optimal Sort or UserZoom’s Card Sorting can be used here. Types of card sorting can be of three types: open, closed, hybrid. Some of the use cases for using card sorting quantitative research techniques are

  1. Designing a new information product or improving one already existing: Open card sorting
  2. Whether different groups of readers use the same approach to finding items: Open
  3. Where new topics fit into an existing website or intranet: Closed
  4. Whether you can reduce the number of categories, by testing whether a smaller set still works well: Closed
  5. Set the tone of the organizational pattern while encouraging participants to generate and name their own categories: Hybrid
  6. Further explore groupings that were unclear in an earlier sort while providing categories for groupings that were clear: Hybrid

Having ~40 cards ensures that one will generate the data needed to make decisions about Information Architecture. Beyond 60 cards, participants will be less likely to complete the card sort

Also one must randomize the order in which participants see the cards and the categories unless there’s a specific order to test. This removes list-primacy bias from the overall result

Two of the techniques to interpret data from card sorting are: dendrogram and similarity matrix

F. Research Methods // Qualitative Research Methods

A qualitative approach requires a small sample however analyzing qualitative data takes more time than the data collection itself. Some of the methods used to collect qualitative data are contextual inquiries/ethnography, interviews, focus groups, user testing, and diary studies

  • To have reliable results with qualitative research, it’s recommended to have a minimum of eight people if you are conducting user testing every two weeks
  • When product is live or in private beta and if one wants to test the end-to-end journey before release, it’s recommended to have between 12 to 24 participants

Contextual inquiries/ethnography

Here the researcher conducts a face-to-face semi-structured/conversational interview within a user’s natural environment. It’s an observational approach. It’s recommended to be used at the start of every project

  1. Understanding their day and which tasks they have to complete at different times helps us understand which information they will require at time t
  2. It enables us to capture not only how they are using the current product but also to identify the other tools/products that they are using
  3. It however can be time-consuming and is more challenging to record


User interviews are frequently used to capture general information about users. There are three types of interviews: structured (pre-defined response), semistructured, and unstructured (natural conversation)

Focus Group

It’s a session that lasts between one and two hours and takes place with six to ten participants who discuss a specific topic. It could be useful when you want your stakeholders involved in the project

Usability Testing

Usability testing/user testing is a face-to-face session with real users who are interacting with the product not only to evaluate the functionalities and the efficiency but also to capture the user’s behavior. However Usability is associated with quantifying success rate, task on time or error rates or satisfaction. User testing can be done as soon as there are some screens available. This is also useful for launching new websites and for evaluating end-to-end user journeys

Some techniques are A/B testing, eye tracking (when there is lot of information – tool Tobii Pro), guerilla testing or pop up research and System Usability Scale (SUS)

System Usability Scale

It is a quick usability test that is now a UX standard. It provides a high-level satisfaction score of the usability of a site, application, or any technological item. SUS is a simple, ten-item scale giving a global view of subjective assessments of usability

SUS scores have a range of 0 to 100. It is not a percentage. A score above 68 is above average, and a score below 68 is below average

Image 7: Interpreting SUS: gold standard for usability testing. Created by John Brooke (1986). Credit Dr.Emmanuelle Savarit

Diary Study

This approach permits us to get insights from users while they are experiencing a situation. The study is longitudinal and captures temporal information

  1. Diary studies are used a lot in the field of education to understand the learning process or seeing how streaming services are being watched
  2. It helps us uncover if participants develop any habits over time
  3. It also allows a peak into user impressions as well as their frustrations
  4. This technique is becoming more democratic now with proliferation of smartphones however making sure that participants remain proactive is a challenge for user researchers

How do we analyse Qualitative Data?

Affinity Diagram

A fast way to analyze data is to create an affinity diagram. Jiro Kawakita created this method that simplifies a large amount of data by grouping it by the themes that emerge from the data

  1. However this analysis should be done with as much objectivity as possible
  2. This method is suitable for brainstorming or early-stage analysis to identify the different steps of a user journey

Thematic Analysis vs. Content Analysis

  1. If we identify that the user makes the same comments while booking a flight or conveys the same issues or frustration, that means we have identified a systematic pattern

Difference between User Profile and User Persona

A user profile is a set of characteristics based on demographics, job role, age, the industry the user is working in. A persona is a typical user who has specific characteristics such are interests, goals, behavior, attitude, habits. A persona is not a real user, it is not a case study, and it is not an imaginative character

G. Few ethical questions in User Research

  • How do we make a website or a digital product accessible so that people with a disability can use it in terms of perception, comprehension, navigation, and interaction?
  • Making respondents comfortable, taking their consent for data use, signing up NDAs for protecting company’s information and also understanding that their participation in the research is voluntary, and one cannot force them to respond to our questions

To sum it up, a general process for conducting user research can be summed up as below

  • Get a brief
  • Refine the scope of the research in order to set up stakeholder expectations
  • Work with your stakeholders to draw up the research questions
  • Choose the relevant method to answer the research questions
  • Identify the right participants
  • Collect and analyze the data
  • Extract meaningful and actionable findings
  • Share your findings with your team and stakeholders

And this process mostly remains the same across industries, org maturity levels, engagement model of the user researcher or geographies!

Book Review: How to Lead in Product Management by Roman Pichler…

The book “How to Lead in Product Management: Practices to Align Stakeholders, Guide Development Teams, and Create Value Together” by Roman Pichler is a quick read which would resonate well with middle to senior product professionals better. The author relies on his past experiences and literature to press home his points. The chapters are organised fairly independently so one can read it backwards if one wants to!

The author starts the book by outlining key challenges for product leaders. He brings forth an important point that a stakeholder is anybody with an interest in your product!

Key challenges that product leaders face are

    • Lack of transactional powers unlike their engineering counterparts
    • Team can be large and heterogeneous with group dynamics in flux
    • Limited influence in choosing who you want to work with!
    • Dual role in terms of being able to guide as well as contribute to team goals
    • Leadership needs at vision, strategy and tactical level are varied

Thereafter author offers a few tactics to influence people and encourage change namely Behavior Change Stairway Model propagated by Voss

Screenshot 2020-08-12 at 10.09.59 PM
Figure 1: Behavior Change Stairway Model by Voss (2016)

Author opines that empathy is probably the most important leadership quality and in order to do that that, he suggests the following (BTW as the author points out developing empathy is the first step in design thinking, an innovation process originally created by IDEO)

    • Cultivate a genuine caring attitude for the people you want to lead, whether you like them or not
    • Improve your expertise as becoming a competent, well-rounded product person requires a continued learning effort
    • Seek the right management support
    • Choose the right leadership style basis people and org context e.g. visionary, democratice, affiliative, delegative, coaching, directing or autocratic. Ultimately one should be flexible in leadership approach and balance the different leadership styles depending on the needs of the stakeholders and the situation at hand

Essentially effective leadership is centred on three components: leader, followers, and situation

After Introduction, the rest of the book is organised as the following: interactions among stakeholders, shared goals, conversations among teams, conflict and its dynamics, decision making / negotiations and self-leadership. This review is also structured along the same lines

A. Interactions

This chapter is focused on scrum master and her responsibilities although general principles could be applied to product managers as well

For effective interactions, gaining people’s trust is vital to guide and align people and to move forward together. Some tactics to build trust are 1. Coming from a place of curiosity and care 2. Listening with an open mind and 3. Being supportive

    • Organise teams around products since component teams tend to have more interdependencies than feature teams —teams that are organised around features—which makes it harder to quickly validate ideas and to offer new or significantly enhanced functionality
    • Form stable teams since teams with stable membership have healthier dynamics and perform better than those that constantly have to deal with the arrival of new members and the departure of veterans
    • Let the Team own the Solution by including the team members in product discovery and user experience work and allow them to directly observe and interact with users
      • Product discovery refers to the work that determines if and why a new product should be developed and how an existing product can become or stay successful
    • Give the Team Time to Experiment and Learn. PMs should engage in enough discovery and strategy work to see things coming and to make the right choices

Lead the Stakeholders by involving the right people. After identifying stakeholders, author refers to power-interest to suggest how to engage with individuals

Screenshot 2020-08-12 at 10.08.13 PM

Figure 2: Power-Interest Grid Ackerman and Eden (2011)

He offers the following tactics to interact with these stakeholders

    • Subjects can make great allies who can help you secure understanding and buy-in for your product across the business. Keep them involved by aligning product roadmaps. Some examples for subjects are product folks from adjacent teams
    • Typical context setters are powerful execs. Regularly consult them to build and maintain a healthy relationship, but don’t allow the context setters to dictate decisions
    • Stakeholders with high interest and high power are called players . These individuals are important partners for PMs. Consequently, one should establish a trustful relationship with them. Attentively listen to what they have to say and empathise with them, but have the courage to decline their suggestions and requests if they are not helpful to create value for the users and business
    • Everyone else is part of the crowd. As these individuals are not particularly interested in the product and don’t have the power to influence product decisions, it’s usually sufficient to keep them informed

Build a stakeholder community. Instead of interacting with the players on a one-on-one basis, aim to build a stakeholder community whose members work together for an extended period of time and who learn to trust, respect, and support each other. In short, move from stakeholder management to stakeholder collaboration

B. Goals

This chapter introduces a set of product-centric, cascading goals as well as guidelines to help one create the right goals
A goal expresses an aim, something we want to achieve. Some goals are big and may never be fully realised, like a vision; others are SMART—specific, measurable, agreed upon, realistic, and timebound

A chain of goals for PMs typically look like this

Screenshot 2020-08-12 at 10.53.21 PM

Figure 3: A Chain of Goals for PMs (Roman Pichler)

Product Vision: It describes the ultimate reason for creating a product and the positive change it should bring about e.g. healthy eating

    • As the vision is an inspirational and visionary goal, it cannot be measured. In fact, one might never fully realise the vision

User and Business Goals: These are strategic goals that are derived from the vision. Also one must make sure that they are specific and measurable. This allows PMs to select the right key performance indicators (KPIs) and understand if the product is meeting its goals

    • Pay particular attention to the vision, user, and business goals. If these goals are not understood and accepted, then getting people to follow product and sprint goals will be challenging

Sprint Goals: These are a step towards the next product goal, and it covers the next one to four weeks. It is a tactical, short-term goal

Objective is a goal that can be measured. One can therefore select metrics to determine progress towards the goal. If the goal is strategic in nature, like a user or business goal, then the metrics would be typically referred to as key performance indicators or KPIs

OKRs (objectives and key results) are the measures used to determine if an objective has been met

How to make goals great and effective?

    • Shared so that people feel responsible for reaching them
    • Neither pressure individuals to agree with you nor leave it up to others to decide the goals
    • Realistic – achievable and measurable. This does not apply for visionary goals
    • Alignment creating
    • Holding other people accountable for meeting agreed goals, and don’t let them getting away with ignoring goals they have agreed to

C. Conversations

This chapter helps reflecting on and improving listening and speaking habits so that one becomes even better at understanding and guiding people

  • Listen deeply. Effective listening not only helps you receive what is being said but also allows you to tune into the speaker’s emotions and empathise with the person

Screenshot 2020-08-12 at 11.09.47 PM
Figure 4: Covey’s Listening Levels

PMs have to listen with the intent to understand, not to critique or convince

  • Listen inwardly. When listening, pay attention to feelings and thoughts. If you are getting distracted or overwhelmed by your reaction to what you are hearing, then pause the conversation
    • Give the other person full attention. Pay close but respectful attention to the other person’s body language and look out for inconsistencies, expressions that don’t match the words
    • Also prefer face-to-face meetings over email and telephone conversations, particularly for important conversations
    • The speaker’s choice of words, pitch, volume, and facial expressions, including eye movements, gestures, and other body language elements, often reveal her or his feelings
    • A great way to discover the needs behind people’s words is to ask why questions

Listen with patience. Learn to be comfortable with silence which is often necessary to encourage the other person to continue to talk and to share something that might be uncomfortable or difficult

How to handle difficult conversations?

    • Positive first. Before you say something critical or negative, first share a positive observation. This must be genuine and not flattery
    • Another way to help people receive difficult messages is flipping and framing
      • Name it: What is the problem?
      • Flip it: What is the positive opposite? Example: instead of saying you always come late to the product backlog meeting,” say, “It would be great if you could be on time for the meeting”
      • Frame it: What is the desired outcome of the positive opposite?

How to be kind in speech?

    • Keep your speech free from anger or other unwholesome emotions
    • Be grateful for the other person’s time and interest, even if you disagree with the individual
    • Apply kind speech not only to the people who are present but also to those who are not
    • Postpone continuing the conversation if you feel that no meaningful progress can be made right now, possibly due to the presence of strong, difficult emotions

D. Conflict

This chapter deals with some tactics to deal with conflict which is perfectly normal at work

Some common pitfalls with conflict

    • Win-lose. Treating conflict like a zero sum game gives rise to the following common but unhelpful conflict strategies a. Competitive confrontation b. Passive aggression c. conflict avoidance d. Passivity which is the opposite of competitive confrontation: You give up what you want and agree to the other person’s requests or demands, thereby trying to appease the individual
    • Blame game. Accepting responsibility and moving from a blame frame to a contribution mindset will help the two individuals stop being caught up in a blame narrative and resolve the conflict
    • Artificial harmony. This is germinated in the workplace because of either fear of confrontation, wrong priorities, work culture or lack of trust

How to resolve conflict?
Conflict resolution is not about winning, retaliating, or putting the other person in her or his place. It’s about developing a shared perspective on what happened, agreeing on the changes required, and re-establishing trust

    • Acknowledge any wrongdoing, but do not allow it to define who you are
    • Author offers compassionate communication template which flows along the lines of

“When I see / hear [observation],
I feel [emotion]
because I need / value [need].
Would you be willing to [request]?”

    • Before one starts the conversation to resolve a conflict, one must let go of negative emotions and thoughts. One must be willing to share observations while being mindful of words
    • Uncover needs. In non-violent communication, needs are considered to be at the root of our feelings; they are the real reason why we feel the way we do and why we want what we want
    • A great guide to your needs is your emotions: Becoming aware of them and asking yourself why they are present will usually lead you to your needs
    • Make Your Request Clear, Specific, and Positive
    • When you make a request, ensure that you ask and not demand

What should one do when they cannot resolve a conflict?
Stop the process, talk to your line manager and HR, and consider involving a neutral and skilled mediator who can help resolve the conflict
When you witness conflict between several stakeholders and the individuals don’t show any sign of successfully resolving the disagreement, don’t ignore the situation since a lingering conflict does not only affect the people involved but also impacts the rest of the group by reducing morale and productivity

E. Decision Making and Negotiation

This chapter shares the techniques to help one develop inclusive solutions and reach sustainable agreements

Benefits of collaborative decision making

    • Better decisions since it harnesses power of collective wisdom
    • Stronger alignment
    • Increased motivation

How to be set up for success?

    • The more important a decision is, and the less people know and trust each other, the more beneficial it is to bring everyone together in the same room
    • Employ a dedicated facilitator in situations when people either don’t know about collaborative decision making or don’t trust each other
    • Set ground rules. Always speak from a place of respect for others and assume good intentions on the part of the group members
    • Don’t bargain over positions
    • Delegate a decision if others are better qualified to decide or if your input is not needed
      • Delegation ensures that the best-qualified people decide, and it frees up your time
      • When applied correctly, it also sends a positive signal to the appointed decision makers
      • If you need to be involved in the decision, then do not delegate it to others, but participate in the decision-making process
    • Choose a decision rule. Such a rule clearly states who decides and how you can tell that the decision has been made. Four common decision rules that facilitate group decisions: unanimity, consent, majority and product person decides after discussion
      • Unanimity and consensus are not synonyms. The former means that everyone agrees; the latter refers to reaching some form of agreement
      • Unanimity should be used when the stakes are high but we wary of it getting degenerate into design by committee
      • Consent is the absence of objections. A decision is made when none objects

Taking the right decision making steps

Screenshot 2020-08-13 at 12.07.10 AM

Figure 5: A Collaborative Decision Making Process based on Kaner et al (2014)

    • Gather diverse perspectives. Be wary of groupthink in well functioning teams. Play devil’s advocate, suggest to get out of the building and adapt the group composition
    • Build shared understanding. Help people understand where they are coming from and encourage them to explore the needs and interests behind people’s perspectives
    • Develop an inclusive solution. While it’s great to care about the decisions you make, don’t search for the perfect decision
      • To develop an inclusive solution, start by considering how you can address people’s needs, interest, and concerns while at the same time moving the product in the right direction

Tips for negotiating successfully
Conversation techniques which can be used during negotiation a. Mirroring (repeating the words) b. Labelling. Acknowledge emotions c. Open-ended question to uncover other person’s needs d. Patience for active listening

    • One should not be desperate to strike a deal. Don’t allow the other person to put you under pressure. No deal is better than a bad deal
    • Behavior Change Stairway Model – explained in Chapter 1
    • Principled negotiation method by Fisher and William (2012)
      • People : Separate the people from the problem
      • Interests: Instead of arguing over positions, look for shared interests and needs
      • Options: Invent multiple options, looking for mutual gains, before deciding what to do. Avoid the mistake of prematurely excluding options and opting for one solution
      • Criteria: Use objective criteria or a fair standard to determine the outcome

F. Self-leadership

Self-leadership is about developing yourself, about becoming a happier individual and a better leader

    • Practise mindfulness. Mindfulness means paying attention to the present moment, to what is happening right now. It helps you become more aware of your feelings, thoughts, and moods
      • “The point is not to get carried away by our feelings and thoughts but to relate to them wisely
      • Benefits of developing mindfulness – greater serenity, increased empathy, better decision making, improved communication
    • Hold personal retrospectives. Every week around 30 mins to ask oneself and note A. What did I get done this week? B. how am I feeling right now? C. What changes do I want to make next week? Select one or two things which you can realistically improve

Leverage Failure. Cultivate self-compassion i.e. being kind towards yourself, without ignoring the shortcomings you might have

Do one thing at a time. To focus your efforts, consider setting yourself a daily goal. Spend a few minutes in the morning planning your day, and decide what the main thing is that you want to achieve and what the desired outcome should be

Don’t neglect important but less urgent work like product discovery. Author suggest Eisenhower Matrix towards the same

Screenshot 2020-08-13 at 12.45.13 AM
Figure 6: The Eisenhower Matrix to prioritise time

The book ends with the advice about taking care of one’s health and not prioritizing one’s job over everything else. As the book says, there is more to life than work!

Overall, I feel this book packs a few punches for tackling practical PM situations and can be read quite enjoyably by middle and senior levels of product professionals. I can however identify a few things which could be improvised upon in future editions

    • The book is probably more biased towards Agile ways of working. It also emphasizes role of Scrum Master in first two chapters which seems out of place with overall theme of the book
    • Although chapters are supposed to be independent from one another, a certain background implicitly is assumed which could be bewildering for new PMs
    • Sometimes I feel the content could be sequenced better. Conversion techniques appear under negotiations, whereas stairway case for behavior change appears in two chapters with similar level of detailing
    • For couple times, points seem to be just enumerated and they don’t necessary gel with the structure of the chapter

The book can be ordered from Amazon. Could not find it on Flipkart!

Book Review: Escaping the Build Trap…

‘Escaping the Build Trap’ by Melissa Perri offers some practical insights into why organisations fall into the habit of just building stuff. She works her approach through a case study format which makes the book an enjoyable and relatable read

Although the mandate to escape build trap often requires CXO-level interventions, there are quite a few takeaways from the book for product people at every level

What is the Build Trap?

The build trap is when organizations become stuck measuring their success by outputs rather than outcomes. It’s when they focus more on shipping and developing features rather than on the actual value of those things

Companies can get out of the build trap by setting themselves up to develop intentional and robust product management practices

Fundamental premise in the book is to view companies as a value exchange system and explaining product management as a key driver of this exchange

Companies which fall in the build trap typically confuse this fundamental exchange. They also cannot define value for their customers since they don’t understand customer problems

Screenshot 2020-06-25 at 3.19.49 PM

Figure 1. Companies as a Value Exchange by Melissa Perri

Community, technology and market typically act as outside constraints on this system, although companies can control their own policies, strategy, incentives and structures to efficiently provide value to customers

The book also argues that the most optimal way for organizations to deliver customer value is to morph into product led organisation which has the following key components

    • Creating a product manager role with the right responsibilities and structure
    • Enabling those product managers with a strategy that promotes good decision making
    • Understanding the process of determining what product to build, through experimentation and optimization
    • Supporting everyone with the right organizational policies, culture, and rewards to allow product management to thrive

A major chunk of the book is organised along these four themes – role, strategy, process and incentives for product managers. I summarize the book in the same buckets

Part 1. ROLE of product managers

A few short chapters focus on the following key questions

1.1 How do you define Product?

The author discriminates between products and projects since the two can be interchangeably used in some orgs. Products are defined as vehicles of value, which can provide recurring value to customers without requiring a company to build something new every time. OTOH services use human labor to primarily deliver value to users and projects are discrete pieces of work towards a specific aim

Product-led companies have the following characteristics

    • They optimise their products to achieve value and desired outcomes
    • Growth is driven by product
    • Organisation is scaled through software products

1.2 What is product management?

Defined as the domain of recognising and investigating known unknowns and reducing the universe around unknown unknowns

1.3 What does it take to be a great PM?

    • Product managers are focused on WHY whereas projects managers on WHEN
    • Goal of a good product manager is to reduce risk by focusing on learning
    • Great product managers always focus on the problem and WHY

1.4 How should the product teams and their roles be organised?

Traditional teams are organised around value streams (think end-to-end customer value prop), features or technology. However best teams are organised around org strategy

Part 2. What is STRATEGY? 

    • It is a framework which helps make decisions. It is not a plan. Sometimes it can also be thought as what a company would NOT do
    • It can also be thought of as a deployable decision-making framework, enabling action to achieve desired outcomes, constrained by current capabilities, coherently aligned to the existing context (Ref. Steven Bungay, The Art of Action)
    • Strategy should not change every year or month, lest it be confused with a plan

2.1 What are the pitfalls of strategy as a plan?

It typically fails owing to divergence between outcomes, plans and actions. This gives rise to three gaps a. Knowledge gap b. Alignment gap c. Effects gap

Organisations try to respond to the knowledge gap by more detailed information, alignment gap by detailed instructions and effects gap by tighter control. And this does not work smoothly across functions unless companies view strategy as an action enabler towards achieving results

Screenshot 2020-06-25 at 4.51.42 PM

Figure 2. The Effects Gap by Stephen Bungay

2.2 What does a good strategy framework contain?

Should be made up of two parts 1. Operational framework for day to day activities 2. Strategy framework detailing how the company realises the vision while strategy deployment is the act of communicating and aligning narratives and stories told through the org

It has four levels in the org 1. Vision 2. Strategic intent 3. Product Initiatives and 4. Options or Solutions

2.3 How to create strategy?

The book introduces a framework called The Product Kata for scientifically creating products

Screenshot 2020-06-25 at 5.02.17 PM

Figure 3. The Product Kata Framework by Melissa Perri

2.4 What is a company Vision?

A good mission explains why the company exists. A vision explains where the company is going based on that purpose. Sometimes the vision is clear, the difficult part is connecting it back to the company’s operations. This is where it’s necessary for company leaders to specify strategic intents. These few, concise, outcome-oriented goals focus the company around how to reach the vision. Strategic intent is always aligned to a company’s current state. They are also at a higher level and business focussed

2.5 Where do product initiatives come in picture?

The product vision communicates why you are building something and what the value proposition is for the customer. OTOH Product initiatives translate the business goals into the problems that we will solve with our product. The product initiatives answer how

If there is no coherent vision, product may not be able to scale 

Part 3. How to uncover the right solutions to build? What is the PROCESS behind it?

Enter the Product Kata as explained earlier, there are 4 steps to this. However understanding which phase in a product under is critical to success

    1. Understanding the direction 
    2. Problem exploration
    3. Solution exploration
    4. Solution optimization

3.1 How do we understand the product direction?

This is where product metrics come in. She introduces two frameworks 

    1. Pirate metrics or AAARR framework as promulgated by Dave McClure. The familiar acquisition, activation, retention, referral and revenue framework does not capture customer happiness and therefore…
    2. HEART framework which stands for happiness, engagement, adoption, retention and task success. However overly relying on retention has the lacunae of it being a lagging indicator. A product may not have that long a runway and therefore need to focus on leading indicators like engagement

3.2 How do we perform problem exploration?

There are well known frameworks like problem based user research aka generative research whose objective is to find which problems to solve. They key here is to ask customers the right questions

3.3 Which solutions to build?

First the PM has to be clear about the distinction between building to learn and building to earn. She defines MVPs as something which is built to learn and therefore should be scoped out as something which contains the minimum effort taken to learn

There are 3 frameworks introduced to explore solutions 1. Concierge which is essentially delivering solutions to customers manually and see if it works. More suited for B2B companies and of course it is labor intensive and won’t scale 2. Wizard of Oz which is real-like product rolled out to a larger set of users. The key here is to not run the experiment longer than it should since the experiment is still essentially manual 3. Concept testing like with a prototype. It is more generative than evaluative in nature 

3.4 When should you not do a design sprint?

Although design testing has become a preferred method to experiment and learn, they should not be conducted before user research and identifying customer problems to solve

3.5 How do we optimise a solution?

This is where product vision helps, in sensing direction of progress and helping teams find alignment in solutions proposed. North Star document is one such tool

North Star document explains the product in a way that can be visualized by the entire team and company. This includes the problem it is solving, the proposed solution, the solution factors that matter for success, and the outcomes the product will result in

3.6 How do we prioritise among different solutions?

Although there are many framework prioritization, she talks about the Cost of Delay in prioritizing work. Cost of Delay is a numeric value that describes the impact of time on the outcomes you hope to achieve. It combines urgency and value so that you can measure impact and prioritize what you should be doing first

Screenshot 2020-06-25 at 5.39.33 PM

Figure 4: Qualitative Cost of Delay by Arnold and Yuce

Part 4. How to transition to being a product-led ORGANISATION in ethos and practice?

In product-led organizations, people are rewarded for learning and achieving goals. Management encourages product teams to get close to their customers, and product management is seen as a critical function that furthers the business

4.1 Why is communication important in organisation?

Visibility in organizations is absolutely key. The more leaders can understand where teams are, the more they will step back and let the teams execute. The more PMs try to hide their progress, the wider the knowledge gap becomes and it becomes difficult to sustain the product for long

4.2 How should one communicate product roadmaps?

Instead of thinking of roadmaps as a Gantt chart, one should view them as an explanation of strategy and the current stage of product. This combines the strategic goals with the themes of work and the emerging product deliverables from it

4.3 Why should one communicate the product phases clearly?

It is important for company wide alignment on what stage a product is in – experiment, alpha, beta or generally available (GA). This helps in reducing friction among different units as well. For example, alpha – a phase in which the objective is to determine whether the solution is desirable to a customer – cannot be included in the sales presentation to customers. Beta products – to see whether the solution is scalable – can be

4.4. How should people be rewarded?

One should be rewarding people for moving the business forward— achieving outcomes, learning about your users, and finding the right business opportunities. Rest is vanity!

4.5. Why should experimentation be encouraged?

Experimentation is the ultimate risk-management strategy because, when you experiment early, you can prevent the failure of something you will have spent billions

4.6 How to budget products?

Fund them like VCs. One should allocate the appropriate funds across product lines for things that are known knowns and ready to be built, and also in discovering new opportunities

Lastly the book ends with tips for identifying whether a company is product-led. My favorites are

    1. Who came up with the last feature or product idea you built? The answer should normally be the team
    2. What was the last product idea that you killed? Killing products is sign of a healthy product culture
    3. When was the last time you talked to your customers?
    4. What are your PMs like?

Overall this is an excellent succinct resource for all product professionals. Some of the things I think could have been better covered because of their relative importance in avoiding the build trap are

    • She could have elaborated more on aligning CXOs towards the need for being product-led org. Also for people below in hierarchy, it becomes quite impossible to manage this chasm and the book does not quite offer tactics to manage this transition without the risk of becoming a product martyr!
    • Delving deeper into which product metrics to adopt since they are critical to assessing the product direction and identify product initiatives
    • Product roadmap’ing and communication have been largely not covered although she does refer excellent resources on the same

The book can be ordered on Flipkart and Amazon!

Book Review: Gamification — Using Game Elements in Serious Contexts…

I have been reading up on how to gamify core product experiences and the current state of art. Came across this book Gamification — Using Game Elements in Serious Contexts, it’s published by Springer whose technical books I tend to like apart from the venerable O’Reilly’s. This book is structured like a compendium of research articles, prose has the same “academic” overtones and each chapter can be read independent of one another. Some familiarity with standard terms in gamification parlance would definitely help in quickly skim through the short but dense book consisting of 11 chapters

Chapter 1

Some points and key definitions are belabored again and again throughout the book, mostly since each chapter is contributed by different researchers. For example, definition of gamification i.e. using game mechanics in a non-gaming / non-serious / business context — so called funware. Also defined at many places are external and intrinsic motivations and how gamification can help generate the latter

Equally important is how affordances for gameful experience may enhance user experience AND outcome. The book also differentiates between a play and game — the latter being rule based and goal oriented form of playing

This 2×2 explains the use of play and games and their integration into products (cited from the book)

Courtesy: “Gamification — Using Game Elements in Serious Contexts

This chapter then introduces well-cited MDA framework i.e. mechanics, dynamics and aesthetics of gamification design. Then various types of game mechanics are enumerated namely points, leaderboards, levels and achievement systems or meta tasks. Game dynamics is defined as the run-time behavior of game mechanics acting on player’s inputs and each other’s outputs while aesthetics describe the desirable emotional responses in a player. In the gamification context, aesthetics should represent goal of the gamified system. The chapter then lists the player types as killers (acting on other players), achievers (acting on the world), socializers (interacting with other players) and explorers (interacting with the world)

The chapter ends with describing different types of achievements per achievement goal theory. Instructors are used to guide users throughout the process, quests are unlocked when users accomplish significant tasks, content discovery achievements encourage users to explore the applications, grinder achievements are unlocked when a task is repeated given number of times, Herculean tasks are like bar raisers — relatively difficult and hard to perform tasks. Trophies are designed to mutually exclude other users from winning them and are therefore rare. Loyalty achievements reward the most loyal users

The chapter sounds familiar — more so when many of the current product engagement hooks built by the likes of Flipkart, Amazon, Swiggy, live quizzing products (MMOs) are using these achievements types as part of core gamification design. Sample these: random often silly sounding games in ecommerce apps’ home screens, product hunts, game design of referral programs, loyalty programs which can only be unlocked at the end of the month, extra lives for watching game till the end….

Chapter 2

This chapter can be termed as core of the book with important inferences as to how user behavior can be influenced by gamification. Motivation (desire to perform an action) is an important factor since it drives human behavior and is used to start an activity. Extrinsic motivation (doing things solely for their outcome) doesn’t create sustainable gamification effect. Use of cultural influences is considered in shaping people’s preferences. Four cultural categories in communication are time (2 mins delay is not the same everywhere!), space (personal or shared), context (explicit or direct) and information flow (e.g. message speed). Culture and design can be combined to create serious games e.g. cultural appropriation can be employed to support game’s mechanics, storylines and interface by designers. Amazon’s Great India shopping festival and the use of elephants in logo!

From this perspective, semiotics, symbolic interactionism and metaphors are important extensions.

The authors then proceed to explain Fogg Behavior Model which shows that human behavior is an outcome of three elements: motivation, ability and trigger happening concurrently. Now famous Nudge Theory covers the last two elements: ability and trigger wherein designer simplifies the environment and the context that lead to a certain action while tiny habits method breaks down the desired method into easy-to-adopt small habits

Chapter 3

This chapter touches on gamification analytics — tools that help to monitor the success of gamification projects, to understand user behavior and to adopt gamification designs

One can sequence activities in gamification analytics as business modelling and requirements, design workflow, implementation, monitoring and adaptation. No surprises there for a product manager!

Some of the key metrics while analyzing gamification:

Gamification feedback rate: while gamification feedback is any state change in the game which the user perceives as success. Feedback rate describes total amount of feedback per time user spent in the gamified application. Too low feedback rate could imply — scope of gamification is too narrow vis-a-vis problem space or game is too hard!

Point distributions over users can help detect flaws in the balance of point amounts for gamified actions

Achievable gamification elements: helps to see overall progression of users in the game. Also useful for making adaptation in gamification design — too easy or too hard game!

User distribution on gamification element state: this should be self explanatory

Temporal statistics — how long users need for the completion of particular gamification elements

User characteristics related cohorts

The chapter ends with evaluating certain tools for gamification analytics e.g. Upsight, HoneyTracks, GAMEhud, GameAnalytics etc. Criteria or product capabilities used for ranking the tools are: ability to monitor app KPIs, gamification elements stats, gamification design adaptation, user group of interest and simulation

Chapter 4

This chapter discusses use of gamification in customer-oriented strategies specifically Open Customer Innovation (OCI). OCI to me seems fancy term for denoting prosumers. More generally, it can be defined as the process for integrating customer feedback into companies’ processes collaboratively

OCI can have different stages basis how much value companies extract from their customers. For example, OCI can just be an idea spotting tool or a tool to identify customer’s interests and preferences like surveys. It can also be used as co-creation concept or coproduction (provide new products on a regular basis). Design thinking is in fact one variant of OCI wherein customers are integrated in design and innovation process. Certain risks of an OCI approach could be: non-awareness on part of customers vis-a-vis their responsibilities concerning the final product, retaining customer motivation to cocreate on a regular basis, lack of creativity on part of customers in the long run

Gamification can help in leading to mental states like self-efficacy and flow. Playing can support divergent thinking while learning / unlearning can be initiated by exposing players to new situations. Customers can also be provided background stories to make customer familiar and acquainted with the company as well as its values and vision thereby mitigating some of the key risks of an OCI approach driven product creation

Chapter 5

This chapter focuses on obstacles and challenges in the use of gamification for virtual idea communities (VICs). VICs are the communities in which distributed groups of individual customers focus on voluntarily sharing and elaborating innovation ideas. As an aside, open source software has developed along these lines from the beginning. Linux Kernel Mailing List ( is legendary we all know! Firms benefit from VICs since 1. These are less complex than face-to-face workshops, focused group studies or panels 2. Customer base can be broader in VICs

However having knowledge about customer motivation to participate in VICs is important for firms. Customer may be motivated by 1. Having fun developing ideas 2. Altruism 3. Product innovation and enhancement 4. Capability-signalling motive or self-marketing motive 5. Recognition motive 6. Learning motive 7. Need motive e.g. lobbying or influencing the firm and its products 8. Contacts-to-peers motive

Some of the game design elements which can be used in VICs are game points (as starting point to drive competitive behavior), social points (for fostering a sense of social belonging and as competitive anchor), redeemable points (autonomy for individual development and differentiation), levels (to increase competitive character of the community and to enable inter-user comparison), leaderboards and high scores (highly competitive game design elements to drive behavior), virtual identities (to overcome hierarchy levels and silo thinking) and collecting (to demonstrate social status)

Some of the challenges behind gamifying VICs are misuse by developers and decision makers, manipulative users and overjustification effect demonstrated by players in which external incentives harm intrinsic motivation

Chapter 6

This chapter focuses on designing social media based on open innovation tool to harness potential of play by applying game mechanics to innovation management. There can be broadly two ways 1. Enriching open innovation tool with game mechanics i.e. gamification 2. Adjusting a multiplayer online game for the purpose of ideation (online ideation game)

There are two challenges organisations face regarding design of social media based on open innovation tool 1. Motivating individuals to participate in a state of high involvement and flow 2. Inspiring individuals to generate creative output. For this purpose, online ideation games OIGs can be of high gamefulness i.e. high extent to which design of the system appears to be a game

How to structure game mechanics in the context of innovation management?

One way is so called genex framework . According to this framework, efficient creativity supporting software tool must encourage four generic behavior: accumulating (learning from previous work), relating (consulting with peers and mentors), creating (exploring and creating possible solutions) and disseminating (spreading elaborated solutions)

Game points can serve as trigger for all four behavior while social points can trigger “relating”. Redeemable points can serve as trigger for creating and relating. Two of the largest idea management systems in the world Hype and Spigit use virtual stores

Levels can trigger accumulating and / or creating. When levels are implemented as sections, it indicates possibility of browsing through information and hence accumulation. While when stages are implemented as stages for example by providing ever-increasing challenges they relate to creating. Leaderboards can trigger accumulating and relating while collecting can be assigned the behavior of accumulating

On the other hand, OIGs follow the idea of game with a purpose. They have two design elements: mission specificity and duration (not necessarily though)

In online communities, minimizing the risk of fraud is very important. Few thumb rules to increase fraud-resistance of a gamified system or OIG are

  1. The larger effort and more time to cheat the system, the lower the probability of fraud. For example, users can receive game points based on their trustworthiness levels. Trustworthiness can be calculated by rating the rater or by using meta data about user behavior. Similarly social points are only allocated if certain persons rated the idea positively. Downside is it can lessen site’s credibility and usability
  2. The more transparent the rules are, easier it is for cheaters to find strategies for cheating. That why Google and Amazon’s rankings are a secret!
  3. The more the true identity of a user is known, the lower the probability of fraud. Completely anonymous virtual identities may trigger fake identities and spam. However if privacy concerns are to be addressed effectively, real identity characteristics of every community member can be made visible to only administrators. Thankfully, recent changes in Aadhar point to adopting similar approach when only virtual Aadhar details are to be shared for authentication!

Chapter 7

This chapter focuses on gamification in the context of social collaboration. The central argument of authors has been that gamification elements have mostly ignored augmenting intrinsic motivation. Game elements that include monetary oriented goals can be seen as extrinsic stimuli whereas achievements to learn or improve in a certain activity lead to intrinsic motivation. In specific elements like points, badges and dashboards could diminish intrinsic motivation of users since these elements only support extrinsic motivation. This effect of secondary extrinsic motivation lowering the primary intrinsic motivation is known as overjustification effect

Authors then review certain market tools in this space e.g. Chatter, IBM connections, SharePoint and JIVE. Assertion is that market for gamification in social collaboration software is dominated by third party plug-ins. Microsoft SharePoint remains an exception

Chapter 8

This chapter is pretty interesting, more so product managers. Think about gamification as a market research method for purchase decision process! This space has been traditionally dominated by classical qualitative market research methods e.g. focus groups, interviews and diary studies. Few “problems” with these methods are: methods may be outdated providing only hypothetical and unrealistic results. Probands (best consumers) are rarely able to describe their purchase decision retrospectively since it’s mainly an emotional process. Besides sample bias may be dominant and problematic. Additionally studies themselves run the risk of influencing the decision process!

The universe of games and related research fields can be visualized in this 2×2 (adapted from the book). Horizontal axis denotes the difference between designing a full game and merely employing some game elements in another context

Courtesy: “Gamification — Using Game Elements in Serious Contexts

Serious games are also games but with another purpose than mere entertainment for example, teaching the player something

Why games in market research studies?

Advantage could be that games allow players to immerse themselves deeply into the topic. As result, player gets in a so called flow and give more valid answers. Players are also more attentive and authentic. There could be higher willingness to participate as well

Authors then describe a hypothetical serious game (a board game) which mimics purchase decision making process by making the users park the car in middle of the board after having reached a predefined number of information fields. One advantage of having smartphone application of this game is insights can be transmitted to researchers in real time. By changing a few design elements, authors claim that it could be adopted to other products or simulate different processes altogether

Chapter 9

This chapter should interest all product managers — use of gamification in requirement engineering (RE) and stakeholder management! Essentially gamification in RE establishes feedback loops that reward the “useful” participants. On the other hand, related trend of crowdsourcing can be employed to maximize the number of participating stakeholders — crowd-centric requirement engineering CCRE. To be unambiguous though, the chapter falls short in suggesting some practicable action points for typical software products

Two pillars that authors have proposed to increasing stakeholder engagement and participation are: gamification as means to improve motivation and finally quality 2. Crowdsourcing to achieve higher and broader involvement by means of an open call

The resulting CCRE method has seven phases

  1. Feasibility analysis (determine applicability of CCRE for the specific situation) including scope of the method. CCRE would be unsuitable for a product with very few clients and products for which PM has clear vision and low openness to crowd. Conversely CCRE would be more suited for products with numerous customers, low vision and high openness
  2. Context analysis: analyses the context of requirement elicitation by identifying candidate stakeholders to involve in crowdsourcing as well as existing channels used to provide feedback
  3. Crowdsourcing preparation: before actually involving the crowd, it has to be formed, its characteristics assessed and members prepared for success. CCRE method should provide enough incentives through gamification elements e.g. roles, points, leaderboards, group formation, exploration and endorsement
  4. Crowd involvement: this is the phase where gamification and crowdsourcing occurs
  5. Requirement identification: one has to distinguish between minority and mainstream requirements as well the requirement priority based on business value, implementation cost and business risk
  6. Focus group execution: organized selectively for requirements with higher business priority and shareholder value

Challenges of applying crowdsourcing for RE

Diversity may render decision making processes harder to achieve (inconsistent requirements for example). Collaboration among stakeholders may also lead to dominance of certain opinions and clustering among the crowd providing the requirements

Challenges of applying crowdsourcing for RE

Gamification of RE processes has technical and organisational costs. Stakeholders may be given an option to opt out of gamification which has its own complexities. Gamification may also negatively influence trustworthiness of elicited requirements as certain users may participate just for the sake of it. Some stakeholders may also lack intrinsic motivation to participate

Chapter 10

This chapter takes example of gamified e-books to discover distinctive rhetorical resources used to enhance reading experience and achieve convergence with original literary world. Convergence i.e. the capacity to enhance rather than distract from the narrative is considered a key element for assessing an interactive e-book. Three dimensions to examine convergence between gamified e-books and literary world are: enacting the material world, navigating the medium of text (this can be measured by ratio of reading to other actions, implemented as animated or interactive calligraphy or adding digital materiality to draw attention to text or by concatenating text as taps and choices on user screen), empathy / perception / emotion and choice

Empathy can be developed through perception for example through animation and interactivity, through choice, through emotions

In specific, multimedia feature of e-books has a small positive effect on young children’s literacy development.

Chapter 11

This chapter analyzes gamification of teaching in higher education and I found it a little underwhelming. Authors have developed a game called GamEducation in a master’s level course over two years. While gamification improved major fields of university teaching e.g. motivation, fun, participation and learning efficiency — work load for both students and teaching staff increased. This has roots in both increasing competitiveness among students as well as increased number of challenges

Typical learning management systems (LMS) are criticized because of their restrictions, refusal of use, complexity and reconstruction of established procedures. However even with integrating game mechanics, projects may not always result in increased learning efficiency. The chapter ends with qualitative evaluation of GamEducation game as described by students themselves

This is a unique book, little theoretical in its approach — nevertheless to their credit authors concede this limitation at few places in the book. However if you can correlate your workplace learning with amazing situations various contributing authors place their readers in, this book could teach a thing or two.

Book Review: The Chief Data Officer’s Playbook…

I chanced upon this book The Chief Data Officer’s Playbook while browsing for the latest practices in the realm of product management specific to data science. It helps that the book got released only in Jan 2018 and how emergence of GDPR in coming May 2018 impacts the role

In summary, the book defines the emerging role of CDO in C-suite and the success formula for first-time, second time CDOs. There is clear understanding of contextual, undefined and blurring role of a CDO vis-v-a-vis CIO and its dependence on org and data maturity levels

Why does an Organisation need a CDO?

In short to help drive business strategy in new age which necessitate data usage! Data quality matters more than a data swamp. Improving data governance across the organisation may require a cultural change. CDO role must allow itself to work across functions. CDOs are to be measured against data management, org efficiency, business outcomes, org agility, profitability and revenue. Please note the order is important here!

Secret Ingredients of a CDO

CDO must have narrative power within an org i.e. the ability to pitch at the right level. CDO needs to master the relationship building. Passion is another must-have quality of a CDO

Self-awareness as a CDO

It helps to know one’s talent, strength, weakness, core values, personality and track record. Secret ingredients of a CDO are: relationship building, communication, credibility, passion for data, specialism (expertise), strategist and tactician and being lucky. I however find this list to be applicable for any leadership role in our times!

To successfully work with people who rely more on their intuition / gut-feel to take decisions, evidence-based decision making approach may work better. This is like adding science to the art of decisions!

Difference between CDO and CIO

Bucket analogy for explaining the difference between CIO and CDO: CIO is responsible for bucket and making sure it’s in safe place. CDO is responsible for fluid in the basket and what happens to it!

The author then provides a handy trick to identify duplicate reporting or data. Just switch off the suspect reports for a while and see who in the org complains. If nobody does, it’s probably not needed. I can vouch that this approach works!

The First 100 Days

Focus has to be on understanding what the business does, how it does it, how it’s treating its data and its understanding about the value of data. Also make sure to understand organisation’s heartbeat, what patterns are there to meetings, what meetings you should be part of and thereafter remaining plugged in those meetings. Be ready to have a team, case of change, vision and makings of a strategy. Also identify and deliver on few quick wins soon!

The book then delves into differentiating immediate data strategy (IDS) and target data strategy (TDS). A SWAT team can focus on IDS and more transformational team on TDS. Calling some of the new projects as initiatives helps!

Avoiding the Hype Cycle and Managing Relationships

This aspects is very important, not just for CDO but for any leadership new hire — how to avoid getting trapped into unrealistic expectations? Some of the tips are: excellent communication, incremental value-add as time passes, early delivery of a piece of advanced analytics and managing leadership teams ahead of time by managing / minimizing shocks

To manage relationships with other C-level executives, make sure you set up meetings when you say you will and do what you say you will do. Trust is an important element. Also other stakeholders need to be made understood that data ownership is a shared ownership while accountability cannot be shared. Being accountable means being responsible and being answerable for your actions

Data, Disruption and Innovation

Mainly four aspects can be looked into when evaluating innovation potential of data: disruption in the data itself, positive effect on the balance sheet, developing new revenue streams and future M&As

First-generation and second-generation CDOs need to balance between risk aversion and value addition. FCDOs may also rely more on POCs and IDS to keep stakeholders glued in (remember hype cycle!) and manage risks. SCDOs may also have an opportunity to be truly innovative since good FCDO would have hopefully done his / her job of laying basics well!

Data, Information Ethics and GDPR

Need to consider unintended consequences irrespective of how ethical our intentions are. We also need to consider data aggregation effect when sharing bulk data in public even if it’s not classic PII. Data governance and incoming GDPR also needs to consider availability, usability, integrity and security of data systems

How to value your data?

It’s very important to understand data valuation especially when a CDO wants to tackle data hoarding mentality across the organisation. Different factors which determine data value could be: 1. Are you the only company that has it? 2. How accurate / complete is it? 3. Is it updated? 4. Is is relevant? 5. Can you use it for more than its current usage? 6. Opportunity cost

How to drive change?

Some tips to overcome resistance are: make people understand why they are being asked to do something differently, share the change load with other leaders, paint optimistic picture of the future, communicate vision using org advocates and be careful of what you measure. What you measure drives behavior!

In short, I found this book to be good as an introductory reading for the uninitiated data leadership professionals. Even for those of us who have been in product data / leadership trenches for some time, some of the suggestions could be really useful irrespective of our functions or data maturity

The book is available from Amazon. I could not find it on Flipkart though 😦

PS: Glossary defines 5Vs of Big Data: Volume, Variety, Velocity, Veracity and Value