The Build Trap, Great Product Managers, Strategic Gaps and Product Kata

Table of content

Table of contents

1. Introduction
2. How Marquetly fell into the build trap
3. Great product managers tend to be pulled back to the problem space until it’s properly explored and validated
4. A good strategy should help organizations avoid the strategic gaps
5. Use Product Kata to get closer to our goals

Introduction

Last month, I finished reading Escaping the build trap as part of an attempt to discover the pillars of product management. Although I did not discover anything of sort, I think it was a great read since the book provides useful frameworks and mental models that I can apply in my daily work. In this blog post, I want to elaborate on some ideas in the book that I found to be interesting.

A theme running throughout the book is about a particular fictitious company called Marquetly and how it escaped the build trap. Before that, we need to set some context by seeing how Marquetly fell into the build trap in the first place.

How Marquetly fell into the build trap

Melissa tells a story about a fictitious company called Marquetly - an education company that provides online training for marketers. Experts in digital marketing, Marquetly professionals create classes through their online platforms that any individual can take for a monthly subscription.

Marquetly was growing rapidly with yearly revenue growth at 30%. They hired a lot of people and assigned them to all sorts of projects. Then they learned that they needed product managers to work with developers after adopting Scrum. They moved marketing people to product management because they knew the audience for the school best.

The product and the sales team were pointing fingers at one another. All of them cited a lack of product management skills as the problem. Melissa got to work with the product managers, assessed their skills, provided them with new frameworks to try. Even though they saw glimpses of early success in how they thought and approached the problem, these moments were short-lived because the company structure incentivized around shipping features rather than values. They were handicapped by poor planning and poor strategy.

Revenue was declining, so the leadership pushed for even more features to be shipped, with the premise that without demonstrating the ability to ship, they would not be able to raise another round of funding.

The product managers reverted to their old ways. They stopped doing user research because it took time away from writing user stories. They began to focus on getting as many features as possible out the door. When the next release came, they had about 10 new features to deliver to their customers.

The calls began pouring in. The site was breaking because the features were not well tested. The teachers were frustrated that the new features got in the way of them trying to create courses and respond to student comments. Many teachers decided to take their courses down, which stressed the account managers to bring them back.

In the end, no one was using the new features that were rushed through the door. The revenue growth was declining, and company was feeling the heat. The company had too many strategic initiatives spread over very few people. The incentive structure is tied to shipping software, not solving problems. People think they have to ship or they won’t get paid. This is what Melissa called the build trap.

The build trap is when organizations become stuck on measuring their success by outputs rather than outcomes. It’s when they focus more on shipping and developing features rather than the actual values those things produce.

Becoming a product-led organization is the way to escape the build trap which, according to Melissa, involves four key components:

• Creating a product manager role with the right responsibilities and structure.
• Enabling those product managers with a strategy that promotes good decision making.
• Understanding the process of determining what product to build, through experimentation and optimization.
• Supporting everyone with the right organizational policies, culture and rewards to allow product management to thrive.

Great product managers tend to be pulled back to the problem space until it’s properly explored and validated

The book presented two bad archetypes of product managers : The Mini-CEO and The Waiter. Both are bad because they are too concerned with what to build, one way or another.

People often say that product managers are like CEOs their products. However, CEOs have a lot of authority over many things: they can fire people, they can change up teams, they can change directions. This myth birthed a very arrogant product manager archetype who doesn’t listen to anyone, and only tells people what to build. He usually does not do enough user research, and therefore relies on more opinions than evidence. To sum it up, The Mini-CEO is too concerned with building what they want to build.

In contrast, The Waiter is someone who swings to the other extreme. They go to their stakeholders, customers and turn their requests into a list of items to be done. There’s no goal, no vision, no decision-making involved. More often than not, the most important person gets their features prioritized. They implement the specific solutions that customers demand, instead of solving their actual problems. Waiters tend to focus on the when, rather than the why. As a result, they easily fall in the trap of managing projects rather than products. The Waiter is too concerned with building what others want to build.

So what’s a great product manager looks like? Let’s follow Melissa’s story of a great product manager in action. Melissa brought in Meghan, a product manager whom she knew, to talk to the team at Maquetly. Meghan worked on a software for consumer mortgages at a large retail bank. She told the team about how she thinks of her role and what does she do on a daily basis.

“I always start with our mortgage division’s vision in mind,” explained Meghan. “That’s our business. The vision is to make it easier and more convenient for mortgage applicants to apply (or for mortgage holders to access), their information from anywhere.”

Here we can see that she starts with what, specifically what does her business promise through offering this software or product. Answering this question leads her to focus on why that promise was made, or more particularly, why the mortgage application process was not easy and convenient.

After spending a lot of time talking and listening to them, she uncovered a lot of user problems that needed to be solved. For example, users have to go to the local bank branch multiple times a month to meet with a loan officer. They have to do a lot of paperwork in the office, which means when they forget necessary documents they have to come back the next day. And even then, they have to wait to see if they’re qualified for the amount they needed.

But how did she decide which problem to solve? Again, she goes back to asking what is the biggest problem that needs to be solved. At the time, 60% of first-time applicants who started the mortgage process did not finish with the bank and instead switched to other competitors. Improving that number was what aligned with the vision of her department.

To do that, the first thing Meghan wanted to understand was why these first-time applications were dropping out. Many people said they were frustrated with the process and willing to make it better. She conducted user research sessions with the team periodically and soon they found a pattern. Because document verification could not be done online, users had to go to the office. Many of them dropped out because they could not find an open appointment to verify their documents. Meghan sent out a survey and found that the problem was prevalent - only 25% who had this problem actually completed their applications with her bank.

She called the team together to generate ideas for what to build. They came up with many solutions and decided to run some experiments to see which one was the best. One experiment is essentially a Wizard of Oz Prototype. They had the applications email in the documents and the bank designated a person to review and approve them. Over time, these applicants completed their applications 90% more often than those who had to come to the office. They then scaled that solution and eventually reaching on the goal of zero verifications.

All the events are kept true to the book, but the framing and emphasis above are mine. In this framing, great product Managers do oscillate between the problem space (the why) and the solution space (the what), but they tend to be pulled back into the problem space until it’s been explored and validated. Only until then the focus is shifted towards ideating, validating and scaling the solution. It’s not a revolutionary idea, but seeing how it’s played out in a narrative is certainly helpful.

A good strategy should help organizations avoid the strategic gaps

Stephen Bungay defines strategy in his book The Art of Action as:

Strategy is a deployable decision-making framework, enabling action to achieve desired outcomes, constrained by current capabilities, coherently aligned to the existing context.

The most important part about the above definition is that it positions strategy as a decision-making framework that enables decisions, instead of a plan to be executed. In other words, a strategy specifies certain essential structures that govern how people should make decisions, but not the decisions themselves. In contrast, a plan is a set of decisions that will be carried out to achieve certain goals or objectives.

Why is this distinction important? Or conversely, what are the consequences of treating strategy as a plan. While studying strategy, Stephen Bungay discovered that when companies approach strategy as a plan, they often fail to achieve what they expected. This failure comes from the actions taken to fill in these gaps:

• The Knowledge Gap.
• The Alignment gap.
• The Effects Gap.

The knowledge gap is the difference between what management would like to know and what the company actually knows. Management desires certain business outcomes but they lack the information necessary to create a plan that would achieve these business outcomes. This creates an impulse for management to demand more details to feel more certain. However, such certainty is often not possible until after experiments with high degree of uncertainty have been carried out. When the team is unable to provide such details, the knowledge gap manifests.

So what happens when management ignores the knowledge gap and proceeds to make plans? The alignment gap appears.

The alignment gap is the difference between what people do and what management wants them to do. Management may come up with their own plans because they believe that executing certain decisions will achieve the desired outcomes. The problem with this is that, as mentioned, they often lack the user understanding and insights required for such plans to be realistic and coherent. If the team has sufficient authority, then they would push back and execute other more informed and thought-out plans. Management is puzzled as to why things do not go as according to their plan, which is when the alignment gap manifests.

What happens, then, if the team doesn’t push back and instead carries out the plans that management provide? The effect gap appears.

The effects gap is the difference between what we expect our plans to achieve and what really happens. When plans based on flawed knowledge (inevitably) fail, management doesn’t understand why they fail and thus commands the team to provide more details & information. As you can see, we have arrived at the knowledge gap once again.

These strategic gaps tend perpetuate themselves and ultimately create friction that slows down progress and eventually grinds the company to a halt. Therefore, the advantage of defining strategy as a decision-making framework is in avoiding these gaps and enable organizations to be more effective.

Use Product Kata to get closer to our goals

Product Kata is my favorite framework from the book, and that’s mostly because Melissa did a great job walking the reader through the how the team at Marquetly actually applied Product Kata in their product development process.

To create strategy, you must understand where you want to go, then identify the obstacles standing in your way and finally experiment to tackle them until you reach the vision. This continuous improvement framework is called the Improvement Kata. The Product Kata is an adapted version that aligns with product terminologies.

There are four stages to the product kata:

1. Understanding the direction.
2. Problem exploration.
3. Solution exploration.
4. Solution optimization.

The first step of the Product Kata is to understand the direction, which is about the company vision and the business goals that the company needs to reach in order to progress towards that vision. They are important but are not too relevant to the ideas I want to elaborate. The next two steps: problem exploration and solution exploration, are particularly interesting.

Melissa showed how the team at Marquetly to explore their problems. They had set the goal to increase the rate of published course to 50% and the number of second courses created by teachers to 30%. After evaluating where they are with respect to these goals (25% and 10%), they determined the biggest obstacle to reaching this goal is the lack of understanding about the problems that teachers face when creating courses.

The next step they took was doing user research. After interviewing 20 teachers, they have discovered numerous problems that users were trying to solve:

• They want to easily transfer their courses from another school.
• They want to import all their content for creating a new course.
• They want an audio-option only to save time creating videos and appeal to podcast lovers.
• They want recommendations on pricing when creating a course.
• They want to know what their potential students want to learn so that they can create relevant content.

They map the problems to the current user journey, and decided that the biggest problem was to get the teachers’ content into the system. They wrote down their hypothesis:

We believe that, by helping teachers get their lesson content into the system painlessly and quickly, we can increase the rate of published courses to 50% and increase the number of second courses created to 30%.

After discovering their most significant problem - getting existing content to the system, they refer back to the Product Kata to see what’s next.

After learning about that teachers have trouble getting the content into the system, their current state with respect to their goals had not changed. But their biggest obstacle did. They needed to know whether they are picky about the format, or they would follow a template.

After reaching out to 20 teachers and offered to get their content into the system, they discovered that the content came in all shapes and forms, Dropbox, Google Docs, Youtube link. The most surprising thing was that the videos came in unedited, only with instructions on how they are to be edited.

It turned out that the teacher were good at developing curriculums, but not necessarily good at making videos. They began to suspect that getting the content into the system might not be the problem, but creating online content was. They went out to talk with the teachers and confirmed that the they were indeed frustrated that they had to learn how to edit and create engaging videos.

After discovering that teachers have troubles editing and creating content which prevent them from publishing more courses, the Marquetly team tried to pitch their video-editing services to the teachers. They set the success metric to be at least 10 of the courses they worked on to be published within a month.

After two weeks, the team further discovered that most teachers had no idea what good videos look like for an online class. They ended advising the teachers to make their videos entertaining. Based on this information, they determined the solution components that mattered to the teachers:

• Recipe or how-to guide for successful video creation.
• The ability to piece together talking-head video, slides, images, audio and Youtube videos.
• The ability to show text on top of the video.
• Introduction slide to the video.

While the team was thinking through how these factors might turn into a scalable offering, they started to see courses go live. Within one week after the videos were edited and uploaded to the site, half the teachers had published their courses. By the end of three weeks, 12 teachers had published, so they succeeded in reaching their metrics for this experiment. The rest of the story is about how they scaled and eventually reached and surpassed their business goals.

The Product Kata for Marquetly looked like this:

I think Product Kata aligns well with the notion that in lean startup, anything that doesn’t produce knowledge is not essential. It does so by:

1. Explicitly recognizing that the outcome of an experiment is knowledge.
2. Tying the knowledge obtained from the previous experiment to the action in the next one.

Conclusion

Overall, while the book was not revolutionary, it has a lot of great ideas and frameworks that are neatly organized and presented. I enjoyed Melissa’s style of writing in this book a lot. Although Marquetly is not an actual company, the way she moves back and forth between presenting an idea and how it manifests at Marquetly was helpful to capture my attention and provide a narrative structure that connects the ideas together.