1. Customer Validation

Going from Idea to Sales Ready Product

This module covers how to go from idea to sellable product by validating your idea with customers to find their urgent need.

No items found.
1.1 Customer Validation
John Vrionis
John Vrionis
Co-Founder and Enterprise Partner

John Vrionis
John Vrionis
Co-Founder and Enterprise Partner

Customer Validation: Going from Idea to Sellable Product

“A value hypothesis is an attempt to articulate the key assumption that underlies why a customer is likely to use your product. Identifying a compelling value hypothesis is what I call finding product/market fit. A value hypothesis identifies the features you need to build, the audience that’s likely to care, and the business model required to entice a customer to buy your product. Companies often go through many iterations before they find product/market fit, if they ever do.” -Andy Rachleff


A startup's lifecycle can be grouped into two phases: pre-product/market fit and post-product/market fit. When your startup is pre-PMF, your singular focus is getting to PMF. But what exactly does that road from seed to PMF look like? 

We sat down with Spiros Xanthos, founder of Omnition, to walk through what he identified as “the five phases from seed to product market fit” and the process he used to take his company Omnition from idea to acquisition in less than 18 months. It’s worth disclaiming that there is no one right route to product market fit. We captured Spiros’ experience and learnings to help founders navigate what is often the toughest leg of the entrepreneurial journey: going from an idea to a sales ready product. For each of the five phases, there is one primary goal and different objectives across customer conversations, product, and team.


Phase 1: Confirm the Market Opportunity


In this first phase, the primary goal is to understand if a real market pain exists. If you’re going to spend the next five to seven years building something, you want to be as sure as possible that a real market opportunity exists. If you’re still employed at another company, this is when you might quit your job and set off on your journey.


  • Customer Conversation Status - For the first few months, your goals should be to talk to 20-30 customers. A good place to start is with people in your network who work in the general space you’re thinking about addressing. You might ask, “What are the problems you face everyday?” “What environment do you work in?” and “What are the common problems in that environment?” What you’re listening for in these conversations is the convergence of the problems you’re hearing. In other words, your product idea has to be consistent with the problems you’re hearing from multiple customers before you build anything. 
  • Product - At this point, your product is nothing more than an idea. Spiros notes that every time he built product features off of pure speculation, it ended up being a waste of time.
  • Team - The team in the first phase is just the co-founders. You can use the notes coming out of Phase 1 customer conversations to help recruit the rest of your team.


Phase 2: Initial Outreach and Early Product Design


This phase is what Spiros likes to call “Interview Plus”. It’s the phase where your customer conversations shift from unstructured conversations to more targeted feedback sessions. The goal is to understand the customer’s specific high priority problems and test initial mockups.


  • Customer Conversation Status - By Phase 2, you’ve probably tapped out your personal network and need to reach out to 40-50 additional prospects. (See our Outreach Playbook for specific outreach tactics.) Your team must agree to a process for collecting, sharing, and analyzing customer feedback. Your customer conversations should have already converged and you’ve confirmed there is a real problem. You probably have some understanding of the common tech stack your customers are working with and a general sense of common problems, but have some remaining open questions. Spiros notes that one way to double-check your assumptions is to draft an internal business plan with the 30-50 questions you still don’t have answers to. Some questions may become irrelevant, but others may become critical to your business. 
  • Product - You might have initial mockups and simple workflows to show in these conversations, but no built product.
  • Team - An early key hire is a UI/UX engineer who can help you design an early product that reflects the customer feedback you’re capturing.


Phase 3: Quick Product Iteration 


By Phase 3, you’ve spoken to what feels like countless customers and gathered detailed, meaningful feedback. Your goal now is to confirm your understanding of your customer’s major problems and design a solution. Rapid iteration is critical to make sure that what the initial workflows you build accurately captures and solves the customer pain you’ve heard in earlier conversations.


  • Customer Conversation Status - Your aim should be to drive conversations to design partner status. One tactic Spiros used to close design partners was offering a 50% discount if they onboarded within the year.
  • Product - You’ve iterated quickly to build out initial workflows you can show in meetings to help close design partners.
  • Team - Hiring another engineer to help with development might make sense to help speed up iteration.


Phase 4: Confirm Design Partners and Launch 1.0


In Phase 4, your goal is to work with your top three design partners to make sure that the full product you’re building meets their needs and is usable in their environments. 


  • Customer Conversation Status - By this point, you should have answers to the 30-50 open questions you drafted in Phase 2. 
  • Product - Working with your top three design partners, you should build a demo product you can show and start selling in customer meetings. Continue iterating and validating the early product.
  • Team - This is when you might consider hiring an SDR (sales development representative) to help scale outreach. 


Phase 5: Sell to Early Adopters


The fifth phase before PMF is all about accelerating your company’s growth. The focus is no longer on validation, but on selling. The goal is to create a repeatable sales process with evangelical supporters of your solution. (A customer reference is something you have to arm twist to get; an evangelist is someone you can’t get off the phone.)


  • Customer Conversation Status - By this point, you should be following a structured sales process with identified decision makers, buyers, and influencers. The focus should be on helping champions navigate buying your software and expanding the pipeline. Scaling and finetuning outreach through marketing should be an additional priority.
  • Product - You have a working 1.0 product with delighted reference customers.
  • Team - This would be the time to bring on additional Engineering, Product, and Marketing hires to round out the team. 


With a full team, early signs of traction, and a working product, raising Series A funding might also be an appealing option to help accelerate growth, but not before those fundamental milestones are achieved. As Steve Blank put it,

“Founders need to ask themselves the hard questions: Have we identified a problem a customer wants solved? Does our product solve these customer needs? If so, do we have a viable and profitable business model? Have we learned enough to go out and consistently sell the product? Are the sales and business plans realistic, scalable, and achievable?”

Founders on their way to PMF work to find those answers as they progress through each phase of the journey. Through extensive customer feedback and quick iteration, they are able to fill in more of the blank space and build something that customers need and are willing to pay for. 

Key Takeaways


  • Don’t waste time building things on speculation. Instead, validate the problem with the customer first and show them early designs to confirm it’s what customers want before building.
  • Use the customer outreach process to disqualify customers and understand the core problem you’re solving. This means being much more disciplined in asking questions where you might not like the answer. This helps you to arrive at the right insights and eliminate prospects who aren’t the right fit.
  • One of the founders needs to spend the majority of their time selling and recruiting. As a founder, you need to learn how to sell as an expert, but also how to actually close deals.
No items found.
1.2 Outreach Playbook
Liam H. Mulcahy
Liam H. Mulcahy
Director, Sales GTM

Liam H. Mulcahy
Liam H. Mulcahy
Director, Sales GTM

Outreach Playbook

This guide is intended to detail the customer validation outreach process we recommend for Unusual Ventures portfolio companies at the seed stage. The goal is to provide a manual for founding teams. At the end of the process the team should be able to deliver:

‣ A crisp articulation of the pain the company solves.
‣ A proven, repeatable outreach methodology to generate meetings with “target” customer types.
‣ At least 10 engaged customer prospects for POC engagements.

Craft Your Outreach Message


All founders eventually reach a point where they tap out their personal network and have to conduct cold outreach. Part of ensuring your outreach is successful and gets the response you want is having the right outreach message.


If your company is in stealth, make use of that because the intrigue encourages people to take the meeting. The outreach message should come from a Founder/C-level executive to make the recipient feel special. Frame the text in the body of the message in a way that asks for feedback with a very crisp statement of the pain/value proposition. 


Example 1: Shujinko

"Hi NAME,

 

Based on your IT compliance leadership at XXXX, I wanted to reach out about a new stealth company. I'm [NAME], CEO of [COMPANY], and we’ve built the first compliance solution designed to solve a critical problem that plagues IT teams – constant monitoring of WHO has access to WHAT, even in dynamic DevOps environments where creating and maintaining appropriate access policies is a tremendous challenge. We're looking for leaders interested in seeing a demo of the solution and giving us feedback.


Backed by Unusual Ventures, the Shujinko solution platform is specifically built for a world where public cloud applications are being built at lightning speed and are often no longer behind corporate firewalls. Shujinko has the unique ability to provide visibility into what is accessible and by who AT ALL TIMES -- that enables IT to move at the same speed as DevOps. The business can create as many public-facing cloud applications as it wants; Compliance teams can finally keep up.

 

We're hoping to get great feedback from security professionals such as yourself and help prepare the platform for a public launch. Would you be willing to take 30 minutes and take a look at what we've built?


Best regards,


Matt Wells, CEO"

Example 2: Omnition


"Hi NAME,

 

Thanks for connecting. I’ve spoken to a few engineers at your company but they weren’t close to observability and monitoring so I’m excited to get connected with you.


I am the founder of a next-generation observability tool called Onnition. We are actually still in stealth [launching publicly next quarter] but I wanted to share our UI with some people who are working on distributed tracing. We are taking a new approach and I think it is really valuable to the space.


Are you open to seeing a demo next week?


Best,

Spiros, CEO"


Example 3: Harness


"We've been working on a startup to bring an innovative approach to Continuous Delivery and DevOps. Our aim is to simplify and fully automate the deployment of complex applications. We are looking for a few engineering and devops leaders in the enterprise for validation and feedback on our approach. We are in stealth today but looking for as much feedback as possible before releasing our GA product. Please let me know if you'd be open to providing us feedback over a quick demo.


Best Regards, 

Jyoti Bansal, CEO"


Key things to note:


  • Include a clear “ask” for the meeting.
  • Experiment with sample subject lines; you want to test to see which works best.
  • Ask if they would like to provide feedback rather than selling them something.
  • Tell them you’d really value feedback from a “world-class” professional. Flattery works.


Organize Your Outreach Process

  • Start with LinkedIn. Use the Navigator tool and search for individuals with key words in their descriptions.  
  • Example: VP Platform, Director of Data or Compliance, Director of IAM, Mulesoft user, Apigee, APIs.
  • Sending emails is better than InMails.
  • Use tools such as Prospect, GetEmail.io, Datanise, Apollo, Connectifier, etc. to find the emails for the people that have been identified.  
  • It’s ok to even guess an email address for high quality targets. This doesn’t scale, but that’s not important at the earliest stage.
  • Consider using tools that allow creating multiple emails so a sequence can be sent over a period of time.  
  • Remember that it can take MANY emails before you ever get a response. (It can take 50 emails to get 1 meeting.)


Ask the Right Questions in Customer Meetings


Congrats, you have a meeting set up! Now what?

  • Plan for 45-minute meetings. The majority of these can be done on the phone. 
  • The early product demo and questions should be your major focus. 
  • Develop “expected comments” from prototypical customers you expect to encounter.
  • What will “Nervous Ned” be most concerned about? What do we need to really emphasize with this persona?


Your customer meeting deck should cover:


Slide 1


  • Present four problem statements. The goal is to present them and ask the prospect “Does this resonate?” and “Which of these is most important to you right now?”  “Any others you would add?”  
  • Example for Shujinko:
  1. Is the cloud a blackbox for you in terms of who is accessing what? 
  2. Worried about passing your GDPR compliance audit?
  3. Concerned internal or external parties are accessing sensitive data they shouldn’t be?
  4. Is your access management team falling behind due to DevOps speed, thereby risking your sensitive data? 
  • NOTE: If none of the above resonate with the participants, there’s no point in continuing the meeting. Move on quickly.  


Slide 2


  • This lays out the magic of what you do.  
  • Use bullet points, but make it VISUAL if possible.
  • Goal is to explain succinctly “Here is what we do and why it is differentiated.”


Slide 3


  • Show early product or demo.
  • Ask Questions. “Do you think this would work in your environment?” 
  • If yes, “How would you sell this to your boss?”
  • Jyoti explained the approach he used for initial meetings for AppDynamics:
  • Tell them your view of the pain and then ask, "Does this resonate with you?” You want to hear, "Yes, we see this all the time at our company..." if they do have that pain.
  • Tell them a bit about your solution (show a quick demo if possible, or slides outlining the solution otherwise). 
  • Ask "Do you think our approach will work in an environment like yours?" 
  • Don’t ask "What do you think of our solution?" People usually want to be polite and spare your feelings.
  • By asking "Will this work in your environment?” the customer may be more comfortable giving honest feedback by saying, "Your product is good…but it won't work in our environment because of XYZ.”


Create a Decision Point


If you get positive answers to a) “Does this resonate with you?” and b) “Do you think our approach will work in an environment like yours?,” immediately ask: 

“How would you make the business case to your boss to pay for this?”

This is one of the most important questions. You want to start hearing them articulate your business case and the value you offer. NOTE: In your first 10-15 calls, until a) and b) are clearly understood, you may never get to this question. When you do, listen to your customer’s articulation of the value proposition: 


  • Ask: “Now that you have heard from us, how would you describe what we’re doing?
  • Follow up: “Is our messaging right or do you think we should change how we describe ourselves?”


Bucket Customer Feedback 


Your team must agree to a process for collecting, sharing, and analyzing customer feedback and conversations. One easy tactic to stay on the same page is designating a leader who takes detailed notes in each engagement and makes them available to the whole team. 

We recommend a shared Google doc or tracking in Confluence/Salesforce that captures:


  • Date and time of call
  • Title and Name of all participants 
  • Questions asked
  • Detailed answers
  • Summarized thoughts on feedback
  • Next steps and action items with assigned responsibilities


Example of Confluence notes:


The goal is to get 40-50 meetings and group them by customer type. Customer type is defined by the “pain” you are hypothesizing they have.


Example 1: Customer Types for Shujinko


  1. Customers with consumer facing web applications that contain sensitive data. (Banking, Retail)
  2. Customers with GDPR compliance top of mind due to industry specific pressure (telcos, service providers, SaaS companies in healthcare).  
  3. Customers who worry that external dev teams (partners or contractors) have access to sensitive data through exposed APIs. (Flextronics, Nike)
  4. Customers who worry that by being on public cloud infrastructure and having services exposed beyond their firewalls, they risk having sensitive data exposed that might embarrass the company. 
  5. Company with DevOps processes who have an upcoming audit and are concerned with regulations or compliance failure. (Jenkins users in Finance or Retail). 

No items found.
1.3 AppDynamics Case Study
John Vrionis
John Vrionis
Co-Founder and Enterprise Partner

John Vrionis
John Vrionis
Co-Founder and Enterprise Partner

AppDynamics Case Study: Finding Your Focus

It was March 2017, and Jyoti Bansal was elated. Cisco Systems had officially acquired AppDynamics for $3.7 billion. As an application performance management (APM) company, AppDynamics developed software to monitor applications across an ever-increasingly complex network of code, third-party interfaces, virtualized servers, and various other inputs. Essentially, the company enabled other companies to monitor applications and resolve performance issues with unprecedented speed. Bansal founded the company in 2008 and served as its CEO until 2015. At the time of its acquisition, AppDynamics had raised more than $300M in funding from several different venture capital firms.


Bansal reflected on his company’s success, recalling how difficult it was to find product-market fit during those early months back in 2008. At the time, they had no customers, additional funding was nonexistent, and several of their assumed markets – finance and banking among them – were not taking any spending risks because of the recent financial crisis. Bansal needed to figure out the profile of his initial customer quickly or he’d be out of business. He undertook a systematic process that helped him (1) find initial customer prospects and (2) refine the messaging for his product so it would resonate. What he learned during this process would subsequently help him build one of the fastest growing startups in Silicon Valley history, and as of 2017, earn the highest ever price paid for a private software company.


The Initial Insight

In 2007, Bansal was working as a software engineer in Silicon Valley. He noticed that applications were becoming more and more distributed, with many more moving parts — and with various aspects getting virtualized into the cloud. He knew these changes meant that companies would have to rethink how they monitored and managed these new systems. Bansal explained:

"People were building apps everywhere. I knew the bigger enterprises had a large number of software applications and the environments that hosted these applications were becoming very complicated. I also knew there would be a lot of code involved with these new applications. And every time a company produces code, it needs to be monitored. The existing solutions would not be good enough for modern applications and the IT operations staff responsible for keeping them running."

Powered by that insight and a $5M investment, Bansal started searching for customer No. 1. He initially cast a wide net, knowing he needed data and feedback to drive prioritization and focus to his efforts:


"I started with LinkedIn cold InMails. I would say: “We are building the next generation of monitoring for Java applications. Would you be interested in talking to us and giving us feedback?” I chose Java for two reasons. First, most critical enterprise applications at that time that had revenue tied to them were written in Java, and second, by focusing on only one technology type, I narrowed down my universe of potential targets and made things simpler for my engineering roadmap. I would also try to target people who were responsible for IT operations, because my insight was that these people were actually the ones who felt the most pain when application performance decreased. Developers produced the code, but it was the IT operations people who scrambled when applications experienced downtime. I initially created a target list of a few hundred potential prospects by crawling LinkedIn, and then I would just send them each an InMail. If I sent out the same message — say, 100 times, and got a hit rate of 13 responses — I’d analyze where the responses were coming from. I started to see that I’d get more responses from companies of a particular size, or more responses from this kind of job title. That’s what informed my focus, and then I’d iterate on what was working and double down on that."

Bansal’s cold emails quickly evolved into a system of testing, something he referred to as his “go broad, then segment” strategy. Some key segments Bansal explored were:


  • Size of business — Small businesses vs. large enterprises
  • Specific roles — Operations engineers vs. network engineers vs. app engineers, etc.
  • Vertical Industries — Banking vs Retail vs Mfg vs Tech vs Media vs Insurance
  • Product Functionality — Visibility and Recommendations and Remediations

Bansal also emphasized the importance of asking for feedback in these cold emails rather than pushing for a sale immediately:

"At that earliest stage, you don’t really exist as a company. You have less than ten people on your team, you don’t have a product yet, and you think you should be trying to sell — but selling can put people in a defensive mindset. You have to engage them in the process, make them feel like they’re partnering with you in solving a problem that they care about. When I was sending emails trying to sell, the response rate was very low. When I sent emails asking for feedback from them as the expert, my response rate was much higher. It was dramatically different. But then I learned another important lesson."

Bansal soon discovered that simply getting responses wasn’t enough. He needed to be able to distinguish the customers who were desperately seeking a solution (because their feedback was the most valuable and they were the most likely to become an initial customer) from the responders who were casually curious about AppDynamics’ technology and the broader changes in the IT monitoring space. One way he accomplished this was by learning to ask initial responders the right set of questions.

Messaging Strategy

Once a prospect was willing to engage with him, Bansal had to go through another set of experiments with his messaging strategy and questions to determine who had a “hair on fire” problem, and who was just kicking tires and learning. He explained:

"I learned that people are too nice. I’d say “What do you think of this?” And they’d go: “Oh, that’s cool…” But when I started to ask people more of a closing question such as: “If we build this product, would you be willing to buy it?” I got a lot further. Other critical qualifying questions I learned to ask were “Why do you need to buy anything?” or “What’s pushing you to buy something now?” If they didn’t have a clear, compelling reason to buy, there was no way they were going to be a good use of our limited time. And there just weren’t enough hours in the day for us to be screwing around educating people. When we found a customer prospect who could articulate a compelling need to buy, we quickly learned to listen carefully to everything they said."

Bansal’s conversations with the right set of potential customers began to unravel some key assumptions he had made about the market and about what his customers might want in the product:


Assumption 1: Customers would be cloud-based in two to three years, and therefore would want a cloud monitoring solution to consider.


Bansal’s first pitch to customers was similar to the pitch that sold his investors. AppDynamics was building a cloud-based platform that would provide companies with the tools to monitor and fix errors within a customer’s network of applications. Bansal, like many others in the Valley at that point, assumed that the adoption of the cloud was going to be a very rapid process that would happen by 2011 at the latest. In contrast, the customers he spoke with saw a much longer timeline for their company transitions to the cloud. As a result, they didn’t see AppDynamics as a solution to their current needs. Bansal elaborated:

"Potential customers] kept saying: 'We like what you’re doing, and it all makes sense. I can use the cloud product you are describing in four to five years, but it doesn’t solve my problem now.' We needed to understand our customer’s current and most severe pain, partner closely with them, and solve it together. That was our ‘a-ha’ moment. If we were going to stay in business, I needed to sell today. There was no funding for being right eventually. I had to solve the problems our customers had now."

Assumption 2: The solution from AppDynamics could have a SaaS management component, not all ‘On Premise.’


Bansal and his team assumed their software had to operate as Software-as-a-Service (SaaS). SaaS business models were in vogue with venture investors at the time as market sentiment was shifting and falling in love with the predictability of recurring revenue. But investors are not customers. And being right too early is the same as being wrong. As Bansal engaged with prospects, he initially suggested customers would have to adjust their approach to be compatible with AppDynamics’ technology. Again, Bansal himself went and spoke directly to his customer prospects and heard things that were opposite to his initial beliefs about their needs:

"I would talk to these customers — some of them were these large banks. And they said, 'We need the entire solution to be on premise. All the data is stored on premise.' And I thought, maybe this is just a traditional company’s stance. Let me talk to some more modern companies. That was in 2009. We called four more companies and all of them said we have to do it on premise… Even Facebook, perhaps the most cutting-edge technology company on the planet at the time, was saying we have to use on premise software. If I hadn’t sat down and had the conversations myself, I’m not sure I would have believed it."


Assumption 3: Customers wanted detection and correction of application issues.


AppDynamics was designing a complex product that would manage the entire monitoring process — from identifying issues in the system to automatically solving them. But discussions with IT departments showed they didn’t need a “find it AND fix it” solution. Customers simply needed a way to visualize app performance so they would know as quickly as possible that there were issues and where those issues were occurring. Bansal and his engineers decided to cut the product roadmap back significantly and instead focus on simple application visibility.


In sum, three of the key hypotheses that Bansal had about what was best for customers turned out to be false. He was asking too much of his early prospects at first: operate in the cloud, change your system architecture to work with our platform, and use services that you find superfluous to your needs. The team was hearing a list of needs from its customers that was very different. Bansal explained how they adjusted for these needs:

"We changed our message based on early feedback. We were now going to our customers and saying “Bring your existing applications — cloud or no cloud — we will get you set up so you will have the best monitoring system you can find. We had been asking far too much at first. We were too far ahead of the customer’s current problems. It should have been about reducing the greatest friction they felt right now, not 2 or 3 or 4 years from now, and it had to be EASY for them to engage with us. Our message evolved to: 'You don’t have to make any changes in application architecture to use our software. You don’t have to make any changes in how you run IT to use our software. You don’t have to be in the cloud to use AppDynamics.' Our messaging evolved from talking about our product to selling by solving problems."

The First Five Customers


By the end of 2009, Bansal had secured his first 5 customers. YAP, a startup speech-to-text platform, was the company’s first customer. A week later, a division of Electronic Arts came on board. Then Netflix, then Priceline, then TiVo. In addition to these customers, Bansal’s initial conversations had established relationships with many more future customers. By casting a wide net, astutely assessing who had urgent need, and using the feedback to focus his efforts on 6 key dimensions — language, functionality, industries, size of company, and buyer/user personas — Bansal and his small team were able to build a product that had an immediate impact and value for their customers — and therefore, gain momentum, additional financing and ultimately massive success.


Exercise for finding your focus

2. Creating Your Product Roadmap

Prioritizing Your Product Roadmap Using the Four Lists

A crucial aspect of every successful startup’s journey is the ability to figure out what to prioritize when it comes to the product roadmap. Jyoti Bansal explains the process he uses to manage competing product requests to deliver a prioritized product roadmap that the engineering team produces on a schedule everyone can count on. He calls this “the Four Lists” process.

No items found.
2.1 The Four Lists
Jyoti Bansal
Jyoti Bansal
Co-Founder and Entrepreneur Partner

Jyoti Bansal
Jyoti Bansal
Co-Founder and Entrepreneur Partner

The Four Lists

List #1: What Sales Wants

The objective of List #1 is to build functionality that enables sales to win new customers. Sales is in the field on a daily basis fighting hand-to-hand combat. To product leaders, the message from sales is usually “if we just had X we could close this six figure deal!” Every competitive situation could turn in our favor if we only had certain features. At Harness, we call these “opportunity gaps.” If you want to stay competitive, you have to close those gaps. Some features you may want to build quickly to compete in that same deal cycle. Others you want to prioritize and build over the next three to six months, so you can keep winning against your competitors over the long term.

When you’re building out this list, ask yourself, “What do you need to win in this market?”

At Harness, we put in place a system for every opportunity in our pipeline and use a tool called Vivun (one of Unusual’s portfolio companies) to track these opportunity gaps. That gives us a dashboard, where we can quantify how “expensive” each opportunity gap is for us. As an example, consider a scenario where one particular opportunity gap came up in five different opportunities in the pipeline and each of those opportunities is worth $1 million of pipeline. Tracking gives us visibility into which product features would be just “nice-to-have” vs. massive costly gaps and allows us to prioritize based on dollar impact. The second trick we always use is the win-loss analysis. Every time we lose a deal to a competitor, we ask the prospective customer for the technical reasons that we lost or whether there were other reasons.

Gaps from the field help drive decisions on multiple dimensions. Revenue is a big one, but these gaps also help prioritize based on severity of need (i.e. will it cause the prospect not to buy vs. just cause friction in the deal?), the segment of the need (i.e. are we meeting the demands of our target market or are we being pulled in new verticals, geographies, company sizes?), and the competitive impact of the need (i.e. should we bolster areas against our top competition?). For example, your sales list will start to look different if you prioritize for moving into the enterprise, expanding internationally, or taking out a competitor.

Keeping track of product requests with List #1 from Sales is incredibly important. But that’s only one consideration — or list — of product features you (as the CTO, CEO, Founder, or Product Leader) should be thinking about.

List #2: Existing Customers

As product leaders, acquiring new customers and tracking the opportunity gaps are unfortunately not the only features that require consideration. If you focus only on closing gaps with competitors, you would neglect your existing customers and what they might need to continue to be successful. List #2 focuses on those users who already use your product and what’s important to them. Often times, existing customers discover they need certain features that they didn’t think of at all in the evaluation process. That’s because as they implement the product and go from being new users to power users over time, they start finding gaps in the product and begin asking for different capabilities. If you don’t listen and demonstrate an ability to offer those features that they need, those customers become dissatisfied and are less likely to renew. One example might be a customer that needs an integration that works with their internal user database, so they can pull in all their users. If you don’t build that feature, their adoption of your product would be limited to a small number of users and they would find decreasing value in your product over time.

The most important thing to consider for this list is being proactive and looking at customer usage and adoption as key metrics. That means examining the data on user engagement and seeing if a customer is becoming increasingly disengaged. You don’t want to wait until they churn, and ask them after the fact why they stopped using your product because by then, it’s too late. Instead, you want to collect product requests from existing customers before they churn and really seek to understand the reasons why customers are not using the product as much as they once did.

At Harness, for every account, one point person is responsible for putting together a list of three things that that customer needs to succeed and three things the customer needs to succeed even more. There are a number of tools that can help you track this. At Harness, we use Natero to keep track of these requests and score customers red, yellow, and green. (Red meaning they’re using our product very little, yellow meaning they’re using the product as expected, and green meaning they’re power users of the product.) We use this scoring system as the leading indicator to determine if our customers are happy and whether or not they’re likely to renew.

Once you have this information about customer satisfaction and requests, there may be several reference points you use to determine which product requests to take on. For instance, you might consider the ARR value of the account or whether the customer is in a strategic vertical you want to dominate. Or maybe it’s a small account, but the customer is really upset. (We refer to this as reading the “temperature” of the customer.) All these things might factor into your prioritization for new product features, but it’s hard to be formulaic about it. At the end of the day, you have to make a judgment call as product leaders.

Delighting customers is extremely important for any startup these days. Almost every successful company has really high metrics for customer retention. So that’s really the second list you have to start tracking and work towards.

List #3: Technical Debt

List #3 is maybe less exciting, but just as important as the other two lists we’ve described. As you quickly iterate and scale as a startup, you start accumulating what people call “ technical debt”, which you chose to take on as a tradeoff to build your product quickly. That technical debt may require you to re-architect a system at some point or slow down and work on test automation. These are not customer-facing features so they don’t impact the sales or customers, but cleaning up technical debt is something you have to keep doing to maintain and/or improve product quality.

List #3 is where your engineers track any accumulating technical debt. This may mean pausing or slowing down while you make improvements. Other times, it may mean redesigning a certain feature or writing more unit tests. Working through technical debt can also mean cleaning up bad code that has been assembled for some time. In terms of how you incorporate this into your sprints, companies dedicate every 3rd or 4th sprint to technical debt. Others prioritize a couple of items from this list in each sprint or allocate a fixed bandwidth in each sprint to technical debt. In any case, don’t go too long without working through your technical debt. If you don’t take care of your technical debt and keep building more features, eventually your product architecture gets so unwieldy that product quality suffers and it creates more serious problems in the long run. That’s why it’s critical to maintain this list and track it as you build out your product.

List #4: Innovations

The fourth and final list comes down to engineering toward future growth. It’s about building innovations that your customers are not yet asking for, but are designed to expand your total addressable market. In Unusual Academy, we teach our founders to find their beachhead market and expand from there. That means constantly thinking about your next innovation.

Ask yourself, “What do I need to do to expand my total addressable market?”

You can’t depend on existing customers to source these innovations because they often aren’t the ideal customers for new features or product lines. They care about what they bought, not about what you don’t have yet.

For example, our first product at AppDynamics was Application Performance Monitoring (APM) for Java applications and our customers who were Java programmers were asking for certain features. That was our beachhead market and we had to deliver against Lists 1 and 2 to stay competitive and delight customers in that space. But at the same time, we had a playbook for expanding into .NET applications and had to build out this capability, which most of those existing customers were not requesting as they didn’t have any .NET applications. Eventually, we built out our third product, End User Monitoring (EUM) and our fourth product, Synthetic Monitoring as we continued innovating and expanding our market.

Essentially, you have to approach this fourth list by working backwards from your revenue goals. Say you want your revenue to be $100 million ARR five years from now, $30 million ARR three years from now, etc. You have to have enough addressable market for your product, so you have to ask yourself what you need to do to keep expanding that market and hit those revenue numbers. That means you have to keep working on product expansion and innovations. Otherwise, your growth will slow down.

Balancing the Four Lists and Making Tradeoffs

Ultimately, whatever you decide to build comes down to those four buckets, which is what any product leader should be weighing when they make tradeoffs and define the priorities for the product roadmap. Some requests on the lists are lighter lift and can be done immediately, like a bug that needs to be fixed. Others take a little more effort and can be accomplished in the medium-term, like slight product enhancements. Then there are the major product requests, which require building out completely new services and features for new customer adoption. There’s variation in the magnitude of requests that the product leader has to balance.

Whether you’re the Head of Engineering, Product Manager, CTO, CPO, or startup CEO (and wearing all those hats at the same time), tracking these four lists can help your team align and ensure they are focusing on the right things. But how exactly do you get the balance right between these four lists? It’s a very tricky balance to pull off and you will have to adjust the balance as business priorities change. That means that at any point, you may want to spend more time on one list over the others. Say you’re losing a lot of leads to competitors and are struggling to get revenue going. That means you probably need to put a lot of your focus on List #1 and make the conscious, deliberate decision to slow down on technical debt or innovations and market expansion.

Or maybe you’ve sold a lot of product, but your customers are unhappy or you are having quality issues and churning customers because you’re not keeping up with the things that customers are asking for. Then you may want to slow down on the other lists and spend your energy on List #2 and List #3. In an ideal world, you want all four lists to be executed extremely well at the same time. But in the world of startups — where time and resources are constrained — finding the right balance right is key.

Structuring Planning Cycles

One thing we’ve found helpful at Harness is putting in place a system where every three months, our team goes through what we want to achieve and creates a planning cycle based on that three-month timeframe. Then, the team does an additional mini planning cycle every two weeks. It might not perfectly match up with the three month planning cycle, but it should align more or less with the broader three-month goals. Finally, we look at what we ship every day. By tracking those daily, two-week, and three-month sprints, we can see how we are making progress across the four lists.

Once a week, our CTO Rishi will meet with our Sales Engineering team to go over the first list and what is important to buyers. Then, he meets weekly with the Customer Success team to understand the second list and what’s important to our existing customers. He also meets weekly with engineering managers and asks for their lists of technical debt. Once he’s accumulated the lists from key stakeholders, he and I talk about the product expansion and innovation list every few months. Then, he balances what goes into a two-week sprint. It’s impossible to implement all ideas across all four lists all at once. Instead, you want to make sure that you are systematically going and adding the next 10% of your vision and going from 30% to 40% and 40% to 50% and 50% to 60% and so on until you reach your full product vision.

Unfortunately, unlike like a social product where millions of users can give you instant feedback, you don’t get immediate feedback on the enterprise side. It may take you months to get the data points to understand if one of your new product features is effective or not. That’s why there’s no exact science behind this process and why it becomes even more important to use the four lists to track high-level trends. If you’re systematic about capturing which requests recur most often across each list, you can get a sense pretty quickly for the most prominent high-impact product features, allowing you to prioritize your engineering and product roadmap.

Using the Four Lists process, you might wonder how the feedback loop works to update the company on the status of what product features were picked and how they were chosen. Sharing this framework/concept with everyone at the company gives everyone the important context that there are multiple requests to balance. At Harness, we have one wiki page in Confluence where everyone can see the lists and we mark progress in Salesforce using Vivun, so the sales team knows which new product features we’re working on and when each will be ready to roll out to customers.

Another thing to keep in mind is that as your company grows, this process changes as well. Usually, you start as a “single product” company and stay that way for the first few years or so. Once you have multiple products, you will likely need to have separate Four Lists for each of your products.

We recognize there are alternative methods to solving this challenge of product prioritization. We wanted to share this concept of “The Four Lists” approach because we’ve seen it work and we hope we can help every founder in some way. Enjoy the journey and good luck! Onward!

Exercise for defining your four lists

Menu
Join the discussion on