5 key questions to ask when you’re ready to invest in analytics
“Without analytics, product development would simply be developing an idea and providing a final outcome that hasn’t considered its consumer expectations. It’s unlikely that the product is going to be successful in meeting consumer expectations when you don’t know what they are to begin with. The statistics don’t lie either. With 21% of products failing to meet customers’ needs, analytics are now more important than ever in contributing to a product’s success.”
––SmartData Collective, “The Growing Role of Analytics in Product Development”
These days you’re likely thinking a lot about growth––where to double down to retain and expand value through your product as quickly as possible. Especially in a tumultuous market, making efficient use of funds and driving scalable growth.
To answer these (potentially) million dollar questions, you may be considering bringing on a product analyst, data scientist, and custom tooling that can deliver the insights you need to make good data-driven decisions. Although there are many business strategies for driving growth, product-led growth (PLG) has taken center stage as a proven way to drive and deliver outsized customer value.
But is the time-consuming and expensive process of building an in-house team worth it? Mengying Li, a former data science manager at Facebook and a data scientist at Microsoft writing for The Review in “The Startup Founder’s Guide to Hiring a Data Scientist,” shares words of caution. “At a startup where time and resources are strapped, diving into data science too quickly can distract from more pressing challenges facing the business on the path to scale.” On the other hand, customer behavior and product data provide the insights you need to make game-changing decisions.
Fuzy aims to help companies at this crossroads take a shortcut to powerful insights. This article will help you evaluate if adding headcount and custom tooling or implementing Product Science as a Service will get you to the answers you’re looking for more quickly and cost-effectively.
Where should you invest to become more data-driven in product development?
For some companies, it makes sense to build an internal department dedicated to product data analysis. As Fuzy’s VP of engineering Treit Le explains, “It comes down to exactly what you want to get out of that business function relative to the cost. Product science is the intersection of domain knowledge, statistics, and computation. Building that team and instrumenting the data and tooling to drive it is extremely expensive. But if you have questions to answer that are very specific to your domain, then that can be a worthwhile financial investment. The goal of Fuzy is to productize this process, making deep insights possible for companies that aren’t willing or don’t need to make a significant financial and time investment: Fuzy handles the statistics and computation that only a handful of experts can perform––and lets the product owner apply their domain knowledge to answer a very wide range of questions.”
Before you go all out in either direction, here’s a checklist of considerations for evaluating if PSaaS or an in-house data science department would better suit your needs.
5 questions for determining if you need a product scientist or Fuzy
What are the core competencies of your company?
Think about whether both applied data science and building internal tooling are core competencies. Do you have an unfair advantage in attracting ML and data science talent? If you can’t check the box on both of these, embarking on a build journey is risky at best. Your ability to maintain and evolve the technology to ensure sustained adoption may not be prioritized and value will rapidly degrade.
Ethan Handel, a product expert at Indeed, discusses a special case when it could be beneficial to DIY data science. “Building your own solution makes the most sense only in highly specialized circumstances where there's no solution that can solve a problem in the way you need it to be solved. From a data perspective, perhaps you need this kind of flexibility, otherwise you can't get the answers you want. But for at least 80% of companies and questions, a custom build isn’t necessary.” (Read our full interview with Ethan)
Does your company culture put stock in data?
Chief Product Officer at Cars.com Matt Crawford gave us his perspective on the cultural prerequisites for becoming data-driven: “You have to have a process-honoring culture. Before investing in any sort of product science capacity, find out if using data to inform decisions is even a topic that’s discussed within the company. In many places, it’s not.” (Read the full interview with Matt here)
Ethan adds from his experience: “Data-based thinking is so deep in the culture at Indeed; and that comes from leadership. There are cultural and explicit expectations for how teams use data, but it’s all about how you incentivize the right folks in your company to use data when and how they should.”
A relentless focus on solving business problems and openness to listening to data are needed regardless of which approach you choose to take. However, there can be cultural implications in implementing PSaaS versus a product science team. First, data may be more likely to be held under lock and key if it is siloed in a single department where only the experts have access. PSaaS encourages data transparency and autonomy, giving teams access to data, documentation, and insights on a self-serve basis. If your goal is for more people in your company to view insights and better understand your users, product cycle, and outcomes, PSaaS may be the right choice for you.
What is your budget for investing in product science?
Your ability and commitment to make significant upfront investments in headcount and infrastructure will largely inform whether you build an in-house team and tool or implement PSaaS.
As Tom Wilbur, a veteran product leader and data scientist puts it, “A third-party tool would have a very different cost structure. People and expertise are incredibly expensive.” (Just one data scientist can cost well over $160k per year, so you’ll need to balance the cost of building a team against the ROI you expect to gain from their work.) “It's a very expensive cost structure to build out teams of fully loaded, very experienced folks,” continues Tom. “Instead, if you can bring on a tool that the product folks on your team can use, then I think that's going to be a much stronger cost model longer term. You also gain a ton of expertise by ‘renting’ a team that's specifically focused on this problem of extracting insights from data.”
As Tom hinted, it’s not just about the money, it’s about finding practitioners with the right skills and experience. And even then, there’s no guarantee of success, according to Ethan Handel. “Throwing people at a problem can work, but it doesn't scale; and in many cases it works up to a point and then it doesn’t. If your goals are collaboration and transparency, you create more murkiness over time with more people. You have to find the sweet spot on the continuum of automation and people. What do we need to get in front of the right people? And what stories do we care most about from our analytics tools?”
PSaaS lies in the sweet spot of cost-effective and powerful insight, requiring a much smaller investment than developing an in-house product science team.
What is your time-to-value equation?
Searching for competitive candidates, hiring and onboarding, building data pipelines and tooling, and optimizing team dynamics take significant time. It could be many months before you reap the benefits of your investment, even without unforeseen delays. You can weigh this against the minimal integration and training time for implementing PSaaS.
PSaaS has a quicker time to value since it sits on top of your existing tool stack and uses machine learning to automatically begin extracting those stories hiding within the raw data. Without PSaaS, it takes time and a ton of trial and error to get a model to a point of usability, even with a data scientist. But Fuzy has already gone through that whole process since building and maintaining the model is our job. We've spent hours learning, building, and rebuilding against datasets and models similar to each customer by the time they start engaging with us.
Additionally, you can connect your tools via API and be off to the races answering your burning questions within days. If you migrate source tools, your historical trends live on, you reconnect your new source system, and get back to work. If you have a very unique product with a complex monetization model, you may lose some customization that you could achieve by building in-house.
Where do you sit on the business maturity curve and how do you plan to grow?
When you are early stage, pre-product market fit, and in the throws of building 0-1 features, your first step is simply to begin collecting data on user behavior and your customers. You're heavily leaning on qualitative insights and user interviews to assess fit and usability at this stage. However, your quantitative data can still be useful in measuring feature-level adoption, reporting, and monitoring trends.
As those of you further along in product maturity and market adoption consider your options, Ethan Handel offers some helpful advice. “A particularly useful question would be, ‘Where do you expect to be in five years?’ As you're growing and solidifying what your business looks like, you should start to shift to investing in data pipelines and analytics tools so you don’t have churn in tooling later, which is just painful.” This is great perspective as you’re weighing whether or not you can invest in the continual improvement of custom tools. PSaaS allows you to outsource the evolution of tooling, helping you stay ahead of the analytics curve without any effort on your part. In other words, Fuzy provides the infrastructure, platform, and expertise so you only need to care about your product needs.
If after asking these questions you decide that Fuzy makes more sense than investing in a data science team, find out why Fuzy could be your product team’s new best friend.