Way back in 2017, The Economist crowned data “the new oil” — and they weren’t wrong.
Tracking data correctly can give your startup a competitive edge, giving you insight into what your customers actually want, how they’re interacting with your product and if they’re about to take their business elsewhere.
Yet many ‘data-driven’ startups still make fundamental mistakes when tracking analytics, especially when it comes to their product.
We spoke to data experts and startups about how they got started with product analytics — and how you can learn from their mistakes.
1. Don’t confuse product analytics with marketing analytics
Marketing analytics shows you how users find your product. Product analytics shows you how users interact with it.
For startups, user growth is obviously extremely important, so the temptation to use market analytics to track everything is huge.
According to Grégoire le Hardy, head of analytics at digital advertising and business intelligence firm Semetis, this is the most common mistake startups make when starting to track analytics — and can lead to a host of problems.
“Google Analytics, for example, is very good for understanding the acquisition part of your funnel, but it lacks a bit of power and efficiency when you want to get to know your users better and understand their behaviour. We often see people trying to use Google Analytics for product analytics purposes — but you simply can’t.”
Google Analytics can tell you where a user found your product in the first place. But, if you want to stretch this kind of software to find out how they’ve interacted with it since, for example, or if your onboarding process is too long, you’ll struggle to get specific and actionable data.
We often see people trying to use Google Analytics for product analytics purposes — but you simply can’t.
One example is Dutch edtech startup StuDocu, a platform for students to collectively revise and share resources. StuDocu CTO Lucas van den Houten says the company tried using marketing analytics software before they switched to Mixpanel, a company that specialises in product analytics and allowed them access to more ‘granular’ data.
This more granular data allowed StuDocu to conduct A/B testing to evaluate their product and UX decisions, such as how students can ‘follow’ different study resources or authors. Van den Houten says their testing gave them a 50% improvement on some of their topline metrics, like the number of daily active users.
2. Don’t underestimate the importance of qualitative data
Collecting lots of data may seem like a sure-fire way of ensuring you have actionable insights, but the startups we spoke to stressed that quantitative data can’t tell you everything.
Qualitative data, which provides qualities or characteristics, can provide detail that numbers can’t. Simply put, while quantitative data can tell you that a user stopped using your product, qualitative data can give you more insight as to why. Both are essential to growing startups, and should be used to complement one another.
Swedish startup EasyPark aims to make EV parking easier and cities more livable by both automating parking payments and allowing drivers to virtually find and reserve parking spots.
"[Startups often] rely on quantitative data — even when there isn't enough of it,” Johan Ocklind, growth hacker at EasyPark told Sifted. “Instead, get qualitative data by interviewing users, sending out surveys and recording [in-app] sessions."
Another example is American dating app and Mixpanel client Hinge, which analysed quantitative data to find out that lots of swiping on its dating app didn’t actually lead to many dates. Just one in 500 swipes led to an exchange in phone numbers.
Without the why, the data is meaningless because it isn’t actionable.
This data, combined with qualitative data from focus groups and surveys, informed its decision to shift from users liking whole profiles to liking specific content blocks on other profiles — a feature which now sets Hinge apart in the dating app market.
In this case, the qualitative data showed them why their app wasn’t leading to dates: “Without the why, the data is meaningless because it isn’t actionable,” Tim MacGougan, chief product officer at Hinge said.
Qualitative and quantitative data can also be contradictory, as quantitative data on its own lacks context, and qualitative data based on human feedback can be biased. When your data is contradictory, this simply signals a need for deeper exploration and experimentation.
3. Think about what you want to track — and then prioritise it
It’s tempting for startups to collect as much data — both qualitative and quantitative — as possible. Joy Allan, Mixpanel’s manager of account management, told Sifted this approach can be damaging to startup growth.
“One of the errors that people make is that they’ll track the whole kitchen sink,” she says. “They’ll just track absolutely everything. It can be expensive and it can result in ‘analysis paralysis’ — which is when you have too much data to know where to look for what’s important or not.”
Keep it simple. Keep it focused.
Which metrics are most important will vary greatly between startups. A recent report from Mixpanel found that while most prioritised engagement, retention, and conversion, different product metrics were more important for different sectors. For example, engagement was most important to media startup products, while retention was most important for B2B products.
Having clearly defined goals before collecting data can prevent this, according to Ocklind. “Digging into data without knowing why is a common error. Use the data to answer specific questions you have. For example, data can be used to answer: ‘What’s the most common exit screen on my app?’”
Diego Rodriguez, cofounder and CEO at Spanish travel app Passporter agreed, and said startups who track too many metrics often think they’ll “extract more information to make better decisions. But this sometimes makes you lose focus on the main metrics that drive real growth.”
“Keep it simple. Keep it focused,” Allan says.
Want to dig into product analytics, like optimising onboarding flows, re-engaging inactive user groups and identifying behaviours of power users? Watch Mixpanel’s demo here.
