← Back to Insights
Lester Leong

Lester Leong

·4 min read

North Star Metrics for AI Products: Why DAU Is the Wrong Choice

The DAU Inheritance Problem

Most AI product teams inherit their north star metric from the SaaS playbook without questioning whether the underlying logic transfers. In SaaS, DAU and WAU are reasonable proxies for value delivery because the products are deterministic. If a project manager logs into Jira, they are almost certainly doing work. If a salesperson opens their CRM, they are managing pipeline. Presence correlates with value. The metric works.

AI products break this correlation completely. A user can log in daily, send dozens of prompts, and extract zero useful output. They are engaged by every traditional measure and served by none of them. The inverse is equally true: a user who logs in twice a week but walks away with a deliverable that would have taken four hours to produce manually is deeply retained. Low frequency, high value. DAU would flag this user as at-risk. In reality, they are your best customer.

The industry is learning this distinction expensively. Teams optimize for DAU, ship features that increase session frequency, celebrate the dashboard moving up and to the right, and then watch retention collapse 60 days later. The metric was never measuring what they thought it was measuring.

Why Presence Does Not Equal Value

The core issue is that AI output quality is variable in a way traditional software output is not. When you run a formula in Excel, it returns the correct answer every time. When you generate a draft, a summary, or an analysis with an AI tool, the output might be excellent, mediocre, or unusable. The user's experience on any given session depends on factors the product team cannot fully control: prompt specificity, task complexity, domain nuance, model limitations.

This variability means that session count is almost entirely decoupled from value received. A user with 30 sessions last month might have gotten useful output in 4 of them. A user with 8 sessions might have gotten useful output in 7. Traditional engagement metrics treat the first user as 3.75x more valuable. Retention data will eventually reveal the opposite.

The problem compounds when teams optimize for the wrong signal. Increasing session frequency through notifications, reminders, or gamified streaks does not increase the probability that any given session delivers value. It increases the denominator without improving the numerator. The ratio of valuable sessions to total sessions actually declines, and the user's cumulative experience degrades even as the dashboard metrics improve.

What a Value-Based North Star Looks Like

The north star for an AI product should answer one question: did the user extract an outcome that mattered to them, and did they come back for it again?

This is structurally harder to measure than DAU. It requires the team to define what "value" means for their specific product. For an AI writing tool, value might mean the user exported or published the generated content. For an AI data analysis tool, value might mean the user shared the output with a colleague or used the finding in a presentation. For an AI coding assistant, value might mean the user accepted the suggested code into their working branch.

The common thread is that value is not measured by what the user asked the product to do. It is measured by what the user did with the product's output. The downstream action is the signal.

A starting framework: "Users who completed a valuable output and returned within 14 days." This metric captures three things DAU ignores. First, it filters for output completion rather than session initiation. Second, it requires the output to meet a value threshold (defined per product). Third, it demands return behavior as evidence that the value was real enough to motivate repeated use.

The Measurement Challenge

Teams default to DAU because it is easy and universally understood. Every analytics platform tracks it out of the box. Every board deck includes it. Every investor knows how to interpret it. Switching to a value-based north star requires work that most early-stage teams deprioritize: instrumenting downstream actions, defining value thresholds, building dashboards around less familiar metrics.

This is a real cost. But it is a one-time cost that compounds in accuracy, while DAU is a free metric that compounds in misdirection.

The practical path starts with identifying the three to five downstream actions that signal genuine value delivery. Instrument those actions. Track the percentage of sessions that produce at least one of them. Then measure how many users who hit a value event return within your product's natural usage interval. For most AI products, that interval is 7 to 14 days, not daily. Forcing a daily measurement cadence onto a product with a weekly value cycle is another inheritance from SaaS that does not transfer.

Once you have a value-based retention metric, validate it against actual churn data. If users who score high on your new metric churn at materially lower rates than users who score high on DAU, you have your answer. In every AI product we have examined, the gap is substantial.

The Cost of Optimizing the Wrong Metric

The most dangerous property of DAU is not that it is inaccurate. It is that it is plausible. The number goes up, the team feels good, and nobody questions the connection between the metric and the business outcome until the cohort data arrives weeks or months later. By then, the product roadmap has been shaped by a signal that was never predictive.

Optimizing for an easy metric that does not predict retention is worse than having no north star at all. No north star forces the team to reason from first principles about what drives their business. A misleading north star gives them false confidence while steering the product in the wrong direction.

Define what value means for your product. Measure whether users receive it. Track whether they return for it. That is your north star. DAU is not.

Want frameworks like this for your company?

I work with 3 to 4 AI-era companies at a time, building the analytics systems that turn data into decisions. If that sounds like what you need, let’s talk.

Get Your Free Diagnosis

Keep Reading