How to Build a Customer Health Score for B2B SaaS

Most Health Scores Are Astrology With a Dashboard

Walk into almost any B2B SaaS company and ask the Customer Success team how their health score works. You will hear some version of this: "It is 30% product usage, 20% support tickets, 20% NPS, 15% sponsor engagement, and 15% contract value." Ask one follow-up question - why those weights and not different ones - and the room goes quiet. The honest answer is that someone picked them in a meeting. The weights were not fit against anything. They were negotiated.

A health score whose weights are invented rather than fit is closer to astrology than analytics. It produces a number that feels rigorous, gets a color attached to it, and drives real decisions about where CS managers spend their time. But the number was never validated against the only outcome that matters: did the account actually renew, churn, or expand. I have watched this pattern play out in three different environments. Through Gradient Growth, where I have consulted for 20+ SMBs and startups, I am routinely handed a health score that nobody trusts and asked why it keeps missing churns. At a financial social media startup before its acquisition, our early account-health logic flagged the wrong accounts until we rebuilt it against outcomes. And on a GenAI squad at a major finance technology company, where the scale of the renewal data finally let me test which signals predict renewal and which ones are decorative. The premise behind all of it is that churn is an outcome you can [measure precisely](/insights/how-to-calculate-churn-rate) - so it is an outcome you can fit a score against.

The good news is that the fix is not exotic. You do not need a deep learning model or a data science org. You need to stop inventing weights and start learning them: collect the candidate signals, label each account by what actually happened, and fit a simple model so the weights are earned. This article is the practical version of that argument - which signals to collect, how to label and fit, a worked example where the arbitrary score and the fitted score disagree, and the caveats that will bite you if you skip them.

Account-Level, Not User-Level

The first thing to get right is the unit of analysis. A customer health score for B2B is an account-level measure. It predicts whether a contract renews, churns, or expands. That is different from a [product engagement score](/insights/product-engagement-score), which is a user-level measure of how deeply an individual person uses the product.

The distinction is not pedantic. In B2B, the buyer, the champion, the admin, and the daily users are often four different people. An account can have one ecstatic power user generating thousands of events while the economic buyer has not logged in since the kickoff call and is quietly evaluating a competitor. A user-level engagement score, aggregated naively, will look healthy. The account is not. Renewal is decided by people whose behavior barely shows up in usage logs.

So the health score rolls up to the account, and the inputs capture the account as an organism: how broadly the seats are used, whether the trajectory is improving or decaying, whether more than one person would fight to keep the tool, and whether the economic sponsor is still in the building. Build the user-level engagement score too - it is a valuable input - but do not confuse it for the health score.

The Signals That Actually Predict Renewal

Across engagements, the same candidate signals keep earning their place. Collect all of them, then let the fitting step decide their weights. Do not assume the ordering below; it varies by product and motion.

1. Usage breadth. The number of distinct core features or modules the account uses meaningfully - the account-level analog of the [feature adoption depth](/insights/feature-adoption-rate-guide) you would track per user. Single-feature accounts are fragile. An account on one module is an account that switches to a point solution the moment a cheaper one appears. 2. Usage depth. Volume of core actions relative to the account's size and plan. Depth without breadth is the power-user trap described below. 3. Usage trend. The direction of usage over the trailing 8 to 12 weeks, not the absolute level. This is the single most underweighted signal in homemade scores. An account at 60% of its peak usage and falling is in more danger than one at 40% and climbing. Trajectory beats level, and a score that reads only the current snapshot is blind to it. 4. Seat activation. The share of purchased seats that are actually active. An account that bought 50 seats and activated 12 has a renewal problem and an expansion ceiling, regardless of how active those 12 are. This is also where the next negotiation starts. 5. Support signal. Ticket volume and, more importantly, sentiment and severity. Raw volume is ambiguous - heavy ticketing can mean deep engagement or deep frustration. Sentiment and escalation patterns disambiguate it. 6. Executive sponsor engagement. Whether the economic buyer or champion is still engaged: attending business reviews, responding to outreach, present in the relationship. Sponsor departure is one of the highest-signal churn predictors in B2B, and it almost never appears in usage data. 7. Renewal proximity. Time to contract renewal. The same risk signal matters far more 60 days out than 300 days out; proximity is a multiplier on everything else. 8. Survey or NPS signal. Include it, but as a minor input. As I have argued at length in [why NPS is overrated](/insights/nps-overrated-alternatives), stated sentiment is a weak, biased, lagging predictor compared to behavior. It earns a small weight, not a starring role.

Notice what this list is built to catch: the multi-threaded versus single-threaded distinction, the trajectory, and the human relationship - none of which a naive usage rollup sees.

Stop Inventing Weights. Fit Them.

Here is the core method, and it is deliberately simple.

Step 1: Assemble the signal table. One row per account, one column per signal above, measured as of a point in time - for example, 90 days before each account's renewal date. Point-in-time discipline matters; you want the signals as they looked before the outcome, not contaminated by it.

Step 2: Label the outcomes. For each historical account, attach what actually happened at renewal: churned, renewed flat, or expanded. Start with the binary version - churned versus retained - because it is the cleanest signal and the easiest to get enough examples of.

Step 3: Fit a simple model. Run a logistic regression predicting churn from the signals. Logistic regression is the right starting point precisely because it is boring: the coefficients are interpretable, it does not need much data, and it tells you the direction and relative strength of each signal. You are not chasing the last point of accuracy. You are replacing guessed weights with learned ones.

Step 4: Read the coefficients as your weights. The fitted model hands you exactly what the meeting was trying to invent - the relative importance of each signal - except now it is grounded in what actually happened. Standardize the inputs first so the coefficients are comparable across signals on different scales.

Step 5: Bucket into red, yellow, green. Convert the model's predicted churn probability into three tiers with thresholds you choose deliberately, then attach a CS playbook to each tier. Red is not a number to admire; it is a trigger for a specific intervention.

A minimal version looks like this:

```python import pandas as pd from sklearn.linear_model import LogisticRegression from sklearn.preprocessing import StandardScaler

SIGNALS = [ "usage_breadth", "usage_depth", "usage_trend", "seat_activation", "support_sentiment", "sponsor_engaged", "days_to_renewal", "nps", ]

def fit_health_model(accounts: pd.DataFrame): """ accounts: one row per historical account, the SIGNALS columns measured as of 90 days pre-renewal, plus a 'churned' label (1/0). """ X = accounts[SIGNALS].fillna(accounts[SIGNALS].median()) y = accounts["churned"].astype(int)

scaler = StandardScaler() Xs = scaler.fit_transform(X)

model = LogisticRegression(max_iter=1000) model.fit(Xs, y)

# learned weights: sign is direction, magnitude is strength (inputs are standardized) coef = pd.Series(model.coef_[0], index=SIGNALS) weights = coef.reindex(coef.abs().sort_values(ascending=False).index) return model, scaler, weights

def score_account(model, scaler, row: pd.Series) -> dict: Xs = scaler.transform(row[SIGNALS].to_frame().T) p_churn = float(model.predict_proba(Xs)[0, 1]) if p_churn >= 0.50: tier = "red" elif p_churn >= 0.20: tier = "yellow" else: tier = "green" return {"churn_probability": round(p_churn, 3), "tier": tier} ```

That is the whole idea. The weights are no longer asserted; they are estimated from accounts that already lived through their renewal. When a stakeholder asks why usage trend outweighs NPS, the answer is no longer "we felt like it." It is "in our last 18 months of renewals, declining usage trend predicted churn roughly four times more strongly than a low NPS did." That sentence is the entire difference between analytics and astrology.

A Worked Example: When the Two Scores Disagree

Consider a real-shaped example with illustrative numbers. An account - call it a mid-market account on a 50-seat contract - looks like this 90 days before renewal:

- Logins are high. One power user logs in daily and generates more events than almost any account in the book. - Usage breadth is narrow: that one user touches 2 of 9 core modules. - Usage trend is negative: account-wide activity is down 35% from its peak six months ago. - Seat activation is 12 of 50 seats active. - The executive sponsor who signed the deal left the company two months ago. No new sponsor has been identified. - NPS from the power user is a 9.

The arbitrary weighted-average score loves this account. Login frequency is high, so the usage component is strong. NPS is a 9, so the sentiment component is maxed. Support tickets are low. The homemade "30% usage + 20% support + 20% NPS + ..." formula returns something like 82 out of 100. Green. The CS manager moves on to accounts that look worse on the dashboard.

The fitted model flags it red. The signals it learned to weight most heavily - usage breadth, usage trend, seat activation, and sponsor engagement - are all flashing. The account is single-threaded (one user, two modules), the trajectory is decaying, only a quarter of the seats are activated, and the economic sponsor is gone. The high login count from one enthusiast is exactly the noise the arbitrary score mistook for signal. The model's predicted churn probability comes out around 0.71. Red, with a playbook: engage a new sponsor, run a value review against the 38 dormant seats, and escalate before the renewal window closes.

This account churned. The weighted-average score never had a chance to catch it, because login volume was carrying a component that should have been carrying breadth and trajectory. The fitted model caught it because the data, not a meeting, decided what mattered. This is the same failure mode I described in [why NPS is overrated](/insights/nps-overrated-alternatives): a metric that captures what one happy person says while missing what the account is actually doing.

From Score to Action: The Tiered Playbook

A score that does not change behavior is a vanity metric. The point of the buckets is that each tier maps to a different CS motion.

- Green. Low predicted churn. The motion is expansion, not defense. Green accounts with high seat activation and rising usage trend are your expansion pipeline, not just your safe renewals. Route them to the expansion play. - Yellow. Elevated but not acute. The motion is diagnosis. Yellow exists to catch accounts early, while there is still time to intervene before renewal proximity turns a manageable risk into a lost deal. Pull the specific signals that drove the yellow rating and address them. - Red. High predicted churn. The motion is an escalation with a named owner and a deadline tied to the renewal date. Red without a deadline is just a worried feeling.

The tiers are only useful if the thresholds are deliberate and the playbooks specific. A red tier that fires on 40% of the book is a miscalibrated threshold, not a crisis. Tune the cutoffs so the red list is short enough that a CS team can actually work it.

A worked sizing example: suppose the model assigns red to 15 of a 200-account book, with an average predicted churn probability of 0.68. That is roughly 10 accounts you expect to lose if you do nothing. If those accounts average 40,000 dollars in annual contract value, the red tier represents about 400,000 dollars of at-risk revenue concentrated in a list a single CS lead can personally work. Recovering even half of it beats spreading attention evenly across 200 accounts that mostly do not need it. That triage - putting effort where the predicted risk and the revenue both concentrate - is the entire economic case for the score.

Caveats That Will Bite You

The method is simple, which makes it easy to deploy badly. Four failure modes to respect.

Small sample. Logistic regression with eight signals needs a reasonable number of churn events to estimate stable coefficients. If you have 40 accounts and 5 churns, you do not have a model; you have an overfit anecdote. Below roughly 50 to 100 outcome events, reduce the signal count, lean on the most interpretable few, and treat the weights as directional rather than precise. A wildly overfit score is confidently wrong, which is worse than the arbitrary one. When the sample is thin, a simpler scorecard built from a handful of high-signal inputs beats a fragile regression.

Leakage. This is the one that quietly inflates every model. If a signal is measured at or after the outcome, it leaks. Cancellation tickets, the sponsor's offboarding email, the seat count after a downsell - these are consequences of churn, not predictors of it. Measure every signal strictly as of the point-in-time cutoff and confirm nothing in the table could only be known later. A model that looks suspiciously accurate is usually leaking, not brilliant.

Recalibration. Your product, your customers, and your motion change. Weights fit on last year's renewals decay. Refit on a rolling basis - quarterly is a reasonable default - and watch whether the coefficient ordering is stable. A signal whose weight swings wildly between refits is telling you either the world changed or your sample is too small to trust.

Do not over-engineer it. Logistic regression is the destination for most teams, not a stepping stone to something fancier. The value is in fitting weights against outcomes at all, not in the sophistication of the fit. A boring, interpretable, well-calibrated logistic model that CS managers trust and act on beats a gradient-boosted black box nobody believes. If the team cannot explain why an account is red, they will not work the red list, and an unworked score is worth nothing.

The One Sentence Version

A customer health score is only worth building if its weights are earned. Collect the account-level signals, label every historical account by what happened at renewal, fit a simple model so the data sets the weights, bucket the output into tiers with real playbooks, and refit as the world moves. Do that, and the score stops being a number people nod at in a meeting and becomes a triage system that puts your CS team in front of the accounts actually about to leave. Skip the fitting step, and no amount of dashboard polish will save you - you will just have astrology with better colors.

---

I help CS and data teams replace invented health scores with account-level models fit to real renewal outcomes. [lester@gradientgrowth.com](mailto:lester@gradientgrowth.com)