Anomaly detection that managers do not mute

Every analytics product ships alerts, and most of them get muted. The mechanism is predictable: a fortnight of notifications where four out of five turn out to be nothing, and the recipient learns that the notification carries no information. After that the channel is dead, and re-earning attention is much harder than getting it the first time.

So the design target is not sensitivity. It is precision high enough that a manager reads the next one.

Rule one: an absolute floor under every percentage

The single biggest source of nonsense alerts is a relative threshold with nothing under it. Fifty percent growth is enormous on a project with 400,000 monthly listeners and meaningless on a track that went from 4 plays to 6.

Every detector needs both a relative condition and an absolute one, and the absolute floor should be set at the level where a human would actually care.

The floors are per-metric and per-project-size, and they need reviewing as a project grows. A floor tuned for a developing act will spam a large one.

Rule two: baselines come from the project's own history

A benchmark taken from another artist is not a baseline, it is a comparison, and it should never trigger an alert. The reason is that music projects differ by an order of magnitude on nearly every metric, and a threshold that is right for one is wrong for the next.

What works is a trailing comparison against the same project: this week against the previous four, or today against the mean of the last 14 days with a minimum sample. That also handles seasonality within a project reasonably well, because a project's own recent history includes its own release cycle.

Two guards on top:

Require enough history. With fewer than about two weeks of readings, there is no baseline and no detector should fire, however dramatic the movement looks. The honest state for a new project is quiet.

Handle the cold start explicitly. The first snapshot of any source is a warm-up reading and can be partial. Treating it as a baseline generates a spectacular fake surge on day two, which is exactly the first impression you do not want.

Rule three: never mix data health with performance

These feel similar and are completely different messages.

"Streams in Germany are up 40 percent this week" is a performance alert for the manager.

"The Meta connection for this project stopped returning data four days ago" is an operational alert, and it needs to reach whoever can fix it, with the run history behind it.

Mixing them does two kinds of damage. The manager gets messages they cannot act on, and the genuine outage hides in a feed of growth notifications. We keep sync health in its own admin surface with per-job run status, duration and per-project failure detail, precisely so it never competes with a surge alert for attention.

The corollary: absence of data is never a performance anomaly. A number going flat because a token expired is not a decline, and a detector that cannot tell the difference will eventually tell a manager their audience collapsed on a day when nothing happened except an expired grant.

Rule four: bias towards silence

Given a choice between surfacing two real things and ten marginal ones, take the two. The maths is simple: a feed with 80 percent precision keeps getting read, and a feed with 30 percent precision gets muted and then the 30 percent is lost too.

Practically that means conservative detectors, and it means saying nothing on a quiet week rather than manufacturing a fact. A weekly digest that says the week was unremarkable is more valuable than one that promotes the largest random fluctuation to a headline.

What is worth detecting

Four families cover most of the value.

Surges against the project's own trailing baseline, with a floor. Streams, followers, a single market, short-form usage.

Milestones on round numbers that people genuinely care about. These are easy, popular and completely deterministic, so they cost nothing in credibility.

Stalls, which are the underrated one. A ticket pacing curve that has not moved for eight days on a show that is four weeks out is more actionable than most growth alerts, because there is still time to respond.

Composite events, where several signals move within a few days. The rarest and the most valuable, and worth a higher priority in the feed, because a single number moving is usually mechanical while five moving together rarely is.

Testing detectors honestly

Write them as pure functions over stored rows with no database access, then feed them synthetic series: a flat project, a project with one gap, a project with a weekly reporting cadence, a brand new project with three days of data, a project whose token expired mid-window. Each of those has a correct answer, and most of the correct answers are silence.

That last part is the test that catches the expensive bugs. It is easy to write a detector that finds real surges. It is harder to write one that stays quiet on a project where nothing is happening, and that is the behaviour that decides whether anyone reads the next alert.