North Signal

Methodology

Our numbers are only worth what our method is. This page describes it fully — the same standard we'd want from anyone selling us measurement.

1 · Prompt taxonomy

What we ask

Each brand gets a segmented question set: branded (your name), category("best X for Y"), competitor("alternatives to Z"), comparison("X vs Y"), and problem("how do I…"). Seeds come from live web research of your market; you add the real phrasings from your sales calls, tickets, and Search Console. Demand tiers are attached with their derivation stated — public signals or your own impressions, never invented volumes.

2 · Repeated measurement

Why one run is never enough

AI engines are non-deterministic: the same prompt yields different answers across runs. We run every prompt N times per engine (default 3, configurable to 10) in fresh sessions, store every raw response immutably, and never compare single snapshots. Cross-engine agreement is low (~10–15% source overlap in published research), so every engine is measured and reported separately.

3 · Extraction

How an answer becomes data

Each response is parsed for: brand and competitor mentions (with position-adjusted prominence); whether the brand is recommended (a stricter bar than mentioned); citations, with each source classified (own domain / competitor / listicle / review / community / video / encyclopedia / editorial); and sentiment. A language-model judge refines recommendation and sentiment calls on live engines; a deterministic rules pipeline is always the fallback, so extraction never silently depends on a model.

4 · Statistics

Every number carries its uncertainty

Proportions (answer share, recommendation rate, citation rate) are reported with 95% Wilson score intervals and the run count. Continuous measures (prominence, sentiment) carry mean ± normal-approximation intervals. Dashboards display the interval, a noise-floor readout, and the run count needed for a target precision. There is no code path that emits a bare point estimate — by design.

5 · Diagnosis & prescriptions

Graded, never mechanism-free

Each engine × segment cell is classified by where it leaks: not mentioned → mentioned-but-not-cited → cited-but-not-recommended → off-site gap. Fixes are drawn from a lever library where every entry carries its mechanism, the strength of evidence behind it (from controlled, peer-reviewed research down to directional industry data), effort, and over-optimization risk. We also publish the non-levers — tactics with measured null or negative effects — and decline to prescribe them.

6 · Controlled experiments

The proof step

A fix under test gets a declared treatment prompt set and a held-out control. After you ship, both are re-measured; pre/post deltas are tested with two-proportion z-tests. The verdict is worked only if the treatment moved significantly beyond the control (which absorbs model updates and drift). Otherwise the verdict is noise or kill — reported as plainly as a win.

7 · Known limits

Stated out loud

  • • Personalization means your buyers' answers can differ from measured answers.
  • • Model updates can reset results overnight; metrics are regime-dependent and trends matter more than levels.
  • • Sentiment flips far more often than mentions — trust it only at larger sample sizes.
  • • Revenue attribution for AI visibility is essentially unsolved industry-wide; we don't claim it.

See it applied to your site