Context quality: read your score, then know what to fix

What the score measures

Open Context Studio and click Context Metrics and you get one score out of 100, with a letter grade. It rolls up four measurements, each a plain count over your own context data. No LLM reads your tables and hands out marks: every number is something you could recompute yourself with a COUNT(*).

A freshly connected warehouse: 44 tables, 5 tagged to an area, 2 referenced by a metric. Nothing is wrong with the data, it's just untouched context.

A score under 50 almost always means the same thing: a warehouse Rig can query, but can't yet explain. That's a curation gap, not a data problem, and it's exactly what the rest of this page walks through fixing.

Quality by area

Below the top-line score, quality by area breaks the same four measurements out per business area, so you can see which topic is ready to trust and which is still raw. Click a row to open that area's tables directly in Studio.

Each area gets its own documentation rate, query volume, eval score and governed-answer rate. The untagged row is everything nobody has sorted yet.

Work the worklist

Scroll down and the same counts turn into a queue: the highest-traffic tables missing an area, a metric, or a description, ranked by how many times people actually queried them in the last 30 days. On a fresh warehouse, one order-items table can absorb over a hundred queries a month while still missing a single metric definition, more than any other gap on the list.

Every row is a link straight into Studio. Nothing on this page writes anything for you. Rig does fill some gaps automatically in the background, joins, mainly, but the worklist is a to-do list for a person on purpose: gaps worth a human decision, not gaps Rig should silently paper over.

Ranked by real 30-day query volume: fct_order_items alone absorbed 146 queries with no metric attached.

Grade real answers with Eval Lab

Coverage and documentation describe the shape of your context. They don't tell you whether Rig actually answers questions well. That's what the Eval Lab is for, and it works differently from the score above: instead of counting rows, it re-runs real questions through the real chat engine and grades what comes back.

It starts from what people ask via MCP, the actual questions people and their AI tools sent over the last 90 days, ranked by frequency. Add a handful to the eval set, click Run benchmark, and each one gets replayed and graded good, flagged, or no answer.

Real questions, mined from real MCP traffic, replayed through the chat engine and graded on this run's date.

That grading is a separate model call from the deterministic score above, and it isn't a pass or fail. It reads the generated SQL and the answer, and marks a question down for a wrong time grain or a plausible-looking but wrong number, not just whether the SQL ran. Good, flagged and no answer are three different outcomes because "the query executed" and "the answer was right" are two different questions.

The run above is the same jaffleshop warehouse from the scorecard earlier, score 28, and the eval agrees with it: one flagged answer, two no answers, nothing good. That's not a broken eval, it's the eval correctly reporting a context layer that hasn't been curated yet. Below a score of about 60, expect the Eval Lab to mostly confirm what the worklist already told you.

If you want more confidence on one question, attach a canonical SQL answer to it. Someone other than whoever wrote that SQL has to approve it before the eval set will grade against it. That four-eyes rule is deliberate: nobody gets to mark their own SQL correct.

A simple weekly workflow

1. Check the score once a week. A ten-point drop usually means new tables landed with no context yet, an opportunity, not a regression.
2. Fix from the worklist, not from a hunch. It's already ranked by real query volume, so the top row is where an hour of work pays back fastest.
3. Start an eval set once the score clears about 60. Below that, most bad answers trace back to missing context, and an eval run will just tell you what the worklist already said.
4. Read flagged answers, don't just count them. A flagged answer with a wrong time grain and one with a wrong join need different fixes, even though both count the same in the summary tile.