Incident file · a reconstruction

    How RBAC saved us from bad artifacts

    One vibecoded dashboard, one direct warehouse connection, one share link, hour by hour. Any resemblance to your company is the point.

    Exhibit A: chain of custody, patient zero → breachscreenshots · decks · CSVexports: unrevokableunfurl previews · link cache ·search index: deletion never reaches thesejulie's laptop"vibe me a dashboard"11:47athe artifact48k rows + creds inside12:03p#cs-team → world"anyone with the link"12:15pdevtools → credsa curious contractor3:40pprodwarehouse02:11a: ghost queries
    Julie · CS ops

    Wanted a churn dashboard by lunch. Got one. Patient zero.

    svc_analytics

    A service account with one password since 2023, shared with dbt and three pipelines. Read-write.

    The Link

    "Anyone with the link can view." It always is. The vector.

    The agent

    Did exactly what it was asked, plus a writeback button nobody asked for. Helpful to a fault.

    The reconstruction · day one

    Twenty-three hours, beat by beat

    11:47a day 1

    The vibecode

    Julie opens Claude Code: "build me a churn dashboard for the QBR." She pastes the warehouse connection string from a two-year-old runbook. It is the only one she could find. The agent inlines postgres://svc_analytics:p4ssw0rd@prod… straight into the page and writes SELECT *, because nobody said not to.

    credential inlinedno review
    12:03p day 1

    The bundle

    It works. It looks finished. The page shows ten rows. The payload underneath carries 48,112: emails, plan, health score, and a free-text notes column. One of them reads "CEO furious. Threatened to walk. Do not escalate." The UI never shows it. The bundle carries it anyway.

    48k rows of PII baked in
    12:09p day 1

    The number

    The dashboard says churn is 9.8%. Finance says 6.5%. The difference: the vibecoded SQL joins subscriptions to accounts and double-counts multi-product customers. Nobody wrote this query and nobody reviews it. It is well formatted, so it reads as reviewed. In the corner, a label says "Last updated: just now." The label is hardcoded.

    fluent but wrongfake freshness
    12:15p day 1

    The share

    #cs-team: "made a quick churn view 🎉". Share settings: anyone with the link. That is the default, and defaults win. It looks finished, so it is treated as finished. Slack unfurls a preview. Someone pins it. A copy now lives in the link cache, out of reach of any future delete.

    treated as officialcache residue
    1:05p day 1

    The border crossing

    Reshared to #leadership. A VP forwards it to an outside advisor, "for context before Thursday." The advisor is in another jurisdiction. The customer data now is too. No one did anything wrong, exactly. Everyone did the normal thing, once.

    external exposurecross-border
    2:20p day 1

    The derivatives

    An AE screenshots a customer's row into a renewal deck. Ops pastes the link into the agency's Slack Connect channel. The agency also serves a competitor. A CSV export lands in a personal Downloads folder. None of these can be recalled. You can rotate a credential. You cannot rotate a screenshot.

    unrevokable copiescompetitor-adjacent
    2:47p day 1

    The second conversation

    The dashboard has a rendering bug on mobile. To debug it, Julie pastes the schema and fifty sample rows (real customers) into a different AI chat, on her personal account. The data now lives in a third place, under someone else's retention policy, in a history nobody audits and no rotation can reach.

    context leakage
    3:40p day 1

    The credential

    At the agency, a junior contractor wonders how the dashboard loads so fast. DevTools, Network tab, and there it is in cleartext: the connection string. He opens a SQL client "just to check something." The dashboard was never the product. The warehouse was.

    warehouse keys leakedevery table reachable
    4:55p day 1

    The meltdown

    The artifact refreshes every 30 seconds, per open tab. Sixty tabs across the company full-scan gold.customers in lockstep. Half the "viewers" are not even human: every unfurl bot and link-preview fetch runs the full query too. The queue melts. The CFO's board dashboard times out mid-meeting. The data team spends two hours blaming dbt.

    four-figure scan billprod degradedbots run queries too
    6:12p day 1

    The fork

    Someone saves a copy and "fixes" a number they were sure was wrong. v2 circulates. By evening, two meetings have quoted two different churn rates, each citing "the dashboard." There is no canonical version because there was never a source of truth. Only snapshots wearing product clothes.

    two official numbers
    7:38p day 1

    The writeback

    The agent scaffolded more than charts. There is a "mark as contacted" button nobody asked for, wired straight to prod. The credential was read-write, so why not. An AE tidying his book after dinner clicks it forty times. Forty UPDATEs land in the CRM-sync table. The nightly job syncs them onward to Salesforce.

    unreviewed write pathprod data mutated
    Day two
    02:11a day 2

    The ghost queries

    svc_analytics runs exploratory queries from an IP nobody recognizes. The audit log records every one of them, attributed to the service account. There are no humans in the log. There were never going to be. (The IP resolves, much later, to someone who found the link on the agency intern's public "useful links" Notion page. A search crawler had already indexed it.)

    no attributionpublicly indexed
    08:14a day 2

    The discovery

    The data lead spots the 2 AM scans and asks the only question that matters: who has access? The honest answer is arithmetic: "anyone with the link," multiplied by every reshare, forward, export, and cache. Unknowable.

    blast radius unknown
    09:00a day 2

    The rotation

    They rotate the credential. It works. It also breaks dbt, three production pipelines, and every copy of the dashboard, because svc_analytics was everyone's password. Now the whole company knows about the incident, because everything is down.

    pipelines downincident goes loud
    11:30a day 2

    The clock

    Legal asks one question: when did the 72-hour breach-notification window open? Best answer: yesterday, 12:15 PM, the moment the link left the room. The notes column is personal data. The clock has been running for twenty-three hours, and nobody was watching it.

    GDPR clock running
    The epilogue
    Day 30 epilogue

    The request

    A customer emails: "delete my data." Under GDPR that is their right. The honest inventory (every copy, cache, export, deck, fork, and personal chat history) is unknowable. The honest answer is "we can't be sure." Somewhere, nine browser tabs still have the dashboard open. The label in the corner still says "Last updated: just now."

    erasure impossible
    The damage, tallied
    • PII of 48,112 customers, including free-text notes, in an unknown number of copies across at least two jurisdictions
    • One live read-write warehouse credential, in cleartext, in every copy of the page
    • A QBR staffing decision made on a churn rate that was double-counted from the first query
    • A four-figure warehouse bill, partly run up by preview bots, and a board meeting that watched a dashboard time out
    • Two conflicting "official" churn rates, both still being quoted
    • Forty phantom “contacted” flags written to prod and synced onward to Salesforce
    • Real customer rows sitting in a personal AI chat history that no rotation or deletion can reach
    • An audit log full of one service account and zero humans
    • A GDPR notification clock that started before anyone knew there was an incident, and a right-to-erasure request that cannot be honestly honored
    The counterfactual

    What RBAC actually changes

    No raw credentials in artifacts, ever. The artifact talks to a governed layer with short-lived, viewer-scoped tokens. DevTools shows a token that expired an hour ago, scoped to one role.

    kills 3:40p the credential · 02:11a ghost queries

    The query runs as the viewer, not a god account. Julie sees CS accounts. The VP sees aggregates. The contractor sees a login wall. Same artifact, three different blast radii.

    kills 1:05p border crossing · 08:14a "who has access?" becomes answerable

    Column and row policies at the layer, not in the SQL. notes is masked, PII is aggregate-only, SELECT * physically can't over-fetch what the role can't see.

    kills 12:03p the bundle · 2:20p the derivatives carry nothing sensitive

    Sandboxed, budgeted queries. Every query is validated and cost-capped before it touches the warehouse. Sixty polling tabs get one cached result, not sixty full scans.

    kills 4:55p the meltdown

    One governed source instead of baked snapshots. The artifact renders live, versioned answers with provenance. There is a canonical number, and forks are visibly stale.

    kills 6:12p the fork · two meetings, one churn rate

    Per-human audit by construction. Every read is attributed to a person, a role, a timestamp. The 72-hour clock starts when the incident starts, because you can see it start. And "delete my data" gets an honest, complete answer.

    kills 11:30a the clock nobody was watching · day 30 the request

    Writes are proposals, not buttons. Mutating ops live behind the action layer: proposed by the artifact, approved by a human, executed by the API with its own audit line. No agent-scaffolded button writes to prod.

    kills 7:38p the writeback

    The AI tools get governed context, not raw rows. Debugging happens against masked samples via MCP: schema yes, real customers no. The chat history holds nothing leakable, on any account.

    kills 2:47p the second conversation · 12:09p certified metrics replace vibecoded joins

    Necessary, not sufficient

    RBAC narrows the blast radius. It does not validate the payload. A perfectly-permissioned dashboard can still show a hallucinated join, a stale metric, or a number that means different things in two meetings. Access control decides who can be wrong at you. Pair it with a context layer (source of truth, provenance, certified definitions) to work on whether it is wrong at all.

    Appendix: the full catalogue of ways this goes wrong

    At creation

    • Read-write creds inlined into client code
    • SELECT * over-fetch: full PII in the bundle
    • Runs as the service account: viewer inherits admin reach
    • Fluent-but-wrong SQL (double-counted joins), unreviewed
    • Snapshot hardcoded as "live" / fake freshness label
    • Agent scaffolds unasked write paths, wired to prod

    At sharing

    • "Anyone with the link" default
    • Unfurl previews, link caches, search indexing
    • Screenshots, decks, CSV exports: unrevokable
    • DevTools exposes the connection string
    • Forks diverge, no canonical version
    • Schema + real rows pasted into other AI chats
    • Link lands on public pages, gets crawled and indexed
    • Data crosses jurisdictions with one forward
    • Reaches an agency that also serves competitors

    Blast radius

    • Polling tabs and preview bots melt the queue
    • Audit log has no humans in it
    • Ghost queries from unknown IPs
    • Unreviewed writes mutate prod, sync onward to CRM
    • Rotation breaks everything sharing the account
    • GDPR clock starts silently at share time
    • Right-to-erasure requests become unanswerable
    • Decisions made on wrong numbers, twice-quoted
    • Customer notes seen by outside eyes
    Moral

    Nobody in this story was malicious. Everyone did the normal thing, once. Julie did not need better judgment. She needed a layer that made the normal thing safe.

    The same day, with Rig

    Every beat of the incident, replayed through a governed layer.

    In the storyThe raw artifactWith Rig
    CredentialsRead-write connection string in the page source, one DevTools tab awayShort-lived, viewer-scoped tokens. Nothing in the page worth stealing
    AccessEveryone with the link inherits the service account's full reachEvery query runs as the viewer. The contractor hits a login wall
    PII48k rows (emails, health scores, free-text notes) baked into the bundleMasked columns and aggregate-only policies enforced at the layer
    The numberA vibecoded join double-counts. 9.8% ships because it looks reviewedCertified metric definitions: churn means one thing, everywhere
    Freshness“Last updated: just now” is hardcoded. Forks diverge silentlyLive governed source with provenance. Forks are visibly stale
    Warehouse loadSixty tabs and every preview bot full-scan prod in lockstepSandboxed, cost-capped queries. Polling hits a cache, not the warehouse
    WritesAn agent-scaffolded button UPDATEs prod forty times after dinnerWrites are proposals: approved by a human, executed via the action layer
    Audit & GDPROne service account, zero humans. The 72-hour clock runs unseenPer-human trail from the first read. “Delete my data” gets a real answer

    Full head-to-head comparisons live on Rig vs. the alternatives.

    Rig is that layer.

    Viewer-scoped access, sandboxed queries, masked PII, and a per-human audit trail. What your team vibecodes ships governed.