Keep your data fresh and add new data

Two kinds of data

• Sources Rig syncs for you, such as your CRM or support desk, are copied into the warehouse on a schedule. They are only as fresh as their last sync, and the rest of this page is mostly about them.
• Tables already in your own warehouse are queried where they are, so an answer reflects whatever your own pipelines last loaded. Rig has nothing to sync for them.

Everything on this page about synced sources happens in the Sources & actions step under Connections → Connectors: the source list, the frequency menus, the statuses, Sync, Full resync and the file upload area. They appear on workspaces where you manage your own connections, for owners and for roles with the Manage connections permission. Everyone else sees Data ingestion for this workspace is managed by Rig. in their place, so ask an owner or Rig support about a source.

Sync schedules

Every source is listed in Sources & actions, and each row has a frequency menu. A new source starts on Nightly.

• Nightly syncs once a day between midnight and 6am UTC. Rig spreads sources across that window, so yours will not all start at midnight.
• Hourly syncs once an hour.
• Manual turns automatic syncs off. The source only syncs when someone clicks Sync.
• Custom… opens a dialog where you choose Monthly on the Nth UK working day (Monday to Friday, excluding England and Wales bank holidays) or a Cron expression, and set the Timezone. Once a source has a custom schedule, the menu shows Custom, and Edit custom… reopens the dialog.

Only the person who connected a source, or a workspace owner, can change its schedule. For anyone else the menu is greyed out.

The full list of connected sources at the top of the page is split into Scheduled and Manual, and a source you switch to Manual moves to the Manual list. When you open one connector's card, its connections are listed there without that split.

Sync now

Click Sync on a source's row to start a sync straight away. While it runs the button reads Cancel. When you have more than one source, Sync all above the list starts every source that is not already running, Manual ones included.

Read a source's status

Each row shows a status next to the time of its last run.

• running, or backfilling full history while older history loads: a sync is in progress.
• Synced: the last sync finished. If an amber note appears under it, part of the data did not load, and the note says what to do.
• Synced · partial: it finished, but not every table loaded.
• partial load · history still owed or full history pending: recent data has loaded and can be queried, and the older history carries on from where it stopped on the next sync.
• failed: for most errors the row explains the problem in plain words and says what to do, and Show details reveals the raw error. When Rig has no explanation for an error, the row shows the raw error with an address to email for help. For an expired or wrong credential there is an Update credentials button, if the source has credential fields to change and you connected it or are an owner. A source that connects through a sign-in popup with nothing to edit does not show it.
• interrupted — retry: the sync stopped part way. It runs again at its next scheduled time; on a Manual source, click Sync. When the sync stopped for one of the reasons below, this label does not appear and the row explains it in grey instead, with nothing for you to do:
  • a deploy on Rig's side: Interrupted by a deploy, then A deploy stopped this run part-way. It retries itself on the next sync.
  • the run reached its time limit: Stopped at its time limit; resumes next run. The progress it made is saved, and a large first load can take several runs.
  • the worker running it stopped: Stopped before it finished; resumes next run. The progress it made is saved here too.
  If you started the sync on the Connections page and still have it open, a notice titled with the source's name and sync interrupted appears. Whatever the reason, it reads A deploy/restart interrupted it — click Sync to retry.Clicking Sync carries on straight away; otherwise the next sync does.
• cancelled: someone stopped it.
• not yet synced: the source is connected but has never run. Click Sync.

Full history and full resync

The first sync of a source pulls its history, which can take several runs on a large account. Each later sync tops it up, and an unfinished history resumes by itself on the next sync, scheduled or manual.

If older data is missing, an owner can re-pull everything: click the icon on the source's row whose tooltip starts Full resync, then Start full resync. Use it sparingly: it can take hours, it counts against the source's API limits, and the source's tables are rebuilt from scratch, so they are incomplete until it finishes. Rows the source no longer returns do not come back. Cancelling part-way rolls nothing back: tables already reset stay reset, with only the rows loaded so far. If a full resync is interrupted, the next sync picks it up again: from its last checkpoint where the source keeps one, and from the beginning where it does not.

Upload files

To add a file, open the CSV / Parquet (S3) tile under Connections → Connectors and click Connect, or Add another connection if you already have one. The label field at the top of the form names a bucket connection, so leave it empty; it reads Label (optional). Once you have a bucket connection it reads Account label, is marked as required and shows This connector is already connected — label this connection to add another. That only matters for a second bucket: uploading and Done work with the field empty. Below it is the upload area: drag files onto it, or click it to browse, then click Done.

Uploaded files go into a source of their own, listed as file_uploads in your connected sources with its own status and schedule. A sync of it starts as soon as the upload finishes. If one is already running, the files land on the next sync instead.

• CSV, TSV, Parquet, JSON, JSONL and NDJSON files each become a table. In a JSON file each object becomes a row, and the file must hold one JSON object per line.
• An Excel file (.xlsx or .xlsm) also becomes one table, with a sheet column saying which sheet each row came from. The folder connectors, such as Google Drive, read each sheet separately and group sheets and files that share the same columns into one table.
• PDF, Word (.docx), Markdown and plain-text files become rows of page and text, so Rig can search and quote them.

Each file can be up to 50 MB. For a larger file, or files that change often, a shared Google Drive folder is less work than uploading by hand. See Connect Google Drive.

When Rig's context catches up

A sync puts new rows in the warehouse at once, and questions about existing tables see them straight away. New tables and columns also need to reach Rig's context layer before Rig knows how to use them.

• If you can edit context, start the sync from the Connections page and keep it open. Rig then updates its context as soon as the sync lands and shows a notice such as New tables/columns — updating context… or Up to date — no context rebuild needed.
• Otherwise the daily context build picks the changes up, so allow up to a day.

Both of these need your workspace's first full context build to have run. Until then there is no daily build, and the notice after a sync says Go to Build context and click Build to create your context layer.

The first time a new source syncs, its tables are visible only to owners. Everyone else sees them once an owner grants their role access in the Access Matrix; see Admin: people, roles and access.

New tables in your own warehouse

When your own pipelines add a table in a schema Rig already reads, the daily context build finds it within about a day. When they add a column to a table Rig already knows, an editor or owner can bring it in sooner: from an AI tool connected to Rig over MCP, ask it to refresh that table's schema. That only rescans tables Rig already knows. A brand-new table waits for the daily build, unless a workspace owner asks the AI tool to add that table to Rig's context.