---
title: "Connect CSV / Parquet (S3) to Rig — Rig guide"
description: "Connect an S3 bucket to Rig with a read-only access key, and load CSV, Parquet, JSONL and PDF files into one warehouse table."
canonical: "https://rig.so/guides/connect-csv-parquet-s3"
format: markdown
---

Search guides, references, playbooks…⌘K

Browse guides

# Connect CSV / Parquet (S3) to Rig

Rig reads the files in an S3 bucket or folder and loads every match into one table you name. Each sync reads only files that are new or changed, so a folder of daily exports builds up into a single table.

## Before you start

- You need an AWS access key for an IAM user that can list and read the bucket. Read access is all Rig needs.
- Know the bucket address and, if the files sit in a folder, its path, such as `s3://my-bucket/exports/`.
- Files that go into one connection should share a layout. Use a second connection for a different kind of export.
- Google Cloud Storage has its own card, so use that for `gs://` buckets.

## Step 1: Create a read-only access key

1. In the AWS console, open `IAM` and choose or create a user for Rig.
2. Give that user a policy that allows `s3:ListBucket` on the bucket and `s3:GetObject` on its objects.
3. Open the user's security credentials, create an access key and copy the access key ID and the secret access key. AWS shows the secret once.

## Step 2: Connect it in Rig

New to Rig? Watch how to find Connections

Adding a source needs a workspace owner, or a role with the Manage connections permission, on a workspace where you manage your own connections. If the Connectors tab says “Data ingestion for this workspace is managed by Rig”, ask an owner or Rig support to add the source.

1. In Rig, open `Connections`, choose the `Connectors` tab and go to `Sources & actions`.
2. Search for **CSV / Parquet (S3)**, open the card and click `Connect`.
3. **Bucket or directory URL**: the bucket and folder, such as `s3://my-bucket/exports/`.
4. **File glob pattern**: which files to read, such as `*.csv`, `exports/*.parquet` or `**/*.jsonl`. It defaults to `*.csv`.
5. **Destination table name**: the table every matching file loads into.
6. **Primary key (optional)**: the column, or comma-separated columns, that identify a row. Set it now if rows can arrive twice, because adding it later leaves existing duplicates in place.
7. **Latest-version column (optional)**: with a primary key, the column that decides which copy wins when several versions of a row arrive together, such as `updated_at`.
8. **AWS access key ID (s3:// only)**: the access key ID you created.
9. **AWS secret access key (s3:// only)**: the matching secret access key.
10. Click `Continue`. Rig checks the credentials and lists the streams it can reach, so choose the ones you want and click `Connect`.
11. The source then syncs on its schedule, which is nightly unless you change it. To load data straight away, click `Sync` on its row; `Test` pulls a small sample first if you want to check the connection.

## What Rig pulls in

CSV / Parquet (S3) lands in its own `csv_filesystem` schema in your warehouse:

- **One table**: every file matching the glob loads into the table you named. CSV, Parquet and JSONL files load as rows, and a PDF loads as one row per page.

## Good to know

- Rig keeps track of each file's modification date, so a re-sync reads only files that are new or changed since the last run.
- Without a primary key, a file that is rewritten has its rows added again. Exports that write a new file each run load cleanly.
- Rig parses files with DuckDB, whose CSV reader copes with byte-order marks, quoted commas and ragged trailing columns.
- To disconnect CSV / Parquet (S3), click the bin icon on the source's row under `Sources & actions`. Only whoever connected the source or a workspace owner can do this, and data already synced stays in the warehouse.

## What people use this for

- Turn a folder of exports into one table
- Keep the latest version of each record
- Join partner and vendor feeds to your data

[More ideas for using CSV / Parquet (S3)](https://rig.so/integrations/csv-parquet-s3)

## CSV / Parquet (S3) documentation

- [Manage access keys for IAM users](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)
- [Identity-based policy examples for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-policies-s3.html)

Was this guide helpful?

Yes No

---

- [This page as HTML](https://rig.so/guides/connect-csv-parquet-s3)
- [Site map for language models](https://rig.so/llms.txt)
- [API and agent documentation](https://rig.so/developers)