Connect Jungle Scout Data Cloud to Rig

    Sync the category insights, category trees, competitive offers and product sales files that Jungle Scout delivers to your Data Cloud S3 bucket into Rig. Jungle Scout sets up the bucket and the AWS keys for you, so there is nothing to create on the Jungle Scout side.

    Before you start

    • You need a Jungle Scout Data Cloud subscription. Jungle Scout provisions a bucket for you, with a name like s3://js-prd-datasharing-yourcompany-01.
    • Jungle Scout shares an AWS service account with you for that bucket. The keys are usually a one-time view, so keep them safe when you receive them.
    • Decide how much history you want. A new source starts from the latest delivery unless you ask for more (see Step 1 below).

    Step 1: Collect the bucket details from Jungle Scout

    This is a different product from the Jungle Scout (Cobalt) API, with different credentials. Jungle Scout's Data Cloud page lists S3 as one delivery method.

    1. Ask your Jungle Scout contact for the bucket URL, the AWS region the bucket lives in and the data share version.
    2. Ask for the AWS access key ID and secret access key of the service account they created for you.
    3. Save the secret access key straight away. It is shown once.

    Step 2: Connect it in Rig

    Adding a source needs a workspace owner, or a role with the Manage connections permission, on a workspace where you manage your own connections. If the Connectors tab says “Data ingestion for this workspace is managed by Rig”, ask an owner or Rig support to add the source.

    1. In Rig, open Connections, choose the Connectors tab and go to Sources & actions.
    2. Search for Jungle Scout Data Cloud, open the card and click Connect.
    3. Bucket URL: the bucket Jungle Scout provisioned, starting with s3://.
    4. AWS access key ID: the access key ID of the service account.
    5. AWS secret access key: the secret access key of the same service account.
    6. AWS region: pre-filled with us-east-1. Leave it as it is unless Jungle Scout gave you another region.
    7. Share version: pre-filled with 3. Leave it as it is unless Jungle Scout tells you the delivered schema has a newer version.
    8. Load history from: leave it blank to start from the latest delivery, which finishes in minutes. Enter a date such as 2024-01-01, or all, to load the archive. That is a large load, but it saves its progress in batches and carries on if it is interrupted.
    9. Click Continue. Rig checks the credentials and lists the streams it can reach, so choose the ones you want and click Connect.
    10. The source then syncs on its schedule, which is nightly unless you change it. To load data straight away, click Sync on its row; Test pulls a small sample first if you want to check the connection.

    What Rig pulls in

    Jungle Scout Data Cloud lands in its own jungle_scout_data_cloud schema in your warehouse:

    • Category insights: monthly and weekly category insight files.
    • Category tree: monthly and weekly category tree files.
    • Competitive offers: offers from competing sellers.
    • Product sales and attributes: monthly and weekly product sales and attribute files.

    Rig reads both the historical folders and the latest delivery with one cursor, so each file is read once and re-delivered files are picked up again when their modification date changes. The marketplace and root category ID exist only in the file path, so Rig adds them as columns.

    Good to know

    • Which markets and categories you see depends on what Jungle Scout put in your share. Size the archive from the files in your bucket, because a full history can run to millions of rows.
    • If Jungle Scout moves you to a new share version, contact Rig before changing the Share version field. Rig remembers how far it has read, so changing the version alone can start the backfill part-way through the history.
    • The files are partitioned in different ways: competitive offers and category trees by marketplace, and the insights and product files by marketplace and root category as well.
    • To disconnect Jungle Scout Data Cloud, click the bin icon on the source's row under Sources & actions. Only whoever connected the source or a workspace owner can do this, and data already synced stays in the warehouse.

    What people use this for

    • Size and track Amazon categories
    • Read product sales and attributes at scale
    • Watch competitive offers

    More ideas for using Jungle Scout Data Cloud

    Jungle Scout Data Cloud documentation

    Was this guide helpful?