Rockset setup

Vendor-supported plugin

Certain core functionality may vary. If you would like to report a bug, request a feature, or contribute, you can check out the linked repository and open an issue.

• Maintained by: Rockset, Inc.
• Authors: Rockset, Inc.
• GitHub repo: rockset/dbt-rockset
• PyPI package: dbt-rockset
• Slack channel: #dbt-rockset
• Supported dbt Core version: v0.19.2 and newer
• dbt Cloud support: Not Supported
• Minimum data platform version: ?

Installing dbt-rockset

Use pip to install the adapter. Before 1.8, installing the adapter would automatically install dbt-core and any additional dependencies. Beginning in 1.8, installing an adapter does not automatically install dbt-core. This is because adapters and dbt Core versions have been decoupled from each other so we no longer want to overwrite existing dbt-core installations. Use the following command for installation:

Configuring dbt-rockset

For Rockset-specific configuration, please refer to Rockset configs.

Connecting to Rockset with dbt-rockset

The dbt profile for Rockset is very simple and contains the following fields:

profiles.yml

rockset:
  target: dev
  outputs:
    dev:
      type: rockset
      workspace: [schema]
      api_key: [api_key]
      api_server: [api_server] # (Default is api.rs2.usw2.rockset.com)

Materializations

Type	Supported?	Details
view	YES	Creates a view.
table	YES	Creates a collection.
ephemeral	YES	Executes queries using CTEs.
incremental	YES	Creates a collection if it doesn't exist, and then writes results to it.

Caveats

1. unique_key is not supported with incremental, unless it is set to _id, which acts as a natural unique_key in Rockset anyway.
2. The table materialization is slower in Rockset than most due to Rockset's architecture as a low-latency, real-time database. Creating new collections requires provisioning hot storage to index and serve fresh data, which takes about a minute.
3. Rockset queries have a two-minute timeout. Any model which runs a query that takes longer to execute than two minutes will fail.