Skip to content

[DOCS] : Fast Follow Connect Fixes #7755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/docs/build/advanced-models/advanced-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ Another use case is when using multiple OLAP engines. This allows you to specify

## Types of Advanced Models

1. [Incremental Models](./incremental-models)
2. [Partitioned Models](./partitions)
3. [Staging Models](./staging)
1. [Incremental Models](/build/advanced-models/incremental-models)
2. [Partitioned Models](/build/advanced-models/partitions)
3. [Staging Models](/build/advanced-models/staging)
4. [DuckDB `pre_exec`/`post_exec` Models](#duckdb-models-pre-exec-sql-post-exec)


Expand Down
2 changes: 1 addition & 1 deletion docs/docs/build/metrics-view/_category_.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
position: 38
label: Create Metrics Views
collapsible: true
collapsed: true
collapsed: true
63 changes: 33 additions & 30 deletions docs/docs/connect/connect.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ Rill is continually evaluating additional OLAP engines that can be added. For a
:::


## Data Warehouse
## Data Warehouses

### Athena
### BigQuery
Expand Down Expand Up @@ -166,9 +166,11 @@ Rill is continually evaluating additional OLAP engines that can be added. For a

## Object Storage

### Azure
### Google Cloud Storage
### Amazon S3
### Google Cloud Storage
### Microsoft Azure Blob Storage



<div className="connector-icon-grid">

Expand All @@ -180,6 +182,15 @@ Rill is continually evaluating additional OLAP engines that can be added. For a
linkLabel="Learn more"
referenceLink="s3"
/>
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-GCS.svg" alt="Google Cloud Storage" />}
header="Google Cloud Storage"
content="Google Cloud Storage for scalable object storage and data lakes."
link="/connect/data-source/gcs"
linkLabel="Learn more"
referenceLink="gcs"
/>

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Azure.svg" alt="Microsoft Azure" />}
header="Azure"
Expand All @@ -188,26 +199,27 @@ Rill is continually evaluating additional OLAP engines that can be added. For a
linkLabel="Learn more"
referenceLink="azure"
/>
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-GCS.svg" alt="Google Cloud Storage" />}
header="Google Cloud Storage"
content="Google Cloud Storage for scalable object storage and data lakes."
link="/connect/data-source/gcs"
linkLabel="Learn more"
referenceLink="gcs"
/>


</div>

## Other Data Connectors

### Google Sheets
### HTTPS
### Local File
### Salesforce


<div className="connector-icon-grid">

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Sheets.svg" alt="Google Sheets" className="sheets-icon" />}
header="Google Sheets"
content="Connect to public Google Sheets to read data from spreadsheets with support for multiple sheets."
link="/connect/data-source/googlesheets"
linkLabel="Learn more"
/>
<ConnectorIcon
icon={<p className="https-icon">https:// </p>}
header="HTTPS"
Expand All @@ -216,7 +228,6 @@ Rill is continually evaluating additional OLAP engines that can be added. For a
linkLabel="Learn more"
referenceLink="https"
/>

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Local.svg" alt="Local File" />}
header="Local File"
Expand All @@ -234,13 +245,6 @@ Rill is continually evaluating additional OLAP engines that can be added. For a
referenceLink="salesforce"
/>

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Sheets.svg" alt="Google Sheets" className="sheets-icon" />}
header="Google Sheets"
content="Connect to public Google Sheets to read data from spreadsheets with support for multiple sheets."
link="/connect/data-source/googlesheets"
linkLabel="Learn more"
/>
</div>

:::tip Missing a connector?
Expand All @@ -249,20 +253,11 @@ We're constantly adding new data connectors. If you don't see what you need, [le

## Other Integrations

### Google Sheets
### Open AI
### Slack

<div className="connector-icon-grid">

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Slack.svg" alt="Slack" className="sheets-icon" />}
header="Slack"
content="Connect to Slack to send alerts and messages from Rill."
link="/connect/data-source/slack"
linkLabel="Learn more"
referenceLink="slack"
/>

<div className="connector-icon-grid">
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-AI.svg" alt="AI" className="sheets-icon" />}
header="AI"
Expand All @@ -271,5 +266,13 @@ We're constantly adding new data connectors. If you don't see what you need, [le
linkLabel="Learn more"
referenceLink="ai"
/>
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Slack.svg" alt="Slack" className="sheets-icon" />}
header="Slack"
content="Connect to Slack to send alerts and messages from Rill."
link="/connect/data-source/slack"
linkLabel="Learn more"
referenceLink="slack"
/>
</div>

6 changes: 6 additions & 0 deletions docs/docs/connect/data-source/athena.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,12 @@ If this project has already been deployed to Rill Cloud and credentials have bee

:::

## Separating Dev and Prod Environments

When ingesting data locally, consider setting parameters in your connector file to limit how much data is retrieved, since costs can scale with the data source. This also helps other developers clone the project and iterate quickly by reducing ingestion time.

For more details, see our [Dev/Prod setup docs](/connect/templating).

## Cloud deployment

When deploying a project to Rill Cloud, Rill requires you to explicitly provide an access key and secret for an AWS service account with access to Athena used in your project. Please refer to our [connector YAML reference docs](/reference/project-files/connectors#athena) for more information.
Expand Down
8 changes: 7 additions & 1 deletion docs/docs/connect/data-source/azure.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Azure Blob Storage
description: Connect to data in Azure Blob Storage
sidebar_label: ABS
sidebar_label: Azure Blob Storage
sidebar_position: 05
---

Expand Down Expand Up @@ -85,6 +85,12 @@ If this project has already been deployed to Rill Cloud and credentials have bee

:::

## Separating Dev and Prod Environments

When ingesting data locally, consider setting parameters in your connector file to limit how much data is retrieved, since costs can scale with the data source. This also helps other developers clone the project and iterate quickly by reducing ingestion time.

For more details, see our [Dev/Prod setup docs](/connect/templating).

## Cloud deployment

When deploying a project to Rill Cloud, Rill requires either an Azure Blob Storage connection string, Azure Storage Key, or Azure Storage SAS token to be explicitly provided for the Azure Blob Storage containers used in your project.
Expand Down
6 changes: 6 additions & 0 deletions docs/docs/connect/data-source/bigquery.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@ If this project has already been deployed to Rill Cloud and credentials have bee

:::

## Separating Dev and Prod Environments

When ingesting data locally, consider setting parameters in your connector file to limit how much data is retrieved, since costs can scale with the data source. This also helps other developers clone the project and iterate quickly by reducing ingestion time.

For more details, see our [Dev/Prod setup docs](/connect/templating).

## Cloud deployment

When deploying a project to Rill Cloud, Rill requires you to explicitly provide a JSON key file for a Google Cloud service account with access to BigQuery used in your project.
Expand Down
80 changes: 42 additions & 38 deletions docs/docs/connect/data-source/data-source.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,45 @@
---
title: "Connect to your Data"
description: Import local files or remote data sources
sidebar_label: "Connectors"
title: "Rill Managed OLAP + Data Ingestion"
description: Import local files or remote data sources into Rill's embedded Analytics Engine
sidebar_position: 00
toc_max_heading_level: 3
className: connect-connect
---


By default, Rill will use a managed embedded analytics engine (**DuckDB** or **ClickHouse**) to support data ingestion. Whether you're working with cloud data warehouses, databases, file storage, or streaming data sources, Rill provides seamless connectivity and data ingestion capabilities. Once this has been ingested, create [downstream models](/build/models), [metrics views](/build/metrics-view) and [visualize your data](/build/dashboards).

import ConnectorIcon from '@site/src/components/ConnectorIcon';

Rill supports importing data from multiple sources to power your analytics and data applications. Whether you're working with cloud data warehouses, databases, file storage, or streaming data sources, Rill provides seamless connectivity and data ingestion capabilities.

## Getting Started with Connectors
:::tip using clickhouse?

Dont forget to [create a managed ClickHouse server](/connect/olap/clickhouse#rill-managed-clickhouse) before getting started!


```yaml
# Connector YAML
# Reference documentation: https://docs.rilldata.com/reference/project-files/connectors

type: connector

driver: clickhouse
managed: true
```

:::


In order to connect and browse through your data, you'll need to create a connector file. Browse through the options below for our supported connectors. Each connector is designed to handle the specific authentication and configuration requirements of your data source.


import ConnectorIcon from '@site/src/components/ConnectorIcon';

### What You Can Do After Connecting

Once the connector is created, you can directly create:
- **Models** - Transform and combine data from multiple sources using SQL
- **Metrics Views** - Define key business metrics and KPIs
- **Dashboards** - Visualize your data with interactive charts and tables
In order to connect and browse through your data, you'll need to create a connector file. Browse through the options below for our supported connectors. Each connector is designed to handle the specific authentication and configuration requirements of your data source.

:::warning OLAP Engine Limitations
Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [ClickHouse](/connect/olap/clickhouse). However, there are still some features in development for managed ClickHouse. For more information see our [managed ClickHouse docs](/connect/olap/clickhouse#rill-managed-clickhouse). If you've still got questions, [contact our team](/contact) for more informaton and scheduled feature releases!
Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [ClickHouse](/connect/olap/clickhouse). However, there are still some features in development for managed ClickHouse. For more information see our [managed ClickHouse docs](/connect/olap/clickhouse#rill-managed-clickhouse). If you've still got questions, [contact our team](/contact) for more information and scheduled feature releases!
:::


## Data Warehouse
## Data Warehouses

### Athena
### BigQuery
Expand Down Expand Up @@ -114,10 +117,11 @@ Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [C


## Object Storage

### Azure
### Google Cloud Storage
### Amazon S3
### Google Cloud Storage
### Microsoft Azure Blob Storage



<div className="connector-icon-grid">

Expand All @@ -129,14 +133,6 @@ Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [C
linkLabel="Learn more"
referenceLink="s3"
/>
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Azure.svg" alt="Microsoft Azure" />}
header="Azure"
content="Connect to Microsoft Azure Blob Storage to read data files with support for various formats."
link="/connect/data-source/azure"
linkLabel="Learn more"
referenceLink="azure"
/>
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-GCS.svg" alt="Google Cloud Storage" />}
header="Google Cloud Storage"
Expand All @@ -145,19 +141,36 @@ Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [C
linkLabel="Learn more"
referenceLink="gcs"
/>
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Azure.svg" alt="Microsoft Azure" />}
header="Azure"
content="Connect to Microsoft Azure Blob Storage to read data files with support for various formats."
link="/connect/data-source/azure"
linkLabel="Learn more"
referenceLink="azure"
/>



</div>

## Other Data Connectors

### Google Sheets
### HTTPS
### Local File
### Salesforce
### Google Sheets
### Slack



<div className="connector-icon-grid">
<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Sheets.svg" alt="Google Sheets" className="sheets-icon" />}
header="Google Sheets"
content="Connect to Google Sheets to read data from spreadsheets."
link="/connect/data-source/googlesheets"
linkLabel="Learn more"
/>
<ConnectorIcon
icon={<p className="https-icon">https:// </p>}
header="HTTPS"
Expand All @@ -166,15 +179,13 @@ Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [C
linkLabel="Learn more"
referenceLink="https"
/>

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Local.svg" alt="Local File" />}
header="Local File"
content="Read data from local files including CSV, JSON, Parquet, and compressed formats."
link="/connect/data-source/local-file"
linkLabel="Learn more"
/>

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Salesforce.svg" alt="Salesforce" />}
header="Salesforce"
Expand All @@ -184,13 +195,6 @@ Rill supports connecting your data to both [DuckDB](/connect/olap/duckdb) and [C
referenceLink="salesforce"
/>

<ConnectorIcon
icon={<img src="/img/connect/icons/Logo-Sheets.svg" alt="Google Sheets" className="sheets-icon" />}
header="Google Sheets"
content="Connect to Google Sheets to read data from spreadsheets with support for multiple sheets."
link="/connect/data-source/googlesheets"
linkLabel="Learn more"
/>
</div>


Expand Down
8 changes: 7 additions & 1 deletion docs/docs/connect/data-source/gcs.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,13 @@ You have now configured Google Cloud access from your local environment. Rill wi
If this project has already been deployed to Rill Cloud and credentials have been set for this source, you can use `rill env pull` to [pull these cloud credentials](/connect/credentials/#rill-env-pull) locally (into your local `.env` file). Please note that this may override any credentials you have set locally for this source.
:::

## Rill Cloud deployment
## Separating Dev and Prod Environments

When ingesting data locally, consider setting parameters in your connector file to limit how much data is retrieved, since costs can scale with the data source. This also helps other developers clone the project and iterate quickly by reducing ingestion time.

For more details, see our [Dev/Prod setup docs](/connect/templating).

## Rill Cloud Deployment

When deploying a project to Rill Cloud, Rill requires a JSON key file to be explicitly provided for a Google Cloud service account with appropriate read access/permissions to the buckets used in your project.

Expand Down
6 changes: 6 additions & 0 deletions docs/docs/connect/data-source/mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ rill env pull
```
:::

## Separating Dev and Prod Environments

When ingesting data locally, consider setting parameters in your connector file to limit how much data is retrieved, since costs can scale with the data source. This also helps other developers clone the project and iterate quickly by reducing ingestion time.

For more details, see our [Dev/Prod setup docs](/connect/templating).

## Cloud deployment

Once a project with a MySQL source has been deployed, Rill requires you to explicitly provide the connection string using the following command:
Expand Down
Loading