Skip to content

Create connector files #7726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 21 commits into
base: main
Choose a base branch
from
+151 −260
Draft

Create connector files #7726

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
50df07c
create postgres connector init
lovincyrus Aug 5, 2025
c232bfc
remove sql and name from postgres driver
lovincyrus Aug 5, 2025
31f94ea
remove sql and name from mysql
lovincyrus Aug 5, 2025
709f29b
remove name from gcs driver
lovincyrus Aug 5, 2025
bf0f577
remove s3 from rewrite to duckdb
lovincyrus Aug 5, 2025
f3b55f7
Revert "remove s3 from rewrite to duckdb"
lovincyrus Aug 6, 2025
3bf311c
set source imported path for connector case
lovincyrus Aug 6, 2025
f225bcd
lint
lovincyrus Aug 6, 2025
e73f867
fix top of file local file yaml compilation
lovincyrus Aug 7, 2025
4e4e597
rm connector set imported file path
lovincyrus Aug 7, 2025
573f4b8
revert
lovincyrus Aug 7, 2025
47d24d3
clean up
lovincyrus Aug 7, 2025
ec6679b
tweak connector form type
lovincyrus Aug 7, 2025
db8331b
modify bigquery configproperties, yupschemas
lovincyrus Aug 8, 2025
1c1e3f7
only test connection for the previous olap connectors
lovincyrus Aug 8, 2025
e67e463
centralize sources and olap connectors to constants
lovincyrus Aug 8, 2025
d5fc2ef
revert for sanity
lovincyrus Aug 8, 2025
e2ec7e6
only set olap connector in rill for olap engines
lovincyrus Aug 8, 2025
d825df4
clean up
lovincyrus Aug 8, 2025
355c72b
fix config properties for athena
lovincyrus Aug 8, 2025
120a199
fix config properties for redshift
lovincyrus Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix config properties for athena
  • Loading branch information
@lovincyrus
lovincyrus committed Aug 8, 2025
commit 355c72b50f1f4fdcc73eb2900ce538df79aa55dc
49 changes: 10 additions & 39 deletions runtime/drivers/athena/athena.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,56 +34,27 @@ var spec = drivers.Spec{
DocsURL: "https://docs.rilldata.com/reference/connectors/athena",
ConfigProperties: []*drivers.PropertySpec{
{
Key: "aws_access_key_id",
Type: drivers.StringPropertyType,
Secret: true,
Key: "aws_access_key_id",
Type: drivers.StringPropertyType,
DisplayName: "AWS access key ID",
Description: "AWS access key ID",
Required: true,
Secret: true,
},
{
Key: "aws_secret_access_key",
Type: drivers.StringPropertyType,
Secret: true,
},
},
// Important: Any edits to the below properties must be accompanied by changes to the client-side form validation schemas.
SourceProperties: []*drivers.PropertySpec{
{
Key: "sql",
Key: "aws_secret_access_key",
Type: drivers.StringPropertyType,
DisplayName: "AWS secret access key",
Description: "AWS secret access key",
Required: true,
DisplayName: "SQL",
Description: "Query to extract data from Athena.",
Placeholder: "select * from catalog.table;",
Secret: true,
},
{
Key: "output_location",
Type: drivers.StringPropertyType,
DisplayName: "S3 output location",
Description: "An output location for query result is required either through the workgroup result configuration setting or set here.",
Placeholder: "s3://bucket-name/path/",
Required: false,
},
{
Key: "workgroup",
Type: drivers.StringPropertyType,
DisplayName: "AWS Athena workgroup",
Description: "AWS Athena workgroup to use for queries. Default is primary",
Placeholder: "primary",
Required: false,
},
{
Key: "region",
Type: drivers.StringPropertyType,
DisplayName: "AWS region",
Description: "AWS region to connect to Athena and the output location.",
Placeholder: "us-east-1",
Required: false,
},
{
Key: "name",
Type: drivers.StringPropertyType,
DisplayName: "Source name",
Description: "The name of the source",
Placeholder: "my_new_source",
Required: true,
},
},
Expand Down
10 changes: 4 additions & 6 deletions web-common/src/features/sources/modal/yupSchemas.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,13 +106,11 @@ export const getYupSchema = {
}),

athena: yup.object().shape({
sql: yup.string().required("sql is required"),
output_location: yup.string(),
workgroup: yup.string(),
name: yup
aws_access_key_id: yup.string().required("AWS access key ID is required"),
aws_secret_access_key: yup
.string()
.matches(VALID_NAME_PATTERN, INVALID_NAME_MESSAGE)
.required("Source name is required"),
.required("AWS secret access key is required"),
output_location: yup.string().required("S3 URI is required"),
}),

redshift: yup.object().shape({
Expand Down