Skip to content

Create connector files #7726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 21 commits into
base: main
Choose a base branch
from
Draft

Create connector files #7726

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
50df07c
create postgres connector init
lovincyrus Aug 5, 2025
c232bfc
remove sql and name from postgres driver
lovincyrus Aug 5, 2025
31f94ea
remove sql and name from mysql
lovincyrus Aug 5, 2025
709f29b
remove name from gcs driver
lovincyrus Aug 5, 2025
bf0f577
remove s3 from rewrite to duckdb
lovincyrus Aug 5, 2025
f3b55f7
Revert "remove s3 from rewrite to duckdb"
lovincyrus Aug 6, 2025
3bf311c
set source imported path for connector case
lovincyrus Aug 6, 2025
f225bcd
lint
lovincyrus Aug 6, 2025
e73f867
fix top of file local file yaml compilation
lovincyrus Aug 7, 2025
4e4e597
rm connector set imported file path
lovincyrus Aug 7, 2025
573f4b8
revert
lovincyrus Aug 7, 2025
47d24d3
clean up
lovincyrus Aug 7, 2025
ec6679b
tweak connector form type
lovincyrus Aug 7, 2025
db8331b
modify bigquery configproperties, yupschemas
lovincyrus Aug 8, 2025
1c1e3f7
only test connection for the previous olap connectors
lovincyrus Aug 8, 2025
e67e463
centralize sources and olap connectors to constants
lovincyrus Aug 8, 2025
d5fc2ef
revert for sanity
lovincyrus Aug 8, 2025
e2ec7e6
only set olap connector in rill for olap engines
lovincyrus Aug 8, 2025
d825df4
clean up
lovincyrus Aug 8, 2025
355c72b
fix config properties for athena
lovincyrus Aug 8, 2025
120a199
fix config properties for redshift
lovincyrus Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
centralize sources and olap connectors to constants
  • Loading branch information
@lovincyrus
lovincyrus committed Aug 8, 2025
commit e67e4636bfb23cc17a9c76a88e501219d7e8ae32
23 changes: 1 addition & 22 deletions web-common/src/features/sources/modal/AddDataModal.svelte
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,34 +39,13 @@
import DuplicateSource from "./DuplicateSource.svelte";
import LocalSourceUpload from "./LocalSourceUpload.svelte";
import RequestConnectorForm from "./RequestConnectorForm.svelte";
import { OLAP_CONNECTORS, SORT_ORDER, SOURCES } from "./constants";

let step = 0;
let selectedConnector: null | V1ConnectorDriver = null;
let requestConnector = false;
let isSubmittingForm = false;

const SOURCES = [
"gcs",
"s3",
"azure",
"bigquery",
"athena",
"redshift",
"duckdb",
"motherduck",
"postgres",
"mysql",
"sqlite",
"snowflake",
"salesforce",
"local_file",
"https",
];

const OLAP_CONNECTORS = ["clickhouse", "druid", "pinot"];

const SORT_ORDER = [...SOURCES, ...OLAP_CONNECTORS];

const ICONS = {
gcs: GoogleCloudStorage,
s3: AmazonS3,
Expand Down
22 changes: 22 additions & 0 deletions web-common/src/features/sources/modal/constants.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,25 @@ export const CONNECTION_TAB_OPTIONS: { value: string; label: string }[] = [
{ value: "parameters", label: "Enter parameters" },
{ value: "dsn", label: "Enter connection string" },
];

export const SOURCES = [
"gcs",
"s3",
"azure",
"bigquery",
"athena",
"redshift",
"duckdb",
"motherduck",
"postgres",
"mysql",
"sqlite",
"snowflake",
"salesforce",
"local_file",
"https",
];

export const OLAP_CONNECTORS = ["clickhouse", "druid", "pinot"];

export const SORT_ORDER = [...SOURCES, ...OLAP_CONNECTORS];
2 changes: 1 addition & 1 deletion web-common/src/features/sources/modal/submitAddDataForm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import { EntityType } from "../../entity-management/types";
import { EMPTY_PROJECT_TITLE } from "../../welcome/constants";
import { isProjectInitialized } from "../../welcome/is-project-initialized";
import { compileSourceYAML, maybeRewriteToDuckDb } from "../sourceUtils";
import { OLAP_CONNECTORS } from "./constants";

interface AddDataFormValues {
// name: string; // Commenting out until we add user-provided names for Connectors
Expand Down Expand Up @@ -174,7 +175,6 @@ export async function submitAddOLAPConnectorForm(

// Test the connection to the OLAP database (only for original OLAP connectors)
// If the connection test fails, rollback the changes
const OLAP_CONNECTORS = ["clickhouse", "druid", "pinot"];
if (OLAP_CONNECTORS.includes(connector.name as string)) {
const result = await testOLAPConnector(
instanceId,
Expand Down