Adding Datastores

Kaspian pipelines support several types of datastores, including popular SQL databases, object datastores like AWS S3 and GCS, and Elasticsearch.

For VPC-locked resources, VPC peering must be setup between the resource’s VPC and Kaspian’s VPC. Contact Kaspian support if assistance is needed.

Postgres, Aurora, and Redshift

Select the Postgres datastore type for all Postgres-compatible data stores, like PostgreSQL, Amazon Aurora, and Amazon Redshift.

Datastore Name: Name for the Postgres-compatible datastore within Kaspian’s pipeline builder
Host: Host name or IP address of the SQL datastore
Port: Port number of the SQL datastore
User: Username used to connect to the SQL datastore
Password: Password for the user to connect to the SQL datastore
Database: Database this connection will used for in the SQL datastore

Datastore Name: Name for the Snowflake datastore within Kaspian’s pipeline builder
Account: Snowflake account URL prefix, e.g., xy12345.us-east-1
User: Username used to connect to the Snowflake datastore
Password: Password for the user to connect to the Snowflake datastore
Warehouse: Specifies the virtual warehouse to use once connected. The specified warehouse should be an existing warehouse for which the specified default role has privileges.
Role: Specifies the default access control role to use in the Snowflake session. Ensure this role is assigned to the user
Database: Specifies the default database to use once connected

Datastore Name: Name for the MySQL datastore within Kaspian’s pipeline builder
Host: Host name or IP address of the MySQL datastore
Port: Port number of the MySQL datastore
User: Username used to connect to the MySQL datastore
Password: Password for the user to connect to the MySQL datastore
Database: Database this connection will used for in the MySQL datastore

Datastore Name: Name for the BigQuery datastore within Kaspian’s pipeline builder
Project: Name of the Google Cloud Platform Project containing the BigQuery datastore
Dataset: Name of the BigQuery dataset storing the pipeline data
Tip
To modify BigQuery Tables in Kaspian pipelines, the Kaspian service IAM role must have the BigQuery Admin role. For assistance, contact Kaspian support.

Datastore Name: Name for the Elasticsearch datastore within Kaspian’s pipeline builder
Host: Host name or IP address of the Elasticsearch datastore
Port: Port number of the Elasticsearch datastore
User: Username used to connect to the Elasticsearch datastore
Password: Password for the user to connect to the Elasticsearch datastore