Adding Integrations

Kaspian supports integrations with several external services, including AWS/GCP, Github, and Slack.

Amazon Web Services (AWS)

Kaspian relies on an existing AWS account where it will host the compute and some elements of data storage. An IAM role for Kaspian services to assume should be created to facilitate this. The IAM Role ARN and AWS region are required for Kaspian to function.

This role requires a limited set of access policies to ensure the services work and have access to Kaspian-specific resources. Please contact Kaspian support for instructions on setting this up.

Google Cloud Platform (GCP)

For those who use GCP, Kaspian runs in its own project. Access to this project is provided via a service account using a Workload Identity Pool, eliminating the need to provide Kaspian with private key JSON files.

Microsoft Azure

Support for Microsoft Azure is coming soon.

Git

GitHub

Code storage for Kaspian pipelines can be hosted on GitHub. In order to access the Github repo, Kaspian needs a GitHub personal access token (PAT) with read permissions for the account. This is a required piece of information.

Warning

GitHub PATs provide access to all repositories accessible by the user who generated the PAT. For this reason, the Kaspian team recommends creating a separate GitHub user which only has access to repositories containing Kaspian pipeline code, and using that separate user to generate the PAT.

Instructions to create a personal access token can be found here. Only repo permissions need to be provided.

GitLab

Code can also be hosted on GitLab. GitLab supports personal access tokens as well as project access tokens (for Premium users and above). When generating a token, you must select the read_api and read_repository scopes.

Bitbucket

Support for Bitbucket is coming soon.

Specifying Repository Entrypoints

GitHub/GitLab integrations can be scoped to individual repositories (or paths within those repositories) using entrypoints.
Start by clicking the highlighted + within the integration form.

In the form that appears, enter the information as described below:

Owner: Github/Gitlab organization or individual who owns the repository, e.g. Kaspian-Inc
Repository: Repository name, as it appears on GitHub or GitLab
Ref (Optional): Specific branch name or commit hash
Path: PYTHONPATH relative to the directory’s root, e.g. / if at the root, or /path/to/python/code/
Regular Expression: Regex specifying which directories to include within the repo, e.g. .* for all repo content

Slack

To send notifications in the event of a pipeline error, Kaspian supports an optional Slack integration. Setting up this integration requires a Slack app OAuth token for the desired workspace (detailed instructions below).

To obtain an app access token, first create a custom Slack app. The most direct way to create an app is via an “app manifest,” which is a YAML-formatted specification for an app’s information and permissions, such as:

_metadata:
  major_version: 1
  minor_version: 1
display_information:
  name: kaspian-bot
features:
  bot_user:
    display_name: kasper
    always_online: false
oauth_config:
  scopes:
    bot:
      - channels:join
      - channels:read
      - chat:write
settings:
  org_deploy_enabled: false
  socket_mode_enabled: false
  token_rotation_enabled: false

Note

In order to send Slack notifications, Kaspian requires, at minimum, channels:join, channels:read, and chat:write permissions as the Bot Token Scope, as shown in the sample manifest above.

To create a custom Slack app, follow the below steps:

Visit the Slack API site, and sign in.
Click Create New App > From an app manifest
Select the workspace to install the app in, and hit Next
Paste the above app manifest YAML and hit Next
Click Create, which will redirect to an app management page
On the left sidebar, go to OAuth & Permissions
Click Install to Workspace under the section titled OAuth Tokens for Your Workspace
Copy the token that appears under the field titled Bot User OAuth Token (it should begin with xoxb)

Once the app is created and linked to a Slack workspace, provide the access token in the Slack integration form.

Note

During pipeline creation, Kaspian will automatically detect which channels the App has access to and present them as options for error notifications.

Microsoft Teams

Kaspian’s Microsoft Teams integration supports the Incoming Webhooks feature. To set up this integration, first create a new incoming webhook in the desired channel. Instructions for this can be found here. In the Kaspian form, simply enter the channel and the configured webhook URL.