Set up a Databricks connector

Capabilities

Resource	Sync	Provision
Accounts
Groups
Roles
Service principals
Users
Workspaces

The Databricks connector supports automatic account provisioning and deprovisioning. This connector syncs non-human identities and displays them on the Identities overview dashboard.

Provisioning account groups requires OAuth authentication. It is not available when authenticating with a workspace token, because the Databricks API does not allow provisioning account groups from a workspace token. Workspace-scoped groups can still be provisioned with a workspace token.

Authentication methods

The connector authenticates with OAuth — an account-level service principal’s client ID and secret. This is the only method currently offered.

Workspace-token (personal access token) authentication is temporarily unavailable. It is not offered when configuring the connector, and configurations that specify it are rejected at startup. Use OAuth instead.The connector’s PAT implementation is intact and the method is expected to return; it is withheld while a platform-side defect is resolved. The defect is not in this connector: a credential declared as a list of secrets is not treated as secret by the configuration layer, so a workspace token supplied through the UI would be stored unencrypted and displayed in clear text.

Gather Databricks credentials

Configuring the connector requires you to pass in credentials generated in Databricks. Gather these credentials before you move on.

A user with the Account admin role in each Databricks workspace you want to sync must perform this task.

Look up your Databricks account ID

Ensure that your Databricks user account has the Account admin role and is assigned to each Databricks workspace you want to sync to C1.

In the Databricks account console, open the menu that appears next to your username in the upper right corner.

Carefully and copy and save account ID.

Generate Databricks credentials

The Databricks connector authenticates with OAuth:

OAuth (syncs info from all Databricks workspaces)
1
Follow the Databricks OAuth authentication documentation to create a service principal and create an OAuth secret.
2
Carefully copy and save the OAuth client ID and secret.

Done. Here’s the set of credentials you’ll need when setting up the connector:

Account ID
OAuth client ID
OAuth client secret

Personal access token (workspace token) authentication is temporarily unavailable and is not offered when configuring the connector, so there is no need to generate one. See Authentication methods above.

Next, move on to the instructions for your chosen setup method.

Configure the Databricks connector

To complete this task, you’ll need:

The Connector Administrator or Super Administrator role in C1
Access to the set of Databricks credentials generated by following the instructions above

Cloud-hosted
Self-hosted

Follow these instructions to use a built-in, no-code connector hosted by C1.

In C1, navigate to Apps > Connectors and click Add connector.

Search for Databricks and click Add.

Choose where to add the connector: Create a new app, or Add to an existing app (then select the app).If you’re creating a new app, choose whether to link it to an application discovered from your identity provider: select Yes and pick the IdP application, or No to continue with just the connector.

Set the connector’s Name and, optionally, a Description.

Click the pencil icon next to Owners to choose who can configure and manage this connector.

Click Add. The connector is created and its configuration page opens.

Find the Settings area of the page and click Edit.

Paste the account ID you looked up in Step 1 into the Account ID field.

Enter your OAuth client ID and client secret into the OAuth2 Client ID and OAuth2 Client Secret fields.

Google Cloud Platform and Azure Databricks customers only: Enter your Databricks account hostname and hostname in the relevant fields.

For more on how to look up these values for GCP, see the GCP Databricks REST API reference and identifiers for workspace objects documentation.
For more on how to look up these values for Azure Databricks, see the the Azure Databricks REST API reference and identifiers for workspace objects documentation.

Click Save.

The connector’s label changes to Syncing, followed by Connected. You can view the logs to ensure that information is syncing.

Done. Your Databricks connector is now pulling access data into C1.

Follow these instructions to use the Databricks connector, hosted and run in your own environment.When running in service mode on Kubernetes, a self-hosted connector maintains an ongoing connection with C1, automatically syncing and uploading data at regular intervals. This data is immediately available in the C1 UI for access reviews and access requests.

Resources

Official download center: For stable binaries (Windows/Linux/macOS) and container images.
GitHub repository: Access the source code, report issues, or contribute to the project.

Step 1: Set up a new Databricks connector

In C1, navigate to Apps > Connectors and click Add connector.

Search for Baton and click Add.

Set the connector’s Name and, optionally, a Description.

Click the pencil icon next to Owners to choose who can configure and manage this connector.

Click Add. The connector is created and its configuration page opens.

In the Settings area of the page, click Edit.

Click Rotate to generate a new Client ID and Secret.Carefully copy and save these credentials. We’ll use them in Step 2.

Step 2: Create Kubernetes configuration files

Create two Kubernetes manifest files for your Databricks connector deployment:

Secrets configuration

# baton-databricks-secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: baton-databricks-secrets
type: Opaque
stringData:
  # C1 credentials
  BATON_CLIENT_ID: <C1 client ID>
  BATON_CLIENT_SECRET: <C1 client secret>
  
  # Databricks account ID (required for both credential options below)
  BATON_ACCOUNT_ID: <Databricks account ID>

  # Databricks credentials: OAuth
  BATON_DATABRICKS_CLIENT_ID: <OAuth client ID>
  BATON_DATABRICKS_CLIENT_SECRET: <OAuth client secret>

  # Optional: limit the sync to specific workspaces, by deployment name.
  # Mutually exclusive with BATON_DATABRICKS_EXCLUDE_WORKSPACES — set one or the other, never both.
  # BATON_WORKSPACES: <deployment name of workspace-a,deployment name of workspace-b>

  # Optional: exclude specific workspaces from the sync
  # (workspace name, deployment name, or numeric ID).
  # Mutually exclusive with BATON_WORKSPACES — set one or the other, never both.
  BATON_DATABRICKS_EXCLUDE_WORKSPACES: <workspace-a,workspace-b>

  # Optional: include if you want C1 to provision access using this connector
  BATON_PROVISIONING: true

BATON_WORKSPACES and BATON_DATABRICKS_EXCLUDE_WORKSPACES cannot both be set. They are mutually exclusive, and a config carrying both is rejected at startup before any API call. Choose one.

OAuth requires a reachable account API. If the account API check fails at startup, the connector fails validation instead of falling back to a workspace-only sync, even when BATON_WORKSPACES is set.

See the connector’s README or run --help to see all available configuration flags and environment variables.

Deployment configuration

# baton-databricks.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: baton-databricks
  labels:
    app: baton-databricks
spec:
  selector:
    matchLabels:
      app: baton-databricks
  template:
    metadata:
      labels:
        app: baton-databricks
        baton: true
        baton-app: databricks
    spec:
      containers:
      - name: baton-databricks
        image: public.ecr.aws/conductorone/baton-databricks:latest
        imagePullPolicy: IfNotPresent
        env:
        - name: BATON_HOST_ID
          value: baton-databricks
        envFrom:
        - secretRef:
            name: baton-databricks-secrets

Step 3: Deploy the connector

Create a namespace in which to run C1 connectors (if desired), then apply the secret config and deployment config files.

Check that the connector data uploaded correctly. In C1, click Apps. On the Managed apps tab, locate and click the name of the application you added the Databricks connector to. Databricks data should be found on the Entitlements and Accounts tabs.

Done. Your Databricks connector is now pulling access data into C1.

Welcome

Manage connectors

Deploy connectors

Upload connector data

Configurable connectors

Pre-built connectors

Set up a Databricks connector

Capabilities

Authentication methods

Gather Databricks credentials

Look up your Databricks account ID

Generate Databricks credentials

Configure the Databricks connector

Resources

Step 1: Set up a new Databricks connector

Step 2: Create Kubernetes configuration files

Secrets configuration

Deployment configuration

Step 3: Deploy the connector

​Capabilities

​Authentication methods

​Gather Databricks credentials

​Look up your Databricks account ID

​Generate Databricks credentials

​Configure the Databricks connector

​Resources

​Step 1: Set up a new Databricks connector

​Step 2: Create Kubernetes configuration files

​Secrets configuration

​Deployment configuration

​Step 3: Deploy the connector

Capabilities

Authentication methods

Gather Databricks credentials

Look up your Databricks account ID

Generate Databricks credentials

Configure the Databricks connector

Resources

Step 1: Set up a new Databricks connector

Step 2: Create Kubernetes configuration files

Secrets configuration

Deployment configuration

Step 3: Deploy the connector