Uniskai can collect and display cost and usage data from Databricks alongside your cloud providers. Databricks is added under Third-party connections in Account manager, and once connected its costs appear in the same Cloudcosts modules as AWS, Azure, GCP, and OCI data.
To connect Databricks, you create a service principal in your Databricks account console, give it access to the workspace and to a SQL warehouse, then add the account in Uniskai and run the GRANT commands it generates for you.
How it works
Object | Where it is created | Purpose |
Service principal | Databricks Account Console | The identity Uniskai authenticates as. Its Client ID also targets the GRANT commands generated in the form. |
OAuth secret | Databricks, on the service principal | Authenticates Uniskai to Databricks. Issued with the sql scope and a lifetime you choose. |
Workspace entitlements | Databricks, on the target workspace | Allow the service principal to sign in to the workspace and use Databricks SQL. |
uniskai_wh | Databricks workspace | The SQL warehouse Uniskai runs its daily read on. 2X-Small, serverless. |
GRANT privileges | Databricks, run in SQL Editor | Read access to the system catalog and to the system.billing and system.compute schemas. |
Prerequisites
Admin access to the Databricks Account Console at accounts.cloud.databricks.com
Permission to manage accounts in Uniskai
A 2X-Small serverless SQL warehouse named uniskai_wh in the workspace you want to connect. Uniskai looks for this exact name, so create it or rename a suitable existing warehouse before you start
How to connect Databricks:
1. Create a service principal and generate a secret
In the Databricks Account Console, open User management in the left sidebar.
Open the Service principals tab and select Add service principal.
Enter a display name and select Add service principal.
On the service principal page, open Credentials & secrets.
In the OAuth secrets section, select Generate secret.
Set Lifetime (days), select Add scope and choose sql, then select Generate.
Copy both the Secret and the Client ID, then select Done.
The secret is shown once. If you close the dialog without copying it, generate a new one. A service principal can hold up to five secrets at a time.
2. Start the connection in Uniskai
In Uniskai, open Account manager and select Add New Account.
Select Databricks.
Enter a name for the connection in Account name.
Paste the Client ID into the Client ID field. The GRANT commands below are generated for that Client ID.
Paste the Secret into the Client secret field.
Leave Purchase channel as Direct from Databricks, unless your Databricks usage is invoiced through a cloud marketplace.
Keep this page open. You return to it after granting access in Databricks.
3. Give the service principal access to the workspace:
In the Databricks Account Console, open Workspaces and select the workspace you want to connect.
Open the Permissions tab and select Add permissions.
In the Identity field, enter the service principal name and select it from the results.
Enable Consumer access, Databricks SQL access and Workspace access. Leave Admin access disabled.
Select Add permissions
A Permission added notification confirms the change, and the service principal appears in the permissions table.
If the service principal was created moments earlier, it may not appear in the search straight away. Refresh the page and search again.
4. Set up the SQL warehouse:
Open the workspace and select SQL Warehouses in the left sidebar.
If a 2X-Small serverless SQL warehouse named uniskai_wh already exists, go to step 4.7.
Select Create SQL warehouse.
In the Name field, enter uniskai_wh. Uniskai looks for this exact name, so any other name is not used.
Set Cluster size to 2X-Small. Leave Auto stop, Scaling and Advanced options at their defaults.
Select Create.
In the SQL warehouses list, open the three-dot menu on the right of the uniskai_wh row and select Permissions.
In the search field, enter the service principal name and select it.
Set the permission level to Can use and select Add.
5. Add the workspace host and run the GRANT commands:
Open the workspace in your browser and copy the host from the address bar, for example dbc-1234abcd-5678.cloud.databricks.com.
Paste it into the Workspace host field in Uniskai. Remove the https:// prefix, any trailing slash, and the ?o= workspace id parameter that the browser adds.
Select the copy icon on the right of the GRANT commands block.
In the Databricks workspace, open SQL Editor, paste the commands and select Run. If the warehouse is stopped, confirm Start, attach and run.
All three statements complete with OK.
6. Connect the account:
Return to Uniskai and select Connect account.
A success notification confirms the connection, and the account appears in Account manager.
View Databricks costs
Costs for the current month appear in Costs analysi right after the account is connected. Historical data is collected separately and can take up to a day to show up in full.
Open Cloudcosts and select Costs analysis.
Open the Overview dashboard
Set Time range to the period you want to review.
In Provider, select Databricks.
Clear the is not filters on Usage Account, Service Category and Service Name if they exclude the data you need.
Select Apply filters.
If the data does not look up to date, select Refresh to reload it.
To build a chart of Databricks spend
Open the Charts tab and create a chart on billing.focus_distributed. (to confirm: the dataset used for Databricks)
Set the metric to SUM(BilledCost).
Add the filter ServiceProviderName IN ('Databricks'). (to confirm: exact provider value stored for Databricks)
To break the cost down by service, set the X-axis (dimension) to ServiceName.
The chart then shows Databricks billed cost and can be added to a dashboard.
Results
The connected account appears in Account manager under the Databricks tab, with Access Read only, Connection Access key, and Status OK. Use Details to view the connection, or the chart icon in Actions to open a cost chart.
After the daily collection has run, Databricks data is available in Costs analysis:
Cost by Provider includes Databricks.
Top Services by Cost includes your Databricks services.
Monthly Total Spend shows Databricks spend over the selected period.
Provider is available as a filter value across Costs analysis modules.
Key notes / limitations
The SQL warehouse must be named uniskai_wh, and the service principal needs Can Use on it.
Run the GRANT commands before selecting Connect account.
One connection per workspace, regardless of which service principal is used.
The secret is shown once. When it expires or is deleted, the daily collection stops updating costs. Generate a new one and enter it through Details, Actions, Update credentials.
The connection is read-only and can be cancelled at any time. Removing it in Uniskai does not delete the objects created in Databricks.
Historical costs are collected gradually. Costs for the current month are available straight after the connection, while earlier periods can take up to a day to fill in.