Databricks Compute Cost Overview

by Tableau

Launch Demo

Description

Unlock the full potential of your Databricks investment with our Tableau-powered dashboard, designed to provide data teams with a comprehensive, easy-to-understand visualisation of their Databricks spending. This intuitive Accelerator delivers real-time insights and granular analytics, enabling informed decision-making and strategic cost management. With secure data integration and customisable features, it's the perfect solution for maintaining financial control and driving efficiency in your Databricks environment.

Answer Key Business Questions

  • What is my quarterly and monthly spend in $USD and DBUs (Databricks Unit)?
  • Which of my workspaces are driving the most consumption?
  • What time of day is driving the most consumption of my Databricks resources?

Monitoring and Improving KPIs

  • Quarterly Databricks account spend
  • Monthly Databricks account spend
  • Cost per SKU
  • Cost by Day of Week & Time

Connect to your Databricks data to enable this Accelerator

Databricks prerequisite

Tableau configuration

  • You will need to use Tableau Desktop to replace the sample data with an extract of your own data, but you can either connect to your own data in Tableau Server/Cloud or directly in Desktop.

Option 1: Create a published data source in Tableau Server or Tableau Cloud

  1. In the Explore tab, select the New drop-down and select Data Source.
  2. Select Databricks as your connection
  3. Enter your credentials and create a relationship between the system.billing.usage table and the system.billing.list_prices table.
  4. Edit the relationship noodle to match the fields shown in the screenshot.
    1. For the Usage Start Time equivalent in list_prices, click the drop-down menu and select Create relationship calculation. Use the syntax from the screenshot – IFNULL([Price End Time], DATE("1/1/3000")).
  5. Publish this data source as an extract on your site.
  6. Open the accelerator workbook in Tableau Desktop.
  7. Create a new data source by clicking on the top menu Data > New Data Source and select Tableau Server as your data source.
  8. Connect to the new extract you generated.
  9. In the top menu, select Data > Replace Data Source and replace the sample data with the new source you created.

Option 2: Create your Databricks connection locally in Tableau Desktop

  1. Open the accelerator workbook in Tableau Desktop.
  2. Create a new data source by clicking on the top menu Data > New Data Source.
  3. Select Databricks as the data source. Install the driver if you haven't done so previously.
  4. Enter your credentials and create a relationship between the system.billing.usage table and the system.billing.list_prices table.
  5. Edit the relationship noodle to match the fields shown in the screenshot. For the Usage Start Time equivalent in list_prices, click the drop-down menu and select Create relationship calculation. Use the syntax from the screenshot – IFNULL([Price End Time], DATE("1/1/3000")).
  6. Select the Extract radio button on the Data Source page
  7. In the top menu, select Data > Replace Data Source and replace the sample data with the new data source you just generated.