Configure Azure Databricks (Service Principal Access)

Step 1: Configure an app in Azure portal

Register an application with the Microsoft Entra ID endpoint in the Azure portal. Alternatively, you can use a Microsoft Entra ID app that is already registered. 

1. Sign in to the Azure portal. 
2. If you have access to multiple tenants, subscriptions, or directories, click the Directories + subscriptions (directory with filter) icon in the top menu to switch to the directory in which you want to register the application. 
3. Search for and select Microsoft Entra ID. 
4. Within Manage, select App registrations > New registration. 
5. For Name, enter a name for the application. 
6. In the Supported account types section, select Accounts in this organizational directory only (Single tenant). 
7. In the Redirect URL (optional) leave empty 
8. Click Register. 
9. On the application page’s Overview page, in the Essentials section, copy the following values: 
   • Application (client) ID 
   • Directory (tenant) ID 
   
10. Add AzureDatabricks to the required permissions of the registered application. You must be an admin user to perform this step. If you encounter a permissions-related issue while you perform this action, contact your administrator for help. 
    1. On the application page’s Overview page, on the Get Started tab, click View API permissions.
    2. Click Add a permission.
11. In the Request API permissions pane, click the APIs my organization uses tab, search for AzureDatabricks, and then select it.
12. Enable the user_impersonation check box, and then click Add permissions.
13. Click Grant admin consent for ### and then Yes. To perform this action, you must be an admin user or have the privilege to grant consent to the application. 

Step 2: Add the Microsoft Entra ID service principal to your Azure Databricks account 

This steps works only if your target Azure Databricks workspace is enabled for identity federation. If your workspace is not enabled for identity federation, skip ahead to Step 3. 

1. In your Azure Databricks workspace, click your username in the top bar and click Manage account. 
   Alternatively, go directly to your Azure Databricks account console at https://accounts.azuredatabricks.net. 
2. Sign in to your Azure Databricks account, if prompted. 
3. On the sidebar, click User management. 
4. Click the Service principals tab. 
5. Click Add service principal. 
6. Enter a Name for the Microsoft Entra ID service principal. 
7. For UUID, enter the Application (client) ID value from Step 1. 
8. Click Add. Your Microsoft Entra ID service principal is added as an Azure Databricks service principal in your Azure Databricks account.  

Step 3: Add the Microsoft Entra ID service principal to your Azure Databricks workspace 

If your workspace is enabled for identity federation: 

1. In your Azure Databricks workspace, click your username in the top bar and click Admin Settings. 
2. Click on the Identity and access tab. 
3. Next to Service principals, click Manage. 
4. Click Add service principal. 
5. Select your Microsoft Entra ID service principal from Step 2 and click Add. Your Microsoft Entra ID service principal is added as an Azure Databricks service principal in your Azure Databricks workspace. 

Skip ahead to Step 4. 

If your workspace is not enabled for identity federation: 

1. In your Azure Databricks workspace, click your username in the top bar and click Admin Settings. 
2. Click on the Identity and access tab. 
3. Next to Service principals, click Manage. 
4. Click Add service principal. 
5. Click Add new. 
6. For ApplicationId, enter the Application (client) ID for your Azure service principal from Step 1. 
7. Enter some Display Name for the new service principal and click Add. Your Microsoft Entra ID service principal is added as an Azure Databricks service principal in your Azure Databricks workspace 

Step 4: Assign workspace-level permissions to the service principal 

1. If the admin console for your workspace is not already opened, click your username in the top bar and click Admin Settings. 
2. Click on the Identity and access tab. 
3. Next to Service principals, click Manage. 
4. Click the name of your service principal to open its settings page. 
5. On the Configurations tab, check the box next to each entitlement that you want your service principal to have for this workspace, and then click Update. Check following check boxes 
   1. Active 
   2. Databricks SQL Access 
   3. Workspace access 
6. On the Permissions tab, grant access to any Azure Databricks users, service principals, and groups that you want to manage and use this service principal. 

Step 5: Enable Personal access token for Service principal 

1. Log into your Databricks workspace 
2. If the admin console for your workspace is not already opened, click your username in the top bar and click Admin Settings. 
3. Click on the Advanced tab. 
4. In Access control 
   1. Enable personal access token 
   2. Click on permission settings 
      1. Search for your service principal name 
         1. Select permission "Can Use" 
      2. Click Add 

Step 6: Enable Hive metadata access for Service principal 

1. Log into your Databricks workspace 
2. Click on the Catalog tab in left menu.
3. Click on catalog that was set earlier.

1. 1. Open permissions tab 
   2. Click on Grant 
      1. Search for your service principal name 
         1. Check the privileges you would like to grant 
            1. Select "All" 
      2. Click Grant 

 

Step 7: Enable cluster access for Service principal 

• To configure the Compute cluster permissions:

1. Navigate to the Permissions section.
2. Select the relevant Service Principal (SP) name.
3. Set the permission to Can Attach To.

• To configure the SQL Warehouse permissions:

1. Navigate to the Permissions
2. Assign the permission level Can Use

 

 

Additional References: 

• https://learn.microsoft.com/en-us/azure/databricks/dev-tools/app-aad-token
• https://learn.microsoft.com/en-us/azure/databricks/dev-tools/service-principals