March 2018

  • Article

Releases are staged. Your Azure Databricks account may not be updated until a week after the initial release date.

Command execution details

March 27 - April 3, 2018: Version 2.68

When you run a command in a notebook, you will now see detailed progress information.

Databricks CLI supports --profile

March 27 - April 3, 2018: Version 2.68

Databricks CLI 0.6.1 supports --profile in all positions.

See Databricks CLI (legacy).

ACLs enabled by default for new Premium SKU customers

March 27 - April 3, 2018: Version 2.68

Access control lists (ACLs) are now enabled by default for all new customers in the Premium SKU. Existing customers must continue to enable ACLs manually.

See Access controls lists can no longer be disabled.

Azure Databricks is now generally available

March 22, 2018

We’re pleased to announce that Azure Databricks is now generally available. In the past weeks, we have added functionality to help make the Azure Databricks experience even better, including:

And of course Azure Databricks continues to provide easy integrations with Azure Blob storage, Azure Data Lake Store, and Azure Cosmos DB.

As a supplement to the documentation provided on this site, learn.microsoft.com provides introductory material, information about Azure account management, and end-to-end tutorials.

New doc site theme

March 21, 2018

We’ve updated the look and feel of our documentation site. We hope you like it!

Autoscaling local storage

March 13-20, 2018: Version 2.67

All clusters on Azure Databricks launch with autoscaling local storage enabled. This means that Azure Databricks automatically attaches additional managed disks to cluster worker VMs whenever they run low on disk.

See Enable autoscaling local storage for more information.

Virtual network (VNet) peering

March 13-20, 2018: Version 2.67

Adds support for virtual network (VNet) peering, which allows the virtual network in which your Azure Databricks resource is running to peer with another Azure virtual network.

See Peer virtual networks for more information.

Cluster event log

Mar 13-20, 2018: Version 2.67

The cluster details page has a new Event Log tab that displays important cluster life cycle events. Historical events can be viewed for 60 days, which is comparable with other data retention times in Azure Databricks.

See Compute event logs for more information.

Databricks CLI: 0.6.0 release

Mar 13, 2018: databricks-cli 0.6.0

Databricks CLI now supports Python 3.

See Databricks CLI (legacy) for more information.

Job run management

Mar 13-20, 2018: Version 2.67

You can now delete a job run in the job details page and the job run page.

The job run Get Output endpoint is GA, and the maximum output returned has been increased to 5 MB.

Edit cluster permissions now requires edit mode

Mar 13-20, 2018: Version 2.67

Previously it was possible to edit a cluster’s permissions without clicking Edit, which was inconsistent with other cluster attributes.

A side effect of this change is that you can no longer edit cluster permissions while a cluster is pending.

Databricks ML Model Export

March 1, 2018

The documentation now covers how to use Databricks ML Model Export, which allows you to export models and full ML pipelines from Apache Spark. These exported models and pipelines can be imported into other (Spark and non-Spark) platforms to do scoring and make predictions. Model Export is targeted at low-latency, lightweight ML-powered applications.

Note

This feature requires Databricks Runtime 4.0+.

See MLeap ML model export for more information.