Machine Learning 的 Databricks Runtime Databricks Runtime for Machine Learning

Databricks Runtime Machine Learning (Databricks Runtime ML) 會自動建立針對機器學習優化的叢集。Databricks Runtime for Machine Learning (Databricks Runtime ML) automates the creation of a cluster optimized for machine learning. Databricks Runtime ML 叢集包含最熱門的機器學習程式庫(例如 TensorFlow、PyTorch、Keras 和 XGBoost),也包含分散式訓練所需的程式庫,例如 Horovod。Databricks Runtime ML clusters include the most popular machine learning libraries, such as TensorFlow, PyTorch, Keras, and XGBoost, and also include libraries required for distributed training such as Horovod. 使用 Databricks Runtime ML 可加速叢集的建立,並確保已安裝的程式庫版本相容。Using Databricks Runtime ML speeds up cluster creation and ensures that the installed library versions are compatible.

如需使用 Azure Databricks 進行機器學習和深度學習的完整資訊,請參閱 機器學習和深度學習指南For complete information about using Azure Databricks for machine learning and deep learning, see Machine learning and deep learning guide.

如需每個 Databricks Runtime ML 版本內容的相關資訊,請參閱 版本資訊。For information about the contents of each Databricks Runtime ML version, see the release notes.

Databricks Runtime ML 建置於 Databricks Runtime 上。Databricks Runtime ML is built on Databricks Runtime. 例如,Machine Learning 的 Databricks Runtime 7.3 LTS 建基於 Databricks Runtime 7.3 LTS。For example, Databricks Runtime 7.3 LTS for Machine Learning is built on Databricks Runtime 7.3 LTS. 基底 Databricks Runtime 中包含的程式庫會列在 Databricks Runtime 版本資訊中。The libraries included in the base Databricks Runtime are listed in the Databricks Runtime release notes.

Machine Learning 的 Databricks Runtime 簡介 Introduction to Databricks Runtime for Machine Learning

本教學課程是專為 Databricks Runtime ML 的新使用者所設計。This tutorial is designed for new users of Databricks Runtime ML. 大約需要10分鐘的時間才能完成,並顯示載入表格式資料、定型模型、分散式超參數微調和模型推斷的完整端對端範例。It takes about 10 minutes to work through, and shows a complete end-to-end example of loading tabular data, training a model, distributed hyperparameter tuning, and model inference. 它也會說明如何使用 MLflow API 和 MLflow 模型登錄。It also illustrates how to use the MLflow API and MLflow Model Registry.

Databricks 教學課程筆記本Databricks tutorial notebook

取得筆記本Get notebook

Databricks Runtime ML 中包含的程式庫 Libraries included in Databricks Runtime ML

注意

Databricks Runtime ML 中不提供連結庫公用程式Library utilities are not available in Databricks Runtime ML.

Databricks Runtime ML 包含各種熱門的 ML 程式庫。The Databricks Runtime ML includes a variety of popular ML libraries. 程式庫會隨著每個版本更新,以包含新的功能和修正程式。The libraries are updated with each release to include new features and fixes.

Azure Databricks 已將支援的程式庫子集指定為最上層程式庫。Azure Databricks has designated a subset of the supported libraries as top-tier libraries. 針對這些程式庫,Azure Databricks 可提供更快速的更新頻率,並隨著每個執行階段版本更新為最新的套件版本, (將相依性衝突) 。For these libraries, Azure Databricks provides a faster update cadence, updating to the latest package releases with each runtime release (barring dependency conflicts). Azure Databricks 也提供最上層程式庫的先進支援、測試和內嵌優化。Azure Databricks also provides advanced support, testing, and embedded optimizations for top-tier libraries.

如需最上層和其他所提供程式庫的完整清單,請參閱下列文章,以瞭解每個可用的執行時間:For a full list of top-tier and other provided libraries, see the following articles for each available runtime:

如何使用 Databricks Runtime MLHow to use Databricks Runtime ML

除了預先安裝的程式庫,Databricks Runtime ML 與叢集設定中的 Databricks Runtime 和管理 Python 套件的方式不同。In addition to the pre-installed libraries, Databricks Runtime ML differs from Databricks Runtime in the cluster configuration and in how you manage Python packages.

使用 Databricks Runtime ML 建立叢集Create a cluster using Databricks Runtime ML

當您 建立叢集時,請從 Databricks Runtime 版本下拉式清單中選取 Databricks Runtime ML 版本。When you create a cluster, select a Databricks Runtime ML version from the Databricks Runtime Version drop-down. 支援 CPU 和已啟用 GPU 的 ML 執行時間。Both CPU and GPU-enabled ML runtimes are available.

選取 Databricks Runtime MLSelect Databricks Runtime ML

如果您選取已啟用 GPU 的 ML 執行時間,系統會提示您選取相容的 驅動程式類型 和背景 工作角色類型If you select a GPU-enabled ML runtime, you are prompted to select a compatible Driver Type and Worker Type. 下拉式清單中不相容的實例類型會呈現灰色。Incompatible instance types are grayed out in the drop-downs. 已啟用 GPU 的實例類型會列在 Gpu 加速 標籤底下。GPU-enabled instance types are listed under the GPU-Accelerated label.

警告

您工作區中 自動安裝到所有 叢集的程式庫,可能會與 Databricks Runtime ML 中包含的程式庫發生衝突。Libraries in your workspace that automatically install into all clusters can conflict with the libraries included in Databricks Runtime ML. 使用 Databricks Runtime ML 建立叢集之前,請清除 [ 在所有叢集上自動安裝 ] 核取方塊。Before you create a cluster with Databricks Runtime ML, clear the Install automatically on all clusters checkbox for conflicting libraries.

管理 Python 套件 Manage Python packages

在 Databricks Runtime ML 中, Conda 套件管理員是用來安裝 Python 套件。In Databricks Runtime ML the Conda package manager is used to install Python packages. 所有 Python 套件都安裝在單一環境內: /databricks/python2 使用 python 2 的叢集,以及 /databricks/python3 使用 python 3 的叢集。All Python packages are installed inside a single environment: /databricks/python2 on clusters using Python 2 and /databricks/python3 on clusters using Python 3. 不支援切換 (或啟用) Conda 環境。Switching (or activating) Conda environments is not supported.

如需管理 Python 程式庫的詳細資訊,請參閱連結 For information on managing Python libraries, see Libraries.

AutoML 支援AutoML support

Databricks Runtime ML 包含可將模型開發程式自動化的工具,並可協助您有效率地找出效能最佳的模型。Databricks Runtime ML includes tools to automate the model development process and help you efficiently find the best performing model.

  • Managed MLFlow 可管理端對端模型生命週期,包括追蹤實驗執行、部署和共用模型,以及維護集中式模型登錄。Managed MLFlow manages the end-to-end model lifecycle, including tracking experimental runs, deploying and sharing models, and maintaining a centralized model registry.
  • Hyperopt,透過類別增強 SparkTrials ,可將 ML 模型參數調整自動化和散發。Hyperopt, augmented with the SparkTrials class, automates and distributes ML model parameter tuning.