程式庫 APILibraries API

程式庫 API 可讓您安裝和卸載程式庫,並取得叢集中的程式庫狀態。The Libraries API allows you to install and uninstall libraries and get the status of libraries on a cluster.

重要

若要存取 Databricks REST API,您必須驗證To access Databricks REST APIs, you must authenticate.

所有叢集狀態 All cluster statuses

端點Endpoint HTTP 方法HTTP Method
2.0/libraries/all-cluster-statuses GET

取得所有叢集上所有程式庫的狀態。Get the status of all libraries on all clusters. 您可以透過 API 或程式庫 UI,以及透過程式庫 UI,將程式庫設定為安裝在所有叢集上的所有程式庫,都可以使用狀態。A status will be available for all libraries installed on clusters via the API or the libraries UI as well as libraries set to be installed on all clusters via the libraries UI. 如果程式庫已設定為要安裝在所有叢集上, is_library_for_all_clusters 則會是 true ,即使該程式庫也安裝在此特定叢集上也一樣。If a library has been set to be installed on all clusters, is_library_for_all_clusters will be true, even if the library was also installed on this specific cluster.

範例回應Example response

{
  "statuses": [
    {
      "cluster_id": "11203-my-cluster",
      "library_statuses": [
        {
          "library": {
            "jar": "dbfs:/mnt/libraries/library.jar"
          },
          "status": "INSTALLING",
          "messages": [],
          "is_library_for_all_clusters": false
        }
      ]
    },
    {
      "cluster_id": "20131-my-other-cluster",
      "library_statuses": [
        {
          "library": {
            "egg": "dbfs:/mnt/libraries/library.egg"
          },
          "status": "ERROR",
          "messages": ["Could not download library"],
          "is_library_for_all_clusters": false
        }
      ]
    }
  ]
}

回應結構 Response structure

欄位名稱Field Name 類型Type 描述Description
狀態statuses ClusterLibraryStatuses的陣列An array of ClusterLibraryStatuses 叢集狀態的清單。A list of cluster statuses.

叢集狀態 Cluster status

端點Endpoint HTTP 方法HTTP Method
2.0/libraries/cluster-status GET

取得叢集中的程式庫狀態。Get the status of libraries on a cluster. 您可以透過 API 或程式庫 UI,以及透過程式庫 UI,將程式庫設定為安裝在所有叢集上的所有程式庫,都可以使用狀態。A status will be available for all libraries installed on the cluster via the API or the libraries UI as well as libraries set to be installed on all clusters via the libraries UI. 如果程式庫已設定為要安裝在所有叢集上, is_library_for_all_clusters 則會是 true ,即使該程式庫也安裝在叢集上也一樣。If a library has been set to be installed on all clusters, is_library_for_all_clusters will be true, even if the library was also installed on the cluster.

範例要求Example request

/libraries/cluster-status?cluster_id=11203-my-cluster

範例回應Example response

{
  "cluster_id": "11203-my-cluster",
  "library_statuses": [
    {
      "library": {
        "jar": "dbfs:/mnt/libraries/library.jar"
      },
      "status": "INSTALLED",
      "messages": [],
      "is_library_for_all_clusters": false
    },
    {
      "library": {
        "pypi": {
          "package": "beautifulsoup4"
        },
      },
      "status": "INSTALLING",
      "messages": ["Successfully resolved package from PyPI"],
      "is_library_for_all_clusters": false
    },
    {
      "library": {
        "cran": {
          "package": "ada",
          "repo": "https://cran.us.r-project.org"
        },
      },
      "status": "FAILED",
      "messages": ["R package installation is not supported on this spark version.\nPlease upgrade to Runtime 3.2 or higher"],
      "is_library_for_all_clusters": false
    }
  ]
}

要求結構 Request structure

欄位名稱Field Name 類型Type 描述Description
cluster_idcluster_id STRING 應抓取其狀態之叢集的唯一識別碼。Unique identifier of the cluster whose status should be retrieved. 此為必要欄位。This field is required.

回應結構 Response structure

欄位名稱Field Name 類型Type 描述Description
cluster_idcluster_id STRING 叢集的唯一識別碼。Unique identifier for the cluster.
library_statuseslibrary_statuses LibraryFullStatus的陣列An array of LibraryFullStatus 叢集中所有程式庫的狀態。Status of all libraries on the cluster.

安裝 Install

端點Endpoint HTTP 方法HTTP Method
2.0/libraries/install POST

在叢集上安裝程式庫。Install libraries on a cluster. 安裝是非同步,它會在要求之後于背景完成。The installation is asynchronous - it completes in the background after the request.

重要

如果叢集已終止,此呼叫將會失敗。This call will fail if the cluster is terminated.

在叢集上安裝滾輪程式庫,就像是 pip 直接在驅動程式和執行程式上對輪子檔案執行命令。Installing a wheel library on a cluster is like running the pip command against the wheel file directly on driver and executors. 系統會安裝程式庫檔案中指定的所有相依性 setup.py ,而這需要程式庫名稱以滿足 滾輪檔案名慣例All the dependencies specified in the library setup.py file are installed and this requires the library name to satisfy the wheel file name convention.

只有在啟動新工作時,才會在執行程式上安裝。The installation on the executors happens only when a new task is launched. 使用 Databricks Runtime 7.1 和以下的程式庫安裝順序不具決定性。With Databricks Runtime 7.1 and below, the installation order of libraries is nondeterministic. 針對滾輪程式庫,您可以建立包含所有滾輪檔案的 zip 檔案,以確保具決定性的安裝順序 .wheelhouse.zipFor wheel libraries, you can ensure a deterministic installation order by creating a zip file with suffix .wheelhouse.zip that includes all the wheel files.

範例要求Example request

{
  "cluster_id": "10201-my-cluster",
  "libraries": [
    {
      "jar": "dbfs:/mnt/libraries/library.jar"
    },
    {
      "egg": "dbfs:/mnt/libraries/library.egg"
    },
    {
      "whl": "dbfs:/mnt/libraries/mlflow-0.0.1.dev0-py2-none-any.whl"
    },
    {
      "whl": "dbfs:/mnt/libraries/wheel-libraries.wheelhouse.zip"
    },
    {
      "maven": {
        "coordinates": "org.jsoup:jsoup:1.7.2",
        "exclusions": ["slf4j:slf4j"]
      }
    },
    {
      "pypi": {
        "package": "simplejson",
        "repo": "https://my-pypi-mirror.com"
      }
    },
    {
      "cran": {
        "package": "ada",
        "repo": "https://cran.us.r-project.org"
      }
    }
  ]
}

要求結構 Request structure

欄位名稱Field Name 類型Type 描述Description
cluster_idcluster_id STRING 要在其上安裝這些程式庫之叢集的唯一識別碼。Unique identifier for the cluster on which to install these libraries. 此為必要欄位。This field is required.
程式庫libraries 連結的陣列An array of Library 要安裝的程式庫。The libraries to install.

卸載 Uninstall

端點Endpoint HTTP 方法HTTP Method
2.0/libraries/uninstall POST

設定要在叢集中卸載的程式庫。Set libraries to be uninstalled on a cluster. 在重新開機叢集之前,不會卸載程式庫。The libraries aren’t uninstalled until the cluster is restarted. 卸載未安裝在叢集上的程式庫沒有任何影響,但不是錯誤。Uninstalling libraries that are not installed on the cluster has no impact but is not an error.

範例要求Example request

{
  "cluster_id": "10201-my-cluster",
  "libraries": [
    {
      "jar": "dbfs:/mnt/libraries/library.jar"
    },
    {
      "cran": "ada"
    }
  ]
}

要求結構 Request structure

欄位名稱Field Name 類型Type 描述Description
cluster_idcluster_id STRING 要卸載這些程式庫之叢集的唯一識別碼。Unique identifier for the cluster on which to uninstall these libraries. 此為必要欄位。This field is required.
程式庫libraries 連結的陣列An array of Library 要卸載的程式庫。The libraries to uninstall.

資料結構 Data structures

本節內容:In this section:

ClusterLibraryStatuses ClusterLibraryStatuses

欄位名稱Field Name 類型Type 描述Description
cluster_idcluster_id STRING 叢集的唯一識別碼。Unique identifier for the cluster.
library_statuseslibrary_statuses LibraryFullStatus的陣列An array of LibraryFullStatus 叢集中所有程式庫的狀態。Status of all libraries on the cluster.

程式庫 Library

欄位名稱Field Name 類型Type 描述Description
jar 或蛋或 .whl、pypi 或 maven 或 cranjar OR egg OR whl OR pypi OR maven OR cran STRING 或(或) STRING STRING PythonPyPiLibraryMavenLibraryRCranLibrarySTRING OR STRING OR STRING OR PythonPyPiLibrary OR MavenLibrary OR RCranLibrary 如果是 jar,則為要安裝之 jar 的 DBFS URI。If jar, DBFS URI of the jar to be installed. 例如:{ "jar": "dbfs:/mnt/databricks/library.jar" }For example: { "jar": "dbfs:/mnt/databricks/library.jar" }.

如果是蛋,請 DBFS 要安裝的蛋的 URI。If egg, DBFS URI of the egg to be installed. 例如:{ "egg": "dbfs:/my/egg" }For example: { "egg": "dbfs:/my/egg" }.

如果是 .whl,則為要安裝的輪子或 zipped 輪子的 URI。If whl, URI of the wheel or zipped wheels to be installed. 只支援 DBFS Uri。Only DBFS URIs are supported. 例如:{ "whl": "dbfs:/my/whl" }For example: { "whl": "dbfs:/my/whl" }.

滾輪檔案名必須使用 正確的慣例The wheel file name needs to use the correct convention. 如果要安裝壓縮輪子,檔案名尾碼應該是 .wheelhouse.zipIf zipped wheels are to be installed, the file name suffix should be .wheelhouse.zip.

如果 pypi,則為要安裝的 PyPI 程式庫規格。If pypi, specification of a PyPI library to be installed. 例如:For example:
{ "package": "simplejson" }

如果 maven,則為要安裝的 Maven 程式庫規格。If maven, specification of a Maven library to be installed. 例如:For example:
{ "coordinates": "org.jsoup:jsoup:1.7.2" }

如果 cran,則為要安裝的 CRAN 程式庫規格。If cran, specification of a CRAN library to be installed.

LibraryFullStatus LibraryFullStatus

特定叢集上程式庫的狀態。The status of the library on a specific cluster.

欄位名稱Field Name 類型Type 描述Description
圖書館library 程式庫Library 程式庫的唯一識別碼。Unique identifier for the library.
statusstatus LibraryInstallStatusLibraryInstallStatus 在叢集中安裝程式庫的狀態。Status of installing the library on the cluster.
messagesmessages STRING 的陣列。An array of STRING 到目前為止針對此文件庫所發生的所有資訊和警告訊息。All the info and warning messages that have occurred so far for this library.
is_library_for_all_clustersis_library_for_all_clusters BOOL 程式庫是否已設定為透過程式庫 UI 在所有叢集上安裝。Whether the library was set to be installed on all clusters via the libraries UI.

MavenLibrary MavenLibrary

欄位名稱Field Name 類型Type 描述Description
座標coordinates STRING Gradle 樣式的 Maven 座標。Gradle-style Maven coordinates. 例如:org.jsoup:jsoup:1.7.2For example: org.jsoup:jsoup:1.7.2. 此為必要欄位。This field is required.
回購repo STRING 要從中安裝 Maven 套件的 Maven 存放庫。Maven repo to install the Maven package from. 如果省略,則會搜尋 Maven 中央存放庫和 Spark 套件。If omitted, both Maven Central Repository and Spark Packages are searched.
排除項exclusions STRING 的陣列。An array of STRING 要排除的相依性清單。List of dependences to exclude. 例如:["slf4j:slf4j", "*:hadoop-client"]For example: ["slf4j:slf4j", "*:hadoop-client"].

Maven 相依性排除項: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.htmlMaven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.

PythonPyPiLibrary PythonPyPiLibrary

欄位名稱Field Name 類型Type 描述Description
套件package STRING 要安裝之 PyPI 封裝的名稱。The name of the PyPI package to install. 也支援選擇性的確切版本規格。An optional exact version specification is also supported. 範例:simplejsonsimplejson==3.8.0Examples: simplejson and simplejson==3.8.0. 此為必要欄位。This field is required.
回購repo STRING 可以找到封裝的存放庫。The repository where the package can be found. 如果未指定,則會使用預設 pip 索引。If not specified, the default pip index is used.

RCranLibrary RCranLibrary

欄位名稱Field Name 類型Type 描述Description
套件package STRING 要安裝之 CRAN 封裝的名稱。The name of the CRAN package to install. 此為必要欄位。This field is required.
回購repo STRING 可以找到封裝的存放庫。The repository where the package can be found. 如果未指定,則會使用預設的 CRAN 存放庫。If not specified, the default CRAN repo is used.

LibraryInstallStatus LibraryInstallStatus

特定叢集上程式庫的狀態。The status of a library on a specific cluster.

狀態Status 描述Description
PENDINGPENDING 尚未採取任何動作來安裝程式庫。No action has yet been taken to install the library. 此狀態的存留期應該非常短。This state should be very short lived.
RESOLVINGRESOLVING 從提供的存放庫抓取安裝程式庫所需的中繼資料。Metadata necessary to install the library is being retrieved from the provided repository.

針對 Jar、蛋和 .Whl 程式庫,此步驟不會有任何作用。For Jar, Egg, and Whl libraries, this step is a no-op.
安裝INSTALLING 您可以藉由將資源新增至 Spark,或在 Spark 節點內執行系統命令,來主動安裝程式庫。The library is actively being installed, either by adding resources to Spark or executing system commands inside the Spark nodes.
安裝INSTALLED 已成功安裝程式庫。The library has been successfully installed.
SKIPPEDSKIPPED 因為 Scala 版本不相容,所以已略過在 Databricks Runtime 7.0 或更新版本的叢集上安裝。Installation on a Databricks Runtime 7.0 or above cluster was skipped due to Scala version incompatibility.
FAILEDFAILED 安裝的某個步驟失敗。Some step in installation failed. 您可以在 [訊息] 欄位中找到詳細資訊。More information can be found in the messages field.
UNINSTALL_ON_RESTARTUNINSTALL_ON_RESTART 程式庫已標示為要移除。The library has been marked for removal. 只有在重新開機叢集時,才能移除程式庫,因此進入此狀態的程式庫會保留到叢集重新開機為止。Libraries can be removed only when clusters are restarted, so libraries that enter this state will remain until the cluster is restarted.