Databricks Runtime 8.1 pour ML (non pris en charge)

Article
03/01/2024

Databricks a publié cette image en mars 2021.

Databricks Runtime 8.1 for Machine Learning fournit un environnement prêt à l'emploi pour l'apprentissage automatique et la science des données basé sur Databricks Runtime 8.1 (non pris en charge). Databricks Runtime ML contient de nombreuses bibliothèques populaires de Machine Learning, notamment TensorFlow, PyTorch et XGBoost. Il prend également en charge la formation de Deep Learning distribué avec Horovod.

Pour plus d’informations, notamment les instructions relatives à la création d’un cluster Databricks Runtime ML, consultez IA et Machine Learning sur Databricks.

Nouvelles fonctionnalités et modifications majeures

Databricks Runtime 8.1 ML est basé sur Databricks Runtime 8.1. Pour plus d'informations sur les nouveautés de Databricks Runtime 8.1, y compris Apache Spark MLlib et SparkR, consultez les notes de publication de Databricks Runtime 8.1 (non pris en charge).

Packages supprimés dans les clusters GPU

Les packages CUDA suivants sont supprimés dans les clusters GPU :

cuda-command-line-tools
cuda-compiler
cuda-cudart-dev
cuda-cufft
cuda-cufft-dev
cuda-cuobjdump
cuda-cupti
cuda-curand
cuda-curand-dev
cuda-cusolver
cuda-cusolver-dev
cuda-cusparse
cuda-cusparse-dev
cuda-documentation
cuda-driver-dev
cuda-gdb
cuda-gpu-library-advisor
cuda-libraries-dev
cuda-license
cuda-memcheck
cuda-minimal-build
cuda-misc-headers
cuda-npp
cuda-npp-dev
cuda-nsight
cuda-nvcc
cuda-nvdisasm
cuda-nvgraph
cuda-nvgraph-dev
cuda-nvjpeg
cuda-nvjpeg-dev
cuda-nvml-dev
cuda-nvprune
cuda-nvrtc-dev
cuda-nvvp
cuda-samples
cuda-sanitizer-api
cuda-toolkit
cuda-tools
cuda-visual-tools
freeglut3
libcublas-dev
libcudnn7-dev
libdrm-dev
libegl1
libegm-mesa0
libgbl1-mesa-dev
libgbm1
libgles1
libgles2
libglu1-mesa
libglu1-mesa-dev
libnccl-dev
libnvinfer-dev
libnvinfer-plugin-dev
libopengl0
libwayland-server0
libx11-xcb-dev
libxcb-dri2-0-dev
libxcb-dri3-dev
libxcb-glx0-dev
libxcb-present-dev
libxcb-randr0
libxcb-randr0-dev
libxcb-render0-dev
libxcb-shape0-dev
libxcb-sync-dev
libxcb-xfixes0
libxcb-xfixes0-dev
libxdamage-dev
libxext-dev
libxfixes-dev
libxi-dev
libxmu-dev
libxmu-headers
libxshmfence-dev
libxxf86vm-dev
mesa-common-dev
nsight-compute
nsight-systems
x11proto-damage-dev
x11proto-fixes-dev
x11proto-input-dev
x11proto-xext-dev
x11proto-xf86vidmode-dev

Modifications majeures apportées à l’environnement Python de Databricks Runtime ML

Consultez Databricks Runtime 8.1 (non pris en charge) pour connaître les modifications majeures apportées à l’environnement Databricks Runtime Python. Pour obtenir la liste complète des packages Python installés et leurs versions, consultez Bibliothèques Python.

Mise à niveau des packages Python

mlflow 1.13.1 -> 1.14.1
plotly 4.14.1 -> 4.14.3
pytz 2020.1 -> 2020.5
shap 0.37.0 -> 0.38.1
tensorflow 2.4.0 -> 2.4.1
torchvision 0.8.1 -> 0.8.2
xgboost 1.3.1 -> 1.3.3

Environnement du système

L’environnement système de Databricks Runtime 8.1 ML diffère de Databricks Runtime 8.1 comme suit :

DBUtils : Databricks Runtime ML n’inclut pas l’utilitaire de bibliothèque (dbutils.library) (hérité). Utilisez plutôt les commandes %pip et %conda. Consultez Bibliothèques Python délimitées à un notebook.
Pour les clusters GPU, Databricks Runtime ML inclut les bibliothèques GPU NVIDIA suivantes :
- CUDA==11.0
- cuDNN 8.0.4.30
- NCCL 2.7.8
- TensorRT 7.1.3

Bibliothèques

Les sections suivantes listent les bibliothèques incluses dans Databricks Runtime ML 8.1 qui diffèrent de celles incluses dans Databricks Runtime 8.1.

Dans cette section :

Bibliothèques de niveau supérieur
Bibliothèques Python
Bibliothèques R
Bibliothèques Java et Scala (cluster Scala 2.12)

Bibliothèques de niveau supérieur

Databricks Runtime 8.1 ML comprend les bibliothèques de niveau supérieur suivantes :

Bibliothèques Python

Databricks Runtime 8.1 ML utilise Conda pour la gestion des packages Python et comprend de nombreux packages ML populaires.

En plus des packages spécifiés dans les environnements Conda dans les sections suivantes, Databricks Runtime 8.1 ML comprend également les packages suivants :

hyperopt 0.2.5.db1
sparkdl 2.1.0.db4

Bibliothèques Python sur les clusters UC

name: databricks-ml
channels:
  - pytorch
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - absl-py=0.11.0=pyhd3eb1b0_1
  - aiohttp=3.7.4=py38h27cfd23_1
  - asn1crypto=1.4.0=py_0
  - astor=0.8.1=py38h06a4308_0
  - async-timeout=3.0.1=py38h06a4308_0
  - attrs=20.3.0=pyhd3eb1b0_0
  - backcall=0.2.0=pyhd3eb1b0_0
  - bcrypt=3.2.0=py38h7b6447c_0
  - blas=1.0=mkl
  - blinker=1.4=py38h06a4308_0
  - boto3=1.16.7=pyhd3eb1b0_0
  - botocore=1.19.7=pyhd3eb1b0_0
  - brotlipy=0.7.0=py38h27cfd23_1003
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2021.4.13=h06a4308_1 # (updated from 2021.1.19 in May 26, 2021 maintenance update)
  - cachetools=4.2.1=pyhd3eb1b0_0
  - certifi=2020.12.5=py38h06a4308_0
  - cffi=1.14.3=py38h261ae71_2
  - chardet=3.0.4=py38h06a4308_1003
  - click=7.1.2=pyhd3eb1b0_0
  - cloudpickle=1.6.0=py_0
  - configparser=5.0.1=py_0
  - cpuonly=1.0=0
  - cryptography=3.1.1=py38h1ba5d50_0
  - cycler=0.10.0=py38_0
  - cython=0.29.21=py38h2531618_0
  - decorator=4.4.2=pyhd3eb1b0_0
  - dill=0.3.2=py_0
  - docutils=0.15.2=py38h06a4308_1
  - entrypoints=0.3=py38_0
  - flask=1.1.2=pyhd3eb1b0_0
  - freetype=2.10.4=h5ab3b9f_0
  - future=0.18.2=py38_1
  - gitdb=4.0.5=py_0
  - gitpython=3.1.12=pyhd3eb1b0_1
  - google-auth=1.22.1=py_0
  - google-auth-oauthlib=0.4.2=pyhd3eb1b0_2
  - google-pasta=0.2.0=py_0
  - gunicorn=20.0.4=py38_0
  - h5py=2.10.0=py38h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - icu=58.2=he6710b0_3
  - idna=2.10=pyhd3eb1b0_0
  - importlib-metadata=2.0.0=py_1
  - intel-openmp=2019.4=243
  - ipykernel=5.3.4=py38h5ca1d4c_0
  - ipython=7.19.0=py38hb070fc8_1
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - isodate=0.6.0=py_1
  - itsdangerous=1.1.0=pyhd3eb1b0_0
  - jedi=0.17.2=py38h06a4308_1
  - jinja2=2.11.2=pyhd3eb1b0_0
  - jmespath=0.10.0=py_0
  - joblib=0.17.0=py_0
  - jpeg=9b=h024ee3a_2
  - jupyter_client=6.1.7=py_0
  - jupyter_core=4.6.3=py38_0
  - kiwisolver=1.3.0=py38h2531618_0
  - krb5=1.17.1=h173b8e3_0
  - lcms2=2.11=h396b838_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20191231=h14c3975_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libpq=12.2=h20c2e04_0
  - libprotobuf=3.13.0.1=hd408876_0
  - libsodium=1.0.18=h7b6447c_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_1
  - libuv=1.40.0=h7b6447c_0
  - lightgbm=3.1.1=py38h2531618_0
  - lz4-c=1.9.2=heb0550a_3
  - mako=1.1.3=py_0
  - markdown=3.3.3=py38h06a4308_0
  - markupsafe=1.1.1=py38h7b6447c_0
  - matplotlib-base=3.2.2=py38hef1b27d_0
  - mkl=2019.4=243
  - mkl-service=2.3.0=py38he904b0f_0
  - mkl_fft=1.2.0=py38h23d657b_0
  - mkl_random=1.1.0=py38h962f231_0
  - more-itertools=8.6.0=pyhd3eb1b0_0
  - multidict=5.1.0=py38h27cfd23_2
  - ncurses=6.2=he6710b0_1
  - networkx=2.5=py_0
  - ninja=1.10.2=py38hff7bd54_0
  - nltk=3.5=py_0
  - numpy=1.19.2=py38h54aff64_0
  - numpy-base=1.19.2=py38hfa32c7d_0
  - oauthlib=3.1.0=py_0
  - olefile=0.46=py_0
  - openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1j in May 26, 2021 maintenance update)
  - packaging=20.4=py_0
  - pandas=1.1.3=py38he6710b0_0
  - paramiko=2.7.2=py_0
  - parso=0.7.0=py_0
  - patsy=0.5.1=py38_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=8.0.1=py38he98fc37_0
  - pip=20.2.4=py38h06a4308_0
  - plotly=4.14.3=pyhd3eb1b0_0
  - prompt-toolkit=3.0.8=py_0
  - prompt_toolkit=3.0.8=0
  - protobuf=3.13.0.1=py38he6710b0_1
  - psutil=5.7.2=py38h7b6447c_0
  - psycopg2=2.8.5=py38h3c74f83_1
  - ptyprocess=0.6.0=pyhd3eb1b0_2
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycparser=2.20=py_2
  - pygments=2.7.2=pyhd3eb1b0_0
  - pyjwt=1.7.1=py38_0
  - pynacl=1.4.0=py38h7b6447c_1
  - pyodbc=4.0.30=py38he6710b0_0
  - pyopenssl=19.1.0=pyhd3eb1b0_1
  - pyparsing=2.4.7=pyhd3eb1b0_0
  - pysocks=1.7.1=py38h06a4308_0
  - python=3.8.8=hdb3f193_4 # (updated from 3.8.5 in May 26, 2021 maintenance update)
  - python-dateutil=2.8.1=pyhd3eb1b0_0
  - python-editor=1.0.4=py_0
  - pytorch=1.7.1=py3.8_cpu_0
  - pytz=2020.5=pyhd3eb1b0_0
  - pyzmq=19.0.2=py38he6710b0_1
  - readline=8.0=h7b6447c_0
  - regex=2020.10.15=py38h7b6447c_0
  - requests=2.24.0=py_0
  - requests-oauthlib=1.3.0=py_0
  - retrying=1.3.3=py_2
  - rsa=4.7.2=pyhd3eb1b0_1
  - s3transfer=0.3.4=pyhd3eb1b0_0
  - scikit-learn=0.23.2=py38h0573a6f_0
  - scipy=1.5.2=py38h0b6359f_0
  - setuptools=50.3.1=py38h06a4308_1
  - simplejson=3.17.2=py38h27cfd23_2
  - six=1.15.0=py38h06a4308_0
  - smmap=3.0.5=pyhd3eb1b0_0
  - sqlite=3.33.0=h62c20be_0
  - sqlparse=0.4.1=py_0
  - statsmodels=0.12.0=py38h7b6447c_0
  - tabulate=0.8.7=py38h06a4308_0
  - threadpoolctl=2.1.0=pyh5ca1d4c_0
  - tk=8.6.10=hbc83047_0
  - torchvision=0.8.2=py38_cpu
  - tornado=6.0.4=py38h7b6447c_1
  - tqdm=4.50.2=py_0
  - traitlets=5.0.5=pyhd3eb1b0_0
  - typing-extensions=3.7.4.3=hd3eb1b0_0
  - typing_extensions=3.7.4.3=pyh06a4308_0
  - unixodbc=2.3.9=h7b6447c_0
  - urllib3=1.25.11=py_0
  - wcwidth=0.2.5=py_0
  - websocket-client=0.57.0=py38_2
  - werkzeug=1.0.1=pyhd3eb1b0_0
  - wheel=0.35.1=pyhd3eb1b0_0
  - wrapt=1.12.1=py38h7b6447c_1
  - xz=5.2.5=h7b6447c_0
  - yarl=1.6.3=py38h27cfd23_0
  - zeromq=4.3.3=he6710b0_3
  - zipp=3.4.0=pyhd3eb1b0_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.4.5=h9ceee32_0
  - pip:
    - astunparse==1.6.3
    - azure-core==1.11.0
    - azure-storage-blob==12.7.1
    - databricks-cli==0.14.1
    - diskcache==5.2.1
    - docker==4.4.4
    - flatbuffers==1.12
    - gast==0.3.3
    - grpcio==1.32.0
    - horovod==0.21.1
    - joblibspark==0.3.0
    - keras-preprocessing==1.1.2
    - koalas==1.6.0
    - llvmlite==0.35.0
    - mleap==0.16.1
    - mlflow==1.14.1
    - msrest==0.6.21
    - numba==0.52.0
    - opt-einsum==3.3.0
    - petastorm==0.9.8
    - pyarrow==1.0.1
    - pyyaml==5.4.1
    - querystring-parser==1.2.4
    - seaborn==0.10.0
    - shap==0.38.1
    - slicer==0.0.7
    - spark-tensorflow-distributor==0.1.0
    - tensorboard==2.4.1
    - tensorboard-plugin-wit==1.8.0
    - tensorflow-cpu==2.4.1
    - tensorflow-estimator==2.4.0
    - termcolor==1.1.0
    - xgboost==1.3.3
prefix: /databricks/conda/envs/databricks-ml

Bibliothèques Python sur les clusters GPU

name: databricks-ml-gpu
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - absl-py=0.11.0=pyhd3eb1b0_1
  - aiohttp=3.7.4=py38h27cfd23_1
  - asn1crypto=1.4.0=py_0
  - astor=0.8.1=py38h06a4308_0
  - async-timeout=3.0.1=py38h06a4308_0
  - attrs=20.3.0=pyhd3eb1b0_0
  - backcall=0.2.0=pyhd3eb1b0_0
  - bcrypt=3.2.0=py38h7b6447c_0
  - blas=1.0=mkl
  - blinker=1.4=py38h06a4308_0
  - boto3=1.16.7=pyhd3eb1b0_0
  - botocore=1.19.7=pyhd3eb1b0_0
  - brotlipy=0.7.0=py38h27cfd23_1003
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2021.4.13=h06a4308_1 # (updated from 2021.1.19 in May 26, 2021 maintenance update)
  - cachetools=4.2.1=pyhd3eb1b0_0
  - certifi=2020.12.5=py38h06a4308_0
  - cffi=1.14.3=py38h261ae71_2
  - chardet=3.0.4=py38h06a4308_1003
  - click=7.1.2=pyhd3eb1b0_0
  - cloudpickle=1.6.0=py_0
  - configparser=5.0.1=py_0
  - cryptography=3.1.1=py38h1ba5d50_0
  - cycler=0.10.0=py38_0
  - cython=0.29.21=py38h2531618_0
  - decorator=4.4.2=pyhd3eb1b0_0
  - dill=0.3.2=py_0
  - docutils=0.15.2=py38h06a4308_1
  - entrypoints=0.3=py38_0
  - flask=1.1.2=pyhd3eb1b0_0
  - freetype=2.10.4=h5ab3b9f_0
  - future=0.18.2=py38_1
  - gitdb=4.0.5=py_0
  - gitpython=3.1.12=pyhd3eb1b0_1
  - google-auth=1.22.1=py_0
  - google-auth-oauthlib=0.4.2=pyhd3eb1b0_2
  - google-pasta=0.2.0=py_0
  - grpcio=1.31.0=py38hf8bcb03_0
  - gunicorn=20.0.4=py38_0
  - h5py=2.10.0=py38h7918eee_0
  - hdf5=1.10.4=hb1b8bf9_0
  - icu=58.2=he6710b0_3
  - idna=2.10=pyhd3eb1b0_0
  - importlib-metadata=2.0.0=py_1
  - intel-openmp=2019.4=243
  - ipykernel=5.3.4=py38h5ca1d4c_0
  - ipython=7.19.0=py38hb070fc8_1
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - isodate=0.6.0=py_1
  - itsdangerous=1.1.0=pyhd3eb1b0_0
  - jedi=0.17.2=py38h06a4308_1
  - jinja2=2.11.2=pyhd3eb1b0_0
  - jmespath=0.10.0=py_0
  - joblib=0.17.0=py_0
  - jpeg=9b=h024ee3a_2
  - jupyter_client=6.1.7=py_0
  - jupyter_core=4.6.3=py38_0
  - kiwisolver=1.3.0=py38h2531618_0
  - krb5=1.17.1=h173b8e3_0
  - lcms2=2.11=h396b838_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libedit=3.1.20191231=h14c3975_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libpng=1.6.37=hbc83047_0
  - libpq=12.2=h20c2e04_0
  - libprotobuf=3.13.0.1=hd408876_0
  - libsodium=1.0.18=h7b6447c_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_1
  - lightgbm=3.1.1=py38h2531618_0
  - lz4-c=1.9.2=heb0550a_3
  - mako=1.1.3=py_0
  - markdown=3.3.3=py38h06a4308_0
  - markupsafe=1.1.1=py38h7b6447c_0
  - matplotlib-base=3.2.2=py38hef1b27d_0
  - mkl=2019.4=243
  - mkl-service=2.3.0=py38he904b0f_0
  - mkl_fft=1.2.0=py38h23d657b_0
  - mkl_random=1.1.0=py38h962f231_0
  - more-itertools=8.6.0=pyhd3eb1b0_0
  - multidict=5.1.0=py38h27cfd23_2
  - ncurses=6.2=he6710b0_1
  - networkx=2.5=py_0
  - nltk=3.5=py_0
  - numpy=1.19.2=py38h54aff64_0
  - numpy-base=1.19.2=py38hfa32c7d_0
  - oauthlib=3.1.0=py_0
  - olefile=0.46=py_0
  - openssl=1.1.1k=h27cfd23_0 # (updated from 1.1.1i in May 26, 2021 maintenance update)
  - packaging=20.4=py_0
  - pandas=1.1.3=py38he6710b0_0
  - paramiko=2.7.2=py_0
  - parso=0.7.0=py_0
  - patsy=0.5.1=py38_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=8.0.1=py38he98fc37_0
  - pip=20.2.4=py38h06a4308_0
  - plotly=4.14.3=pyhd3eb1b0_0
  - prompt-toolkit=3.0.8=py_0
  - prompt_toolkit=3.0.8=0
  - protobuf=3.13.0.1=py38he6710b0_1
  - psutil=5.7.2=py38h7b6447c_0
  - psycopg2=2.8.5=py38h3c74f83_1
  - ptyprocess=0.6.0=pyhd3eb1b0_2
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycparser=2.20=py_2
  - pygments=2.7.2=pyhd3eb1b0_0
  - pyjwt=1.7.1=py38_0
  - pynacl=1.4.0=py38h7b6447c_1
  - pyodbc=4.0.30=py38he6710b0_0
  - pyopenssl=19.1.0=pyhd3eb1b0_1
  - pyparsing=2.4.7=pyhd3eb1b0_0
  - pysocks=1.7.1=py38h06a4308_0
  - python=3.8.8=hdb3f193_4 # (updated from 3.8.5 in May 26, 2021 maintenance update)
  - python-dateutil=2.8.1=pyhd3eb1b0_0
  - python-editor=1.0.4=py_0
  - pytz=2020.5=pyhd3eb1b0_0
  - pyzmq=19.0.2=py38he6710b0_1
  - readline=8.0=h7b6447c_0
  - regex=2020.10.15=py38h7b6447c_0
  - requests=2.24.0=py_0
  - requests-oauthlib=1.3.0=py_0
  - retrying=1.3.3=py_2
  - rsa=4.7.2=pyhd3eb1b0_1
  - s3transfer=0.3.4=pyhd3eb1b0_0
  - scikit-learn=0.23.2=py38h0573a6f_0
  - scipy=1.5.2=py38h0b6359f_0
  - setuptools=50.3.1=py38h06a4308_1
  - simplejson=3.17.2=py38h27cfd23_2
  - six=1.15.0=py38h06a4308_0
  - smmap=3.0.5=pyhd3eb1b0_0
  - sqlite=3.33.0=h62c20be_0
  - sqlparse=0.4.1=py_0
  - statsmodels=0.12.0=py38h7b6447c_0
  - tabulate=0.8.7=py38h06a4308_0
  - threadpoolctl=2.1.0=pyh5ca1d4c_0
  - tk=8.6.10=hbc83047_0
  - tornado=6.0.4=py38h7b6447c_1
  - tqdm=4.50.2=py_0
  - traitlets=5.0.5=pyhd3eb1b0_0
  - typing-extensions=3.7.4.3=hd3eb1b0_0
  - typing_extensions=3.7.4.3=pyh06a4308_0
  - unixodbc=2.3.9=h7b6447c_0
  - urllib3=1.25.11=py_0
  - wcwidth=0.2.5=py_0
  - websocket-client=0.57.0=py38_2
  - werkzeug=1.0.1=pyhd3eb1b0_0
  - wheel=0.35.1=pyhd3eb1b0_0
  - wrapt=1.12.1=py38h7b6447c_1
  - xz=5.2.5=h7b6447c_0
  - yarl=1.6.3=py38h27cfd23_0
  - zeromq=4.3.3=he6710b0_3
  - zipp=3.4.0=pyhd3eb1b0_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.4.5=h9ceee32_0
  - pip:
    - astunparse==1.6.3
    - azure-core==1.11.0
    - azure-storage-blob==12.7.1
    - databricks-cli==0.14.1
    - diskcache==5.2.1
    - docker==4.4.4
    - flatbuffers==1.12
    - gast==0.3.3
    - horovod==0.21.1
    - joblibspark==0.3.0
    - keras-preprocessing==1.1.2
    - koalas==1.6.0
    - llvmlite==0.35.0
    - mleap==0.16.1
    - mlflow==1.14.1
    - msrest==0.6.21
    - numba==0.52.0
    - opt-einsum==3.3.0
    - petastorm==0.9.8
    - pyarrow==1.0.1
    - pyyaml==5.4.1
    - querystring-parser==1.2.4
    - seaborn==0.10.0
    - shap==0.38.1
    - slicer==0.0.7
    - spark-tensorflow-distributor==0.1.0
    - tensorboard==2.4.1
    - tensorboard-plugin-wit==1.8.0
    - tensorflow==2.4.1
    - tensorflow-estimator==2.4.0
    - termcolor==1.1.0
    - torch==1.7.1
    - torchvision==0.8.2
    - xgboost==1.3.3
prefix: /databricks/conda/envs/databricks-ml-gpu

Packages Spark contenant des modules Python

Package Spark	Module Python	Version
graphframes	graphframes	0.8.1-db2-spark3.1

Bibliothèques R

Les bibliothèques R sont identiques aux bibliothèques R fournies dans Databricks Runtime 8.1.

Bibliothèques Java et Scala (cluster Scala 2.12)

En plus des bibliothèques Java et Scala de Databricks Runtime 8.1, Databricks Runtime 8.1 ML contient les fichiers JAR suivants :

Clusters UC

ID de groupe	ID d’artefact	Version
com.typesafe.akka	akka-actor_2.12	2.5.23
ml.combust.mleap	mleap-databricks-runtime_2.12	0.17.3-4882dc3
ml.dmlc	xgboost4j-spark_2.12	1.3.1
ml.dmlc	xgboost4j_2.12	1.3.1
org.mlflow	mlflow-client	1.14.1
org.scala-lang.modules	scala-java8-compat_2.12	0.8.0
org.tensorflow	spark-tensorflow-connector_2.12	1.15.0

Clusters GPU

ID de groupe	ID d’artefact	Version
com.typesafe.akka	akka-actor_2.12	2.5.23
ml.combust.mleap	mleap-databricks-runtime_2.12	0.17.3-4882dc3
ml.dmlc	xgboost4j-spark-gpu_2.12	1.3.1
ml.dmlc	xgboost4j-gpu_2.12	1.3.1
org.mlflow	mlflow-client	1.14.1
org.scala-lang.modules	scala-java8-compat_2.12	0.8.0
org.tensorflow	spark-tensorflow-connector_2.12	1.15.0