Databricks Runtime 5,5 LTS för Machine Learning
Databricks släpptes den här avbildningen i juli 2019.
Databricks Runtime 5,5 LTS för Machine Learning är en färdig miljö för maskin inlärning och data vetenskap baserat på Databricks Runtime 5,5 LTS. Databricks Runtime ML innehåller många populära maskin inlärnings bibliotek, inklusive TensorFlow, PyTorch, keras och XGBoost. Den har också stöd för distribuerad djup inlärnings utbildning med Horovod.
Mer information, inklusive anvisningar för hur du skapar ett Databricks Runtime ML-kluster, finns Databricks runtime för Machine Learning.
Nya funktioner
Databricks Runtime 5,5 LTS för Machine Learning skapas ovanpå Databricks Runtime 5,5 LTS. Information om vad som är nytt i Databricks Runtime 5,5-LTS finns i viktig information om Databricks Runtime 5,5 LTS .
Förutom biblioteks uppdateringarhar DATABRICKS runtime 5,5 LTS för Machine Learning introducerar följande nya funktioner:
- MLflow 1,0 python-paketet har lagts till
Bättre
Uppgraderade Machine Learning-bibliotek
- Tensorflow har uppgraderats från 1.12.0 till 1.13.1
- PyTorch har uppgraderats från 0.4.1 till 1.1.0
- scikit – lär dig uppgraderade från 0.19.1 till 0.20.3
Åtgärd med en nod för HorovodRunner
Aktiverade HorovodRunner som bara körs på driv rutins-noden. Tidigare skulle du behöva köra en driv rutin och minst en arbetsnod för att använda HorovodRunner. Med den här ändringen kan du nu distribuera utbildning inom en enda nod (det vill säga en nod med flera GPU: n) och därmed använda beräknings resurserna på ett mer effektivt sätt.
Utfasning
I hyperopt -biblioteket har vi föråldrat följande egenskaper för hyperopt.SparkTrials
:
SparkTrials.successful_trials_count
SparkTrials.failed_trials_count
SparkTrials.cancelled_trials_count
SparkTrials.total_trials_count
och ersatte egenskaperna med följande funktioner:
SparkTrials.count_successful_trials()
SparkTrials.count_failed_trials()
SparkTrials.count_cancelled_trials()
SparkTrials.count_total_trials()
Systemmiljö
System miljön i Databricks Runtime 5,5 LTS för Machine Learning skiljer sig från Databricks Runtime 5,5 på följande sätt:
- Python: 3.6.5 för python 3-kluster och 2.7.15 för python 2-kluster.
- DBUtils: innehåller inte biblioteks verktyg.
- För GPU-kluster är följande NVIDIA GPU-bibliotek:
- CUDA 10,0
- CUDNN 7.6.0
Bibliotek
I följande avsnitt listas de bibliotek som ingår i Databricks Runtime 5,5-LTS för Machine Learning som skiljer sig från de som ingår i Databricks Runtime 5,5.
Bibliotek på översta nivån
Databricks Runtime 5,5-LTS för Machine Learning innehåller följande bibliotekpå den översta nivån:
Python-bibliotek
Databricks Runtime 5,5 LTS för Machine Learning använder Conda för hantering av python-paket. Därför finns det stora skillnader i installerade python-bibliotek jämfört med Databricks Runtime. I följande avsnitt beskrivs Conda-miljöer för Databricks Runtime 5,5-LTS för Machine Learning kluster med python 2 eller 3, och processor-eller GPU-aktiverade datorer.
Python 3 på CPU-kluster
name: null
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=2.0=cpu_0
- _tflow_select=2.3.0=mkl
- absl-py=0.7.1=py36_0
- asn1crypto=0.24.0=py36_0
- astor=0.7.1=py36_0
- backcall=0.1.0=py36_0
- backports=1.0=py_2
- bcrypt=3.1.6=py36h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py36_0
- boto=2.48.0=py36_1
- boto3=1.7.62=py36h28b3542_1
- botocore=1.10.62=py36h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py36_0
- cffi=1.11.5=py36he75722e_1
- chardet=3.0.4=py36_1
- click=7.0=py36_0
- cloudpickle=0.8.0=py36_0
- colorama=0.3.9=py36h489cec4_0
- configparser=3.7.3=py36_1
- cryptography=2.2.2=py36h14c3975_0
- cycler=0.10.0=py36h93f1223_0
- cython=0.28.2=py36h14c3975_0
- decorator=4.3.0=py36_0
- docutils=0.14=py36hb0f60f5_0
- entrypoints=0.2.3=py36_2
- et_xmlfile=1.0.1=py36hd6bccc3_0
- flask=1.0.2=py36_1
- freetype=2.8=hab7d2ae_1
- gast=0.2.2=py36_0
- gitdb2=2.0.5=py36_0
- gitpython=2.1.11=py36_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py36hdbcaa40_0
- gunicorn=19.9.0=py36_0
- h5py=2.8.0=py36h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py36_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py36h82fb2a8_1
- intel-openmp=2018.0.0=8
- ipython=6.4.0=py36_1
- ipython_genutils=0.2.0=py36_0
- itsdangerous=0.24=py36_1
- jdcal=1.4=py36_0
- jedi=0.12.0=py36_1
- jinja2=2.10=py36_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py36_0
- jupyter_client=5.2.3=py36_0
- jupyter_core=4.4.0=py36_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py36_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=he6710b0_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- llvmlite=0.23.1=py36hdbcaa40_0
- lxml=4.2.1=py36h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py36_0
- markupsafe=1.0=py36h14c3975_1
- mistune=0.8.3=py36h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py36ha843d7b_0
- mkl_random=1.0.2=py36hd81dba3_0
- mock=3.0.5=py36_0
- msgpack-python=0.5.6=py36h6bb024c_1
- nbconvert=5.3.1=py36_0
- nbformat=4.4.0=py36h31c9010_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py36hfd86e86_0
- numba=0.38.0=py36h637b7d7_0
- numpy=1.16.2=py36h7e9f1db_0
- numpy-base=1.16.2=py36hde5b4d6_0
- olefile=0.45.1=py36_0
- openpyxl=2.5.3=py36_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py36h637b7d7_0
- pandocfilters=1.4.2=py36_1
- paramiko=2.4.2=py36_0
- parso=0.2.0=py36_0
- pathlib2=2.3.2=py36_0
- patsy=0.5.0=py36_0
- pexpect=4.5.0=py36_0
- pickleshare=0.7.4=py36_0
- pillow=5.1.0=py36h3deb7b8_0
- pip=10.0.1=py36_0
- ply=3.11=py36_0
- prompt_toolkit=1.0.15=py36h17d85b1_0
- protobuf=3.8.0=py36he6710b0_0
- psycopg2=2.7.5=py36hb7f436b_0
- ptyprocess=0.5.2=py36h69acd42_0
- py-xgboost=0.90=py36he6710b0_0
- py-xgboost-cpu=0.90=py36_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py36_1
- pygments=2.2.0=py36_0
- pynacl=1.3.0=py36h7b6447c_0
- pyopenssl=18.0.0=py36_0
- pyparsing=2.2.0=py36_1
- pysocks=1.6.8=py36_0
- python=3.6.5=hc3d631a_2
- python-dateutil=2.7.3=py36_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py36_0
- pyyaml=5.1=py36h7b6447c_0
- pyzmq=17.0.0=py36h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py36he2e5f8d_1
- s3transfer=0.1.13=py36_0
- scikit-learn=0.20.3=py36hd81dba3_0
- scipy=1.1.0=py36h7c811a0_2
- setuptools=39.1.0=py36_0
- simplegeneric=0.8.1=py36_2
- simplejson=3.16.0=py36h14c3975_0
- singledispatch=3.4.0.3=py36_0
- six=1.11.0=py36_1
- smmap2=2.0.5=py36_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py36h035aef0_0
- tabulate=0.8.3=py36_0
- tensorboard=1.13.1=py36hf484d3e_0
- tensorflow=1.13.1=mkl_py36h27d456a_0
- tensorflow-base=1.13.1=mkl_py36h7ce6ba3_0
- tensorflow-estimator=1.13.0=py_0
- tensorflow-mkl=1.13.1=h4fcabd2_0
- termcolor=1.1.0=py36_1
- testpath=0.3.1=py36h8cadb63_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py36h14c3975_0
- traitlets=4.3.2=py36_0
- urllib3=1.22=py36hbe7ace6_0
- virtualenv=16.0.0=py36_0
- wcwidth=0.1.7=py36hdf4376a_0
- webencodings=0.5.1=py36_1
- werkzeug=0.14.1=py36_0
- wheel=0.31.1=py36_0
- wrapt=1.11.1=py36h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch-cpu=1.1.0=py3.6_cpu_0
- torchvision-cpu=0.3.0=py36_cuNone_1
- pip:
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- future==0.17.1
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- tensorboardx==1.7
- torchvision==0.3.0
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python3
Python 3 på GPU-kluster
name: null
channels:
- pytorch
- Databricks
- defaults
dependencies:
- tensorflow=1.13.1.db1=gpu_py36h2903d8e_0
- tensorflow-base=1.13.1.db1=gpu_py36he292aa2_0
- tensorflow-gpu=1.13.1.db1=h0d30ee6_0
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=1.0=gpu_0
- _tflow_select=2.1.0=gpu
- absl-py=0.7.1=py36_0
- asn1crypto=0.24.0=py36_0
- astor=0.7.1=py36_0
- backcall=0.1.0=py36_0
- backports=1.0=py_2
- bcrypt=3.1.6=py36h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py36_0
- boto=2.48.0=py36_1
- boto3=1.7.62=py36h28b3542_1
- botocore=1.10.62=py36h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py36_0
- cffi=1.11.5=py36he75722e_1
- chardet=3.0.4=py36_1
- click=7.0=py36_0
- cloudpickle=0.8.0=py36_0
- colorama=0.3.9=py36h489cec4_0
- configparser=3.7.3=py36_1
- cryptography=2.2.2=py36h14c3975_0
- cudnn=7.6.0=cuda10.0_0
- cupti=10.0.130=0
- cycler=0.10.0=py36_0
- cython=0.28.2=py36h14c3975_0
- decorator=4.3.0=py36_0
- docutils=0.14=py36_0
- entrypoints=0.2.3=py36_2
- et_xmlfile=1.0.1=py36hd6bccc3_0
- flask=1.0.2=py36_1
- freetype=2.8=hab7d2ae_1
- gast=0.2.2=py36_0
- gitdb2=2.0.5=py36_0
- gitpython=2.1.11=py36_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py36hdbcaa40_0
- gunicorn=19.9.0=py36_0
- h5py=2.8.0=py36h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py36_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py36h82fb2a8_1
- intel-openmp=2018.0.0=8
- ipython=6.4.0=py36_1
- ipython_genutils=0.2.0=py36hb52b0d5_0
- itsdangerous=0.24=py36_1
- jdcal=1.4=py36_0
- jedi=0.12.0=py36_1
- jinja2=2.10=py36_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py36_0
- jupyter_client=5.2.3=py36_0
- jupyter_core=4.4.0=py36_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py36_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=h688424c_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- llvmlite=0.23.1=py36hdbcaa40_0
- lxml=4.2.1=py36h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py36_0
- markupsafe=1.0=py36h14c3975_1
- mistune=0.8.3=py36h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py36ha843d7b_0
- mkl_random=1.0.2=py36hd81dba3_0
- mock=3.0.5=py36_0
- msgpack-python=0.5.6=py36h6bb024c_1
- nbconvert=5.3.1=py36_0
- nbformat=4.4.0=py36h31c9010_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py36hfd86e86_0
- numba=0.38.0=py36h637b7d7_0
- numpy=1.16.2=py36h7e9f1db_0
- numpy-base=1.16.2=py36hde5b4d6_0
- olefile=0.45.1=py36_0
- openpyxl=2.5.3=py36_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py36h637b7d7_0
- pandocfilters=1.4.2=py36_1
- paramiko=2.4.2=py36_0
- parso=0.2.0=py36_0
- pathlib2=2.3.2=py36_0
- patsy=0.5.0=py36_0
- pexpect=4.5.0=py36_0
- pickleshare=0.7.4=py36h63277f8_0
- pillow=5.1.0=py36h3deb7b8_0
- pip=10.0.1=py36_0
- ply=3.11=py36_0
- prompt_toolkit=1.0.15=py36_0
- protobuf=3.8.0=py36he6710b0_0
- psycopg2=2.7.5=py36hb7f436b_0
- ptyprocess=0.5.2=py36h69acd42_0
- py-xgboost=0.90=py36h688424c_0
- py-xgboost-gpu=0.90=py36h28bbb66_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py36_1
- pygments=2.2.0=py36_0
- pynacl=1.3.0=py36h7b6447c_0
- pyopenssl=18.0.0=py36_0
- pyparsing=2.2.0=py36_1
- pysocks=1.6.8=py36_0
- python=3.6.5=hc3d631a_2
- python-dateutil=2.7.3=py36_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py36_0
- pyyaml=5.1=py36h7b6447c_0
- pyzmq=17.0.0=py36h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py36he2e5f8d_1
- s3transfer=0.1.13=py36_0
- scikit-learn=0.20.3=py36hd81dba3_0
- scipy=1.1.0=py36h7c811a0_2
- setuptools=39.1.0=py36_0
- simplegeneric=0.8.1=py36_2
- simplejson=3.16.0=py36h14c3975_0
- singledispatch=3.4.0.3=py36h7a266c3_0
- six=1.11.0=py36_1
- smmap2=2.0.5=py36_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py36h035aef0_0
- tabulate=0.8.3=py36_0
- tensorboard=1.13.1=py36hf484d3e_0
- tensorflow-estimator=1.13.0=py_0
- termcolor=1.1.0=py36_1
- testpath=0.3.1=py36_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py36h14c3975_0
- traitlets=4.3.2=py36h674d592_0
- urllib3=1.22=py36hbe7ace6_0
- virtualenv=16.0.0=py36_0
- wcwidth=0.1.7=py36hdf4376a_0
- webencodings=0.5.1=py36_1
- werkzeug=0.14.1=py36_0
- wheel=0.31.1=py36_0
- wrapt=1.11.1=py36h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch=1.1.0=py3.6_cuda10.0.130_cudnn7.5.1_0
- torchvision=0.3.0=py36_cu10.0.130_1
- pip:
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- future==0.17.1
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- tensorboardx==1.7
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python3
Python 2 på CPU-kluster
name: null
channels:
- pytorch
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=2.0=cpu_0
- _tflow_select=2.3.0=mkl
- absl-py=0.7.1=py27_0
- asn1crypto=0.24.0=py27_0
- astor=0.7.1=py27_0
- backports=1.0=py_2
- backports.shutil_get_terminal_size=1.0.0=py27_2
- backports.weakref=1.0.post1=py_1
- backports_abc=0.5=py_0
- bcrypt=3.1.6=py27h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py27_0
- boto=2.48.0=py27_1
- boto3=1.7.62=py27h28b3542_1
- botocore=1.10.62=py27h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py27_0
- cffi=1.11.5=py27he75722e_1
- chardet=3.0.4=py27_1
- click=7.0=py27_0
- cloudpickle=0.8.0=py27_0
- colorama=0.3.9=py27h5cde069_0
- configparser=3.7.3=py27_1
- cryptography=2.2.2=py27h14c3975_0
- cycler=0.10.0=py27hc7354d3_0
- cython=0.28.2=py27h14c3975_0
- decorator=4.3.0=py27_0
- docutils=0.14=py27_0
- entrypoints=0.2.3=py27_2
- enum34=1.1.6=py27_1
- et_xmlfile=1.0.1=py27_0
- flask=1.0.2=py27_1
- freetype=2.8=hab7d2ae_1
- funcsigs=1.0.2=py27_0
- functools32=3.2.3.2=py27_1
- future=0.17.1=py27_0
- futures=3.2.0=py27_0
- gast=0.2.2=py27_0
- gitdb2=2.0.5=py27_0
- gitpython=2.1.11=py27_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py27hdbcaa40_0
- gunicorn=19.9.0=py27_0
- h5py=2.8.0=py27h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py27_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py27h5722d68_1
- intel-openmp=2018.0.0=8
- ipaddress=1.0.22=py27_0
- ipython=5.7.0=py27_0
- ipython_genutils=0.2.0=py27_0
- itsdangerous=0.24=py27_1
- jdcal=1.4=py27_0
- jinja2=2.10=py27_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py27h7ed5aa4_0
- jupyter_client=5.2.3=py27_0
- jupyter_core=4.4.0=py27_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py27_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=he6710b0_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- linecache2=1.0.0=py27_0
- llvmlite=0.23.1=py27hdbcaa40_0
- lxml=4.2.1=py27h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py27_0
- markupsafe=1.0=py27h14c3975_1
- mistune=0.8.3=py27h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py27ha843d7b_0
- mkl_random=1.0.2=py27hd81dba3_0
- mock=3.0.5=py27_0
- msgpack-python=0.5.6=py27h6bb024c_1
- nbconvert=5.3.1=py27_0
- nbformat=4.4.0=py27hed7f2b2_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py27hfd86e86_0
- numba=0.38.0=py27h637b7d7_0
- numpy=1.16.2=py27h7e9f1db_0
- numpy-base=1.16.2=py27hde5b4d6_0
- olefile=0.45.1=py27_0
- openpyxl=2.5.3=py27_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py27h637b7d7_0
- pandocfilters=1.4.2=py27_1
- paramiko=2.4.2=py27_0
- pathlib2=2.3.2=py27_0
- patsy=0.5.0=py27_0
- pexpect=4.5.0=py27_0
- pickleshare=0.7.4=py27_0
- pillow=5.1.0=py27h3deb7b8_0
- pip=10.0.1=py27_0
- ply=3.11=py27_0
- prompt_toolkit=1.0.15=py27_0
- protobuf=3.8.0=py27he6710b0_0
- psycopg2=2.7.5=py27hb7f436b_0
- ptyprocess=0.5.2=py27h4ccb14c_0
- py-xgboost=0.90=py27he6710b0_0
- py-xgboost-cpu=0.90=py27_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py27_1
- pygments=2.2.0=py27_0
- pynacl=1.3.0=py27h7b6447c_0
- pyopenssl=18.0.0=py27_0
- pyparsing=2.2.0=py27_1
- pysocks=1.6.8=py27_0
- python=2.7.15=h1571d57_0
- python-dateutil=2.7.3=py27_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py27_0
- pyyaml=5.1=py27h7b6447c_0
- pyzmq=17.0.0=py27h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py27hc5b0589_1
- s3transfer=0.1.13=py27_0
- scandir=1.7=py27h14c3975_0
- scikit-learn=0.20.3=py27hd81dba3_0
- scipy=1.1.0=py27h7c811a0_2
- setuptools=39.1.0=py27_0
- simplegeneric=0.8.1=py27_2
- simplejson=3.16.0=py27h14c3975_0
- singledispatch=3.4.0.3=py27_0
- six=1.11.0=py27_1
- smmap2=2.0.5=py27_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py27h035aef0_0
- tabulate=0.8.3=py27_0
- tensorboard=1.13.1=py27hf484d3e_0
- tensorflow=1.13.1=mkl_py27h74ee40f_0
- tensorflow-base=1.13.1=mkl_py27h7ce6ba3_0
- tensorflow-estimator=1.13.0=py_0
- tensorflow-mkl=1.13.1=h4fcabd2_0
- termcolor=1.1.0=py27_1
- testpath=0.3.1=py27hc38d2c4_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py27h14c3975_0
- traceback2=1.4.0=py27_0
- traitlets=4.3.2=py27_0
- unittest2=1.1.0=py27_0
- urllib3=1.22=py27ha55213b_0
- virtualenv=16.0.0=py27_0
- wcwidth=0.1.7=py27h9e3e1ab_0
- webencodings=0.5.1=py27_1
- werkzeug=0.14.1=py27_0
- wheel=0.31.1=py27_0
- wrapt=1.11.1=py27h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch-cpu=1.1.0=py2.7_cpu_0
- torchvision-cpu=0.3.0=py27_cuNone_1
- pip:
- backports.functools-lru-cache==1.5
- backports.ssl-match-hostname==3.7.0.1
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- subprocess32==3.5.4
- tensorboardx==1.7
- torchvision==0.3.0
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python2
Python 2 på GPU-kluster
name: null
channels:
- Databricks
- pytorch
- defaults
dependencies:
- tensorflow=1.13.1.db1=gpu_py27h8e347d7_0
- tensorflow-base=1.13.1.db1=gpu_py27he292aa2_0
- tensorflow-gpu=1.13.1.db1=h0d30ee6_0
- _libgcc_mutex=0.1=main
- _py-xgboost-mutex=1.0=gpu_0
- _tflow_select=2.1.0=gpu
- absl-py=0.7.1=py27_0
- asn1crypto=0.24.0=py27_0
- astor=0.7.1=py27_0
- backports=1.0=py_2
- backports.shutil_get_terminal_size=1.0.0=py27_2
- backports.weakref=1.0.post1=py_1
- backports_abc=0.5=py_0
- bcrypt=3.1.6=py27h7b6447c_0
- blas=1.0=mkl
- bleach=2.1.3=py27_0
- boto=2.48.0=py27_1
- boto3=1.7.62=py27h28b3542_1
- botocore=1.10.62=py27h28b3542_0
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py27_0
- cffi=1.11.5=py27he75722e_1
- chardet=3.0.4=py27_1
- click=7.0=py27_0
- cloudpickle=0.8.0=py27_0
- colorama=0.3.9=py27_0
- configparser=3.7.3=py27_1
- cryptography=2.2.2=py27h14c3975_0
- cudnn=7.6.0=cuda10.0_0
- cupti=10.0.130=0
- cycler=0.10.0=py27_0
- cython=0.28.2=py27h14c3975_0
- decorator=4.3.0=py27_0
- docutils=0.14=py27hae222c1_0
- entrypoints=0.2.3=py27_2
- enum34=1.1.6=py27_1
- et_xmlfile=1.0.1=py27h75840f5_0
- flask=1.0.2=py27_1
- freetype=2.8=hab7d2ae_1
- funcsigs=1.0.2=py27_0
- functools32=3.2.3.2=py27_1
- future=0.17.1=py27_0
- futures=3.2.0=py27_0
- gast=0.2.2=py27_0
- gitdb2=2.0.5=py27_0
- gitpython=2.1.11=py27_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.12.1=py27hdbcaa40_0
- gunicorn=19.9.0=py27_0
- h5py=2.8.0=py27h989c5e5_3
- hdf5=1.10.2=hba1933b_1
- html5lib=1.0.1=py27_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py27h5722d68_1
- intel-openmp=2018.0.0=8
- ipaddress=1.0.22=py27_0
- ipython=5.7.0=py27_0
- ipython_genutils=0.2.0=py27h89fb69b_0
- itsdangerous=0.24=py27_1
- jdcal=1.4=py27_0
- jinja2=2.10=py27_0
- jmespath=0.9.4=py_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py27h7ed5aa4_0
- jupyter_client=5.2.3=py27_0
- jupyter_core=4.4.0=py27_0
- keras=2.2.4=0
- keras-applications=1.0.8=py_0
- keras-base=2.2.4=py27_0
- keras-preprocessing=1.1.0=py_1
- krb5=1.16.1=hc83ff2d_6
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.3.0=hdf63c60_0
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libpq=10.4=h1ad7b7a_0
- libprotobuf=3.8.0=hd408876_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.3.0=hdf63c60_0
- libtiff=4.0.9=he85c1e1_2
- libxgboost=0.90=h688424c_0
- libxml2=2.9.8=h26e45fe_1
- libxslt=1.1.32=h1312cb7_0
- linecache2=1.0.0=py27_0
- llvmlite=0.23.1=py27hdbcaa40_0
- lxml=4.2.1=py27h23eabaa_0
- mako=1.0.10=py_0
- markdown=3.1.1=py27_0
- markupsafe=1.0=py27h14c3975_1
- mistune=0.8.3=py27h14c3975_1
- mkl=2019.4=243
- mkl_fft=1.0.12=py27ha843d7b_0
- mkl_random=1.0.2=py27hd81dba3_0
- mock=3.0.5=py27_0
- msgpack-python=0.5.6=py27h6bb024c_1
- nbconvert=5.3.1=py27_0
- nbformat=4.4.0=py27hed7f2b2_0
- ncurses=6.1=he6710b0_1
- ninja=1.9.0=py27hfd86e86_0
- numba=0.38.0=py27h637b7d7_0
- numpy=1.16.2=py27h7e9f1db_0
- numpy-base=1.16.2=py27hde5b4d6_0
- olefile=0.45.1=py27_0
- openpyxl=2.5.3=py27_0
- openssl=1.0.2o=h14c3975_1
- pandas=0.23.0=py27h637b7d7_0
- pandocfilters=1.4.2=py27_1
- paramiko=2.4.2=py27_0
- pathlib2=2.3.2=py27_0
- patsy=0.5.0=py27_0
- pexpect=4.5.0=py27_0
- pickleshare=0.7.4=py27h09770e1_0
- pillow=5.1.0=py27h3deb7b8_0
- pip=10.0.1=py27_0
- ply=3.11=py27_0
- prompt_toolkit=1.0.15=py27_0
- protobuf=3.8.0=py27he6710b0_0
- psycopg2=2.7.5=py27hb7f436b_0
- ptyprocess=0.5.2=py27h4ccb14c_0
- py-xgboost=0.90=py27h688424c_0
- py-xgboost-gpu=0.90=py27h28bbb66_0
- pyasn1=0.4.5=py_0
- pycparser=2.18=py27_1
- pygments=2.2.0=py27_0
- pynacl=1.3.0=py27h7b6447c_0
- pyopenssl=18.0.0=py27_0
- pyparsing=2.2.0=py27_1
- pysocks=1.6.8=py27_0
- python=2.7.15=h1571d57_0
- python-dateutil=2.7.3=py27_0
- python-editor=1.0.4=py_0
- pytz=2018.4=py27_0
- pyyaml=5.1=py27h7b6447c_0
- pyzmq=17.0.0=py27h14c3975_3
- readline=7.0=h7b6447c_5
- requests=2.18.4=py27hc5b0589_1
- s3transfer=0.1.13=py27_0
- scandir=1.7=py27h14c3975_0
- scikit-learn=0.20.3=py27hd81dba3_0
- scipy=1.1.0=py27h7c811a0_2
- setuptools=39.1.0=py27_0
- simplegeneric=0.8.1=py27_2
- simplejson=3.16.0=py27h14c3975_0
- singledispatch=3.4.0.3=py27h9bcb476_0
- six=1.11.0=py27_1
- smmap2=2.0.5=py27_0
- sqlite=3.23.1=he433501_0
- sqlparse=0.3.0=py_0
- statsmodels=0.9.0=py27h035aef0_0
- tabulate=0.8.3=py27_0
- tensorboard=1.13.1=py27hf484d3e_0
- tensorflow-estimator=1.13.0=py_0
- termcolor=1.1.0=py27_1
- testpath=0.3.1=py27_0
- tk=8.6.7=hc745277_3
- tornado=5.0.2=py27h14c3975_0
- traceback2=1.4.0=py27_0
- traitlets=4.3.2=py27hd6ce930_0
- unittest2=1.1.0=py27_0
- urllib3=1.22=py27ha55213b_0
- virtualenv=16.0.0=py27_0
- wcwidth=0.1.7=py27_0
- webencodings=0.5.1=py27_1
- werkzeug=0.14.1=py27_0
- wheel=0.31.1=py27_0
- wrapt=1.11.1=py27h7b6447c_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=hf484d3e_1
- zlib=1.2.11=h7b6447c_3
- pytorch=1.1.0=py2.7_cuda10.0.130_cudnn7.5.1_0
- torchvision=0.3.0=py27_cu10.0.130_1
- pip:
- backports.functools-lru-cache==1.5
- backports.ssl-match-hostname==3.7.0.1
- databricks-cli==0.8.7
- docker==4.0.2
- fusepy==2.0.4
- horovod==0.16.4
- hyperopt==0.1.2.db6
- kiwisolver==1.1.0
- matplotlib==2.2.2
- mleap==0.8.1
- mlflow==1.0.0
- msgpack==0.5.6
- networkx==2.2
- nose==1.3.7
- nose-exclude==0.5.0
- psutil==5.6.3
- pyarrow==0.13.0
- pymongo==3.8.0
- querystring-parser==1.2.3
- seaborn==0.8.1
- subprocess32==3.5.4
- tensorboardx==1.7
- tqdm==4.32.2
- websocket-client==0.56.0
prefix: /databricks/python2
Spark-paket som innehåller python-moduler
Spark-paket | Python-modul | Version |
---|---|---|
graphframes | graphframes | 0.7.0-DB1-Spark 2.4 |
Spark-djupgående-inlärning | sparkdl | 1.5.0-db4-Spark 2.4 |
tensorframes | tensorframes | 0.7.0-s_2.11 |
R-bibliotek
R-biblioteken är identiska med r-biblioteken i Databricks Runtime 5,5.
Java-och Scala-bibliotek (Scala 2,11-kluster)
Förutom Java-och Scala-biblioteken i Databricks Runtime 5,5 innehåller Databricks Runtime 5,5 LTS för Machine Learning följande jar v7:
Grupp-ID | Artefakt-ID | Version |
---|---|---|
com. databricks | Spark-djupgående-inlärning | 1.5.0-db4-Spark 2.4 |
com. typesafe. Akka | Akka-actor_2.11 | 2.3.11 |
ml. förbrännings. mleap | mleap-databricks-runtime_2.11 | 0.13.0 |
ml. dmlc | xgboost4j | 0,90 |
ml. dmlc | xgboost4j-Spark | 0,90 |
org. graphframes | graphframes_2.11 | 0.7.0-DB1-Spark 2.4 |
org. tensorflow | libtensorflow | 1.13.1 |
org. tensorflow | libtensorflow_jni | 1.13.1 |
org. tensorflow | Spark-tensorflow-connector_2.11 | 1.13.1 |
org. tensorflow | tensorflow | 1.13.1 |
org. tensorframes | tensorframes | 0.7.0-s_2.11 |