Databricks Runtime 3.4 (不支援)

Databricks 於 2017 年 11 月發行此映射。

重要

此版本已於 2018 年 7 月 30 日淘汰。 如需 Databricks 執行時間淘汰原則和排程的詳細資訊,請參閱 Databricks 運行時間支援生命週期

下列版本資訊提供 Apache Spark 所提供的 Databricks Runtime 3.4 相關信息。

變更和改進

  • 引進了適用於 PySpark 的新向量化 UDF。
  • 能夠限制查詢監督程式中的數據分割數目上限。
  • 已改善使用 Databricks 連結庫 UI 和 API 安裝 CRAN 套件時的錯誤訊息通訊。
  • MySQL JDBC 驅動程式已由 MariaDB JDBC 驅動程式取代。
  • 將 Python 連結庫安裝程式工具從 36.2.7 升級至 36.6.0,轉輪從 0.30.0a0 升級至 0.30.0。
  • 將 R 從 3.2.3 升級至 3.4.2。
  • 已升級數個預安裝的 R 連結庫。 如需詳細資訊,請參閱 預安裝的 R 連結庫
  • 已為需要符合 HIPAA 規範叢集的客戶新增 AES 加密

Apache Spark

Databricks Runtime 3.4 包含 Apache Spark 2.2.0。 Databricks Runtime 3.4 也包含下列額外的 Bug 修正和 Spark 的改善:

  • [SPARK-21718][SQL]類型繁重的記錄:「根據統計數據略過分割區...」
  • [SPARK-22278][SS]在 GroupState 中公開目前的事件時間浮水印和目前的處理時間
  • [SPARK-22187][SS]更新儲存狀態的 unsaferow 格式,以便我們可以在狀態為 Null 時設定逾時
  • [SPARK-22136][SS]在串流聯結早期評估單邊條件
  • [SPARK-20396][SQL][PYSPARK] groupby(.apply() with pandas udf
  • [SPARK-22124][SQL]範例和限制也應該延遲 codegen 下的輸入評估
  • [SPARK-21782][CORE]當 numPartitions 為 2 的電源時,重新分割會建立扭曲
  • [SPARK-21527][CORE]使用緩衝區限制以使用 JAVA NIO Util 的 buffercache
  • [SPARK-22176][SQL]修正 Dataset.show 中的溢位問題
  • [SPARK-21410][CORE]如果 RDD.count() 小於 分割區,請為 RangePartitioner 建立較少的分割區
  • [SPARK-21517][CORE]避免在遠端傳輸區塊時複製記憶體
  • [SPARK-22238]修正 EnsureStatefulOpPartitioning 所造成的計劃解決錯誤
  • [SPARK-22159][SQL]讓組態名稱一致地以「已啟用」結尾。
  • [SPARK-20331][SQL]增強的Hive分割區剪除述詞下推
  • [SPARK-22223][SQL]ObjectHashAggregate 不應該引入不必要的隨機顯示
  • [SPARK-22263][SQL]將決定性重構為延遲值
  • [SPARK-22257][SQL]在 ExpressionSet 中保留所有不具決定性的表達式
  • [SPARK-21907][CORE]修正溢出期間的陰鬱
  • [SPARK-22252][SPARK-21165]FileFormatWriter 錯誤修正
  • [SPARK-22230]在狀態存放區還原中交換每一數據列順序。
  • [SPARK-21988][SS]實作 StreamingRelation.computeStats 以修正說明
  • [SPARK-22170][SQL]減少廣播聯結中的記憶體耗用量
  • [SPARK-22206][SQL][SPARKR] R 中的間距無法在空白群組數據行上運作
  • [SPARK-20466][CORE]HadoopRDD#addLocalConfiguration 會擲回 NPE
  • [SPARK-22178][SQL]透過 REFRESH TABLE 命令重新整理持續性檢視
  • [SPARK-22158][SQL] convertMetastore 不應該忽略數據表屬性
  • [SPARK-22146]讀取包含特殊字元的 ORC 檔案時,FileNotFoundException
  • [SPARK-22161][SQL]新增 Impala 修改的 TPC-DS 查詢
  • [SPARK-22129] [SPARK-22138] 發行腳本改善
  • [SPARK-22143] [SQL]修正 OffHeapColumnVector 中的記憶體流失
  • [SPARK-22140] 新增 TPCDSQuerySuite
  • [SPARK-22214] [SQL]重構清單 Hive 分割區程式碼
  • [SPARK-22136] [SS]實作資料流程外部聯結。
  • [SPARK-22053] [SS]附加模式中的資料流程串流內部聯結
  • [SPARK-22153] [SQL]重新命名 ShuffleExchange -ShuffleExchangeExec
  • [SPARK-22160] [SQL]讓每個資料分割的樣本點 (在範圍分割器中) 可設定,並將預設值顛簸至 100
  • [SPARK-22159] [SQL]讓組態名稱一致地以「已啟用」結尾。
  • [SPARK-22141] [SQL]檢查笛卡兒產品之前傳播空的關聯性
  • [SPARK-22120] [SQL]TestHiveSparkSession.reset() 應該清除 Hive 倉儲目錄
  • [SPARK-22083] [CORE]在 MemoryStore.evictBlocksToFreeSpace 中釋放鎖定
  • [SPARK-22107] 變更為 python 快速入門中的別名
  • [SPARK-22109] [SQL]解決資料分割資料行中字串與時間戳記之間的類型衝突
  • [SPARK-22092] OffHeapColumnVector.reserveInternal 中的重新配置會損毀結構和陣列資料
  • [SPARK-18136] 修正 Windows 上 Python pip 安裝的SPARK_JARS_DIR
  • [SPARK-22094] [SS] processAllAvailable 應該檢查查詢狀態
  • [SPARK-21928] [CORE]在 SerializerManager 的私人 kryo 上設定 classloader
  • [SPARK-22076] [SQL]Expand.projections 不應該是 Stream
  • [SPARK-22052] MetricsReporter.scala 中指派的計量不正確
  • [SPARK-22043] [PYTHON]改善show_profiles和dump_profiles的錯誤訊息
  • [SPARK-21953] 如果其中一個記憶體和磁片位元組存在,則顯示溢出的記憶體和磁片位元組
  • [SPARK-21985] [PYSPARK]雙壓縮 RDD 的 PairDeserializer 已中斷
  • [SPARK-18608] [ML][FOLLOWUP]修正 PySpark OneVsRest 的雙重快取。
  • [SPARK-21980] [SQL]群組函式中的參考應使用 semanticEquals 編制索引
  • [SPARK-22103] 將 HashAggregateExec 父系移至個別函式
  • [SPARK-22141] [SQL]檢查笛卡兒產品之前傳播空的關聯性
  • [SPARK-21977] SinglePartition 優化會中斷特定串流具狀態匯總需求

已知問題

  • 執行程式頁面上的記錄連結未正確設定。 請使用背景工作頁面來存取執行程式的 stdout 和 stderr 連結。

維護更新

請參閱 Databricks Runtime 3.4 維護更新

系統環境

  • 作業系統 :Ubuntu 16.04.3 LTS
  • JAVA :1.8.0_131
  • Scala :2.10.6 (Scala 2.10 叢集版本)/2.11.8 (Scala 2.11 叢集版本)
  • Python :2.7.12(如果使用 Python 3,則為 3.5.2)
  • R :R 3.4.2 版 (2017-09-28)
  • 針對 GPU 叢集,已安裝下列 NVIDIA GPU 程式庫:
    • CUDA 8.0
    • CUDNN 6.0

預先安裝的 Python 程式庫

程式庫 版本 程式庫 版本 程式庫 版本
ansi2html 1.1.1 argparse 1.2.1 backports-abc 0.5
boto 2.42.0 boto3 1.4.1 botocore 1.4.70
brewer2mpl 1.4.1 certifi 2016.2.28 cffi 1.7.0
chardet 2.3.0 colorama 0.3.7 configobj 5.0.6
密碼編譯 1.5 cycler 0.10.0 Cython 0.24.1
decorator 4.0.10 docutils 0.14 enum34 1.1.6
et-xmlfile 1.0.1 freetype-py 1.0.2 funcsigs 1.0.2
fusepy 2.0.4 期貨 3.1.1 ggplot 0.6.8
html5lib 0.999 idna 2.1 ipaddress 1.0.16
ipython 2.2.0 ipython-genutils 0.1.0 jdcal 1.2
Jinja2 2.8 jmespath 0.9.0 llvmlite 0.13.0
lxml 3.6.4 MarkupSafe 0.23 matplotlib 1.5.3
mpld3 0.2 msgpack-python 0.4.7 ndg-httpsclient 0.3.3
numba 0.28.1 numpy 1.11.1 openpyxl 2.3.2
pandas 0.18.1 pathlib2 2.1.0 patsy 0.4.1
pexpect 4.0.1 pickleshare 0.7.4 Pillow 3.3.1
pip 9.0.1 3.9 prompt-toolkit 1.0.7
psycopg2 2.6.2 ptyprocess 0.5.1 py4j 0.10.3
pyarrow 0.4.1 pyasn1 0.1.9 pycparser 2.14
Pygments 2.1.3 PyGObject 3.20.0 pyOpenSSL 16.0.0
pyparsing 2.2.0 pypng 0.0.18 Python 2.7.12
python-dateutil 2.5.3 python-geohash 0.8.5 pytz 2016.6.1
requests 2.11.1 s3transfer 0.1.9 scikit-learn 0.18.1
scipy 0.18.1 沖刷 0.32 seaborn 0.7.1
setuptools 36.6.0 simplejson 3.8.2 simples3 1.0
singledispatch 3.4.0.3 六次 1.10.0 statsmodels 0.6.1
tornado 4.5.2 traitlets 4.3.0 urllib3 1.19.1
virtualenv 15.0.1 wcwidth 0.1.7 wheel 0.30.0
wsgiref 0.1.2

預安裝的 R 連結庫

程式庫 版本 程式庫 版本 程式庫 版本
abind 1.4-5 assertthat 0.2.0 backports 1.1.1
base 3.4.2 BH 1.65.0-1 bindr 0.1
bindrcpp 0.2 bit 1.1-12 bit64 0.9-7
bitops 1.0-6 blob 1.1.0 boot 1.3-20
brew 1.0-6 汽車 2.1-5 caret 6.0-77
chron 2.3-51 class 7.3-14 叢集 2.0.6
codetools 0.2-15 colorspace 1.3-2 commonmark 1.4
compiler 3.4.2 crayon 1.3.4 curl 3.0
CVST 0.2-1 data.table 1.10.4 datasets 3.4.2
DBI 0.7 ddalpha 1.3.1 DEoptimR 1.0-8
遞減 1.1.1 devtools 1.13.3 dichromat 2.0-0
digest 0.6.12 dimRed 0.1.0 doMC 1.3.4
dplyr 0.7.4 DRR 0.0.2 foreach 1.4.3
foreign 0.8-69 Gbm 2.1.3 ggplot2 2.2.1
git2r 0.19.0 glmnet 2.0-13 glue 1.1.1
gower 0.1.2 graphics 3.4.2 grDevices 3.4.2
grid 3.4.2 gsubfn 0.6-6 gtable 0.2.0
H2o 3.14.0.3 httr 1.3.1 hwriter 1.3.2
hwriterPlus 1.0-3 ipred 0.9-6 iterators 1.0.8
jsonlite 1.5 kernlab 0.9-25 KernSmooth 2.23-15
labeling 0.3 lattice 0.20-35 Lava 1.5.1
lazyeval 0.2.0 littler 0.3.2 lme4 1.1-14
lubridate 1.6.0 magrittr 1.5 mapproj 1.2-5
maps 3.2.0 MASS 7.3-47 矩陣 1.2-11
MatrixModels 0.4-1 備忘 1.1.0 方法 3.4.2
mgcv 1.8-22 mime 0.5 minqa 1.2.4
ModelMetrics 1.1.0 蒙塞爾 0.4.3 mvtnorm 1.0-6
nlme 3.1-131 nloptr 1.0.4 nnet 7.3-12
numDeriv 2016.8-1 openssl 0.9.7 parallel 3.4.2
pbkrtest 0.4-7 pkgconfig 2.0.1 pkgKitten 0.1.4
plogr 0.1-1 plyr 1.8.4 讚美 1.0.0
pROC 1.10.0 prodlim 1.6.1 proto 1.0.0
purrr 0.2.3 quantreg 5.33 R.methodsS3 1.7.1
R.oo 1.21.0 R.utils 2.5.0 R6 2.2.2
randomForest 4.6-12 RColorBrewer 1.1-2 Rcpp 0.12.13
RcppEigen 0.3.3.3.0 RcppRoll 0.2.2 RCurl 1.95-4.8
recipes 0.1.0 reshape2 1.4.2 rlang 0.1.2
robustbase 0.92-7 RODBC 1.3-15 roxygen2 6.0.1
rpart 4.1-11 rprojroot 1.2 Rserve 1.7-3
RSQLite 2.0 rstudioapi 0.7 scales 0.5.0
sfsmisc 1.1-1 sp 1.2-5 SparkR 2.2.0
疏鬆M 1.77 spatial 7.3-11 splines 3.4.2
sqldf 0.4-11 statmod 1.4.30 stats 3.4.2
stats4 3.4.2 stringi 1.1.5 stringr 1.2.0
survival 2.41-3 tcltk 3.4.2 TeachingDemos 2.10
testthat 1.0.2 tibble 1.3.4 tidyselect 0.2.0
timeDate 3012.100 tools 3.4.2 utils 3.4.2
viridisLite 0.2.0 晶須 0.3-2 withr 2.0.0
xml2 1.1.1

預安裝的 Java 和 Scala 連結庫 (Scala 2.10 叢集版本)

群組識別碼 成品標識碼 版本
antlr antlr 2.7.7
com.amazonaws amazon-kinesis-client 1.7.3
com.amazonaws aws-java-sdk-autoscaling 1.11.126
com.amazonaws aws-java-sdk-cloudformation 1.11.126
com.amazonaws aws-java-sdk-cloudfront 1.11.126
com.amazonaws aws-java-sdk-cloudhsm 1.11.126
com.amazonaws aws-java-sdk-cloudsearch 1.11.126
com.amazonaws aws-java-sdk-cloudtrail 1.11.126
com.amazonaws aws-java-sdk-cloudwatch 1.11.126
com.amazonaws aws-java-sdk-cloudwatchmetrics 1.11.126
com.amazonaws aws-java-sdk-codedeploy 1.11.126
com.amazonaws aws-java-sdk-cognitoidentity 1.11.126
com.amazonaws aws-java-sdk-cognitosync 1.11.126
com.amazonaws aws-java-sdk-config 1.11.126
com.amazonaws aws-java-sdk-core 1.11.126
com.amazonaws aws-java-sdk-datapipeline 1.11.126
com.amazonaws aws-java-sdk-directconnect 1.11.126
com.amazonaws aws-java-sdk-directory 1.11.126
com.amazonaws aws-java-sdk-dynamodb 1.11.126
com.amazonaws aws-java-sdk-ec2 1.11.126
com.amazonaws aws-java-sdk-ecs 1.11.126
com.amazonaws aws-java-sdk-efs 1.11.126
com.amazonaws aws-java-sdk-elasticache 1.11.126
com.amazonaws aws-java-sdk-elasticbeanstalk 1.11.126
com.amazonaws aws-java-sdk-elasticloadbalancing 1.11.126
com.amazonaws aws-java-sdk-elastictranscoder 1.11.126
com.amazonaws aws-java-sdk-emr 1.11.126
com.amazonaws aws-java-sdk-glacier 1.11.126
com.amazonaws aws-java-sdk-iam 1.11.126
com.amazonaws aws-java-sdk-importexport 1.11.126
com.amazonaws aws-java-sdk-kinesis 1.11.126
com.amazonaws aws-java-sdk-kms 1.11.126
com.amazonaws aws-java-sdk-lambda 1.11.126
com.amazonaws aws-java-sdk-logs 1.11.126
com.amazonaws aws-java-sdk-machinelearning 1.11.126
com.amazonaws aws-java-sdk-opsworks 1.11.126
com.amazonaws aws-java-sdk-rds 1.11.126
com.amazonaws aws-java-sdk-redshift 1.11.126
com.amazonaws aws-java-sdk-route53 1.11.126
com.amazonaws aws-java-sdk-s3 1.11.126
com.amazonaws aws-java-sdk-ses 1.11.126
com.amazonaws aws-java-sdk-simpledb 1.11.126
com.amazonaws aws-java-sdk-simpleworkflow 1.11.126
com.amazonaws aws-java-sdk-sns 1.11.126
com.amazonaws aws-java-sdk-sqs 1.11.126
com.amazonaws aws-java-sdk-ssm 1.11.126
com.amazonaws aws-java-sdk-storagegateway 1.11.126
com.amazonaws aws-java-sdk-sts 1.11.126
com.amazonaws aws-java-sdk-support 1.11.126
com.amazonaws aws-java-sdk-js-libraries 1.11.22
com.amazonaws aws-java-sdk-workspaces 1.11.126
com.amazonaws jmespath-java 1.11.126
com.carrotsearch hppc 0.7.1
com.chuusai shapeless_2.10 2.3.2
com.clearspring.analytics stream 2.7.0
com.databricks Rserve 1.8-3
com.databricks dbml-local_2.10 0.2.2-db1-spark2.2
com.databricks dbml-local_2.10-tests 0.2.2-db1-spark2.2
com.databricks jets3t 0.7.1-0
com.databricks.scalapb compilerplugin_2.10 0.4.15-9
com.databricks.scalapb scalapb-runtime_2.10 0.4.15-9
com.esotericsoftware kryo-shaded 3.0.3
com.esotericsoftware minlog 1.3.0
com.fasterxml 同學 1.0.0
com.fasterxml.jackson.core jackson-annotations 2.6.7
com.fasterxml.jackson.core jackson-core 2.6.7
com.fasterxml.jackson.core jackson-databind 2.6.7.1
com.fasterxml.jackson.dataformat jackson-dataformat-cbor 2.6.7
com.fasterxml.jackson.datatype jackson-datatype-joda 2.6.7
com.fasterxml.jackson.module jackson-module-paranamer 2.6.7
com.fasterxml.jackson.module jackson-module-scala_2.10 2.6.7.1
com.github.fommil jniloader 1.1
com.github.fommil.netlib 核心 1.1.2
com.github.fommil.netlib native_ref-java 1.1
com.github.fommil.netlib native_ref-java-natives 1.1
com.github.fommil.netlib native_system java 1.1
com.github.fommil.netlib native_system-java-natives 1.1
com.github.fommil.netlib netlib-native_ref-linux-x86_64-natives 1.1
com.github.fommil.netlib netlib-native_system-linux-x86_64-natives 1.1
com.github.rwl jtransforms 2.4.0
com.google.code.findbugs jsr305 2.0.1
com.google.code.gson gson 2.2.4
com.google.guava 番 石榴 15.0
com.google.protobuf protobuf-java 2.6.1
com.googlecode.javaewah JavaEWAH 0.3.2
com.h2database h2 1.3.174
com.jamesmurty.utils java-xmlbuilder 1.0
com.jcraft jsch 0.1.50
com.jolbox bonecp 0.8.0.RELEASE
com.mchange c3p0 0.9.5.1
com.mchange mchange-commons-java 0.2.10
com.microsoft.azure azure-data-lake-store-sdk 2.0.11
com.microsoft.sqlserver mssql-jdbc 6.1.0.jre8
com.ning compress-lzf 1.0.3
com.sun.mail javax.mail 1.5.2
com.thoughtworks.paranamer paranamer 2.6
com.trueaccord.lenses lenses_2.10 0.3
com.twitter chill-java 0.8.0
com.twitter chill_2.10 0.8.0
com.twitter parquet-hadoop-bundle 1.6.0
com.twitter util-app_2.10 6.23.0
com.twitter util-core_2.10 6.23.0
com.twitter util-jvm_2.10 6.23.0
com.typesafe config 1.2.1
com.typesafe scalalogging-slf4j_2.10 1.1.0
com.univocity univocity-parsers 2.2.1
com.vlkan flatbuffers 1.2.0-3f79e055
com.zaxxer HikariCP 2.4.1
commons-beanutils commons-beanutils 1.7.0
commons-beanutils commons-beanutils-core 1.8.0
commons-cli commons-cli 1.2
commons-codec commons-codec 1.10
commons-collections commons-collections 3.2.2
commons-configuration commons-configuration 1.6
commons-dbcp commons-dbcp 1.4
commons-digester commons-digester 1.8
commons-HTTPclient commons-HTTPclient 3.1
commons-io commons-io 2.4
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-net commons-net 2.2
commons-pool commons-pool 1.5.4
info.ganglia.gmetric4j gmetric4j 1.0.7
io.dropwizard.metrics metrics-core 3.1.2
io.dropwizard.metrics metrics-ganglia 3.1.2
io.dropwizard.metrics metrics-graphite 3.1.2
io.dropwizard.metrics metrics-healthchecks 3.1.2
io.dropwizard.metrics metrics-jetty9 3.1.2
io.dropwizard.metrics metrics-json 3.1.2
io.dropwizard.metrics metrics-jvm 3.1.2
io.dropwizard.metrics metrics-log4j 3.1.2
io.dropwizard.metrics metrics-servlet 3.1.2
io.netty netty 3.9.9.Final
io.netty netty-all 4.0.43.Final
io.prometheus simpleclient 0.0.16
io.prometheus simpleclient_common 0.0.16
io.prometheus simpleclient_dropwizard 0.0.16
io.prometheus simpleclient_servlet 0.0.16
io.prometheus.jmx 收藏家 0.7
javax.activation 啟用 1.1.1
javax.annotation javax.annotation-api 1.2
javax.el javax.el-api 2.2.4
javax.jdo jdo-api 3.0.1
javax.servlet javax.servlet-api 3.1.0
javax.servlet.jsp jsp-api 2.1
javax.transaction Jta 1.1
javax.validation validation-api 1.1.0.Final
javax.ws.rs javax.ws.rs-api 2.0.1
javax.xml.bind jaxb-api 2.2.2
javax.xml.stream stax-api 1.0-2
javolution javolution 5.5.1
jline jline 2.11
joda-time joda-time 2.9.3
Log4j apache-log4j-extras 1.2.17
Log4j Log4j 1.2.17
mx4j mx4j 3.0.2
net.hydromatic eigenbase-properties 1.1.5
net.iharder base64 2.3.8
net.java.dev.jets3t jets3t 0.9.3
net.jpountz.lz4 lz4 1.3.0
net.razorvine pyolite 4.13
net.sf.jpam jpam 1.1
net.sf.opencsv opencsv 2.3
net.sf.supercsv super-csv 2.2.0
net.sourceforge.f2j arpack_combined_all 0.1
org.acplt oncrpc 1.0.7
org.antlr ST4 4.0.4
org.antlr antlr-runtime 3.4
org.antlr antlr4-runtime 4.5.3
org.antlr stringtemplate 3.2.1
org.apache.ant ant 1.9.2
org.apache.ant ant-jsch 1.9.2
org.apache.ant ant-launcher 1.9.2
org.apache.arrow arrow-format 0.4.0
org.apache.arrow arrow-memory 0.4.0
org.apache.arrow arrow-vector 0.4.0
org.apache.avro avro 1.7.7
org.apache.avro avro-ipc 1.7.7
org.apache.avro avro-ipc-tests 1.7.7
org.apache.avro avro-mapred-hadoop2 1.7.7
org.apache.calcite calcite-avatica 1.2.0-incubating
org.apache.calcite calcite-core 1.2.0-incubating
org.apache.calcite calcite-linq4j 1.2.0-incubating
org.apache.commons commons-compress 1.4.1
org.apache.commons commons-crypto 1.0.0
org.apache.commons commons-lang3 3.5
org.apache.commons commons-math3 3.4.1
org.apache.curator curator-client 2.6.0
org.apache.curator curator-framework 2.6.0
org.apache.curator 策展人-食譜 2.6.0
org.apache.derby Derby 10.10.2.0
org.apache.directory.api api-asn1-api 1.0.0-M20
org.apache.directory.api api-util 1.0.0-M20
org.apache.directory.server apacheds-i18n 2.0.0-M15
org.apache.directory.server apacheds-kerberos-codec 2.0.0-M15
org.apache.hadoop hadoop-annotations 2.7.3
org.apache.hadoop hadoop-auth 2.7.3
org.apache.hadoop hadoop-client 2.7.3
org.apache.hadoop hadoop-common 2.7.3
org.apache.hadoop hadoop-hdfs 2.7.3
org.apache.hadoop hadoop-mapreduce-client-app 2.7.3
org.apache.hadoop hadoop-mapreduce-client-common 2.7.3
org.apache.hadoop hadoop-mapreduce-client-core 2.7.3
org.apache.hadoop hadoop-mapreduce-client-jobclient 2.7.3
org.apache.hadoop hadoop-mapreduce-client-shuffle 2.7.3
org.apache.hadoop hadoop-yarn-api 2.7.3
org.apache.hadoop hadoop-yarn-client 2.7.3
org.apache.hadoop hadoop-yarn-common 2.7.3
org.apache.hadoop hadoop-yarn-server-common 2.7.3
org.apache.htrace htrace-core 3.1.0-incubating
org.apache.httpcomponents httpclient 4.5.2
org.apache.httpcomponents httpcore 4.4.4
org.apache.ivy 常春藤 2.4.0
org.apache.parquet parquet-column 1.8.2-databricks1
org.apache.parquet parquet-common 1.8.2-databricks1
org.apache.parquet parquet-encoding 1.8.2-databricks1
org.apache.parquet parquet-format 2.3.1
org.apache.parquet parquet-hadoop 1.8.2-databricks1
org.apache.parquet parquet-jackson 1.8.2-databricks1
org.apache.thrift libfb303 0.9.3
org.apache.thrift libthrift 0.9.3
org.apache.xbean xbean-asm5-shaded 4.4
org.apache.zookeeper zookeeper 3.4.6
org.bouncycastle bcprov-jdk15on 1.51
org.codehaus.jackson jackson-core-asl 1.9.13
org.codehaus.jackson jackson-jaxrs 1.9.13
org.codehaus.jackson jackson-mapper-asl 1.9.13
org.codehaus.jackson jackson-xc 1.9.13
org.codehaus.janino commons-compiler 3.0.0
org.codehaus.janino janino 3.0.0
org.datanucleus datanucleus-api-jdo 3.2.6
org.datanucleus datanucleus-core 3.2.10
org.datanucleus datanucleus-rdbms 3.2.9
org.eclipse.jetty jetty-client 9.3.11.v20160721
org.eclipse.jetty jetty-continuation 9.3.11.v20160721
org.eclipse.jetty jetty-http 9.3.11.v20160721
org.eclipse.jetty jetty-io 9.3.11.v20160721
org.eclipse.jetty jetty-jndi 9.3.11.v20160721
org.eclipse.jetty jetty-plus 9.3.11.v20160721
org.eclipse.jetty jetty-proxy 9.3.11.v20160721
org.eclipse.jetty jetty-security 9.3.11.v20160721
org.eclipse.jetty jetty-server 9.3.11.v20160721
org.eclipse.jetty jetty-servlet 9.3.11.v20160721
org.eclipse.jetty jetty-servlets 9.3.11.v20160721
org.eclipse.jetty jetty-util 9.3.11.v20160721
org.eclipse.jetty jetty-webapp 9.3.11.v20160721
org.eclipse.jetty jetty-xml 9.3.11.v20160721
org.fusesource.jansi jansi 1.4
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.4.0-b34
org.glassfish.hk2 hk2-locator 2.4.0-b34
org.glassfish.hk2 hk2-utils 2.4.0-b34
org.glassfish.hk2 osgi-resource-locator 1.0.1
org.glassfish.hk2.external aopalliance-repackaged 2.4.0-b34
org.glassfish.hk2.external javax.inject 2.4.0-b34
org.glassfish.jersey.bundles.repackaged 澤西-瓜瓦 2.22.2
org.glassfish.jersey.containers jersey-container-servlet 2.22.2
org.glassfish.jersey.containers jersey-container-servlet-core 2.22.2
org.glassfish.jersey.core jersey-client 2.22.2
org.glassfish.jersey.core 澤西-通用 2.22.2
org.glassfish.jersey.core jersey-server 2.22.2
org.glassfish.jersey.media jersey-media-jaxb 2.22.2
org.hibernate hibernate-validator 5.1.1.Final
org.iq80.snappy snappy 0.2
org.javassist javassist 3.18.1-GA
org.jboss.logging jboss-logging 3.1.3.GA
org.jdbi jdbi 2.63.1
org.joda joda-convert 1.7
org.jodd jodd-core 3.5.2
org.jpmml pmml-model 1.2.15
org.jpmml pmml-schema 1.2.15
org.json4s json4s-ast_2.10 3.2.11
org.json4s json4s-core_2.10 3.2.11
org.json4s json4s-jackson_2.10 3.2.11
org.mariadb.jdbc mariadb-java-client 2.1.2
org.mockito mockito-all 1.9.5
org.objenesis objenesis 2.1
org.postgresql postgresql 9.4-1204-jdbc41
org.roaringbitmap RoaringBitmap 0.5.11
org.rocksdb rocksdbjni 5.2.1
org.rosuda.REngine REngine 2.1.0
org.scala-lang jline 2.10.6
org.scala-lang scala-compiler_2.10 2.10.6
org.scala-lang scala-library_2.10 2.10.6
org.scala-lang scala-reflect_2.10 2.10.6
org.scala-lang scalap_2.10 2.10.6
org.scala-sbt test-interface 1.0
org.scalacheck scalacheck_2.10 1.12.5
org.scalamacros quasiquotes_2.10 2.0.0
org.scalanlp breeze-macros_2.10 0.13.2
org.scalanlp breeze_2.10 0.13.2
org.scalatest scalatest_2.10 2.2.6
org.slf4j jcl-over-slf4j 1.7.16
org.slf4j jul-to-slf4j 1.7.16
org.slf4j slf4j-api 1.7.16
org.slf4j slf4j-log4j12 1.7.16
org.spark-project.hive hive-beeline 1.2.1.spark2
org.spark-project.hive hive-cli 1.2.1.spark2
org.spark-project.hive hive-exec 1.2.1.spark2
org.spark-project.hive hive-jdbc 1.2.1.spark2
org.spark-project.hive hive-metastore 1.2.1.spark2
org.spark-project.spark 閒置 1.0.0
org.spire-math spire-macros_2.10 0.13.0
org.spire-math spire_2.10 0.13.0
org.springframework spring-core 4.1.4.RELEASE
org.springframework spring-test 4.1.4.RELEASE
org.tukaani xz 1.0
org.typelevel machinist_2.10 0.6.1
org.typelevel macro-compat_2.10 1.1.1
org.xerial sqlite-jdbc 3.8.11.2
org.xerial.snappy snappy-java 1.1.2.6
org.yaml 蛇形 1.16
oro oro 2.0.8
software.amazon.ion ion-java 1.0.2
Stax stax-api 1.0.1
xmlenc xmlenc 0.52

預先安裝的 JAVA 和 Scala 程式庫 (Scala 2.11 叢集版本)

群組識別碼 成品識別碼 版本
antlr antlr 2.7.7
com.amazonaws amazon-kinesis-client 1.7.3
com.amazonaws aws-java-sdk-autoscaling 1.11.126
com.amazonaws aws-java-sdk-cloudformation 1.11.126
com.amazonaws aws-java-sdk-cloudfront 1.11.126
com.amazonaws aws-java-sdk-cloudhsm 1.11.126
com.amazonaws aws-java-sdk-cloudsearch 1.11.126
com.amazonaws aws-java-sdk-cloudtrail 1.11.126
com.amazonaws aws-java-sdk-cloudwatch 1.11.126
com.amazonaws aws-java-sdk-cloudwatchmetrics 1.11.126
com.amazonaws aws-java-sdk-codedeploy 1.11.126
com.amazonaws aws-java-sdk-cognitoidentity 1.11.126
com.amazonaws aws-java-sdk-cognitosync 1.11.126
com.amazonaws aws-java-sdk-config 1.11.126
com.amazonaws aws-java-sdk-core 1.11.126
com.amazonaws aws-java-sdk-datapipeline 1.11.126
com.amazonaws aws-java-sdk-directconnect 1.11.126
com.amazonaws aws-java-sdk-directory 1.11.126
com.amazonaws aws-java-sdk-dynamodb 1.11.126
com.amazonaws aws-java-sdk-ec2 1.11.126
com.amazonaws aws-java-sdk-ecs 1.11.126
com.amazonaws aws-java-sdk-efs 1.11.126
com.amazonaws aws-java-sdk-elasticache 1.11.126
com.amazonaws aws-java-sdk-elasticbeanstalk 1.11.126
com.amazonaws aws-java-sdk-elasticloadbalancing 1.11.126
com.amazonaws aws-java-sdk-elastictranscoder 1.11.126
com.amazonaws aws-java-sdk-emr 1.11.126
com.amazonaws aws-java-sdk-glacier 1.11.126
com.amazonaws aws-java-sdk-iam 1.11.126
com.amazonaws aws-java-sdk-importexport 1.11.126
com.amazonaws aws-java-sdk-kinesis 1.11.126
com.amazonaws aws-java-sdk-kms 1.11.126
com.amazonaws aws-java-sdk-lambda 1.11.126
com.amazonaws aws-java-sdk-logs 1.11.126
com.amazonaws aws-java-sdk-machinelearning 1.11.126
com.amazonaws aws-java-sdk-opsworks 1.11.126
com.amazonaws aws-java-sdk-rds 1.11.126
com.amazonaws aws-java-sdk-redshift 1.11.126
com.amazonaws aws-java-sdk-route53 1.11.126
com.amazonaws aws-java-sdk-s3 1.11.126
com.amazonaws aws-java-sdk-ses 1.11.126
com.amazonaws aws-java-sdk-simpledb 1.11.126
com.amazonaws aws-java-sdk-simpleworkflow 1.11.126
com.amazonaws aws-java-sdk-sns 1.11.126
com.amazonaws aws-java-sdk-sqs 1.11.126
com.amazonaws aws-java-sdk-ssm 1.11.126
com.amazonaws aws-java-sdk-storagegateway 1.11.126
com.amazonaws aws-java-sdk-sts 1.11.126
com.amazonaws aws-java-sdk-support 1.11.126
com.amazonaws aws-java-sdk-js-libraries 1.11.22
com.amazonaws aws-java-sdk-workspaces 1.11.126
com.amazonaws jmespath-java 1.11.126
com.carrotsearch hppc 0.7.1
com.chuusai shapeless_2.11 2.3.2
com.clearspring.analytics stream 2.7.0
com.databricks Rserve 1.8-3
com.databricks dbml-local_2.11 0.2.2-db1-spark2.2
com.databricks dbml-local_2.11-tests 0.2.2-db1-spark2.2
com.databricks jets3t 0.7.1-0
com.databricks.scalapb compilerplugin_2.11 0.4.15-9
com.databricks.scalapb scalapb-runtime_2.11 0.4.15-9
com.esotericsoftware kryo-shaded 3.0.3
com.esotericsoftware minlog 1.3.0
com.fasterxml 同學 1.0.0
com.fasterxml.jackson.core jackson-annotations 2.6.7
com.fasterxml.jackson.core jackson-core 2.6.7
com.fasterxml.jackson.core jackson-databind 2.6.7.1
com.fasterxml.jackson.dataformat jackson-dataformat-cbor 2.6.7
com.fasterxml.jackson.datatype jackson-datatype-joda 2.6.7
com.fasterxml.jackson.module jackson-module-paranamer 2.6.7
com.fasterxml.jackson.module jackson-module-scala_2.11 2.6.7.1
com.github.fommil jniloader 1.1
com.github.fommil.netlib 核心 1.1.2
com.github.fommil.netlib native_ref-java 1.1
com.github.fommil.netlib native_ref-java-natives 1.1
com.github.fommil.netlib native_system java 1.1
com.github.fommil.netlib native_system-java-natives 1.1
com.github.fommil.netlib netlib-native_ref-linux-x86_64-natives 1.1
com.github.fommil.netlib netlib-native_system-linux-x86_64-natives 1.1
com.github.rwl jtransforms 2.4.0
com.google.code.findbugs jsr305 2.0.1
com.google.code.gson gson 2.2.4
com.google.guava 番 石榴 15.0
com.google.protobuf protobuf-java 2.6.1
com.googlecode.javaewah JavaEWAH 0.3.2
com.h2database h2 1.3.174
com.jamesmurty.utils java-xmlbuilder 1.0
com.jcraft jsch 0.1.50
com.jolbox bonecp 0.8.0.RELEASE
com.mchange c3p0 0.9.5.1
com.mchange mchange-commons-java 0.2.10
com.microsoft.azure azure-data-lake-store-sdk 2.0.11
com.microsoft.sqlserver mssql-jdbc 6.1.0.jre8
com.ning compress-lzf 1.0.3
com.sun.mail javax.mail 1.5.2
com.thoughtworks.paranamer paranamer 2.6
com.trueaccord.lenses lenses_2.11 0.3
com.twitter chill-java 0.8.0
com.twitter chill_2.11 0.8.0
com.twitter parquet-hadoop-bundle 1.6.0
com.twitter util-app_2.11 6.23.0
com.twitter util-core_2.11 6.23.0
com.twitter util-jvm_2.11 6.23.0
com.typesafe config 1.2.1
com.typesafe.scala-logging scala-logging-api_2.11 2.1.2
com.typesafe.scala-logging scala-logging-slf4j_2.11 2.1.2
com.univocity univocity-parsers 2.2.1
com.vlkan flatbuffers 1.2.0-3f79e055
com.zaxxer HikariCP 2.4.1
commons-beanutils commons-beanutils 1.7.0
commons-beanutils commons-beanutils-core 1.8.0
commons-cli commons-cli 1.2
commons-codec commons-codec 1.10
commons-collections commons-collections 3.2.2
commons-configuration commons-configuration 1.6
commons-dbcp commons-dbcp 1.4
commons-digester commons-digester 1.8
commons-HTTPclient commons-HTTPclient 3.1
commons-io commons-io 2.4
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-net commons-net 2.2
commons-pool commons-pool 1.5.4
info.ganglia.gmetric4j gmetric4j 1.0.7
io.dropwizard.metrics metrics-core 3.1.2
io.dropwizard.metrics metrics-ganglia 3.1.2
io.dropwizard.metrics metrics-graphite 3.1.2
io.dropwizard.metrics metrics-healthchecks 3.1.2
io.dropwizard.metrics metrics-jetty9 3.1.2
io.dropwizard.metrics metrics-json 3.1.2
io.dropwizard.metrics metrics-jvm 3.1.2
io.dropwizard.metrics metrics-log4j 3.1.2
io.dropwizard.metrics metrics-servlet 3.1.2
io.netty netty 3.9.9.Final
io.netty netty-all 4.0.43.Final
io.prometheus simpleclient 0.0.16
io.prometheus simpleclient_common 0.0.16
io.prometheus simpleclient_dropwizard 0.0.16
io.prometheus simpleclient_servlet 0.0.16
io.prometheus.jmx 收藏家 0.7
javax.activation 啟用 1.1.1
javax.annotation javax.annotation-api 1.2
javax.el javax.el-api 2.2.4
javax.jdo jdo-api 3.0.1
javax.servlet javax.servlet-api 3.1.0
javax.servlet.jsp jsp-api 2.1
javax.transaction Jta 1.1
javax.validation validation-api 1.1.0.Final
javax.ws.rs javax.ws.rs-api 2.0.1
javax.xml.bind jaxb-api 2.2.2
javax.xml.stream stax-api 1.0-2
javolution javolution 5.5.1
jline jline 2.11
joda-time joda-time 2.9.3
Log4j apache-log4j-extras 1.2.17
Log4j Log4j 1.2.17
mx4j mx4j 3.0.2
net.hydromatic eigenbase-properties 1.1.5
net.iharder base64 2.3.8
net.java.dev.jets3t jets3t 0.9.3
net.jpountz.lz4 lz4 1.3.0
net.razorvine pyolite 4.13
net.sf.jpam jpam 1.1
net.sf.opencsv opencsv 2.3
net.sf.supercsv super-csv 2.2.0
net.sourceforge.f2j arpack_combined_all 0.1
org.acplt oncrpc 1.0.7
org.antlr ST4 4.0.4
org.antlr antlr-runtime 3.4
org.antlr antlr4-runtime 4.5.3
org.antlr stringtemplate 3.2.1
org.apache.ant ant 1.9.2
org.apache.ant ant-jsch 1.9.2
org.apache.ant ant-launcher 1.9.2
org.apache.arrow arrow-format 0.4.0
org.apache.arrow arrow-memory 0.4.0
org.apache.arrow arrow-vector 0.4.0
org.apache.avro avro 1.7.7
org.apache.avro avro-ipc 1.7.7
org.apache.avro avro-ipc-tests 1.7.7
org.apache.avro avro-mapred-hadoop2 1.7.7
org.apache.calcite calcite-avatica 1.2.0-incubating
org.apache.calcite calcite-core 1.2.0-incubating
org.apache.calcite calcite-linq4j 1.2.0-incubating
org.apache.commons commons-compress 1.4.1
org.apache.commons commons-crypto 1.0.0
org.apache.commons commons-lang3 3.5
org.apache.commons commons-math3 3.4.1
org.apache.curator curator-client 2.6.0
org.apache.curator curator-framework 2.6.0
org.apache.curator 策展人-食譜 2.6.0
org.apache.derby Derby 10.10.2.0
org.apache.directory.api api-asn1-api 1.0.0-M20
org.apache.directory.api api-util 1.0.0-M20
org.apache.directory.server apacheds-i18n 2.0.0-M15
org.apache.directory.server apacheds-kerberos-codec 2.0.0-M15
org.apache.hadoop hadoop-annotations 2.7.3
org.apache.hadoop hadoop-auth 2.7.3
org.apache.hadoop hadoop-client 2.7.3
org.apache.hadoop hadoop-common 2.7.3
org.apache.hadoop hadoop-hdfs 2.7.3
org.apache.hadoop hadoop-mapreduce-client-app 2.7.3
org.apache.hadoop hadoop-mapreduce-client-common 2.7.3
org.apache.hadoop hadoop-mapreduce-client-core 2.7.3
org.apache.hadoop hadoop-mapreduce-client-jobclient 2.7.3
org.apache.hadoop hadoop-mapreduce-client-shuffle 2.7.3
org.apache.hadoop hadoop-yarn-api 2.7.3
org.apache.hadoop hadoop-yarn-client 2.7.3
org.apache.hadoop hadoop-yarn-common 2.7.3
org.apache.hadoop hadoop-yarn-server-common 2.7.3
org.apache.htrace htrace-core 3.1.0-incubating
org.apache.httpcomponents httpclient 4.5.2
org.apache.httpcomponents httpcore 4.4.4
org.apache.ivy 常春藤 2.4.0
org.apache.parquet parquet-column 1.8.2-databricks1
org.apache.parquet parquet-common 1.8.2-databricks1
org.apache.parquet parquet-encoding 1.8.2-databricks1
org.apache.parquet parquet-format 2.3.1
org.apache.parquet parquet-hadoop 1.8.2-databricks1
org.apache.parquet parquet-jackson 1.8.2-databricks1
org.apache.thrift libfb303 0.9.3
org.apache.thrift libthrift 0.9.3
org.apache.xbean xbean-asm5-shaded 4.4
org.apache.zookeeper zookeeper 3.4.6
org.bouncycastle bcprov-jdk15on 1.51
org.codehaus.jackson jackson-core-asl 1.9.13
org.codehaus.jackson jackson-jaxrs 1.9.13
org.codehaus.jackson jackson-mapper-asl 1.9.13
org.codehaus.jackson jackson-xc 1.9.13
org.codehaus.janino commons-compiler 3.0.0
org.codehaus.janino janino 3.0.0
org.datanucleus datanucleus-api-jdo 3.2.6
org.datanucleus datanucleus-core 3.2.10
org.datanucleus datanucleus-rdbms 3.2.9
org.eclipse.jetty jetty-client 9.3.11.v20160721
org.eclipse.jetty jetty-continuation 9.3.11.v20160721
org.eclipse.jetty jetty-http 9.3.11.v20160721
org.eclipse.jetty jetty-io 9.3.11.v20160721
org.eclipse.jetty jetty-jndi 9.3.11.v20160721
org.eclipse.jetty jetty-plus 9.3.11.v20160721
org.eclipse.jetty jetty-proxy 9.3.11.v20160721
org.eclipse.jetty jetty-security 9.3.11.v20160721
org.eclipse.jetty jetty-server 9.3.11.v20160721
org.eclipse.jetty jetty-servlet 9.3.11.v20160721
org.eclipse.jetty jetty-servlets 9.3.11.v20160721
org.eclipse.jetty jetty-util 9.3.11.v20160721
org.eclipse.jetty jetty-webapp 9.3.11.v20160721
org.eclipse.jetty jetty-xml 9.3.11.v20160721
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.4.0-b34
org.glassfish.hk2 hk2-locator 2.4.0-b34
org.glassfish.hk2 hk2-utils 2.4.0-b34
org.glassfish.hk2 osgi-resource-locator 1.0.1
org.glassfish.hk2.external aopalliance-repackaged 2.4.0-b34
org.glassfish.hk2.external javax.inject 2.4.0-b34
org.glassfish.jersey.bundles.repackaged 澤西-瓜瓦 2.22.2
org.glassfish.jersey.containers jersey-container-servlet 2.22.2
org.glassfish.jersey.containers jersey-container-servlet-core 2.22.2
org.glassfish.jersey.core jersey-client 2.22.2
org.glassfish.jersey.core 澤西-通用 2.22.2
org.glassfish.jersey.core jersey-server 2.22.2
org.glassfish.jersey.media jersey-media-jaxb 2.22.2
org.hibernate hibernate-validator 5.1.1.Final
org.iq80.snappy snappy 0.2
org.javassist javassist 3.18.1-GA
org.jboss.logging jboss-logging 3.1.3.GA
org.jdbi jdbi 2.63.1
org.joda joda-convert 1.7
org.jodd jodd-core 3.5.2
org.jpmml pmml-model 1.2.15
org.jpmml pmml-schema 1.2.15
org.json4s json4s-ast_2.11 3.2.11
org.json4s json4s-core_2.11 3.2.11
org.json4s json4s-jackson_2.11 3.2.11
org.mariadb.jdbc mariadb-java-client 2.1.2
org.mockito mockito-all 1.9.5
org.objenesis objenesis 2.1
org.postgresql postgresql 9.4-1204-jdbc41
org.roaringbitmap RoaringBitmap 0.5.11
org.rocksdb rocksdbjni 5.2.1
org.rosuda.REngine REngine 2.1.0
org.scala-lang scala-compiler_2.11 2.11.8
org.scala-lang scala-library_2.11 2.11.8
org.scala-lang scala-reflect_2.11 2.11.8
org.scala-lang scalap_2.11 2.11.8
org.scala-lang.modules scala-parser-combinators_2.11 1.0.2
org.scala-lang.modules scala-xml_2.11 1.0.2
org.scala-sbt test-interface 1.0
org.scalacheck scalacheck_2.11 1.12.5
org.scalanlp breeze-macros_2.11 0.13.2
org.scalanlp breeze_2.11 0.13.2
org.scalatest scalatest_2.11 2.2.6
org.slf4j jcl-over-slf4j 1.7.16
org.slf4j jul-to-slf4j 1.7.16
org.slf4j slf4j-api 1.7.16
org.slf4j slf4j-log4j12 1.7.16
org.spark-project.hive hive-beeline 1.2.1.spark2
org.spark-project.hive hive-cli 1.2.1.spark2
org.spark-project.hive hive-exec 1.2.1.spark2
org.spark-project.hive hive-jdbc 1.2.1.spark2
org.spark-project.hive hive-metastore 1.2.1.spark2
org.spark-project.spark 閑置 1.0.0
org.spire-math spire-macros_2.11 0.13.0
org.spire-math spire_2.11 0.13.0
org.springframework spring-core 4.1.4.RELEASE
org.springframework spring-test 4.1.4.RELEASE
org.tukaani xz 1.0
org.typelevel machinist_2.11 0.6.1
org.typelevel macro-compat_2.11 1.1.1
org.xerial sqlite-jdbc 3.8.11.2
org.xerial.snappy snappy-java 1.1.2.6
org.yaml 蛇形 1.16
oro oro 2.0.8
software.amazon.ion ion-java 1.0.2
Stax stax-api 1.0.1
xmlenc xmlenc 0.52