Databricks Connect 发行说明Databricks Connect release notes

此页列出了为 Databricks Connect的版本和维护更新。This page lists releases and maintenance updates issued for Databricks Connect.

Databricks Connect 的主要和次要包版本必须始终与 Databricks Runtime 版本匹配。The Databricks Connect major and minor package version must always match your Databricks Runtime version. Databricks 建议始终使用与 Databricks Runtime 版本相匹配的 Databricks Connect 的最新补丁版本。Databricks recommends that you always use the most recent patch version of Databricks Connect that matches your Databricks Runtime version. 例如,使用 Databricks Runtime 7.3 群集时,请使用最新的 databricks-connect==7.3.* 包。For example, when you use a Databricks Runtime 7.3 cluster, use the latest databricks-connect==7.3.* package.

适用于 Databricks Runtime 7.3 LTS 的 Databricks ConnectDatabricks Connect for Databricks Runtime 7.3 LTS

Databricks Connect 7.3.6Databricks Connect 7.3.6

2021 年 1 月 12 日January 12, 2021

  • [ES-59054] 通过 Databricks Connect 修复 Delta 优化表命令[ES-59054] Fix Delta Optimize Table command through Databricks Connect
  • [ES-53497] 修复 Databricks 容器服务 (DCS) 上的 Databricks Connect[ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)

Databricks Connect 7.3.5Databricks Connect 7.3.5

2020年 12 月 9 日December 9, 2020

Databricks Connect 7.3 现已正式发布。Databricks Connect 7.3 is now GA.

  • [ES-49865] 防止在将 Databricks Connect 用于 SBT 时由于依赖库自动同步而发生 500 内部服务器错误。[ES-49865] Prevent 500 Internal Server Errors due to automatic synchronization of dependent libraries when using Databricks Connect with SBT.

Databricks Connect 7.3.4 Beta 版本Databricks Connect 7.3.4 Beta

2020 年 11 月 24 日November 24, 2020

  • [ES-44382] 将 spark.databricks.service.allowTerminatedClusterStart=false 设置为阻止 Databricks Connect 启动已终止的 Databricks Runtime 群集。[ES-44382] Set spark.databricks.service.allowTerminatedClusterStart=false to prevent Databricks Connect from starting a terminated Databricks Runtime cluster.
  • 修复了在禁用 INFO 日志记录级别的情况下客户端无法与服务器正确同步状态的错误。Fix an error where client could not synchronize state with server correctly if INFO logging level was disabled.

Databricks Connect 7.3.3 Beta 版本Databricks Connect 7.3.3 Beta

2020 年 11 月 3 日November 3, 2020

适用于 Databricks Runtime 7.3 的初始 Databricks Connect 版本。Initial Databricks Connect release for Databricks Runtime 7.3. 此版本包括:This release includes:

  • 支持 Azure Active Directory 凭据直通。Support for Azure Active Directory credential passthrough. 通过 Databricks Connect,现在可以对标准群集使用 Azure Active Directory 凭据直通。You can now use Azure Active Directory credential passthrough with Standard clusters using Databricks Connect. 这样就可以通过 Databricks Connect 使用对 Azure Databricks 进行身份验证所用的 Azure Active Directory 标识对 Azure Data Lake Storage Gen1Azure Data Lake Storage Gen2 自动进行身份验证。This enables you to authenticate automatically to Azure Data Lake Storage Gen1 and Azure Data Lake Storage Gen2 from Databricks Connect using the same Azure Active Directory identity that you use to authenticate to Azure Databricks.

  • 支持 Delta Lake 按时间顺序查看。Support for Delta Lake time travel.

  • 使用 DBUtils 时,在 Databricks Connect 客户端与 Databricks Runtime 作业或笔记本之间轻松转换。Easy transition between Databricks Connect clients and Databricks Runtime jobs or notebooks when using DBUtils. 请参阅访问 DBUtilsSee Access DBUtils.

已知问题:Known issues:

Databricks Runtime 版本验证在此 beta 版本中被禁用。Databricks Runtime version validation is disabled in this beta release. 请确保将此客户端连接到 7.3 Databricks Runtime 群集。Make sure to connect this client to a 7.3 Databricks Runtime cluster.

适用于 Databricks Runtime 7.1 的 Databricks ConnectDatabricks Connect for Databricks Runtime 7.1

Databricks Connect 7.1.12Databricks Connect 7.1.12

2021 年 1 月 12 日January 12, 2021

  • [ES-59054] 通过 Databricks Connect 修复 Delta 优化表命令[ES-59054] Fix Delta Optimize Table command through Databricks Connect
  • [ES-53497] 修复 Databricks 容器服务 (DCS) 上的 Databricks Connect[ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)

Databricks Connect 7.1.11Databricks Connect 7.1.11

2020年 12 月 9 日December 9, 2020

  • [ES-49865] 防止在将 Databricks Connect 用于 SBT 时由于依赖库自动同步而发生 500 内部服务器错误。[ES-49865] Prevent 500 Internal Server Errors due to automatic synchronization of dependent libraries when using Databricks Connect with SBT.

Databricks Connect 7.1.10Databricks Connect 7.1.10

2020 年 11 月 24 日November 24, 2020

  • 防止状态轮询重新启动群集。Prevent status polls from restarting the cluster.
  • [ES-43656] 修复与 Azure Data Lake Storage Gen2 支持相关的序列化错误。[ES-43656] Fix serialization error related to Azure Data Lake Storage Gen2 support.
  • [ES-30594] 允许更大的 HTTP 标头并扩大 Databricks Connect 中的缓冲区。[ES-30594] Allow larger HTTP headers and enlarge buffers in Databricks Connect.
  • [ES-34186] 避免发生与日志记录相关的内存不足异常。[ES-34186] Avoid out of memory exception related to logging.
  • [ES-39493] 防止 databricks-connect test中出现递归加载错误。[ES-39493] Prevent recursive load error in databricks-connect test.
  • [ES-44382] 将 spark.databricks.service.allowTerminatedClusterStart=false 设置为阻止 Databricks Connect 启动已终止的 Databricks Runtime 群集。[ES-44382] Set spark.databricks.service.allowTerminatedClusterStart=false to prevent Databricks Connect from starting a Databricks Runtime cluster that is terminated.
  • 修复了在禁用 INFO 日志记录级别的情况下客户端无法与服务器正确同步状态的错误。Fix an error in which client could not synchronize state with server correctly if INFO logging level was disabled.
  • 一些小问题修复。Minor fixes.

Databricks Connect 7.1.1Databricks Connect 7.1.1

2020 年 9 月 23 日September 23, 2020

  • [ES-32536] 修复与嵌套时区感知表达式的序列化相关的问题。[ES-32536] Fix an issue with serialization of nested timezone-aware expressions.
  • [ES-33705] 支持 DButils 中的 Azure Data Lake Storage Gen2 文件系统。[ES-33705] Support Azure Data Lake Storage Gen2 filesystem in DButils.

Databricks Connect 7.1.0Databricks Connect 7.1.0

2020 年 8 月 13 日August 13, 2020

适用于 Databricks Runtime 7.1 的初始 Databricks Connect 版本。Initial Databricks Connect release for Databricks Runtime 7.1.

  • 支持新的 Spark 3.0 DataSource V2 API。Support new Spark 3.0 DataSource V2 APIs.
  • 支持在单个 Databricks Connect 应用程序中使用多个 SparkSession。Support using multiple SparkSessions in a single Databricks Connect application.
  • 优化客户端和服务器之间的 Databricks Connect 状态同步。Optimize Databricks Connect state synchronization between client and server. 当服务器处于空闲状态时,请避免过多轮询状态更新。Avoid excessive polling for status updates when the server is idle.

适用于 Databricks Runtime 6.4 的 Databricks ConnectDatabricks Connect for Databricks Runtime 6.4

Databricks Connect 6.4.22Databricks Connect 6.4.22

2021 年 1 月 12 日January 12, 2021

  • [ES-59054] 通过 Databricks Connect 修复 Delta 优化表命令[ES-59054] Fix Delta Optimize Table command through Databricks Connect
  • [ES-53497] 修复 Databricks 容器服务 (DCS) 上的 Databricks Connect[ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)

Databricks Connect 6.4.21Databricks Connect 6.4.21

2020年 12 月 9 日December 9, 2020

  • [ES-25890] 修复以下错误:在某些情况下,数据帧 cache() 无效并再次执行查询。[ES-25890] Fix an error where in some cases DataFrame cache() would have no effect and the query would be executed again.
  • [ES-26196] 支持在联接中使用 NATURAL 和 USING 语法。[ES-26196] Support using NATURAL and USING syntax in joins.
  • [ES-33475][ES-34186] 避免发生与日志记录相关的内存不足异常。[ES-33475][ES-34186] Avoid out of memory exception related to logging.
  • [ES-39493] 防止 databricks-connect 测试中出现递归加载错误。[ES-39493] Prevent recursive load error in databricks-connect test.
  • [ES-43656] 修复与 Azure Data Lake Storage Gen2 支持相关的序列化错误。[ES-43656] Fix serialization error related to Azure Data Lake Storage Gen2 support.
  • [ES-30594] 允许更大的 HTTP 标头并扩大 Databricks Connect 中的缓冲区。[ES-30594] Allow larger HTTP headers and enlarge buffers in Databricks Connect.
  • 一些小问题修复Minor fixes

Databricks Connect 6.4.2Databricks Connect 6.4.2

2020 年 11 月 5 日November 5, 2020

  • [ES-47216] 通过 Databricks Runtime 6.4 2020 年 11 月 3 日的更新,修复了版本检查兼容性问题。[ES-47216] Fix version check compatibility issue with Databricks Runtime 6.4 November 3, 2020 maintenance update.

Databricks Connect 6.4.1Databricks Connect 6.4.1

2020年 4 月 8 日April 8, 2020

次要修复和更新。Minor fixes and updates.

Databricks Connect 6.4.0Databricks Connect 6.4.0

2020 年 3 月 7 日March 7, 2020

适用于 Databricks Runtime 6.4 的初始 Databricks Connect 版本。Initial Databricks Connect release for Databricks Runtime 6.4.