取消流式处理单元后无法运行 笔记本命令 Cannot run notebook commands after canceling streaming cell

问题Problem

在连接到 Databricks Runtime 5.0 群集的笔记本中取消正在运行的流式处理单元后,不能在笔记本中运行任何后续命令。After you cancel a running streaming cell in a notebook attached to a Databricks Runtime 5.0 cluster, you cannot run any subsequent commands in the notebook. 这些命令将保留在 "正在等待运行" 状态,并且必须清除笔记本的状态,或者分离并重新附加群集,然后才能成功在笔记本上运行命令。The commands are left in the “waiting to run” state, and you must clear the notebook’s state or detach and reattach the cluster before you can successfully run commands on the notebook.

请注意,仅当取消单个单元时,才会出现此问题。当你运行所有单元格并取消所有单元时,此功能不适用。Note that this issue occurs only when you cancel a single cell; it does not apply when you run all and cancel all cells.

版本Version

此问题会影响 Databricks Runtime 5.0 群集。This problem affects Databricks Runtime 5.0 clusters. 它还会影响其 Spark 配置已 spark.databricks.chauffeur.enableIdleContextTracking 设置为 Databricks Runtime 4.3 群集 trueIt also affects Databricks Runtime 4.3 clusters whose Spark Configuration spark.databricks.chauffeur.enableIdleContextTracking has been set to true.

原因Cause

Databricks Runtime 4.3 引入了可选的空闲执行上下文功能(默认情况下在 Databricks Runtime 5.0 中启用),该功能允许执行上下文跟踪流执行顺序以确定它们是否处于空闲状态。Databricks Runtime 4.3 introduced an optional idle execution context feature, which is enabled by default in Databricks Runtime 5.0, that allows the execution context to track streaming execution sequences to determine if they are idle. 遗憾的是,这会导致在您取消流式处理单元格时,基础执行上下文会处于无效状态。Unfortunately, this introduced an issue that causes the underlying execution context to be left in an invalid state when you cancel a streaming cell. 这会阻止其他命令运行,直到重置笔记本状态。This prevents additional commands from being run until the notebook state is reset. 此行为特定于交互式笔记本,不影响作业。This behavior is specific to interactive notebooks and does not affect jobs.

有关空闲执行上下文的详细信息,请参阅执行上下文For more information about idle execution contexts, see Execution contexts.

解决方案Solution

Azure Databricks 正在努力解决此问题并发布 Databricks Runtime 5.0 的维护更新。Azure Databricks is working to resolve this issue and release a maintenance update for Databricks Runtime 5.0. 同时,你可以执行以下任一操作:In the meantime, you can do either of the following:

  • 若要在不重新启动群集的情况下修正受影响的笔记本,请单击笔记本的 "清除" 菜单并选择 "清除状态":To remediate an affected notebook without restarting the cluster, go to the notebook’s Clear menu and select Clear State:

    no-alternative-textno-alternative-text

  • 如果重新启动群集是可接受的,则可以通过关闭空闲上下文跟踪来解决该问题。If restarting the cluster is acceptable, you can solve the issue by turning off idle context tracking. 在群集上设置以下Spark 配置值:Set the following Spark configuration value on the cluster:

    spark.databricks.chauffeur.enableIdleContextTracking false
    

    然后重新启动该群集。Then restart the cluster.