如何检查是否可以在笔记本中修改某个 Spark 属性How to check if a spark property is modifiable in a notebook

问题Problem

可以通过设置各种配置来优化应用程序。You can tune applications by setting various configurations. 某些配置必须设置在群集级别,而某些配置则在笔记本或应用程序中设置。Some configurations must be set at the cluster level, whereas some are set inside notebooks or applications.

解决方案Solution

若要检查是否可以在笔记本中设置特定的 Spark 配置,请在笔记本单元中运行以下命令:To check if a particular Spark configuration can be set in a notebook, run the following command in a notebook cell:

spark.conf.isModifiable("spark.databricks.preemption.enabled")

如果 true 返回了,则可以在笔记本中设置属性。If true is returned, then the property can be set in the notebook. 否则,必须在群集级别设置。Otherwise, it must be set at the cluster level.