流式处理作业性能下降Streaming job has degraded performance

问题Problem

您有一个流式处理作业,它的性能随着时间的推移而下降。You have a streaming job which has its performance degrade over time.

你使用相同的配置和相同的源启动新的流式处理作业,其性能比现有作业更好。You start a new streaming job with the same configuration and same source, and it performs better than the existing job.

原因Cause

旧的检查点问题可能会导致长时间运行的流式处理作业性能下降。Issues with old checkpoints can result in performance degradation in long running streaming jobs.

如果作业从同一个检查点间歇停止并重新启动,则可能会发生这种情况。This can happen if the job was intermittently halted and restarted from the same checkpoint.

您可以通过查看最新的微批处理偏移序列号来验证问题。You can validate the issue by reviewing the latest micro batch offset sequence number.

解决方案Solution

  • 更改检查点目录。Change the checkpoint directory.
  • 避免重新启动具有相同检查点目录的旧流式处理作业。Avoid restarting old streaming jobs with the same checkpoint directories.
  • 如果无法更改检查点目录,请增加群集容量。If you cannot change the checkpoint directory, increase the cluster capacity.