Redaguoti

Bendrinti naudojant

Schedule and run an Apache Spark job definition

  • Straipsnis

Learn how to run a Microsoft Fabric Apache Spark job definition and find the job definition status and details.

Prerequisites

Before you get started, you must:

How to run a Spark job definition

There are two ways you can run a Spark job definition:

  • Run a Spark job definition manually by selecting Run from the Spark job definition item in the job list.

    Screenshot showing where to select Run.

  • Schedule a Spark job definition by setting up a schedule plan on the Settings tab. Select Settings on the toolbar, then select Schedule.

    Screenshot showing where to select Schedule on the Settings tab.

Important

To run, a Spark job definition must have a main definition file and a default lakehouse context.

Tip

For a manual run, the account of the currently logged in user is used to submit the job. For a run triggered by a schedule, the account of the user who created the schedule plan is used to submit the job.

Three to five seconds after you've submitted the run, a new row appears under the Runs tab. The row shows details about your new run. The Status column shows the near real-time status of the job, and the Run kind column shows if the job is manual or scheduled.

Screenshot of Runs list details.

For more information on how to monitor a job, see Monitor your Apache Spark job definition.

How to cancel a running job

Once the job is submitted, you can cancel the job by selecting Cancel active run from the Spark job definition item in the job list.

Screenshot showing where to select Cancel active run.

Spark job definition snapshot

The Spark job definition stores its latest state. To view the snapshot of the history run, select View Snapshot from the Spark job definition item in the job list. The snapshot shows the state of the job definition when the job is submitted, including the main definition file, the reference file, the command line arguments, the referenced lakehouse, and the Spark properties.

Screenshot showing where to select View Snapshot.

From a snapshot, you can take three actions:

  • Save as a Spark job definition: Save the snapshot as a new Spark job definition.
  • Open Spark job definition: Open the current Spark job definition.
  • Restore: Restore the job definition with the snapshot. The job definition is restored to the state when the job was submitted.

Screenshot of the snapshot menu options.

Related content