MLflow Experiment

Note

Available in Databricks Runtime 6.0 ML.

The MLflow experiment data source provides a standard API to load MLflow experiment run data:

val df = spark.read.format("mlflow-experiment").load("...")

Configuration

The load method supports an optional string of comma separated experiment IDs. Leaving it blank loads the MLflow experiment associated with the notebook.

Schema

The schema of the DataFrame returned by the data source is:

root
|-- run_id: string
|-- experiment_id: string
|-- metrics: map
|    |-- key: string
|    |-- value: double
|-- params: map
|    |-- key: string
|    |-- value: string
|-- tags: map
|    |-- key: string
|    |-- value: string
|-- start_time: timestamp
|-- end_time: timestamp
|-- status: string
|-- artifact_uri: string

MLflow experiment data source notebook

Get notebook