how to extend using Init script on DBFS

Amesh Jeram 20 Reputation points
2024-04-29T16:34:14.41+00:00

is there a way for me to extend the life of using current set-up of init scripts on DBFS?
reason is I am getting errors now and i have not migrated my init scripts to UC/Workspace or Cloud storage and nothing runs now.

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,469 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,955 questions
0 comments No comments
{count} votes

Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 78,986 Reputation points Microsoft Employee
    2024-04-30T03:17:46.8733333+00:00

    @Amesh Jeram - Thanks for the question and using MS Q&A platform.

    Is there a way for me to extend the life of using current set-up of init scripts on DBFS?

    No, you need to migrate from init scripts on DBFS, see Migrate init scripts from DBFS.

    I am getting errors now and i have not migrated my init scripts to UC/Workspace or Cloud storage and nothing runs now.

    Init scripts on DBFS have reached end-of-life and can no longer be used. You must migrate your init scripts to a supported location before you can start compute. Store init scripts on Unity Catalog Volumes, as workspace files, or in cloud object storage.

    Note: You can store and configure init scripts from workspace files, Unity Catalog volumes, and cloud object storage, but init scripts are not supported on all cluster configurations and not all files can be referenced from init scripts.

    The following table indicates the support for init scripts based on the source location and the cluster access mode.

    User's image

    Users that need to migrate init scripts from DBFS can use the following guides. Make sure you’ve identified the correct target for your configuration. See Recommendations for init scripts.

    For more details, refer to Azure Databricks- Where can init scripts be installed?

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    0 comments No comments

0 additional answers

Sort by: Most helpful