Error in Azure Synapse Notebook Unable to Authenticate to Access Azure Managed Identity

Question

I am trying to authenticate to access other Azure resource (Azure Digital Twins) from Azure Synapse without explicitly using secrets, so I tried to use the Azure Managed Identity and followed this tutorial. After following the tutorial, I am getting the error below (unable to authenticate):

ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. No identity has been assigned to this resource.
SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
VisualStudioCodeCredential: Failed to get Azure user details from Visual Studio Code.
AzureCliCredential: Azure CLI not found on path

Below is the code snippet:

from azure.identity import DefaultAzureCredential  
from azure.digitaltwins.core import DigitalTwinsClient  
  
# azure_managed_identity_client_id, adt_url change to corresponding values  
credential = DefaultAzureCredential(managed_identity_client_id=<azure_managed_identity_client_id>)  
service_client = DigitalTwinsClient(<adt_url>, credential)  
  
relationship_query = 'SELECT * FROM RELATIONSHIPS'  
relationships = service_client.query_twins(relationship_query)  
  
relationships_df = pd.DataFrame()  
  
for relationship in relationships:  
    print(relationship)  

Steps to reproduce:

1. Create a managed identity instance in Azure Portal
2. In access control (IAM) of the other Azure resource (ADT in our case; blob storage in demo case), grant access to the managed identity created in step 1
3. Go to the identity pane of the destination service (Synapse in our case; Azure Functions in demo case) in Azure Portal to add the user assigned identity (in this case, we add the managed identity instance)
4. Run the code snippet above in Azure Synapse notebook attached to a Spark Pool

Is there anything I did incorrectly? What's the best practice for accessing Azure resources that don't support Linked Service in Synapse without referencing the secret in the code (inside the Synapse notebook)?