Where to store secret token retrieved during runtime?

Question

Hi Team,

I'm trying to visualize and best way to implement a very common enterprise use case.

I'm using ADF, Key Vault, Azure SQL data base as inventories.

Requirement is to fetch some data from an exposed API. Before calling the API I need to generate one access token by calling a different API from the same vendor. This token has TTL 3 months. So I prefer to store it in my inventories.

Now I'm confused where to store the runtime secret?

In KV if we try to store, I need to give ADF Service Principle, set access to KV which may not be a good solution. Even after saving I need to store the secret version for retrieving the value again.

Within ADF I can't store as after ADF execution completes this secret will be destroyed.

Can we store it in Azure SQL by allowing masing? Can SQL protect the secure token properly?

Please share your views.

Regards,

Partha

Answer 1

Hello,

Storing runtime secrets securely is crucial for maintaining the integrity and security of your application. Azure provides several options for managing secrets, and here are some best practices you might consider:

Azure Key Vault (KV): It’s designed for securely storing and accessing secrets. You can store the access token in KV and give your Azure Data Factory (ADF) Managed Identity the necessary permissions to retrieve the secret. This avoids having to use the ADF Service Principal directly and is considered a secure approach.

Secrets Rotation: Since your access token has a TTL of 3 months, you should implement a rotation policy for your secrets. Azure Key Vault supports this by allowing you to set expiration dates on secrets and provides mechanisms to update them regularly.

Access Control: Ensure that only trusted identities have access to the secrets. Use Azure’s role-based access control to assign the minimum required permissions.

Secrets in Azure SQL Database: Storing secrets in your database can be done, but it’s not recommended due to the complexity My Milestone Card of ensuring encryption and proper access control. If you choose this route, you must ensure that the data is encrypted using Transparent Data Encryption (TDE) and that access is tightly controlled.

Caching Secrets: To minimize calls to Key Vault, you can cache the secret in your application’s memory, but ensure that you implement secure caching strategies to prevent unauthorized access.

Network Isolation: Configure your firewall to restrict access to the Key Vault, allowing only your applications and services to retrieve the secrets.

Remember, the key to managing secrets is not just about where you store them but also how you manage access and rotation. Azure Key Vault is a robust solution that, when used correctly, can help you manage your application secrets effectively and securely.

I hope the solution may helps you.

Answer 2

@Pratim Das, Partha C

Answering to your query: If I set key vault secret value in runtime, the version will change. How can I know going forward which version to use?

As per :https://learn.microsoft.com/en-us/azure/data-factory/how-to-use-azure-key-vault-secrets-pipeline-activities

The secret-version is optional; the latest version is returned when not specified. It is often desirable to specify a secret URI in a pipeline without a specific version so that the pipeline always uses the latest version of the secret.

Please "Accept the answer (Yes)" and "share your feedback ". This will help us and others in the community as well.

Thanks,

Akshay Kaushik