question

IanLittle-1760 avatar image
3 Votes"
IanLittle-1760 asked thangluu edited

Possible cache-set-value issue with Redis and API Management consumption tier

I am using API Management consumption tier and taking an approach to pull a shared secret out of Azure KeyVault and store in a cache to minimise key vault lookups.
The approach I am taking is very similar to the following :

[https://blog.eldert.net/implementing-smart-caching-of-secrets-in-azure-api-management-policies/][1]

Since we are using the consumption tier, I needed to configure API Management to use an external (Azure Redis cache) to avail of caching.
We are already have deployed an Azure Redis cache instance for other non APIM parts of our cloud architecture so that was convenient.
I can shared parts of the APIM ARM template if that helps.

I am providing a snippet of the inbound policy definition that we are using to check the cache, pull a secret from key vault if cache misses.
I've set a long TTL onto the cache entry.

What I am seeing is 2 things.
1. The key doesn't end up as apimSharedSecret in Redis - it is consistently written to cache as 2_apimSharedSecret (checked this via SCAN 0 on the Azure Redis console )
2. The value written to the cache has several non-printable byte/chars before the actual secret as a sort of preamble followed by the value we are putting in via cache-store-value

This is very surprising. I have not attempted to try this on one of the non consumption tiers (using inbuilt cache).
I am working around this now by retrieving the cache value in Azure function app by matching the key lookup as 2_apimSharedSecret.
Also, I am using C# regex to split out the preamble and I've prefixed our actual secret with a distinct anchoring string so that I skip over these bizarre preambly characters in the secret pulled out of Redis.

When running the test on the APIM operation(s) the testing trace log in APIM shows 2_apimSharedSecret as the key in the trace for cache-get-value as well as cache-set-value and in cache-set-value it shows the value being stored to cache as the secret value pulled from the keyvault with no non-printable preamble characters.

I am looking for pointers as to whether what I am seeing is expected in terms of how APIM integrates with Redis or whether I may have possibly mis-configured something in deploying the APIM resource.

Inbound policy snippet given below (the KeyVault URL is fictitious in my example below)



 <cache-lookup-value key="apimSecret" variable-name="processedSecret" />
         <choose>
             <when condition="@(!context.Variables.ContainsKey("processedSecret"))">
                 <!-- Retrieve secret from Key Vault -->
                 <send-request mode="new" response-variable-name="cached-secret" timeout="20" ignore-error="false">
                     <set-url>https://dev-foobar-kv.vault.azure.net/secrets/SharedSecret/?api-version=7.0</set-url>
                     <set-method>GET</set-method>
                     <authentication-managed-identity resource="https://vault.azure.net" />
                 </send-request>
                 <!-- Store response body in context variable as a string -->
                 <set-variable name="processedSecret" value="@((string)((IResponse)context.Variables["cached-secret"]).Body.As<JObject>()["value"])" />
                 <!-- Store result in cache -->
                 <cache-store-value key="apimSecret" value="@((string)context.Variables["processedSecret"])" duration="1200000" />
             </when>
         </choose>
         <set-header name="Apim-Secret" exists-action="override">
             <value>@((string)context.Variables["processedSecret"])</value>
         </set-header>

azure-api-managementazure-key-vaultazure-cache-redis
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello @IanLittle-1760, Welcome to the Q&A platform. I went through the policy snippet above, on line number 11 the property "set-variable" I am not sure if the property is getting stored as a string. Can you please try the syntax mentioned below instead.
<set-variable name="processedSecret" value="@(((IResponse)context.Variables["cached-secret"]).Body.As<string>())" />
You can find more information in this example here. Please let me know if this does not help, I will be glad to continue with our discussion.


0 Votes 0 ·
IanLittle-1760 avatar image
0 Votes"
IanLittle-1760 answered IanLittle-1760 edited

Hi Chaitanya,

Many thanks for the suggestion.
However, unfortunately that by itself doesn't work for me.

The reason I have used JObject and then pulling out the "Value" attribute is because KeyVault returns the secret as a JSON response body within the RESTful call to retreive the secret from vault.
So , when I changed the set-variable line to

 <set-variable name="processedSecret" value="@(((IResponse)context.Variables["cached-secret"]).Body.As<string>())" />

.. and then stored in cache (which is Azure Redis in my case) the value was stored (and retrieved) similar to below :

 "\x18\x01\x00\x00\xda\x01\x94\x02\n\x91\x02{\"value\":\"fictonalcompany-secret-dev-cloudws07aa259c-43f1-4e3f-98ab-e3cc7d4422f8\",\"id\":\"https://dev-fictionalproduct-kv.vault.azure.net/secrets/apimSharedSecret/3ed1fafed7da4c3f98d160d7a14bd656\",\"attributes\":{\"enabled\":true,\"created\":1602681357,\"updated\":1602681357,\"recoveryLevel\":\"Purgeable\"}}

I also tested the cache store/retrieval in isolation (just to eliminate any confusion regarding the Keyvault value retrieval) by adding the following simple additional key/store line in the inbound policy as follows :

<cache-store-value key="d1" value="123456" duration="12000" />

The cache value stored from the above policy line , when examined in Redis was as follows :

 "\x0b\x00\x00\x00\xda\x01\b\n\x06123456"

... it was stored in a key named 2_d1

So, it is clear to me that the external cache integration between APIM and Redis is somehow serializing even basic string values with this preamble/binary format/header preceding the actual value that the policy code is supposed to be storing in the cache.

It looks to me like the first two bytes are perhaps some indication of content length but I could be wrong.
The 0x18 , 0x01 that was generated when storing the longer (full JSON body) looks like some least significant byte, most significant byte encoding to indicate content length or offsets.

Also, I managed to deploy an API Management resource on my Azure subscription on the development pricing tier (which includes inbuilt cache - no need for Redis).
I verified that strings are stored in the internal cache as expected with no binary preamble/header bytes.
Though, interestingly the key name that I request when storing the value using <cache-store-value> always gets prefixed with 2_ so that appears to be a general API Management/Cache implementation behaviour.







5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

thangluu avatar image
0 Votes"
thangluu answered thangluu edited

Hi @ChaitanyaNaykodiMSFT-9638 , @MayankBargali-MSFT
I have same this issue. Please help our how to config to ignore prefix and remove auto gen prefix bytes in value of cache?
Because I want use this cache for many service share value and key together.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.