Challenge with maximum allowed size of WAVEFORMATEXTENSIBLE structure in Media Foundation
You might consider to extend the extensible format to store custom data for your MFT, but wait, you have size limitation here. I am not going to share the exact size, this is something I am leaving for you to figure out . By the way, on Windows 8.1 we have reduced this size further. The good news is in near future we will eliminate this restriction from Windows 8.1. I don’t know about Windows 7 though. Anyway, keep an eye on this blog for further update.
So, what can you do to workaround this?
I would suggest you to use a different subtype for your custom format and register your decryptor to handle that format. This may require you to decrypt and decode in the same MFT.