CComActivator::DOCreateInstance throws access violation in 32-bit code.

Norman Viljoen 1

Hi All,

I am seeing an exception running 32-bit code using CComActivator::DOCreateInstance. This call causes an access violation when invoking CoCreateInstance to instantiate a device enumerator object in a 32-bit DLL. The same code compiled 64-bit invoking the same enumerator in a 64-bit DLL works perfectly.

IMMDeviceEnumerator* pDeviceEnumerator = NULL;
CoCreateInstance(__uuidof(MMDeviceEnunerator), NULL, CLCTX_INPROC_SERVER, __uuidof(IMMDeviceEnumerator), (LPVOID*)&pDeviceEnumerator);

push 0
push eax
push 1
push 0
push dword ptr [ebp+10h]
mov ecx, ebx
call combase!CComActivator::DOCreateInstance
(3bc.1cf4): Access violation - code c0000005 (first chance)

BTW. The calling convention used in the DLL for all the enumerator methods is STDMETHODCALLTYPE, which is the required calling convention through class inheritance. What am I missing here? Is it a calling convention problem? Any ideas?

Sincerely,
Norman

RLWA32 40,276 Reputation points

2022-05-12T19:32:00.55+00:00

(3bc.1cf4): Access violation - code c0000005 (first chance)

The debugger sees a first chance exception before the application being debugged. A first chance exception isn't necessarily fatal if the exception is handled by the application. Was there a subsequent crash due to an access violation or some other problem?

I could not reproduce the problem using a console application. I assume that when you talk about 32-bit and 64-bit DLLs you are referring to the in-process COM server.
Norman Viljoen 1 Reputation point

2022-05-12T20:40:54.21+00:00

Hi,
Thanks for the response. The DLL is a custom audio enumerator available as 32-bit and 64-bit. A client application has no problem accessing the 64-bit DLL, but a 32-bit client throws an exception when trying to access the audio enumerator methods. For instance the 32-bit wmplayer accessing the 32-bit endpoint audio enumerator DLL fails while the 64-bit wmplayer works correctly with the 64-bit audio enumerator DLL. Note that the custom audio enumerator C++ source code is identical for both 32 and 64 bit. Also in case you are wondering the DLL is in the correct folders. (\Windows\SysWOW64 for the 32-bit dll and \Windows\System32 for the 64-bit dll)

Anyhow, I created a simple C++ application to help debug the problem - all that it does is use CoCreateInstance to return an instance of the enumerator then call the various methods like get the EnumAudioEndpoints and GetDefaultAudioEndpoint and access the audio device and endpoint methods. In 64-bit everything everything works fine. But the 32-bit code fails in the CoCreateInstance call.

Furthermore, I also tested the endpoint enumerator DLL by bypassing COM and dynamically loading the DLL to call the GetTSAudioEndpointEnumeratorForSession directly to get the enumerator instance and call all the methods - this works without issue for both 32-bit and 64-bit. Ultimately, It is when accessing the 32-bit audio endpoint DLL via COM that there is a problem.
Thanks,
Norman
RLWA32 40,276 Reputation points

2022-05-12T22:19:00.503+00:00

Maybe the problem is in the class factory code since that would be the first place that CoCreateInstance would go. Of course, that's just a guess.
Junjie Zhu - MSFT 14,446 Reputation points Microsoft Vendor

2022-05-13T07:31:36.637+00:00

In CoCreateInstance , Have you tried use CLSCTX_LOCAL_SERVER ?
Norman Viljoen 1 Reputation point

2022-05-13T12:55:57.55+00:00

I am using CLSCTX_INPROC_SERVER because the audio enumerator must run in the same context as the caller. Doing this lets my test application emulate what the audio clients are doing. Nonetheless, for interest's sake changing to CLSCTX_LOCAL_SERVER causes both 32-bit and 64-bit to fail with error code: 0x80040154.
RLWA32 40,276 Reputation points

2022-05-13T13:01:23.127+00:00

Can you share your code with us so that we have some visibility into what is happening? You need not provide your implementation of interfaces since the problem seems to be the COM object instantiation. For test purposes interface implementations could simply return E_NOTIMPL.
Norman Viljoen 1 Reputation point

2022-05-16T13:05:08.997+00:00

Hi,
Below is a concise version of the code. In 32-bit executables the exception occurs in the "App code" when CoCreateInstance returns. In the debugger I can see that GetTSAudioEndpointEnumeratorForSession is called. This problem does not occur in 64-bit.
Thanks,
Norman

// DLL code
/#define DLL_EXPORT(type) extern "C" __declspec(dllexport) type
DLL_EXPORT(HRESULT) GetTSAudioEndpointEnumeratorForSession(In DWORD SessionId, Out IMMDeviceEnumerator **ppEndpointEnumerator)
{
HRESULT hr = E_NOINTERFACE;
*ppEndpointEnumerator = NULL;
return hr;
}

// App code
int Test() {
HRESULT hr;
hr = CoInitialize(nullptr);
if (hr == S_OK) {
IMMDeviceEnumerator *deviceEnumerator = NULL;
hr = CoCreateInstance(__uuidof(MMDeviceEnumerator), NULL, CLSCTX_INPROC_SERVER, __uuidof(IMMDeviceEnumerator), (LPVOID *)&deviceEnumerator);
...
}
...
}
RLWA32 40,276 Reputation points

2022-05-16T13:11:21.507+00:00

What I had hoped was for you to share the relevant COM server code so that we might attempt to recreate the problem. Without some code from your custom enumerator we are left to guess at possible causes.
Norman Viljoen 1 Reputation point

2022-05-16T14:27:35.82+00:00

The error occurs with the GetTSAudioEndpointEnumeratorForSession as shown above. The code implementing the custom audio enumerator isn't needed to recreate the problem.
RLWA32 40,276 Reputation points

2022-05-16T14:31:30.187+00:00

I haven't created a custom audio enumerator myself so I guess I don't understand enough about the topic to assist you further.
RLWA32 40,276 Reputation points

2022-05-16T14:40:11.273+00:00

One thing I did read in the MS documentation was that your DLL needs to be signed. Are both the 64-bit and 32-bit DLLs signed?
Norman Viljoen 1 Reputation point

2022-05-16T14:43:22.517+00:00

Yeah both are Microsoft signed
Norman Viljoen 1 Reputation point

2022-05-16T14:45:52.227+00:00

Nonetheless, thank you for spending time looking at this and responding.
RLWA32 40,276 Reputation points

2022-05-16T14:55:19.803+00:00

One more thought -- Have you tested on multiple versions of Windows (8.1, 10, 11) and different builds of Windows 10/11? Can the problem be isolated to a particular environment?

1 answer

Norman Viljoen 1 Reputation point

2022-05-17T10:56:03.91+00:00

I found the solution to this. The 32-bit call is expected to use the __stdcall calling convention. But using this convention causes the CoCreateInstance to not find the GetTSAudioEndpointEnumeratorForSession. To fix that I used a pragma:
#pragma comment(linker, "/export:GetTSAudioEndpointEnumeratorForSession=_GetTSAudioEndpointEnumeratorForSession@8")
Now the 32-bit code works as well as the 64-bit code.
Please sign in to rate this answer.
RLWA32 40,276 Reputation points

2022-05-17T11:03:38.907+00:00

This strikes me as a documentation bug. The documented prototype for the exported function does not specify the use of the __stdcall calling convention. It would make no difference for 64-bit builds since calling convention specifications are ignored for them.

Then again, if the code that calls GetProcAddress for the exported function is incorrect the problem is bigger than mere documentation.

Norman Viljoen 1 Reputation point

2022-05-19T16:14:20.163+00:00

As I have mentioned before, bypassing COM completely and loading the 32-bit audio endpoint DLL using loadLibrary and then getting the GetTSAudioEndpointEnumeratorForSession with GetProcAddress I could access the endpoint methods without a problem. Anyhow, using the #pragma solved the COM problem for 32-bit and now 32-bit clients like wmplayer work correctly. So it must be a Documentation issue where the 32-bit detail is missing. Perhaps, since 32-bit may be considered legacy it is just ignored :)

RLWA32 40,276 Reputation points

2022-05-19T16:17:01.057+00:00

Interestingly, if you use a def file to export GetTSAudioEndpointEnumeratorForSession there are no problems in either 64-bit or 32-bit. Maybe that should be in the documentation.
Sign in to comment