question

hotmailcombangu00-1467 avatar image
0 Votes"
hotmailcombangu00-1467 asked MotoX80 commented

iis crash dump analysis (Related Event 5011)

Dear all
i'm dying. my iis system occurred crash dump with event 5011.
I attached the result of analyzing the dump file with Windbg.
Please anybody help me what is root cause of the problems.


Opened log file 'd:\logs.txt

Symbol Path validation summary
Response Time (ms) Location
Deferred SRV*c:\symbols*http://msdl.microsoft.com/download/symbols
0:280> .reload
................................................................
................................................................
................................................................
...........................................................
Loading unloaded module list
................................................................
0:280> !analyze -v



                  Exception Analysis                                   *

ERROR: Symbol file could not be found. Defaulted to export symbols for crxf_pdf.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for crpe32.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for cslibu-3-0.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for u312frko.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for etc-1-0-12-6.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for cxlibw-5-0.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for secSSO.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for dtsagent.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for sacommlayer.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for rptcontrollers.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for clientdoc.dll -
ERROR: Symbol file could not be found. Defaulted to export symbols for cxlib-5-0.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for rptdefmodel.dll -
!ip2md 6223e460

Failed to request MethodData, not in JIT code range
GetUrlPageData2 (WinHttp) failed: 12002.

DUMP_CLASS: 2

DUMP_QUALIFIER: 400

CONTEXT: (.ecxr)
.ecxr
rax=0000000000000040 rbx=0000000000000412 rcx=0000000007c2ccd0
rdx=fffffffff83d3330 rsi=0000000000000000 rdi=0000000000000001
rip=000000006223e460 rsp=0000000c9afafa38 rbp=0000000041dd52b0
r8=fffffff3699d2100 r9=07ffffff9b4ce908 r10=0000000c9e25ab80
r11=0000000c9e25abd0 r12=0000000041dd52b0 r13=0000000000000001
r14=0000000c9e25abd0 r15=0000000007c2ccd0
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010200
msvcr80!memcpy+0x2c0:
00000000`6223e460 4c8b4c0af8 mov r9,qword ptr [rdx+rcx-8] ds:ffffffff`fffffff8=????????????????
.cxr
Resetting default scope

FAULTING_IP:
msvcr80!memcpy+2c0 [F:\dd\vctools\crt_bld\SELF_64_AMD64\crt\src\AMD64\memcpy.asm @ 376]
00000000`6223e460 4c8b4c0af8 mov r9,qword ptr [rdx+rcx-8]

EXCEPTION_RECORD: (.exr -1)
.exr -1
ExceptionAddress: 000000006223e460 (msvcr80!memcpy+0x00000000000002c0)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: fffffffffffffff8
Attempt to read from address fffffffffffffff8

DEFAULT_BUCKET_ID: INVALID_POINTER_READ

PROCESS_NAME: w3wp.exe

ERROR_CODE: (NTSTATUS) 0xc0000005 - <Unable to get error code text>

EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - <Unable to get error code text>

EXCEPTION_CODE_STR: c0000005

EXCEPTION_PARAMETER1: 0000000000000000

EXCEPTION_PARAMETER2: fffffffffffffff8

FOLLOWUP_IP:
msvcr80!memcpy+2c0 [F:\dd\vctools\crt_bld\SELF_64_AMD64\crt\src\AMD64\memcpy.asm @ 376]
00000000`6223e460 4c8b4c0af8 mov r9,qword ptr [rdx+rcx-8]

READ_ADDRESS: fffffffffffffff8

WATSON_BKT_PROCSTAMP: 5215df96

WATSON_BKT_PROCVER: 8.5.9600.16384

PROCESS_VER_PRODUCT: Internet Information Services

WATSON_BKT_MODULE: msvcr80.dll

WATSON_BKT_MODSTAMP: 520b0ac2

WATSON_BKT_MODOFFSET: 1e460

WATSON_BKT_MODVER: 8.0.50727.8428

BUILD_VERSION_STRING: 6.3.9600.19994 (winblue_ltsb_escrow.210331-1613)

MODLIST_WITH_TSCHKSUM_HASH: 022ba75515edd5b880b2478ee3e1bfa5d86a1f87

MODLIST_SHA1_HASH: 8bb64d6e8508dc802ea0c15f4c6da5636d78c168

NTGLOBALFLAG: 0

PROCESS_BAM_CURRENT_THROTTLED: 0

PROCESS_BAM_PREVIOUS_THROTTLED: 0

APPLICATION_VERIFIER_FLAGS: 0

PRODUCT_TYPE: 3

SUITE_MASK: 272

DUMP_FLAGS: 8000c07

DUMP_TYPE: 3

MISSING_CLR_SYMBOL: 0

ANALYSIS_SESSION_HOST: DESKTOP-V3P2R65

ANALYSIS_SESSION_TIME: 06-14-2021 10:52:53.0147

ANALYSIS_VERSION: 10.0.15063.468 amd64fre

MANAGED_CODE: 1

MANAGED_ENGINE_MODULE: clr

MANAGED_ANALYSIS_PROVIDER: SOS

THREAD_ATTRIBUTES:
OS_LOCALE: KOR

PROBLEM_CLASSES:

ID: [0n292]
Type: [@ACCESS_VIOLATION]
Class: Addendum
Scope: BUCKET_ID
Name: Omit
Data: Omit
PID: [Unspecified]
TID: [0x1710]
Frame: [0] : msvcr80!memcpy
ID: [0n264]
Type: [INVALID_POINTER_READ]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Add
Data: Omit
PID: [Unspecified]
TID: [0x1710]
Frame: [0] : msvcr80!memcpy

BUGCHECK_STR: APPLICATION_FAULT_INVALID_POINTER_READ

PRIMARY_PROBLEM_CLASS: APPLICATION_FAULT

LAST_CONTROL_TRANSFER: from 00000000622a6a70 to 000000006223e460

STACK_TEXT:
0000000c`9afafa38 00000000`622a6a70 : 00000000`00000004 00000000`62226e97 0000000c`915358b0 0000000c`00000002 : msvcr80!memcpy+0x2c0
0000000c`9afafa40 00000000`41d6a6d4 : 00000000`00000000 00000000`00000001 00000000`000001c2 00000000`00000002 : msvcr80!memmove_s+0x80
0000000c`9afafa80 00000000`41d6bbdc : 0000000c`9d20ebd0 0000000c`9afafb20 0000000c`9e25abd0 00000000`00000000 : crxf_pdf!UXFInitializeW+0x1834
0000000c`9afafb00 00000000`37b18a60 : 0000000c`9d20ebd0 0000000c`00000412 0000000c`9e1ca560 00000000`0042beae : crxf_pdf!UXFGetExportFormatsExW+0x10c
0000000c`9afafb50 00000000`004b6e95 : 0000000c`9d20ebd0 00000000`00000000 00000000`004b37c0 00000000`00000000 : crpe32!PEGetNthTableQualifier+0x16c2d0
0000000c`9afafba0 00000000`004b3898 : 0000000c`903f63d0 00000000`00000537 00000000`00000537 00007ffe`00000002 : cslibu_3_0!CSLib300::CSThreadSafeDLL::UninitForThread+0x965
0000000c`9afafbd0 00007ffe`f2e424fd : 00000000`00000000 00000000`00000000 00000000`00000537 00000000`010a0152 : cslibu_3_0!CSLib300::CSDLLServerThreadWnd::ServerWindowProc+0xd8
0000000c`9afafc00 00007ffe`f2e42357 : 00000002`5166ace0 0000000c`741c8b78 00000000`010a0152 00000000`5c9e1a37 : user32!UserCallWinProcCheckWow+0x149
0000000c`9afafcd0 00000000`5c9e1922 : 00000000`00000000 0000000c`a2e7c990 00000000`00000001 00000000`38313040 : user32!DispatchMessageWorker+0x1a7
0000000c`9afafd50 00000000`5c9e22a4 : 0000000c`a2e7c990 00000000`00000000 00000000`00000000 00000000`00000001 : mfc80u!AfxInternalPumpMessage+0x52
0000000c`9afafd80 00000000`5c9e17fd : 00000000`00000994 00000000`00000000 0000000c`9adabe70 0000000c`a2e7c990 : mfc80u!CWinThread::Run+0x70
0000000c`9afafdc0 00000000`622237d7 : 0000000c`9adabe70 0000000c`a2e7c990 0000000c`a2d90e20 0000000c`a2d90e20 : mfc80u!_AfxThreadEntry+0x131
0000000c`9afafec0 00000000`62223894 : 00000000`622d95c0 0000000c`a2d90e20 00000000`00000000 00000000`00000000 : msvcr80!_callthreadstartex+0x17
0000000c`9afafef0 00007ffe`f33613f2 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : msvcr80!_threadstartex+0x84
0000000c`9afaff20 00007ffe`f4ec54f4 : 00007ffe`f33613d0 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x22
0000000c`9afaff50 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x34


THREAD_SHA1_HASH_MOD_FUNC: 8682d5f8aeda4082adb9fdd5eb4274a4f4e1fbc3

THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 2e19ac7c6f430330c5ac3eccc3eb37cc080def85

THREAD_SHA1_HASH_MOD: 0535aa0d03f77259e7bd031ea8c92d0abf210428

FAULT_INSTR_CODE: a4c8b4c

FAULTING_SOURCE_LINE: F:\dd\vctools\crt_bld\SELF_64_AMD64\crt\src\AMD64\memcpy.asm

FAULTING_SOURCE_FILE: F:\dd\vctools\crt_bld\SELF_64_AMD64\crt\src\AMD64\memcpy.asm

FAULTING_SOURCE_LINE_NUMBER: 376

SYMBOL_STACK_INDEX: 0

SYMBOL_NAME: msvcr80!memcpy+2c0

FOLLOWUP_NAME: MachineOwner

MODULE_NAME: msvcr80

IMAGE_NAME: msvcr80.dll

DEBUG_FLR_IMAGE_TIMESTAMP: 520b0ac2

STACK_COMMAND: .ecxr ; kb

FAILURE_BUCKET_ID: INVALID_POINTER_READ_c0000005_msvcr80.dll!memcpy

BUCKET_ID: APPLICATION_FAULT_INVALID_POINTER_READ_msvcr80!memcpy+2c0

FAILURE_EXCEPTION_CODE: c0000005

FAILURE_IMAGE_NAME: msvcr80.dll

BUCKET_ID_IMAGE_STR: msvcr80.dll

FAILURE_MODULE_NAME: msvcr80

BUCKET_ID_MODULE_STR: msvcr80

FAILURE_FUNCTION_NAME: memcpy

BUCKET_ID_FUNCTION_STR: memcpy

BUCKET_ID_OFFSET: 2c0

BUCKET_ID_MODTIMEDATESTAMP: 520b0ac2

BUCKET_ID_MODCHECKSUM: cd8fa

BUCKET_ID_MODVER_STR: 8.0.50727.8428

BUCKET_ID_PREFIX_STR: APPLICATION_FAULT_INVALID_POINTER_READ_

FAILURE_PROBLEM_CLASS: APPLICATION_FAULT

FAILURE_SYMBOL_NAME: msvcr80.dll!memcpy

WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/w3wp.exe/8.5.9600.16384/5215df96/msvcr80.dll/8.0.50727.8428/520b0ac2/c0000005/0001e460.htm?Retriage=1

TARGET_TIME: 2021-05-23T02:20:26.000Z

OSBUILD: 9600

OSSERVICEPACK: 19994

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

OSPLATFORM_TYPE: x64

OSNAME: Windows 8.1

OSEDITION: Windows 8.1 Server TerminalServer SingleUserTS

USER_LCID: 0

OSBUILD_TIMESTAMP: 2021-04-01 12:23:02

BUILDDATESTAMP_STR: 210331-1613

BUILDLAB_STR: winblue_ltsb_escrow

BUILDOSVER_STR: 6.3.9600.19994

ANALYSIS_SESSION_ELAPSED_TIME: 8ee8

ANALYSIS_SOURCE: UM

FAILURE_ID_HASH_STRING: um:invalid_pointer_read_c0000005_msvcr80.dll!memcpy

FAILURE_ID_HASH: {e117c5a4-11fe-47f8-fd1e-fc9c06ae830a}

Followup: MachineOwner


0:280> lmvm msvcr80
Browse full module list
start end module name
00000000`62220000 00000000`622e9000 msvcr80 (private pdb symbols) c:\symbols\msvcr80.AMD64.pdb\526D792BD9BA4E5CBB9104FE0BF437781\msvcr80.AMD64.pdb
Loaded symbol image file: msvcr80.dll
Image path: C:\Windows\WinSxS\amd64_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.8428_none_88dcdb0b2fb19957\msvcr80.dll
Image name: msvcr80.dll
Browse all global symbols functions data
Timestamp: Wed Aug 14 13:42:42 2013 (520B0AC2)
CheckSum: 000CD8FA
ImageSize: 000C9000
File version: 8.0.50727.8428
Product version: 8.0.50727.8428
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Visual Studio® 200 InternalName: MSVCR80.DLL
OriginalFilename: MSVCR80.DLL
ProductVersion: 8.00.50727.8428
FileVersion: 8.00.50727.8428
FileDescription: Microsoft® C Runtime Library LegalCopyright: ? Microsoft Corporation. All rights reserved.
0:280> lmvm crxf_pdf
00000000`41d30000 00000000`41de3000 crxf_pdf (export symbols) crxf_pdf.dll
Loaded symbol image file: crxf_pdf.dll
Image path: C:\Program Files (x86)\SAP BusinessObjects\Crystal Reports for .NET Framework 4.0\Common\SAP BusinessObjects Enterprise XI 4.0\win64_x64\crxf_pdf.dll
Image name: crxf_pdf.dll
Browse all global symbols functions data
Timestamp: Tue Apr 18 18:59:41 2017 (58F5E38D)
CheckSum: 000BDCC7
ImageSize: 000B3000
File version: 13.0.20.2399
Product version: 13.0.20.2399
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: SAP SE
ProductName: SBOP Portable Document Format DLL for Crystal Reports
InternalName: crxf_pdf.dll
OriginalFilename: crxf_pdf.dll
ProductVersion: 13.0.20.2399
FileVersion: 13.0.20.2399
FileDescription: Portable Document Format DLL for Crystal Reports
LegalCopyright: Copyright 2015 SAP SE. All rights reserved.
LegalTrademarks: Crystal Reports is a trademark of Business Objects or one of its subsidiaries

windows-serverwindows-server-iis
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

please anybody help me

0 Votes 0 ·
MotoX80 avatar image
0 Votes"
MotoX80 answered

I don't know how much help we will be to you. In the docs I see it crashing in Crystal Reports.


Thread 280

msvcr80!memcpy+0x2c0
msvcr80!memmove_s+0x80
crxf_pdf!UXFInitializeW+0x1834
crxf_pdf!UXFGetExportFormatsExW+0x10c
crpe32!PEGetNthTableQualifier+0x16c2d0
cslibu_3_0!CSLib300::CSThreadSafeDLL::UninitForThread+0x965
cslibu_3_0!CSLib300::CSDLLServerThreadWnd::ServerWindowProc+0xd8
user32!UserCallWinProcCheckWow+0x149
user32!DispatchMessageWorker+0x1a7
mfc80u!AfxInternalPumpMessage+0x52
mfc80u!CWinThread::Run+0x70
mfc80u!_AfxThreadEntry+0x131
msvcr80!_callthreadstartex+0x17
msvcr80!_threadstartex+0x84
kernel32!BaseThreadInitThunk+0x22
ntdll!RtlUserThreadStart+0x34


crxf_pdf - Crystal Reports export to pdf
crpe32 - Crystal Reports print engine
cslibu_3_0 - Crystal Reports



I would think that the problem is most likely data related. That is, you have a user running some giant query and Crystal Reports cannot process it or has some bug where memory gets overwritten when it tries to print to PDF.

How often does this happen? One time? Every 2 hours?

Check the logs in C:\Windows\System32\LogFiles\HTTPERR. I would expect find a "Connection_Abandoned" error that occurs at the time of the crash. Get the source IP and then query your IIS logs to find the user. Ask them what they were doing.

Based on some of the file names, it would appear that you are running something called Apriso Portal (Flexnet?) . You might have more luck troubleshooting this problem if you contact the vendors support team and ask them for help. The information that you provided looks more like an application problem and not a common Windows\IIS bug.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

SamWu-MSFT avatar image
0 Votes"
SamWu-MSFT answered SamWu-MSFT edited

@hotmailcombangu00-1467 It is impossible to get useful information from your dump, you need to use Debug Diagnostic Tool.

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

hotmailcombangu00-1467 avatar image
0 Votes"
hotmailcombangu00-1467 answered hotmailcombangu00-1467 commented

Hi~SamWu-MSFT
Thank a lot.

I have already tried the analysis using the debug diag tool.(please refer to attached a file)

[1]: /answers/storage/attachments/107573-w3wpexe-210523-112025-crashhanganalysis.pdf


· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

For some reason, I cannot download this file, I suggest you open a support case via http://support.microsoft.com to contact Microsoft.

0 Votes 0 ·

hi~~

can you download this file?
could you please check again linked file
onto above message :
w3wpexe-210523-112025-crashhanganalysis.pdf (556.8 KiB)


and I am not contracted with ms. i'm so sed :-((



0 Votes 0 ·
hotmailcombangu00-1467 avatar image
0 Votes"
hotmailcombangu00-1467 answered MotoX80 commented

Dear MotoX80
Thank you very very much.
I'm so happy, you are my savior. :---))

· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Glad i could help.

0 Votes 0 ·