Can't see call stack in WinDbg

[Alan Williams-Key]

My application creates a minidump using MiniDumpWriteDump() if it crashes. I
am receiving some crash reports from users and I would like to see where the
problem is using WinDbg. When I load the crash dump it appears to load
successfully and I get a line that states
"This dump file has an exception of interest stored in it."
I then go to View-Call Stack and the call stack is displayed showing calls
to functions in ntdll, dbghelp and kernel32 but then
"WARNING Frame IP not in any known module. Following frames may be wrong."
Just when it gets to the interesting part! I've set my symbol path. What
could be the cause of this inability to decode the call stack? Thanks for any
help.

[Alex Blekhman]

Post by Alan Williams-Key
I then go to View-Call Stack and the call stack is displayed
showing calls to functions in ntdll, dbghelp and kernel32 but
then "WARNING Frame IP not in any known module. Following frames
may be wrong."
Just when it gets to the interesting part! I've set my symbol
path. What could be the cause of this inability to decode the
call stack?

Most likely it is due the dreadful /Oy compiler switch, which gets
enabled by default for optimized code (/Ox, /O1, /O2 switches).
Benefits of the /Oy switch are doubtful but debugging really sucks
as you can see by yourself. You can disable frame pointer
omissions by specifying the /Oy- _after_ all other optimizations.

HTH
Alex

[Alan Williams-Key]

Post by Alex Blekhman

Post by Alan Williams-Key
I then go to View-Call Stack and the call stack is displayed
showing calls to functions in ntdll, dbghelp and kernel32 but
then "WARNING Frame IP not in any known module. Following frames
may be wrong."
Just when it gets to the interesting part! I've set my symbol
path. What could be the cause of this inability to decode the
call stack?

Most likely it is due the dreadful /Oy compiler switch, which gets
enabled by default for optimized code (/Ox, /O1, /O2 switches).
Benefits of the /Oy switch are doubtful but debugging really sucks
as you can see by yourself. You can disable frame pointer
omissions by specifying the /Oy- _after_ all other optimizations.
HTH
Alex

Thanks. I looked this up and according to the help, optimisations are only
available in Professional and Enterprise editions of my Visual Studio (V6.0
C++) and I have the standard version. Certainly there seems to be no way to
change optimisation settings in the project settings dialog. Does this mean
the frame pointers are optimised out and I can do nothing about it?

[Alex Blekhman]

Post by Alan Williams-Key
Thanks. I looked this up and according to the help,
optimisations are only available in Professional and Enterprise
editions of my Visual Studio (V6.0 C++) and I have the standard
version.

That's interesting. The Standard Edition of VC6 doesn't have
optimizer at all (or at least that's what MSDN says). You can
check whether frame pointer omission (FPO) optimization is enabled
quite easy. Create simple console project and make a function
call. Then build the project in Realease mode and see the
generated code in disassembler. If FPO is enabled you won't see

push ebp
mov ebp,esp

at the beginning of the function and

pop ebp

at the end of the function.

You could try to control the FPO with #pragma:

#pragma optimize("y", off|on)

But as I mentioned above, the Standard Edition shouldn't have any
optimizations enabled to begin with.

I wish I could be more helpful.

Alex

[Alan Williams-Key]

Post by Alex Blekhman
That's interesting. The Standard Edition of VC6 doesn't have
optimizer at all (or at least that's what MSDN says). You can
check whether frame pointer omission (FPO) optimization is enabled
quite easy. Create simple console project and make a function
call. Then build the project in Realease mode and see the
generated code in disassembler. If FPO is enabled you won't see
push ebp
mov ebp,esp

Well I checked this in my application and those statements are there in the
assembler. Does anyone have any other ideas?

[Scot T Brennecke]

Post by Alan Williams-Key

Post by Alex Blekhman

Post by Alan Williams-Key
I then go to View-Call Stack and the call stack is displayed
showing calls to functions in ntdll, dbghelp and kernel32 but
then "WARNING Frame IP not in any known module. Following frames
may be wrong."
Just when it gets to the interesting part! I've set my symbol
path. What could be the cause of this inability to decode the
call stack?

Most likely it is due the dreadful /Oy compiler switch, which gets
enabled by default for optimized code (/Ox, /O1, /O2 switches).
Benefits of the /Oy switch are doubtful but debugging really sucks
as you can see by yourself. You can disable frame pointer
omissions by specifying the /Oy- _after_ all other optimizations.
HTH
Alex

Thanks. I looked this up and according to the help, optimisations are only
available in Professional and Enterprise editions of my Visual Studio (V6.0
C++) and I have the standard version. Certainly there seems to be no way to
change optimisation settings in the project settings dialog. Does this mean
the frame pointers are optimised out and I can do nothing about it?

Alan,
If you have FPO getting in the way (as was suspected), you should be able to tell by displaying
the call stack using the 'kv' variation. The 'v' tells it to display in similar form to the 'kb',
but indicating if FPO (or not) is relevant on that frame.
You say you have specified the symbols path. Did you confirm that symbols were actually matched
to your module for the frame(s) in question? Also, just because it says "may be wrong" doesn't
necessarily mean "are wrong". Do they look wrong to you?

Scot

[Alan Williams-Key]

"Scot T Brennecke" wrote:

message

Post by Scot T Brennecke
Alan,
If you have FPO getting in the way (as was suspected), you should be able to tell by displaying
the call stack using the 'kv' variation. The 'v' tells it to display in similar form to the 'kb',
but indicating if FPO (or not) is relevant on that frame.
You say you have specified the symbols path. Did you confirm that symbols were actually matched
to your module for the frame(s) in question? Also, just because it says "may be wrong" doesn't
necessarily mean "are wrong". Do they look wrong to you?
Scot

1) FPO can't be a problem - I've confirmed in the disassembly window in VS
that the instructions are there.
2) No, I don't know if the symbols have loaded. How can I tell? All I know
is I get a sensible decodign of the stack for system calls (function name,
parameters) then the message about the frame pointers not matching any known
module and then there is just numbers.
3) Do they look wrong? No idea - I'm really confused here because in WinDbg
I clicked the addrs button which I think shows the stack addresses but these
do not correspond in any way to the addresses shown in the stack print in my
ERRORLOG.TXT file. eg, stack in ERRLOG.TXT is 0x0012f538 to 0x00130000 and
in WinDbg teh frame pointers go from 0012dbf4 to 0012ddc4. Could they be
reporting different threads?

[Alan Williams-Key]

"Alan Williams-Key" wrote:
Sorry folks - I've found the solution from somewhere else. I needed to issue
the .ecxr command.

Alan

[Scot T Brennecke]

I'm glad you found a resolution with .ecxr.
To know if the symbols are loaded for a specific module in WinDbg, issue the "lmvm" command with the
module name, and it will say what symbols, if any, are loaded. If you see just numbers (addresses)
for stack frames where you expect your own function names to be, that indicates the symbols don't
match. Make sure to set the symbol path to include the location of your PDB file that matches your
running module.
Those stack ranges certainly could be different threads, but without examining your process (or
dump), I can't say for sure. If you have an answer to your immediate issue, we can wait until
another time to address that question.

Post by Alan Williams-Key
message

Post by Scot T Brennecke
Alan,
If you have FPO getting in the way (as was suspected), you should be able to tell by
displaying
the call stack using the 'kv' variation. The 'v' tells it to display in similar form to the
'kb',
but indicating if FPO (or not) is relevant on that frame.
You say you have specified the symbols path. Did you confirm that symbols were actually
matched
to your module for the frame(s) in question? Also, just because it says "may be wrong" doesn't
necessarily mean "are wrong". Do they look wrong to you?
Scot

1) FPO can't be a problem - I've confirmed in the disassembly window in VS
that the instructions are there.
2) No, I don't know if the symbols have loaded. How can I tell? All I know
is I get a sensible decodign of the stack for system calls (function name,
parameters) then the message about the frame pointers not matching any known
module and then there is just numbers.
3) Do they look wrong? No idea - I'm really confused here because in WinDbg
I clicked the addrs button which I think shows the stack addresses but these
do not correspond in any way to the addresses shown in the stack print in my
ERRORLOG.TXT file. eg, stack in ERRLOG.TXT is 0x0012f538 to 0x00130000 and
in WinDbg teh frame pointers go from 0012dbf4 to 0012ddc4. Could they be
reporting different threads?

[Alan Williams-Key]

Post by Scot T Brennecke
I'm glad you found a resolution with .ecxr.

Looks like I spoke too soon. Still having problems. This is the latest:


Loading Dump File [D:\Crash\CRASH.DMP]
User Mini Dump File: Only registers, stack and portions of memory are
available

Symbol search path is:
SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols;D:\Crash
Executable search path is: D:\Crash
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt, suite: SingleUserTS Personal
Machine Name:
Debug session time: Thu Apr 16 17:21:06.000 2009 (GMT+2)
System Uptime: not available
Process Uptime: 0 days 0:00:26.000
.............................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(ed98.eda0): Integer divide-by-zero - code c0000094 (first/second chance not
available)
eax=7c82fbf0 ebx=00164f70 ecx=011a0ea0 edx=59a63d70 esi=00164f48 edi=00164fa0
eip=7c90e4f4 esp=0012c7c4 ebp=0012c7d4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
7c90e4f4 c3 ret
0:000> .ecxr
eax=00000002 ebx=00000001 ecx=0000015d edx=00000000 esi=60425bf0 edi=60459270
eip=602fa0a6 esp=0012e038 ebp=0012f380 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206
Unable to load image D:\Program Files\Microsoft Visual
Studio\MyProjects\atcsim\Release\atcwindows.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for atcwindows.exe
*** ERROR: Module load completed but symbols could not be loaded for
atcwindows.exe
atcwindows+0xa0a6:
602fa0a6 ?? ???

You will see that the executable search path is "D:\Crash" (line 4) so why
is WinDbg trying and failing to load from a completely different path (and
why fail anyway since the executable does exit there)?

Alan

[Scot T Brennecke]

When you executed the program at the time the dump was generated, the full path to the .exe was
already known and saved in the dump. It did not need to search the .exepath to get the path of the
Exe.
It says the symbols could not be loaded for Atcwindows.exe. Did you put the PDB for that executable
in D:\Crash? If so, is it the exact same PDB that was generated at the time the .exe was built?
Which version of WinDbg is this? Have you installed the one that was released a few weeks ago?
Install Debugging Tools for Windows 32-bit Version:
http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a

This dump is just a small minidump. Can you gather one using the adplus.vbs script that comes with
the DTW package? something like this:
cscript adplus.vbs -crash -pn atcwindows.exe -o D:\Crash

Post by Alan Williams-Key

Post by Scot T Brennecke
I'm glad you found a resolution with .ecxr.

Loading Dump File [D:\Crash\CRASH.DMP]
User Mini Dump File: Only registers, stack and portions of memory are
available
SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols;D:\Crash
Executable search path is: D:\Crash
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt, suite: SingleUserTS Personal
Debug session time: Thu Apr 16 17:21:06.000 2009 (GMT+2)
System Uptime: not available
Process Uptime: 0 days 0:00:26.000
.............................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(ed98.eda0): Integer divide-by-zero - code c0000094 (first/second chance not
available)
eax=7c82fbf0 ebx=00164f70 ecx=011a0ea0 edx=59a63d70 esi=00164f48 edi=00164fa0
eip=7c90e4f4 esp=0012c7c4 ebp=0012c7d4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
7c90e4f4 c3 ret
0:000> .ecxr
eax=00000002 ebx=00000001 ecx=0000015d edx=00000000 esi=60425bf0 edi=60459270
eip=602fa0a6 esp=0012e038 ebp=0012f380 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206
Unable to load image D:\Program Files\Microsoft Visual
Studio\MyProjects\atcsim\Release\atcwindows.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for atcwindows.exe
*** ERROR: Module load completed but symbols could not be loaded for
atcwindows.exe
602fa0a6 ?? ???
You will see that the executable search path is "D:\Crash" (line 4) so why
is WinDbg trying and failing to load from a completely different path (and
why fail anyway since the executable does exit there)?
Alan

[Alan Williams-Key]

Post by Scot T Brennecke
When you executed the program at the time the dump was generated, the full path to the .exe was
already known and saved in the dump. It did not need to search the .exepath to get the path of the
Exe.

[Alan] OK, but - 1) when I get a crash dump from a user, it is looking for
an exe path that does not exist on my PC. How do I force it to search the
path in the image path? 2) When the crash dump is from my PC and the .exe
still exists it still fails to load (as in the example I gave)!

Post by Scot T Brennecke
It says the symbols could not be loaded for Atcwindows.exe. Did you put the PDB for that executable
in D:\Crash?

[Alan] Yes.

Post by Scot T Brennecke
If so, is it the exact same PDB that was generated at the time the .exe was built?

[Alan] To the best of my knowledge, yes (stored it using symstore).

Post by Scot T Brennecke
Which version of WinDbg is this? Have you installed the one that was released a few weeks ago?

[Alan] I downloaded the debug tools for win x86 on 20th April.

Post by Scot T Brennecke
This dump is just a small minidump. Can you gather one using the adplus.vbs script that comes with
cscript adplus.vbs -crash -pn atcwindows.exe -o D:\Crash

[Alan] How would I do that from within my C++ program?

Post by Scot T Brennecke

Post by Alan Williams-Key
Loading Dump File [D:\Crash\CRASH.DMP]
User Mini Dump File: Only registers, stack and portions of memory are
available
SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols;D:\Crash
Executable search path is: D:\Crash
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt, suite: SingleUserTS Personal
Debug session time: Thu Apr 16 17:21:06.000 2009 (GMT+2)
System Uptime: not available
Process Uptime: 0 days 0:00:26.000
.............................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(ed98.eda0): Integer divide-by-zero - code c0000094 (first/second chance not
available)
eax=7c82fbf0 ebx=00164f70 ecx=011a0ea0 edx=59a63d70 esi=00164f48 edi=00164fa0
eip=7c90e4f4 esp=0012c7c4 ebp=0012c7d4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
7c90e4f4 c3 ret
0:000> .ecxr
eax=00000002 ebx=00000001 ecx=0000015d edx=00000000 esi=60425bf0 edi=60459270
eip=602fa0a6 esp=0012e038 ebp=0012f380 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206
Unable to load image D:\Program Files\Microsoft Visual
Studio\MyProjects\atcsim\Release\atcwindows.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for atcwindows.exe
*** ERROR: Module load completed but symbols could not be loaded for
atcwindows.exe
602fa0a6 ?? ???
You will see that the executable search path is "D:\Crash" (line 4) so why
is WinDbg trying and failing to load from a completely different path (and
why fail anyway since the executable does exit there)?
Alan

[Scot T Brennecke]

When you're opening a full minidump file, you really don't need to be seeking executables. The dump
contains the loaded code already.
I neglected to observe that you stated in your original post you were using MiniDumpWriteDump, so
ignore my ADPlus comments.
Try turning on !sym noisy to get more information about the failure to load symbols. It is possibly
finding the .exe in the D:\Crash after failing to load it from the original path, but not finding
matching symbols.

Post by Alan Williams-Key

Post by Scot T Brennecke
When you executed the program at the time the dump was generated, the full path to the .exe was
already known and saved in the dump. It did not need to search the .exepath to get the path of
the
Exe.

[Alan] OK, but - 1) when I get a crash dump from a user, it is looking for
an exe path that does not exist on my PC. How do I force it to search the
path in the image path? 2) When the crash dump is from my PC and the .exe
still exists it still fails to load (as in the example I gave)!

Post by Scot T Brennecke
It says the symbols could not be loaded for Atcwindows.exe. Did you put the PDB for that
executable
in D:\Crash?

[Alan] Yes.

Post by Scot T Brennecke
If so, is it the exact same PDB that was generated at the time the .exe was built?

[Alan] To the best of my knowledge, yes (stored it using symstore).

Post by Scot T Brennecke
Which version of WinDbg is this? Have you installed the one that was released a few weeks ago?

[Alan] I downloaded the debug tools for win x86 on 20th April.

Post by Scot T Brennecke
This dump is just a small minidump. Can you gather one using the adplus.vbs script that comes
with
cscript adplus.vbs -crash -pn atcwindows.exe -o D:\Crash

[Alan] How would I do that from within my C++ program?

Post by Scot T Brennecke

Post by Alan Williams-Key
Loading Dump File [D:\Crash\CRASH.DMP]
User Mini Dump File: Only registers, stack and portions of memory are
available
SRV*c:\websymbols*http://msdl.microsoft.com/download/symbols;D:\Crash
Executable search path is: D:\Crash
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt, suite: SingleUserTS Personal
Debug session time: Thu Apr 16 17:21:06.000 2009 (GMT+2)
System Uptime: not available
Process Uptime: 0 days 0:00:26.000
.............................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(ed98.eda0): Integer divide-by-zero - code c0000094 (first/second chance not
available)
eax=7c82fbf0 ebx=00164f70 ecx=011a0ea0 edx=59a63d70 esi=00164f48 edi=00164fa0
eip=7c90e4f4 esp=0012c7c4 ebp=0012c7d4 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
7c90e4f4 c3 ret
0:000> .ecxr
eax=00000002 ebx=00000001 ecx=0000015d edx=00000000 esi=60425bf0 edi=60459270
eip=602fa0a6 esp=0012e038 ebp=0012f380 iopl=0 nv up ei pl nz na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206
Unable to load image D:\Program Files\Microsoft Visual
Studio\MyProjects\atcsim\Release\atcwindows.exe, Win32 error 0n2
*** WARNING: Unable to verify timestamp for atcwindows.exe
*** ERROR: Module load completed but symbols could not be loaded for
atcwindows.exe
602fa0a6 ?? ???
You will see that the executable search path is "D:\Crash" (line 4) so why
is WinDbg trying and failing to load from a completely different path (and
why fail anyway since the executable does exit there)?
Alan