How do debuggers work?

[jacob navia]

In a recent discussion, we have seen the "regulars" group say they do
not use a debugger. Just reading the source code is enough. Wild claims
are being done. For instance a certain Mr "Heathfield" claims

<quote >
<[email protected]>

On a number of occasions I have (correctly) debugged code that I haven't
even seen, let alone run through a debugger.

<end quote>

This series of installments are dedicated to people that aren't
super heroes, "regulars in clc" or similar, and will use a debugger.
I will assume then, that you need a debugger, and want to know how
it works.

------------------------------------------------------------------------

A debugger is a normal program, that starts another program (the
"debuggee", or using gdb terminology "the "inferior") that is going
to run under the supervision of the debugger.

The debugger uses the informations generated by the compiler to
associate the source code with the machine addresses that the program
is executing, and show the corresponding source code.

The debugger achieves this goal by establishing a *breakpoint*. This
can be established in (basically) two ways:

1: By inserting a breakpoint instruction into special positions of the
code.
2: By using the compiler to generate stop instructions that will make
the program stop and wait instructions from a supervisor program.

The first solution is used when the debugger can have access to the
code stream and can modify it. This is the most usual solution in
workstations and advanced CPUs.

The second solution is used when the code stream is not writable (for
instance, the code is in ROM or EPROM and it is not possible to insert
anything into it).

The second solution is also the only solution available when the CPU
has no "break" instruction and no single step execution mode. In that
case, the debugger must provide this functionality.

Debug information formats
-------------------------
The informations generated by the compiler are specific from compiler to
compiler. Under windows, we have the NBXX series of standards produced
by Microsoft corp for their debugger "Code view", one of the most
innovative debuggers written to date.

The debug information under Unix can be from several standards (this is
an old Unix story: there are just too many standards to choose from).

The COFF standard debug info is generated by compilers in certain
debugging environment, specifically for the DBX debugger under AIX,
as far as I know, even if the format seems extended COFF and not just
COFF that is very simple as debugging formats go. (COFF means Common
Object code File Format, or something like that, I forgot)

We have under Unix also the "stabs" debug info format. This is a format
used by older versions of gcc (and lcc-win under linux) that writes
"strings" (i.e. String TABleS), that describe the program.

And we have more recently, the DWARF standard. This format i used by
more recent versions of gcc, and its a VERY complex debug information
format, as complex as its target language, C++. Obviously DWARF people
will tell you that DWARF is language independent, but its main use is
within C++/C.

Under windows, lcc-win uses the NB09 format from Microsoft Visual C++
version 4. I decided to stay within that format, and not "upgrade" to
the more "advanced" versions NB10, and following, since the effort
needed was not worth the few functionalities more that the new format
provided. This is a binary format (unlike "stabs") that is composed of
variable length records, with two bytes for length of record, and then
a "Type" field that describes what the record should contain.

With the change from 32 to 64 bits, I needed to tweak this format, and
I see a major rewrite soon, since there are some fields needed that
I do not yet support.

Operating system support
------------------------

The debugger needs to be called by the OS when something happens, and
obviously, it needs to be called before the inferior program starts, so
that breakpoints can be established at startup.

Under windows there is a complex API that provides this essential
service to debuggers. It allows everything a debugger needs, like
getting notified when an event occurs, writing/reading the inferior
program data space, etc.

Under Linux/Unix, there is the "ptrace" system call that provides this
service. It provides the same functionalities you would want from a
debugger: controlled execution, step by step, read/write program memory,
etc.

Setting a breakpoint.
---------------------
To set a breakpoint, the debugger reads the code at the address where
it wants to stop, and overwrites that code instructions with a
breakpoint instruction. This instruction will work like an interrupt,
stop the program execution, and give control to the OS to service that
interrupt. The OS saves the context of the breakpoint (registers, etc)
and gives control to the debugger.

To erase a breakpoint, the debugger writes the instructions that were
previously stored at the breakpoint address, and either executes them
in single step mode, rewriting the breakpoint after control has passed
over those instructions (that is a *permanent* breakpoint) or just
goes on (that was a *temporary* breakpoint).


Changing the point where execution resumes
------------------------------------------
The debugger can write any value into the inferior program registers.
This means that the instruction pointer can be changed, making the
program resume at a different source line than the last one executed.


In a next installment we will discuss about what happens when you crash.
You will see what other options are available besides "reading the
source code".

 

[Richard Heathfield]

jacob navia said:

In a recent discussion, we have seen the "regulars" group say they do
not use a debugger. Just reading the source code is enough. Wild claims
are being done. For instance a certain Mr "Heathfield" claims

<quote >
<[email protected]>

On a number of occasions I have (correctly) debugged code that I haven't
even seen, let alone run through a debugger.

<end quote>

Far from being a "wild claim", it's actually true. What's more, what I did
wasn't particularly difficult. A sufficiently experienced programmer *can*
sometimes solve another programmer's simple problems remotely (e.g. over
the phone) without ever seeing the source code. I would guess that those
who read this will react in one of three different ways: (a) disbelief and
scorn (like yours), (b) "one day I'll be able to do that", or (c) "yeah,
that's obvious, I do it in my sleep, so what?" All three reactions are
completely understandable, but (c) is the reaction I'd expect from
experienced programmers.

This series of installments are dedicated to people that aren't
super heroes, "regulars in clc" or similar, and will use a debugger.
I will assume then, that you need a debugger,

That's the problem - your assumption that a debugger is needed. It isn't.
It's a nice tool to have in your box, but it's not essential, even to
non-superheroes such as myself.

In a next installment we will discuss about what happens when you crash.
You will see what other options are available besides "reading the
source code".

Yes, and nobody denies it. What you seem to fail to realise is that
"reading the source code" really *is* a perfectly feasible option. Nobody
has claimed that no other options exist.

 

[Ben Bacarisse]

This series of installments are dedicated to people that aren't
super heroes, "regulars in clc" or similar, and will use a debugger.
I will assume then, that you need a debugger, and want to know how
it works.

This (and several of your other new threads) really belong in
comp.programming. Why not post there?

 

[Kenny McCormack]

jacob navia said:


Far from being a "wild claim", it's actually true.

Guess what? We don't believe you.

Any more than we believe anything that GWB says.

 

[Antoninus Twink]

Guess what? We don't believe you.

Any more than we believe anything that GWB says.

I think RJH and GWB have a lot in common - the stupid dogmatism, the
irrational certainty that their world-view is correct, and above all the
same belligerence directed against those who question their actions.

 

[Antoninus Twink]

That's the problem - your assumption that a debugger is needed. It isn't.
It's a nice tool to have in your box, but it's not essential, even to
non-superheroes such as myself.

But you're too modest - if all the claims you make for yourself in this
newsgroup are true, then you really are nothing less than a superhero.

 

[Kenny McCormack]

But you're too modest - if all the claims you make for yourself in this
newsgroup are true, then you really are nothing less than a superhero.

Shucks... 'Tweren't nothin', ma'am.

 

[Kenny McCormack]

I think RJH and GWB have a lot in common - the stupid dogmatism, the
irrational certainty that their world-view is correct, and above all the
same belligerence directed against those who question their actions.

All very true. The analogy was not chosen by accident.
Note further that most of us give up this kind of certainty about the
world at about age 12 - when we begin to see how complex the world
really is. RJH & GWB both exhibit classical signs of stunted growth.

I am certain that it won't be much longer until references to GWB are
treated the same way as references to Hitler. I.e., as an emblem of
corrupt government. And there will probably be something akin to
Godwin's Law in reference to "the first mention of GWB in a thread".

 

[Kenneth Brody]

jacob navia said:


Far from being a "wild claim", it's actually true. What's more, what I did
wasn't particularly difficult. A sufficiently experienced programmer *can*
sometimes solve another programmer's simple problems remotely (e.g. over
the phone) without ever seeing the source code. I would guess that those
who read this will react in one of three different ways: (a) disbelief and
scorn (like yours), (b) "one day I'll be able to do that", or (c) "yeah,
that's obvious, I do it in my sleep, so what?" All three reactions are
completely understandable, but (c) is the reaction I'd expect from
experienced programmers.

I have to agree with Me. Heathfield on this one, falling pretty close to
group (c), above. (AKA "BTDT".)

While most ("nearly all"?) debugging is more involved, I, too, have, on
more than one occasion, debugged someone else's code without ever seeing
it, based solely on the symptoms.

That's the problem - your assumption that a debugger is needed. It isn't.
It's a nice tool to have in your box, but it's not essential, even to
non-superheroes such as myself.

Well, even though some debugging can be done over the phone / via e-mail,
without seeing any source, that's certainly the exception rather than the
rule. (Consider the case where a database index corruption is detected
150,000 records into a report, and the corruption occurred some 20,000
records earlier. I'd hate to think about tracking that one down without
a debugger. "Could" it have been done without one? Yes, which I guess
technically means the debugger wasn't "essential". Then again, you could
say that a C compiler isn't "essential" either, as an experienced
programmer could hand-compile the code without one.)

Yes, and nobody denies it. What you seem to fail to realise is that
"reading the source code" really *is* a perfectly feasible option. Nobody
has claimed that no other options exist.

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[Dik T. Winter]

> While most ("nearly all"?) debugging is more involved, I, too, have, on
> more than one occasion, debugged someone else's code without ever seeing
> it, based solely on the symptoms.

Indeed. In one instance the output from a test program was sufficient
to determine what the bug was. I may note that the but was in a
proprietary part of the software, so source was not even available,
neither was there a possibility to use a debugger.

 

[Richard Heathfield]

Kenneth Brody said:

While most ("nearly all"?) debugging is more involved, I, too, have, on
more than one occasion, debugged someone else's code without ever seeing
it, based solely on the symptoms.

Right.

Well, even though some debugging can be done over the phone / via e-mail,
without seeing any source, that's certainly the exception rather than the
rule.

Right again.

(Consider the case where a database index corruption is detected
150,000 records into a report, and the corruption occurred some 20,000
records earlier. I'd hate to think about tracking that one down without
a debugger. "Could" it have been done without one? Yes, which I guess
technically means the debugger wasn't "essential". Then again, you could
say that a C compiler isn't "essential" either, as an experienced
programmer could hand-compile the code without one.)

When a debugger is the right tool for the job, I advocate using one. I
sense a strawman here. I do not hold the position that debuggers are never
useful. I *do* hold the position that they are only one tool amongst
several, and that some people don't need them as often as others.

<snip>

 

[Richard Tobin]

The debugger needs to be called by the OS when something happens, and
obviously, it needs to be called before the inferior program starts, so
that breakpoints can be established at startup.

This doesn't quite reflect the sequence of events on systems I'm
familiar with. The debugger is run by the user in the normal way.
And then it starts, or attaches to, the program to be debugged. And
when that progam starts or something goes wrong, the debugger is not
*called*, but either receives a signal or returns from a system call
(wait() in unix).

-- Richard

 

[jacob navia]

This doesn't quite reflect the sequence of events on systems I'm
familiar with. The debugger is run by the user in the normal way.
And then it starts, or attaches to, the program to be debugged. And
when that progam starts or something goes wrong, the debugger is not
*called*, but either receives a signal or returns from a system call
(wait() in unix).

-- Richard

Yes, there is a "Wait for debug event" function that is
awaken when there is a debug event. This is not (obviously)
a normal function call, I was using the word "call" in a
generic sense as to when you call somebody to do some
task... It is not a function call.

 

[Morris Dovey]

This series of installments are dedicated to people that aren't
super heroes, "regulars in clc" or similar, and will use a debugger.
I will assume then, that you need a debugger, and want to know how
it works.

This could be a useful series. Will you also address
non-interactive debug programs such as those who identify
dynamically-allocated memory drains, etc for post-execution
analysis?

Operating system support
------------------------

The debugger needs to be called by the OS when something happens, and
obviously, it needs to be called before the inferior program starts, so
that breakpoints can be established at startup.

will you address debuggers used in non-hosted embedded
environments and those that can also be used to debug hardware,
such as those used to provide JTAG/EJTAG manipulation for
examining and modifying the states of internal CPU signals,
latches, and registers?

I would expect that a reasonably complete, high-quality effort
will keep you fairly busy for at least a week or so...

In fact, if you can manage to present a complete and accurate
version of "How do debuggers work" in less than three months, I
think you should have "super hero" status yourself.

Go for it!

 

[Malcolm McLean]

I think RJH and GWB have a lot in common - the stupid dogmatism, the
irrational certainty that their world-view is correct, and above all the
same belligerence directed against those who question their actions.

Both have also got a comon touch that can easily be misinterpreted as
stupidity. RH you know for yourself, GWB is far from an idiot.

 

[Malcolm McLean]

When a debugger is the right tool for the job, I advocate using one. I
sense a strawman here. I do not hold the position that debuggers are never
useful. I *do* hold the position that they are only one tool amongst
several, and that some people don't need them as often as others.

My view on this is that the debugger creates a very intimate relationship
with the programmer. When I first learnt C I used one. The run / debug cycle
becomes a natural way of working.
Then In my first job I got a Unix box and suddenly the debugger was taken
away. Then I got another one for a games console, but it wasn't well
designed. I decided at that point that even a good debugger was too
seductive. So now I use diagnostics almost exclusively. If I really can't
find a bug in a Windows program I might try to debug.

 

[Antoninus Twink]

Both have also got a comon touch that can easily be misinterpreted as
stupidity. RH you know for yourself, GWB is far from an idiot.

The levels of sycophancy in this group never cease to amaze me. If
Heathfield fell in shit, half the people here would swear blind he
smelled of roses.

 

[Kenneth Brody]

Antoninus Twink wrote:
[...]

The levels of sycophancy in this group never cease to amaze me. If
Heathfield fell in shit, half the people here would swear blind he
smelled of roses.

.... right after they've been fertilized.

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[Kenny McCormack]

The levels of sycophancy in this group never cease to amaze me. If
Heathfield fell in shit, half the people here would swear blind he
smelled of roses.

It happens every day.

And they do.

 

[Kenny McCormack]

Both have also got a comon touch that can easily be misinterpreted as
stupidity. RH you know for yourself, GWB is far from an idiot.

Keep in mind that McLean is religious. That kinda binds him to GWB in a
way that most rational people simply cannot grok.