how to modify code while debugging it without having to stop and then restart debugger

[python]

hello and thanks for reading this,

i have been a dos/windows user using some form of the basic language for 30 years now.
i own and run a small programming company and there is one feature that keeps me in the windows/basic world.

while i will agree that it has not evolved well, it does have one awesome feature that i have yet to see replicated in
any linux product that i know about so far.


i am a long time windows user and have had a great way to learn new api.
to write some code and then run it.
if there is an error, the debugger will load.
then i can figure out what the eror is, just touch up the ocde and continue to run the code.
i do not have to stop the code, modify the code, rerun the code.
often an error will only happen after a complex set of conditions and not have to completely stop the app is a fantastic
way to debug.

there are several applications that can do this.
in fact, the free version of the visual studio 2005, which is free, have this ability.

so how can i use python to debug code and change that code without having to restart the code.

thanks so much,
dave

 

[James Stroud]

so how can i use python to debug code and change that code without having
to restart the code.

look into reload()



--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

 

[Mike Meyer]

i am a long time windows user and have had a great way to learn new api.

There's a better way. See below.

to write some code and then run it.
if there is an error, the debugger will load.
then i can figure out what the eror is, just touch up the ocde and continue to run the code.
i do not have to stop the code, modify the code, rerun the code.
often an error will only happen after a complex set of conditions and not have to completely stop the app is a fantastic
way to debug.

Yup. It's been around for decades. The cooler implementations will
interpose a stage that offers you a set of proposed fixes as well as
the ability to get to the debugger.

so how can i use python to debug code and change that code without having to restart the code.

Well, as James mentioned, you can use "reload" to reload your
modules. But in general, this kind of thing doesn't work very well in
OO languages in general, and in Python in particular. Here's an
example of why:
.... print "Version 1"
.... .... print "Version 2"
....
I changed the function f on the fly - just like you want to - but all
the existing references to it will still refer to the *old* version of
the function(*). To do the right thing, you need to fix all the
references to the old code to refer to the new code as well. Unless
your language has some ability to capture a code reference - first
class functions, closures, or objects - that won't be a problem. But
Python ha all those things, and every one of them causes problems like
this. So you really can't "continue" your program from the debugger;
you need to restart it to get everything that references code to
reference the edited code.

But if the point is to learn an API, then there's something a lot
better than tweaking code inside a debugger. That's testing code
inside the interpreter. When writing web scraping software with
BeautifulSoup, writing the first guess at the scrape sequence and then
tweaking it in the debugger would be ok. But being able to get the
soup object in the interpreter, and try scrape sequences to see what
they return directly is even better. Doing it in an Emacs buffer means
I have a complete log of what I did to create the scrape, so I can
reconstruct even complicated sequences if the need arises.

If you hang out in this group long enough, you'll hear a lot about
"unit testing". It's an excellent idea, and I recommend it
highly. However, it's always struck me as a very batch oriented
approach. A more interactive approach is "exploratory proramming",
which is closer to what you're describing, and is appropriate when you
don't know the problem space very well, or if - as when learning a new
api - you don't know the tool set very well. Python is an excellent
tool for this. It's not exactly like what you described, but different
isn't necessarily inferior.

<mike

*) This is a feature. Classic use is:
oldfoo = foo
def foo(*args, *kwds):
# preprocess the arguments
return oldfoo(*args, *kwds)

 

[python]

thanks for all that have replied so far.
i still find it __very__ hard to believe that i cannot edit code inside a function while debugging it.
as i mentioned even micro$soft can do this using statically type languages like visual basic and csharp.
also, both visualbasic and csharp have goto statements, which i do to not use in final code but can be real handy when
used with the ability to change debugged code on the fly while inside the function being debugged.

for example,
if i am inside a function and there is some an error on a line and that is where the debugger is currently pointing at,
i simple copy and paste the bad code line just below the actual code line.
i fix this copied code line.
then i just turn the bad line into a comment line and the debugger will move the current focus to the next time, which
is the fixed code.

how can such a dynamic language like python not be able to do this.

i have seen functions like exec that can even run dynamically generated text on the fly.

very strange..

any other ideas,

thanks so much,

dave

 

[Fabio Zadrozny]

Hi Dave,

Currently there is no python debugger (that I know of) that does it,
altough tools are beggining to get to it (another example outside of the
python world is that eclipse already does it for java). If you use the
pdb (that is the command-line debugger that comes along with python),
you can do it, but it would require a considerable effort because you
would have to modify it manually (with reload, assignments, etc.) -- not
really a good thing in my opinion -- and you wouldn't have the easiness
that is provided by visual debuggers.

Now, I'm the developer of pydev (http://pydev.sf.net), that is a plugin
that aims at enabling python development within eclipse (and is already
used a lot in the python community), and I believe that given some time
pydev will reach the functionality you're describing -- as others will
probably do too -- and that's one of the reasons why pydev is being
developed (the current state of tools for editing python still has space
for LOTS of opportunities).

So, making it short... currently you can do it but it is not easy
because tools are still catching up to what python provides (I
personally believe this is because it is so easy to edit python and
understand the code that some people don't look for other alternatives,
and end up just 'accepting' what they have because it is already better
than alternatives -- altough this is only true for some given scenarios).

Cheers,

Fabio

 

[Magnus Lycka]

so how can i use python to debug code and change that code without having to restart the code.

I don't know how well the commercial GUIs, such as Wing IDE
manage to handle debugging. Perhaps that's worth looking into.

It's my impression that debugger support in Python is weaker
than e.g. VB, because Python programmers don't need and use
debuggers so much.

I think there are several reasons for this:
- It's easy to experiment with code in the interactive
interpreter.
- Python programs don't dump. There is rarely a need to put
a breakpoint at some known safe place and single-step from
there until it crashes, and then redo everything, trying to
find at what place before the crash you really had your bug.
You'll almost always get a controlled exception in Python.
- The tracebacks you get when exceptions appear are very
informative, and typically enough to spot the bugs more or
less at once. I debugged python programs I've never seen
before last night and today. There were maybe half a dozen
bugs, and in all cases, the tracebacks showed me exactly what
I needed to do to fix the problems at once.
- Due to Python's expressiveness, typical Python programs are
shorter and simpler than comparable programs written in
other languages. If you have spaghetti code, you really
need to single-step to understand what is going on. Python
code is typically well structured.
- With Python, it's common that people write unit tests
using e.g. the unittest or doctest libraries. With a test
driven approach as described in Extreme Programming, you run
your tests very often, with small changes in the code between
each test run.
- With object-oriented programming, it's easier to structure
your code so that each chunk of code (e.g. method) is
easy to understand. In other words, the divide and conquer
approach to problem solving works better.

I guess another reason is that Microsoft has put a lot of money
into making VB and friends user friendly. These products are
very much geared into accomodating beginners, and a nice looking
GUI has been a very high priority. For an open source tool such
as Python, where the people who drive development are the people
who need to use the tool, being beginner friendly isn't the top
priority (even though Python has succeeded well in that regard
anyway). Aspects such as stability and consistency in semantics
is considered much more important. (VB has a prettier GUI, but
Python is a much prettier language...)

 

[Steven D'Aprano]

thanks for all that have replied so far.
i still find it __very__ hard to believe that i cannot edit code inside a function while debugging it.

You write a function:

def myfunct(s):
# input arg s is a string
foo = s*3
bar = s.upper() + foo # LINE 2
blob = foo.lower() + bar
return blob

You enter the debugger and single-step to the marked line LINE 2. Then you
edit the code to this:

def myfunct(n):
# input arg n is an int
foo = n + 1
bar = foo*2 # LINE 2
blob = foo**bar
return blob

What should Python do when you step the debugger, and why is it useful?

as i mentioned even micro$soft can do this using statically type languages like visual basic and csharp.
also, both visualbasic and csharp have goto statements, which i do to not use in final code but can be real handy when
used with the ability to change debugged code on the fly while inside the function being debugged.

Better and better. Yes, I can see how the ability to jump around a
function on the fly would really help you understand how the function is
supposed to work when you take the gotos out.

for example,
if i am inside a function and there is some an error on a line and that is where the debugger is currently pointing at,
i simple copy and paste the bad code line just below the actual code line.
i fix this copied code line.
then i just turn the bad line into a comment line and the debugger will move the current focus to the next time, which
is the fixed code.

how can such a dynamic language like python not be able to do this.

Do you try to ignore the syntax and grammar of the programming language
you are coding in too, or only English?


[snip]

Just out of curiosity, how much is the free version of Visual Studio 2005?

 

[Steve Holden]

Better and better. Yes, I can see how the ability to jump around a
function on the fly would really help you understand how the function is
supposed to work when you take the gotos out.

I must admit I had been wondering just how far the OP wanted to go in
mangling the code. I suspect that the interesting bit to the OP is
having a visual editor available to alter functions and class
definitions "on the fly" rather than having to completely re-enter the
definition as you would at the interactive interpreter prompt. He or
she'd probably be a bit unhappy about the need to reload() modules too,
I suppose.

Do you try to ignore the syntax and grammar of the programming language
you are coding in too, or only English?

That's rather unkind. I'd judge we are plainly dealing with someone who
is working hard to express questions in a foreign language. Funny,
perhaps, but definitely unkind. Take two demerits and smack yourself on
the wrist.

Just out of curiosity, how much is the free version of Visual Studio 2005?

I'm not positive, but i think they're currently giving it away.

regards
Steve

 

[Mike Meyer]

I must admit I had been wondering just how far the OP wanted to go in
mangling the code. I suspect that the interesting bit to the OP is
having a visual editor available to alter functions and class
definitions "on the fly" rather than having to completely re-enter the
definition as you would at the interactive interpreter prompt. He or
she'd probably be a bit unhappy about the need to reload() modules
too, I suppose.

In that case, you're using the wrong IDE. I run the Python interpeter
inside of Emacs. I edit my code in another buffer. In the source code
buffer, I hit M-C-x, and the current version of the function I'm
currently editing gets sent to the interpreter. Reload is pretty easy
as well - C-c RETURN, and the module I'm editing gets reloaded.

<mike

 

[Neil Hodgson]

Steven D'Aprano:

You write a function:

def myfunct(s):
# input arg s is a string
foo = s*3
bar = s.upper() + foo # LINE 2
blob = foo.lower() + bar
return blob

You enter the debugger and single-step to the marked line LINE 2. Then you
edit the code to this:

def myfunct(n):
# input arg n is an int
foo = n + 1
bar = foo*2 # LINE 2
blob = foo**bar
return blob

What should Python do when you step the debugger,

Python should throw the power switch and sulk for at least an hour.

> and why is it useful?

Teaches the user who's boss.

Debug time code modification is useful to me in C++ for two reasons.
The first is where there is a simple bug: step, step, step, aah!,
fiddle, step, works! The second is where you want to perturb the code to
produce some unusual situation within a test run: how would the calling
code cope if this code allowed a duplicate element through? fiddle,
step, step, crash! Mmm, that looks like the fault report, maybe there is
another way that duplicates are possible. The first type of change
become permanent parts of the code while the second type are ephemeral.

There are limitations to the technology: if you add too much code it
won't fit in the allocation (which has been padded a bit for debugging
but not much) or the function is being reentered. I'm not sure about all
the limitations and they change between releases but it probably fails
about one time in ten for me. This doesn't stop the session, just leaves
your change unapplied.

Neil

 

[Magnus Lycka]

In that case, you're using the wrong IDE. I run the Python interpeter
inside of Emacs. I edit my code in another buffer. In the source code
buffer, I hit M-C-x, and the current version of the function I'm
currently editing gets sent to the interpreter. Reload is pretty easy
as well - C-c RETURN, and the module I'm editing gets reloaded.

As far as I understand, the OP wanted to do this while single-stepping
through the program he's editing. While this might work as a kind of
exploration, it's probably not an optimal development strategy. It
might be difficult to predict how the program will run the next time
if you manipulate it during execution.

I think test-driven development as described e.g. in my EPC presentation
last year is more rewarding: http://www.thinkware.se/epc2004test/
(See e.g. the log.html)

I suppose different languages and tools foster different styles of
work, and I can understand that it's frustrating if a favoured style
of development isn't really supported by the Python tools--even though
few Python programmers bother about single-stepping through their
code.

In general, it's clearly non-optimal to run code many magnitudes
slower than the nominal speed, and I suspect that few people would
care to do that unless the structure of the code they work with
was messy.

I guess it's a bit like driving an old crappy car, and then getting
into a new Toyota. I can understand that it seems strange not to
have the trunk filled with tools if you're about to take a long trip,
but it's probably a mistake to think that this will make the journey
with the Toyota more problematic than the trip would have been with
a car that you need to repair every now and then.

 

[Ed Singleton]

I used Visual Basic a long time in the past and I know what you mean.

The ability to step through code line by line was very useful in a
language where you often didn't know what was happening. I
particularly loved the ability to hover the mouse over any variable or
expression and see the value at that point in the code.

As a learning tool it would be excellent for Python, as would the
ability to step through the code, hit an error, step back one line,
change the line that caused the error and then continue stepping
forward again.

However I have to say that since using Python, I haven't needed these
features as much (though I still would have liked to have them
available).

Ed

 

[Steven D'Aprano]

That's rather unkind. I'd judge we are plainly dealing with someone who
is working hard to express questions in a foreign language. Funny,
perhaps, but definitely unkind. Take two demerits and smack yourself on
the wrist.

Your judgement is very different from mine. The poster's name is Dave, and
to my eyes his writing is very good English, albeit with lots of typos,
except for refusal to use capital letters where required ("Oh, I'll just
leave out braces in C because I feel like it") and deliberately incorrect
use of punctuation ("I don't feel like using '.' for attribute references,
I'll use '?' instead").

Programmers are supposed to be precise in their use of language --
failure to write what you intend is a bug in natural language just as
much as it is in C, VB, Lisp or Python. It just aggravates me to see
supposedly precise and accurate programmers *deliberately* breaking syntax
and grammar of natural language for no good reason.

(It is, of course, possible to break the rules of natural language for
good reason. Good writers do it all the time.)

I'm not positive, but i think they're currently giving it away.

Hmmm... this free version they give away... how much are they giving it
away for?

*wink* (we could keep this up all day...)

 

[Mike Meyer]

As far as I understand, the OP wanted to do this while single-stepping
through the program he's editing. While this might work as a kind of
exploration, it's probably not an optimal development strategy. It
might be difficult to predict how the program will run the next time
if you manipulate it during execution.

Yes, that's what he wanted. I was pointing out that there are
alternatives between "changing the function while you're debugging it"
and "retyping the function completely at the interactive prompt."

I think test-driven development as described e.g. in my EPC presentation
last year is more rewarding: http://www.thinkware.se/epc2004test/
(See e.g. the log.html)

Depends on what you're doiing. If you know the subject area well
enough that you casn design all the objects and methods in advance so
you can write your unit tests, then this is indeed very rewarding.

If, on the other hand, you are doing something where you don't have a
clear understanding of what all the components are, and how they
interact, then it's more important to try things out to gain that
understanding than it is to have tests for components or methods that
you may well discard or morph beyond recognition tomorrow. The dynamic
nature of Python, coupled with the bundled interactive interpreter,
makes it particularly good for this type of programming.

I guess it's a bit like driving an old crappy car, and then getting
into a new Toyota. I can understand that it seems strange not to
have the trunk filled with tools if you're about to take a long trip,
but it's probably a mistake to think that this will make the journey
with the Toyota more problematic than the trip would have been with
a car that you need to repair every now and then.

Nice analogy.

<mike