How to write verbose scripts

[Steven D'Aprano]

I find myself writing command line tools in Python where I wish to
include "verbose" output to stdout.

I start with a helper function:


def print_(obj, level=0):
if _verbosity >= level:
print obj


And then I end up with functions or methods looking like this:


def parrot(x)
print_("precondition", level=2)
do_something()
print_("status is good...", level=1)
print_("parrot is squawking strongly now", level=2)
do_something_else()
print_("squawk squawk squawk", level=3)
do_more()
print_("postcondition", level=1)
return something


That often means that my functions end up with more message printing code
than actual code. The whole thing seems messy and hard to manage for all
but the smallest scripts.

Worst of all, sometimes the messages I wish to print may be expensive to
compute, and I don't want to waste time computing them if they aren't
going to be printed because the verbosity is too low. But nor do I wish
to fill my code with this:

if _verbosity >= 3:
x = calculate_complicated_thing()
print_(x, level=3)



Is there a better way of doing this than the way I am going about it?

 

[Joe Riopel]

Is there a better way of doing this than the way I am going about it?

Would the logging module help, and just print the output to the stdout
(or a file) instead?

 

[Mensanator]

I find myself writing command line tools in Python where I wish to
include "verbose" output to stdout.

I start with a helper function:

def print_(obj, level=0):
if _verbosity >= level:
print obj

And then I end up with functions or methods looking like this:

def parrot(x)
print_("precondition", level=2)
do_something()
print_("status is good...", level=1)
print_("parrot is squawking strongly now", level=2)
do_something_else()
print_("squawk squawk squawk", level=3)
do_more()
print_("postcondition", level=1)
return something

That often means that my functions end up with more message printing code
than actual code. The whole thing seems messy and hard to manage for all
but the smallest scripts.

Worst of all, sometimes the messages I wish to print may be expensive to
compute, and I don't want to waste time computing them if they aren't
going to be printed because the verbosity is too low. But nor do I wish
to fill my code with this:

if _verbosity >= 3:
x = calculate_complicated_thing()
print_(x, level=3)

Is there a better way of doing this than the way I am going about it?

I do something like this, although I don't know if it would
be an improvement.

def collatz(a,p):
""" 3x+1 sequencer, no loop detection

collatz(a,p)
a: starting value
p: print options (binary)
bit 0 print even numbers (turns off power division)
bit 1 print odd numbers
bit 2 print debug (if any)
returns: CollatzSequenceParameters [R1count, R2count]
"""
ONE = gmpy.mpz(1)
TWO = gmpy.mpz(2)
TWE = gmpy.mpz(3)
a = gmpy.mpz(a)
t = 0
u = 0
done = 0

if (p & 1)==1:
print_evens = True
else:
print_evens = False
if (p & 2)==2:
print_odds = True
else:
print_odds = False
if (p & 4)==4:
print_debug = True
else:
print_debug = False

while done==0:
f = gmpy.scan1(a,0) # locate LS 1-bit
if f>0: # it's even
if print_evens:
print a,
a = a >> 1 # no power division
u += 1
else:
a = a >> f # power division
u += f
else:
if print_odds:
print a,
if a==1:
done = 1
seq_end = t + u
else:
a = a*TWE + ONE
t += 1
return [u,t]

 

[Diez B. Roggisch]

I find myself writing command line tools in Python where I wish to
include "verbose" output to stdout.

I start with a helper function:


def print_(obj, level=0):
if _verbosity >= level:
print obj


And then I end up with functions or methods looking like this:


def parrot(x)
print_("precondition", level=2)
do_something()
print_("status is good...", level=1)
print_("parrot is squawking strongly now", level=2)
do_something_else()
print_("squawk squawk squawk", level=3)
do_more()
print_("postcondition", level=1)
return something


That often means that my functions end up with more message printing code
than actual code. The whole thing seems messy and hard to manage for all
but the smallest scripts.

Worst of all, sometimes the messages I wish to print may be expensive to
compute, and I don't want to waste time computing them if they aren't
going to be printed because the verbosity is too low. But nor do I wish
to fill my code with this:

if _verbosity >= 3:
x = calculate_complicated_thing()
print_(x, level=3)



Is there a better way of doing this than the way I am going about it?

I use the logging-module.

Regarding the expensive computations: maysomething like this help:

class DeferredToString(object):

def __init__(self, func):
self.func = func

def __repr__(self):
return repr(self.func())

def __str__(self):
return str(self.func())



dts = DeferredToString

Because then you can do

logger.debug("Some text for an: %r", dts(lambda: long_computation()))

Because AFAIK the string is only interpolated if the logging level is
actually put out.

Diez

 

[MRAB]

I find myself writing command line tools in Python where I wish to
include "verbose" output to stdout.

I start with a helper function:

def print_(obj, level=0):
    if _verbosity >= level:
        print obj

And then I end up with functions or methods looking like this:

def parrot(x)
    print_("precondition", level=2)
    do_something()
    print_("status is good...", level=1)
    print_("parrot is squawking strongly now", level=2)
    do_something_else()
    print_("squawk squawk squawk", level=3)
    do_more()
    print_("postcondition", level=1)
    return something

That often means that my functions end up with more message printing code
than actual code. The whole thing seems messy and hard to manage for all
but the smallest scripts.

Worst of all, sometimes the messages I wish to print may be expensive to
compute, and I don't want to waste time computing them if they aren't
going to be printed because the verbosity is too low. But nor do I wish
to fill my code with this:

if _verbosity >= 3:
    x = calculate_complicated_thing()
    print_(x, level=3)

Is there a better way of doing this than the way I am going about it?

How about:

def print_(obj, level=0):
if _verbosity >= level:
if callable(obj):
obj = obj()
print obj

so that you could then use:

print_("precondition", level=2)

and:

print_(calculate_complicated_thing, level=3)

or:

print_(lambda: calculate_complicated_thing(argument), level=3)

 

[John Machin]

On Sep 2, 11:55 am, Steven D'Aprano <st...@REMOVE-THIS-

if (p & 1)==1:
print_evens = True
else:
print_evens = False
if (p & 2)==2:
print_odds = True
else:
print_odds = False
if (p & 4)==4:
print_debug = True
else:
print_debug = False

No, no, no, you've taken "How to write verbose scripts" rather too
literally; try this:

print_evens = p & 1
print_odds = p & 2
print_debug = p & 4

 

[Mensanator]

No, no, no, you've taken "How to write verbose scripts" rather too
literally; try this:

print_evens = p & 1
print_odds = p & 2
print_debug = p & 4

Thanks for that. Luckily, the code only does that once per call.

 

[Bruno Desthuilliers]

John Machin a écrit :

No, no, no, you've taken "How to write verbose scripts" rather too
literally;

KEYBOARD !

 

[Uwe Schmitt]

I find myself writing command line tools in Python where I wish to
include "verbose" output to stdout.

I start with a helper function:

def print_(obj, level=0):
    if _verbosity >= level:
        print obj

And then I end up with functions or methods looking like this:

def parrot(x)
    print_("precondition", level=2)
    do_something()
    print_("status is good...", level=1)
    print_("parrot is squawking strongly now", level=2)
    do_something_else()
    print_("squawk squawk squawk", level=3)
    do_more()
    print_("postcondition", level=1)
    return something

That often means that my functions end up with more message printing code
than actual code. The whole thing seems messy and hard to manage for all
but the smallest scripts.

Worst of all, sometimes the messages I wish to print may be expensive to
compute, and I don't want to waste time computing them if they aren't
going to be printed because the verbosity is too low. But nor do I wish
to fill my code with this:

if _verbosity >= 3:
    x = calculate_complicated_thing()
    print_(x, level=3)

Is there a better way of doing this than the way I am going about it?

You can save some code if you use function decorators for logging
input and output values of
functions.
So write lots of functions containing one statement and your problem
is solved ;-)

Greetings, Uwe

 

[Steven D'Aprano]

Would the logging module help, and just print the output to the stdout
(or a file) instead?

Thank you to everyone who answered.

As I feared, it seems that there's no really simple way of dealing with
arbitrary messages at arbitrary parts of my code.

I'm currently playing around with a few ideas, one of which is a
decorator to print information about function calls. However, what I'd
like to avoid is having to explicitly call the decorator on every method.
I guess that there's metaclass trickery I can play. Does anyone have any
pointers on how I can apply a decorator to every method in a class
automatically?

 

[Lie]

You can save some code if you use function decorators for logging
input and output values of
functions.
So write lots of functions containing one statement and your problem
is solved ;-)

Greetings, Uwe

That would be perfect if python is a fully functional language (read
that as: purely functional language). Functional language dictates no
side effects, so logging input and output of functions is pretty much
everything needed to do debugging (which is the main reason for having
verbose mode). It would still be feasible, though, if you keep
yourself from using functions with side-effects (or use as few of them
as possible).

Bjorn Lindqvist says:

One big downside with that approach is that it becomes much harder to
grep for the log message. Usually when I go through logs, I don't care
what exactly the message says, just that it is easily googleable (uses
some kind of identifying text) and that it is unique so I can know
exactly where it was emitted.

Actually, I'd prefer a log code (perhaps also as an option, to use -e
for showing up log code besides the log message). The log code would
be unique and referenced only once in the whole application (to nail
down who said what to a single point). The code would make a call to
the log printer with the log code (not the error string) and a few
arguments to be interpolated to the error string (taken from a
dictionary indexed by error code).

The downside is loss of inline documentation by the logging codes.