Optparse and help formatting?

[Tim Chase]

I've been learning the ropes of the optparse module and have been
having some trouble getting the help to format the way I want.

I want to specify parts of an option's help as multiline.
However, the optparse formatter seems to eat newlines despite my
inability to find anything in optparse.py that does something
obvious like .replace("\n", " ") to eat the newlines I stick in
the help. It also seems to mung tabs. An example from my code:

parser.add_option("-x", "--extended",
action="callback",
callback=callback_test,
type="string", # required to get metavar to show in help
dest="test",
metavar="CONF",
help="a comma-separated list of options.\n"
"s=[YYYY]MMDD,n=NAME,c=COLOR\n"
"s,start=[YYYY]MMDD\n"
"\tstart day of period (default, current day)\n"
...
)

which would accept options that looked something like

test.py -x s=20070401,n=XR71,c=black

(my callback_test breaks apart parser.rargs[0] to deal with the
funky params).

However, when I display the help, its reformatting eats the
newlines in my help-string and strangely looks like it converts
my tabs to spaces.

Any hints on how to either work around the problem or fix it?

As a side note, I couldn't find documented anywhere that if you
have a "metavar" setting for a "callback" parameter, you have to
specify the "type" to get the metavar to show up in the help as
"-x CONF, --extended=CONF". If you don't specify the type, it
will display as "-x, --extended" without the metavar, even if you
specify one. Hackishly setting type="string" made the metavar
appear.

Thanks for any hints,

-tkc
(python2.4 on Debian, FWIW)

 

[Ben Finney]

I've been learning the ropes of the optparse module and have been
having some trouble getting the help to format the way I want.

A quick perusal of the 'optparse.py' code shows me this:

========
[...]
class OptionParser([...]):
def __init__([...],
formatter=None,
[...]):
[...]
if formatter is None:
formatter = IndentedHelpFormatter()
[...]
========

So, the OptionParser init method accepts the help formatter as the
'formatter' argument, defaulting to a new instance of
IndentedHelpFormatter.

Presumably, it's a matter of subclassing 'optparse.HelpFormatter' and
overriding the behaviour you want to change, then passing an instance
of your new class as the 'formatter' argument to the 'OptionParser()'
invocation.

 

[Steven Bethard]

I've been learning the ropes of the optparse module and have been
having some trouble getting the help to format the way I want.

I want to specify parts of an option's help as multiline.
However, the optparse formatter seems to eat newlines despite my
inability to find anything in optparse.py that does something
obvious like .replace("\n", " ") to eat the newlines I stick in
the help. It also seems to mung tabs. An example from my code:

parser.add_option("-x", "--extended",
action="callback",
callback=callback_test,
type="string", # required to get metavar to show in help
dest="test",
metavar="CONF",
help="a comma-separated list of options.\n"
"s=[YYYY]MMDD,n=NAME,c=COLOR\n"
"s,start=[YYYY]MMDD\n"
"\tstart day of period (default, current day)\n"
...
)

which would accept options that looked something like

test.py -x s=20070401,n=XR71,c=black

(my callback_test breaks apart parser.rargs[0] to deal with the
funky params).

However, when I display the help, its reformatting eats the
newlines in my help-string and strangely looks like it converts
my tabs to spaces.

Any hints on how to either work around the problem or fix it?

I guess this is just an optparse-y week on c.l.py. ;-) Ben Finney
pointed you in the right direction for optparse.

If you care to try out argparse (http://argparse.python-hosting.com/)
which has a similar API, it has a builtin RawTextHelpFormatter formatter
class::

import argparse

def my_type(string):
# split string and turn it into appropriate object(s)
return 'foo(%s)' % string

parser = argparse.ArgumentParser(
formatter_class=argparse.RawTextHelpFormatter
)
parser.add_argument(
"-x", "--extended",
type=my_type, dest="test", metavar="CONF",
help="a comma-separated list of options.\n"
"s=[YYYY]MMDD,n=NAME,c=COLOR\n"
"s,start=[YYYY]MMDD\n"
"\tstart day of period (default, current day)\n"
)

args = parser.parse_args()
print args.test

This program will produce output like::

$ python script.py -x sdfsdfas
foo(sdfsdfas)

$ python script.py --help
usage: script.py [-h] [-x CONF]

optional arguments:
-h, --help show this help message and exit
-x CONF, --extended CONF
a comma-separated list of options.
s=[YYYY]MMDD,n=NAME,c=COLOR
s,start=[YYYY]MMDD
start day of period ...

HTH,

STeVe

 

[Tim Chase]

I've been learning the ropes of the optparse module and have been

having some trouble getting the help to format the way I want.

A quick perusal of the 'optparse.py' code shows me this:

========
[...]
class OptionParser([...]):
def __init__([...],
formatter=None,
[...]):
[...]
if formatter is None:
formatter = IndentedHelpFormatter()
[...]
========

So, the OptionParser init method accepts the help formatter as the
'formatter' argument, defaulting to a new instance of
IndentedHelpFormatter.

Presumably, it's a matter of subclassing 'optparse.HelpFormatter' and
overriding the behaviour you want to change, then passing an instance
of your new class as the 'formatter' argument to the 'OptionParser()'
invocation.

Ben, thanks for pointing me in the right direction. Digging in
this code, it looks like textwrap.[fill|wrap] method calls were
what was eating my NL and munging my tabs.

While not a terribly elegant solution, as there's no easy way to
intercept the two bits I want without copying and pasting a large
bit of the format_description and format_option methods just to
change the behavior of one call to textwrap.*, I did manage to
get it working as desired.

In the event that anybody else needs the solution I hacked
together, I've pasted it below which, as Ben suggests, can be
used with

parser = OptionParser(...
formatter=IndentedHelpFormatterWithNL()
)

Hope this helps somebody else too.

-tim



class IndentedHelpFormatterWithNL(IndentedHelpFormatter):
def format_description(self, description):
if not description: return ""
desc_width = self.width - self.current_indent
indent = " "*self.current_indent
# the above is still the same
bits = description.split('\n')
formatted_bits = [
textwrap.fill(bit,
desc_width,
initial_indent=indent,
subsequent_indent=indent)
for bit in bits]
result = "\n".join(formatted_bits) + "\n"
return result

def format_option(self, option):
# The help for each option consists of two parts:
# * the opt strings and metavars
# eg. ("-x", or "-fFILENAME, --file=FILENAME")
# * the user-supplied help string
# eg. ("turn on expert mode", "read data from FILENAME")
#
# If possible, we write both of these on the same line:
# -x turn on expert mode
#
# But if the opt string list is too long, we put the help
# string on a second line, indented to the same column it would
# start in if it fit on the first line.
# -fFILENAME, --file=FILENAME
# read data from FILENAME
result = []
opts = self.option_strings[option]
opt_width = self.help_position - self.current_indent - 2
if len(opts) > opt_width:
opts = "%*s%s\n" % (self.current_indent, "", opts)
indent_first = self.help_position
else: # start help on same line as opts
opts = "%*s%-*s " % (self.current_indent, "", opt_width, opts)
indent_first = 0
result.append(opts)
if option.help:
help_text = self.expand_default(option)
# Everything is the same up through here
help_lines = []
for para in help_text.split("\n"):
help_lines.extend(textwrap.wrap(para, self.help_width))
# Everything is the same after here
result.append("%*s%s\n" % (
indent_first, "", help_lines[0]))
result.extend(["%*s%s\n" % (self.help_position, "", line)
for line in help_lines[1:]])
elif opts[-1] != "\n":
result.append("\n")
return "".join(result)

 

[Steven D'Aprano]

def format_option(self, option):
# The help for each option consists of two parts:
# * the opt strings and metavars

[snip]

Tim, I notice you're using lots of # lines as comments to describe the
function. Perhaps you should consider using docstrings instead.

Pardon me if I'm telling you what you already know...

A docstring is a string that immediately follows a class, function or
method declaration, or at the beginning of a module:


def parrot():
"This is a doc string."
s = "this is not a docstring"


The advantage of docstrings is that unlike # comments, they aren't
discarded at compile time, and can be accessed by the caller:
'This is a doc string'

This is especially useful in the interactive interpreter, where
help(object) will grab the docstring and format it nicely on screen.

 

[Tim Chase]

def format_option(self, option):

# The help for each option consists of two parts:
# * the opt strings and metavars

[snip]

Tim, I notice you're using lots of # lines as comments to describe the
function. Perhaps you should consider using docstrings instead.

Pardon me if I'm telling you what you already know...

Yes, I do know about and use docstrings, and no, no offense
taken. You'll have to persuade the [Debian/Python] maintainer of
optparse.py from whom I pilfered the code.

tim@rubbish:~$ sed -n '/def format_option(/{N;N;N;N;N;p}'
/usr/lib/python2.4/optparse.py

def format_option(self, option):
# The help for each option consists of two parts:
# * the opt strings and metavars
# eg. ("-x", or "-fFILENAME, --file=FILENAME")
# * the user-supplied help string
# eg. ("turn on expert mode", "read data from FILENAME")


-tim