Python Usability Bugs: Formatting argparse subcommands

Suppose you want to build a tool with a simple interface:

usage: sub <command>

commands:

  status -  show status
  list   -  print list

Python proposes to use argparse module. And if you follow documentation, the best you can get will be this output:

usage: sub {status,list} ...

positional arguments:
  {status,list}
    status       show status
    list         print list

And it you implement proper formatting, your code will look like this:

import argparse
import sys

class CustomHelpFormatter(argparse.HelpFormatter):
  def _format_action(self, action):
    if type(action) == argparse._SubParsersAction:
      # inject new class variable for subcommand formatting
      subactions = action._get_subactions()
      invocations = [self._format_action_invocation(a) for a in subactions]
      self._subcommand_max_length = max(len(i) for i in invocations)

    if type(action) == argparse._SubParsersAction._ChoicesPseudoAction:
      # format subcommand help line
      subcommand = self._format_action_invocation(action) # type: str
      width = self._subcommand_max_length
      help_text = ""
      if action.help:
          help_text = self._expand_help(action)
      return "  {:{width}} -  {}\n".format(subcommand, help_text, width=width)

    elif type(action) == argparse._SubParsersAction:
      # process subcommand help section
      msg = '\n'
      for subaction in action._get_subactions():
          msg += self._format_action(subaction)
      return msg
    else:
      return super(CustomHelpFormatter, self)._format_action(action)


def check():
  print("status")
  return 0

parser = argparse.ArgumentParser(usage="sub <command>", add_help=False,
             formatter_class=CustomHelpFormatter)

subparser = parser.add_subparsers(dest="cmd")
subparser.add_parser('status', help='show status')
subparser.add_parser('list', help='print list')

# custom help messge
parser._positionals.title = "commands"

# hack to show help when no arguments supplied
if len(sys.argv) == 1:
  parser.print_help()
  sys.exit(0)

args = parser.parse_args()

if args.cmd == 'list':
  print('list')
elif args.cmd == 'status':
  sys.exit(check())

Here you may see the failure of OOP (object oriented programming). The proper answer to this formatting problem is just to define a data structure for command line help in JSON or similar format and let people dump and process it with templates. Once option definition is parsed, the information there is static, so there is no need in those intertwined recursive method calls. So just do it in 2 pass - get dataset and render template.

Using getopt with optparse (or how to move from getopt gradually)

TL;DR: https://bitbucket.org/techtonik/scons/commits/bcb60b

SCons has a very old and interesting codebase with a lots of outdated and unusual stuff that makes it more difficult to extend. One such thing is getopt library, which is a predessor for Optik library (written by Greg Ward) now better known as optparse.

So I wanted to replace getopt with optparse, but didn't want to change everything in one step, because I didn't have time to check every option. Instead I decided to parse options I needed with optparse and leave everything else to the old getopt engine.

getopt only needs a list of arguments to work. sys.argv[1:] to be exact. This is also the second half of result returned by OptionParser.parse_args() function. The only problem was to teach OptionParser to ignore unknown options and leave them in arguments. Strange thing, but Optik examples included this user story, completely ignored in optparse documentation. To make this long user story short, you need to subclass OptionParser to use getopt with optparse:

# "Pass-through" option parsing -- an OptionParser that ignores
# unknown options and lets them pile up in the leftover argument
# list.  Useful to gradually port getopt to optparse.

from optparse import OptionParser, BadOptionError

class PassThroughOptionParser(OptionParser):
    def _process_long_opt(self, rargs, values):
        try:
            OptionParser._process_long_opt(self, rargs, values)
        except BadOptionError, err:
            self.largs.append(err.opt_str)
    def _process_short_opts(self, rargs, values):
        try:
            OptionParser._process_short_opts(self, rargs, values)
        except BadOptionError, err:
            self.largs.append(err.opt_str)

parser = PassThroughOptionParser(add_help_option=False)
parser.add_option('-a', '--all', action='store_true',
                      help="Run all tests.")
(options, args) = parser.parse_args()

#print "options:", options
#print "args:", args

Now pass args down to the getopt call and you're all set.

P.S. In argparse you can use ArgumentParser.parse_known_args() function.

Update 2013-02: For humane option parsing you should definitely see docopt library.