multiple invocations of OptionParser.parse_args() within the same python execution

Question

I have the following example setup:

|-- main_script.py
`-- module
    |-- __init__.py
    `-- submodule.py

where the contents of main_script are:

import optparse
import module

parser = optparse.OptionParser()
group = optparse.OptionGroup("submodules options")
group.add_option("","--main_script.bar", dest="bar", action="store_true")
parser.add_option_group(group)

opts,args = parser.parse_args()

if opts.bar:
    print ("Bar")

and the contents of submodule.py are:

import optparse
parser = optparse.OptionParser()
group = optparse.OptionGroup(parser, "submodules options")
group.add_option("","--module.submodule.foo", dest="foo", action="store_true")
parser.add_option_group(group)

opts,args = parser.parse_args()

if opts.foo:
    print ("Foo")

Since main_script imports submodule the parse_args from submodule is the called. Is there anyway to combine these instances of OptionParser and raise an error if there are option conflicts?

Answer 1

The easiest way is to break your logic into functions. You should not execute logic at the global module scope in the first place, but use a if name == "__main__" wrapper construct.

You could define a function add_options(parser) to each module that must define options, and call this at the root level before calling parse_args:

import optparse
import submodule

def add_options(parser):
    parser.add_option(...)

def main():
    parser = optparse.OptionParser()
    add_options(parser)
    submodule.add_options(parser)
    opts, args = parser.parse_args()
    #...

if __main__ == "__main__":
    main()

Answer 2

In an attempt to solve this problem from the outside in - I have an initial implementation that patches the optparse module by replacing the OptionParser class with a subclass of OptionParser and overloads the parse_args method + exposes a new delayed_parse_args method. I'm sharing snippets from this solution in case anyone else finds its useful or can make an improvement to it.

optparse_patch.py

import optparse

def patch_opt_parser():
    optparse.stashed_parsers = {}

    class OptionParserEx(optparse.OptionParser):
        def delayed_parse_args(self, callback):
            optparse.stashed_parsers[self] = callback

        def parse_args(self, args=None, values=None):
            for parser, callback in getattr(optparse,"stashed_parsers").items():
                # add all the option_list & option_groups from the individual
                # parsers in stashed_parsers to `self`
            for parser, callback in getattr(optparse,"stashed_parsers").items():
                # update `parser` to inherit the option_lists and option_groups from 
                # self. then ....
                _o, _a = optparse._OptionParser.parse_args( parser, args, values )
                callback( _o, _a t)
            return getattr(optparse,"_OptionParser").parse_args(self, args, values)

    optparse._OptionParser = optparse.OptionParser
    optparse.OptionParser = OptionParserEx

patch_opt_parser()

This allows submodules to 'stash' their expected options and have the command line options evaluated at a later stage when the client of the module actually provides an OptionParser and calls the parse_args method on it's own OptionParser. An example of such a use case is as follows:

module.py

import optparse_patch
import optparse
parser = optparse.OptionParser()
group = optparse.OptionGroup(parser, "module options")
group.add_option("-f", dest="flip", action="store_true")
parser.add_option_group(group)
def cli_callback ( opts, args ):
    if opts.flip: 
        print "flip"
opts, args = parser.delayed_parse_args ( cli_callback ) 

main.py

import module
import optparse


myparser = optparse.OptionParser()
mygroup = optparse.OptionGroup(myparser, "main options")
mygroup.add_option("-j", dest="jump", action="store_true")
myparser.add_option_group(mygroup)
opts, args = myparser.parse_args()
if opts.jump:
     print "jump"

calling the program main.py results in the following outputs:

python main.py --help

Usage: main.py [options]

Options:
  -h, --help  show this help message and exit

  module options:
    -f        

  main options:
    -j        

python main.py -j -f

flip
jump