April 21, 2004
Command line parsing

I mentioned Michele Simionato's lovely command line parser the other day. The nice thing about this is that it takes the command line option definitions from a human readable definition in the module's docstring.

David Boddie commented on this recipe, pointing out his lovely command line parser. The nice thing about this one is that it can throw up an automatically generated GUI form allowing the options to be specified in addition to accepting the options from the command line. (There's nothing new in the world - this is similar in concept to the way the iSeries' commands work.) But the syntax that David's parser uses isn't too user friendly.

So, what I'd really like would be a combination of the two approaches; David's GUI generating parser, but driven from a human readable docstring. Well, I have the code to both, so I'll have a bash at a cut-n-shunt job.

Posted to Python by Simon Brunning at April 21, 2004 01:18 PM
Comments

Wow, I was just thinking yesterday that I needed something like David's. Please ping me if you throw this together.

Posted by: Bob Lee on April 21, 2004 03:35 PM

I'm an idiot. Do you know of a Java library like David's?

Posted by: Bob Lee on April 21, 2004 03:39 PM

Nope. I doubt it's possible - Java's javadocs aren't around at runtime, are they?

Posted by: Simon Brunning on April 21, 2004 04:30 PM

If you do it in Python, I think minor abuse of the class syntax would work quite nicely for defining an interface.

Posted by: Ian Bicking on April 21, 2004 05:12 PM

"Java's javadocs aren't around at runtime"

I don't think your compiler is going to explode if you put the usage text in any other string...

Posted by: Fredrik on April 21, 2004 05:19 PM

Ironically, I just posted about using Mono.GetOptions over on my blog, I really like this way of doing it.

http://www.talios.com/read/579685.htm

Posted by: Mark Derricutt on April 22, 2004 04:39 AM

Oh, I don't know, Fredrik - the Java compiler seems to object to all sorts of reasonable things. ;-)

Seriously, of *course* you could keep the usage information in another string, use it to define the available options, and print it out on request. But, frankly, I never call Java from the command line anyway, so it's only the Python version that I'm interested in.

No work on it yet, though - I was so tired that I went to bed at 9:30 last night!

Posted by: Simon Brunning on April 22, 2004 08:51 AM

The main reason for writing the library was to avoid the tedious post-parsing phase in which you end up checking for things like the presence of options, missing arguments and contradictory options. There are also issues to deal with related to the various combinations of options and arguments a program might allow. The GUI generation and user input correction code just followed fairly naturally from the initial design of the library.

Before I wrote the library, I had to write loads of program logic to sanity check the specified options before even getting to the point where the actual input could be verified. All of that can be done by the library, so that you can write (in a trivial case) a usage/syntax string like

"--verbose|--quiet"

and rely on idiotic combinations of options such as

--verbose --quiet

to be rejected outright, leaving you to get on with connecting various options with the relevant functionality.

A more complex example is treated at

http://www.boddie.org.uk/david/Projects/Python/CMDSyntax/Bake-off/

and a reference guide can be found in the following location:

http://www.boddie.org.uk/david/Projects/Python/CMDSyntax/Reference/reference.html

Regarding the syntax: is it the API or the usage string syntax that people find uncomfortable? The usage string syntax was chosen because it's fairly close to the form seen in common command line tools, making it fairly easy to wrap existing tools in a GUI, or convert them to use the library.

Posted by: David Boddie on April 22, 2004 11:57 AM

There' nothing *wrong* with your syntax, David, nothing at all. It's very powerful, supporting things like synonyms and mutually exclusive options. But it's not quite what a user wants to see if they ask for help at the command line.

Michele's syntax, OTOH, is *exactly* what you'd want to see on the command line - but it's pretty simple.

Two possibilities occur to me. One is to extend Michele's syntax to make it as powerful as yours, and to plug a parser for this into your library, along with the option to pluck the command line options definition from the docstring. But I'm not at all sure that Michele's syntax can be extended in this way.

The other possibility would be to generate the user-friendly usage stuff from your syntax, and to append that to the docstring. This wouldn't be hard at all. Probably a better approach.

Posted by: Simon Brunning on April 22, 2004 12:16 PM

The problem is a fundamental usability problem with command line tools: if you have too many options then you probably need a better way of expressing them. I'd agree that using an "[options]" placeholder is better when you have a load of truly optional things to specify, but it's also an abdication of responsibility to say

myprogram [options]

then include pages of options in which the description doesn't really do justice to the complexities of the "real" syntax accepted by "myprogram". Ideally, there'd be a compromise between the two ways, with automatically-generated examples for clarity.

Anyway, to get to your ideal solution, I'd recommend looking at the test.py file in the CMDSyntax distribution, and make it output a more friendly description. I do actually have a function to do this sort of thing, but it isn't finished. Mail me if you want it.

Posted by: David Boddie on April 22, 2004 01:12 PM

I've been fiddling with my variation on this for a decade. The option definitions are language independent, but I don't have a Python template for emitting a Python parser. I *do* have templates for C, C++, Scheme, man pages and even a .texi output. Python would be simple enough. A GUI input would be really cool! I mostly work with drivers, so I don't do much GUI stuff...

See: http://autogen.sf.net/autoopts.html

Posted by: Bruce Korb on May 23, 2004 06:37 PM

I'm a complete *tyro* at the command line parsing, just trying to get to the point where i can parse one line off the keyboard...i'm working in c++...and know i need to do a getline and parse from there...but haven't figured it out yet. any help would be appreciated...i'm sorry i'm such a novice, and can't really contribute anything to the board...

Posted by: Jill Jones on October 23, 2004 03:22 AM

A couple of people were asking about option parsing for Java and C#. I have a library that I've developed over the last few years which supports "traditional", K&R style as well as a more "hands-off" approach including specifying required and conflicting options, auto help and various types of options, GNU-style, POSIX-style, short options, etc.

More information as well as some articles on use can be found at http://te-code.sourceforge.net/. Hope this helps.

Posted by: Andrew S. Townley on December 3, 2004 08:52 AM
Post a comment
Name:


Email Address:


URL:



Comments:


Remember info?