Zsh Completion Syntax

zsh, an alternative to bash, is known for its superior completion system. This provides the user with a convenient way to input arguments and options to a command. It really is a time saver and more tools should come with a completion script. For bioinformatics I have been assembling completion script in the biozsh project. More general tools can go into the zsh users completions repository. With this blog post I would like to document the process I use to write a completion script for a simple tool. If you find errors or bad style, feel free to drop me a line.

For demonstration purposes wee need a simple tool. In this case I use my own andi. It has a simple user interface, but covers most basic components needed for a successful completion.

Getting Started #

Create a file called _andi with the following contents.

#compdef andi

_andi() {
}

_andi

This script, when executed calls a function _andi which in turn, does nothing. The first line in the file tells zsh that this script completes options for the command andi. Put the file in any directory, execute fpath+=($PWD) and start a new subshell. If you now type andi as a command and hit TAB, nothing should happen. The shell will not even give you a list of files. This way you know that the completion is working (_andi() is a noop, at the moment).

Boilerplate #

To make the completion script behave nicely, we have to add some boilerplate code. For one, _andi should return 0 iff successful.

#compdef andi

_andi() {
    integer ret=1
    local -a args
    args+=(
        # …
    )
    _arguments $args[@] && ret=0
    return ret
}

_andi

We also define an empty array args and populate it with nothing. This array is passed to a built-in function called _arguments. If that function does not error, ret becomes 0 (success) and that value is returned.

Populating the Arguments #

Here is the help message for andi as of version 0.11.

Usage: andi [-jlv] [-b INT] [-p FLOAT] [-m MODEL] [-t INT] FILES...
    FILES... can be any sequence of FASTA files. If no files are supplied, stdin is used instead.
    Options:
  -b, --bootstrap <INT>  Print additional bootstrap matrices
      --file-of-filenames <FILE>  Read additional filenames from FILE; one per line
  -j, --join           Treat all sequences from one file as a single genome
  -l, --low-memory     Use less memory at the cost of speed
  -m, --model <Raw|JC|Kimura>  Pick an evolutionary model; default: JC
  -p <FLOAT>           Significance of an anchor; default: 0.025
  -t, --threads <INT>  Set the number of threads; by default, all available processors are used
      --truncate-names Truncate names to ten characters
  -v, --verbose        Prints additional information
  -h, --help           Display this help and exit
      --version        Output version information and acknowledgments

You can see that the options are ordered alphabetically. It is good style to keep this order in the completion script. For sake of demonstration we will start with the easiest case: The option --truncate-names is a simple on/off switch. To make zsh aware of this option, we have to add the following code:

args+=(
    '--truncate-names[Truncate names to ten characters]'
    )

First comes the option name, then in square brackets the description. Everything is in single quotes, because it does not contain any shell variables we want expanded.

Let’s go for a slightly more difficult case, -p.

args+=(
    '-p[Significance of an anchor; default: 0.025]:float:'
    '--truncate-names[Truncate names to ten characters]'
    )

The basic syntax is the same. Now -p takes an argument. So after the option description we put :argument description:. In this case the argument is a float value and we leave it at that. Also, providing the default value is a matter of taste.

More Options To make andi print additional bootstrapped matrices, one can supply an integer with either -b or --bootstrap. Both options have the same functionality. Here is one possible way of writing it.

args+=(
    '-b[Print additional bootstrap matrices]:int:'
    '--bootstrap[Print additional bootstrap matrices]:int:'
    '-p[Significance of an anchor; default: 0.025]:float:'
    '--truncate-names[Truncate names to ten characters]'
    )

This has two problems. One, it is repetitive, and two, andi can only take one bootstrapping argument. Thus, after -b has been provided by the user it is not sensible to advertise --bootstrap anymore. This gets us to the matter of exclusion lists: options given in parenthesis before an option, will no longer be completed, if that option is already in the buffer.

args+=(
    '(--bootstrap)-b[Print additional bootstrap matrices]:int:'
    '(-b)--bootstrap[Print additional bootstrap matrices]:int:'
    '-p[Significance of an anchor; default: 0.025]:float:'
    '--truncate-names[Truncate names to ten characters]'
    )

Now things are looking even more repetitive. Luckily, we can use the fact, that those are just strings. With brace expansion it is easy to group common things.

args+=(
    '(-b --bootstrap)'{-b,--bootstrap}'[Print additional bootstrap matrices]:int:'
    '-p[Significance of an anchor; default: 0.025]:float:'
    '--truncate-names[Truncate names to ten characters]'
    )

Note the difference in syntax: exclusion lists use a space as separator, whereas brace expansion uses commas.

By default, it is assumed, that most options may appear only once on the command line. However, andi supports multiple levels of verbosity, indicated by the flag given up to two times. To make zsh not remove -v from the suggestions, if it has already been given, simply add a star in the right place.

'(-v --verbose)*'{-v,--verbose}'[Prints additional information]'

Files When an option, such as --file-of-filenames takes a path as an argument, zsh should provide the user with a list of applicable files.

'--file-of-filenames[Read additional filenames from FILE; one per line]:file:_files'

In this case _files is a special function that only completes files, and directories. It can also be restricted to only completing files with a certain extension, or only directories. See zsh docs for details.

andi also takes filenames as regular arguments on the command line. To support this, we add as the last thing in the args array:

':file:_files'

Values #

Multiple models of sequence evolution are supported by andi. We can either let the user input the name of a model (bad), or supply them with a list of all available options (good).

'(-m --model)'{-m,--model}'[Pick an evolutionary model]:model:(Raw JC Kimura)'

We can also be even nicer and supply the user with a description for each option. Note the excessive need for escaping.

'(-m --model)'{-m,--model}'[Pick an evolutionary model]:model:((
    Raw\\:Uncorrected\\ distances
    JC\\:Jukes\\-Cantor\\ corrected
    Kimura\\:Kimura\\-two\\-parameter
    ))'

DRY #

Most reasonable programs support one or more of the options -h, --help, -V, --version, --usage, etc., with andi being no exception. However, given one of these flags, the program takes a different path, prints a message and then immediately exits. Thus if either of these options is on the command line, it does not make sense to complete other options anymore (-) nor any files (*).

'(- *)'{-h,--help}'[Display help and exit]'
'(- *)--version[Output version information and acknowledgments]'

But also, if any command is given, neither of these three options should be completed. One could add -h --help --version to the exclusion list of all commands so far, but that would be tedious. Instead we make use of variable expansion in the shell.

local I="-h --help --version"
local ret=1
local -a args

args+=(
    "($I -b --bootstrap)"{-b+,--bootstrap=}'[Print additional bootstrap matrices]:int:'
    "($I)*--file-of-filenames=[Read additional filenames from file; one per line]:file:_files"
    "($I -j --join)"{-j,--join}'[Treat all sequences from one file as a single genome]'
    "($I -l --low-memory)"{-l,--low-memory}'[Use less memory at the cost of speed]'
    "($I -m --model)"{-m+,--model=}'[Pick an evolutionary model]:model:((
        Raw\\:Uncorrected\\ distances
        JC\\:Jukes\\-Cantor\\ corrected
        Kimura\\:Kimura\\-two\\-parameter
    ))'
    "($I)-p+[Significance of an anchor; default\\: 0.025]:float:"
    "($I -t --threads)"{-t+,--threads=}'[The number of threads to be used; by default, all available processors are used]:num_threads:'
    "($I)--truncate-names[Print only the first ten characters of each name]"
    "($I)*"{-v,--verbose}'[Prints additional information]'
    '(- *)'{-h,--help}'[Display help and exit]'
    '(- *)--version[Output version information and acknowledgments]'
    '*:file:_files'
    )

Note how double quotes "" have to be used to allow expansion.

Getopt #

In the previous block of code you can also see that I added something to options taking an argument. If the option is a single letter, it is followed by a + otherwise a =. This is in accordance with the argument parser used, getopt.

andi -b 100
andi --bootstrap 100
andi -b100
andi --bootstrap=100

These four calls are equivalent. By default, zsh assumes that an option has to be whitespace separated from its argument. Given a + there can also be no separation, and an = allows for an equality sign.

Furthermore, getopt allows short option stacking and -- as a separator. To enable this, we do:

_arguments -w -s -S $args[@] && ret=0

Result #

The final result should look more or less like the following.

#compdef andi

_andi() {
    local I="-h --help --version"
    local ret=1
    local -a args

    args+=(
        "($I -b --bootstrap)"{-b+,--bootstrap=}'[Print additional bootstrap matrices]:int:'
        "($I)*--file-of-filenames=[Read additional filenames from file; one per line]:file:_files"
        "($I -j --join)"{-j,--join}'[Treat all sequences from one file as a single genome]'
        "($I -l --low-memory)"{-l,--low-memory}'[Use less memory at the cost of speed]'
        "($I -m --model)"{-m+,--model=}'[Pick an evolutionary model]:model:((
            Raw\\:Uncorrected\\ distances
            JC\\:Jukes\\-Cantor\\ corrected
            Kimura\\:Kimura\\-two\\-parameter
        ))'
        "($I)-p+[Significance of an anchor; default\\: 0.025]:float:"
        "($I -t --threads)"{-t+,--threads=}'[The number of threads to be used; by default, all available processors are used]:num_threads:'
        "($I)--truncate-names[Print only the first ten characters of each name]"
        "($I)*"{-v,--verbose}'[Prints additional information]'
        '(- *)'{-h,--help}'[Display help and exit]'
        '(- *)--version[Output version information and acknowledgments]'
        '*:file:_files'
    )

    _arguments -w -s -S $args[@] && ret=0

    return ret
}

_andi

If I ever get round writing a Part II, it will be about adding more logic to the completion. How to enable some arguments given others, or how to write subcommands a la git commit. But this post should be enough to get you started.