I talked about this a bit on Discord, but I think it's time to actually start talking about a new library I've been working on. So here goes: introducing the very creatively named uutils-args, a derive-based argument parsing library.

Why a custom argument parser

For a while now, we've been using clap as our argument parser, but over time I've been become increasingly frustrated with it. Not because it's a bad library, but because it doesn't fit the very specific needs of the coreutils and we constantly have to work around clap with a lot of verbose code. I looked into basically every Rust argument parser out there and none really fit our needs. So about a month ago, I started working on a parser based on lexopt specifically for uutils.

The whole design is too complex to explain here, so I wrote some documents in the repo. It has three subpages:

• an overview of all argument types in coreutils
• a page listing problems we ran into with clap
• and how uutils-args works.

The library also integrates very nicely with some of the plans I had earlier for the --help pages (#3181, #2816), because the help text is read as markdown from a file in this library.

Current Status

The library supports many utils already, but a few important features are still missing. Some low-hanging fruit:

• The deprecated arguments for head, tail and uniq,
• Hidden arguments,
• Setting exit codes.

And more complicated:

• better and more complete markdown rendering,
• argument completion.

There are also rough edges that can be smoothed out like improving the error messages and the output of --help.

It is also very much a product of me working on it alone, which means that most of the naming conventions make sense to me but maybe not to others. I would very much welcome feedback on any part of the API.

If you want to see the library in action, I suggest to take a look at some of the coreutils for which I have created tests (more to come in the future):

• basename
• cat
• mktemp
• ls

A full list is here: https://github.com/tertsdiepraam/uutils-args/tree/main/tests/coreutils. I found that most are relatively easy to port.

Downsides

Porting to this new library will have a few downsides.

1. New contributors won't be familiar with this library.
2. We won't have argument completion (until we implement ourselves in this library)
3. We'll have to update uudoc.
4. We don't get all the features and years of iterative improvement that clap has.

Proposed roadmap

Of course, I'm aware that this is gonna be a huge effort to port and we should carefully how (and even if) we want to do that. For now, I've come up with the following steps:

• Figure out whether the migration is going to be worth it.
• Collect feedback and accept PRs to get the library ready for use in uutils. This includes
  • Reevaluating the names for crates, traits and functions.
  • Implementing missing features.
  • Adding more documentation.
  • Stabilize the API.
  • Add more coreutils APIs as tests to prove the feasibility of the library.
• Fix up administration
  • Move the repo to the uutils organisation
  • Publish the crates in the repo
• Port utils one by one¹
• Change uudoc to use the info from the new library.

How to help out

For now, I need lots of feedback on this library, so I can get it ready, feel free to open an issue with whatever feedback you have or comment below. You can also help by picking up any of the open issues.

I'd be happy to answer any questions about this library :)

cc @sylvestre, @rivy, @jfinkels