march 2018

What is Left to do After your Open Source Project is Done

So I have this little library, really simple, portable, in a single source file for super easy deployment. A little web page to document the thing, “just copy those 2 files” installation instructions, then take over the world.

This was never going to work.

To have an impact, you need to address tons of side stuff: test suite, build system, manual, distribution channels, even a touch of marketing. You won’t go far if you don’t.

Testing

This one is kinda obvious: whatever the project, you need to ensure it does the job. This often means writing an automated test suite. Be careful not to underestimate how thorough that test suite must be. I did, and that didn’t go so well.

Building and installing

The whole point of making single file libraries is to avoid messing with build systems. Users would just copy & paste the source file in their project, and their own build system would do the rest. This is a pretty popular approach.

Yet having a build & installation procedure is still important:

• Not everyone uses C or C++. Some languages don’t even require C code for their FFI. If they already have an object file, they can just use it directly.

• Some projects want to update their dependencies without recompiling themselves. When yours is such a dependency, compiling it as a .so or .dll file enables easier system wide updates. Most useful for security patches.

• Installing a library system wide can be more convenient than integrating the code, especially more so if you provide pkgconfig support.

• Package managers are even more convenient. But then your project must be integrated to the most popular ones. Using a standard build system eases this burden, which is important whether you shoulder it yourself or not.

Thus, Monocypher grew a nice standard GNU Makefile with all the right variables (CC, CFLAGS, DESTDIR, and PREFIX). This helps package managers override them as they see fit. I could support other build systems as well, but chose not to. Instead, if someone comes up with an alternate way to build Monocypher, I link to their work.

Manual

My library is simple. The API is simple and pretty obvious. So I figured the manual could also be small and simple. A web page with an informal explanation and a few examples ought to do it, right?

Well…

The public interface exposes 47 functions. Most users won’t use more than 18 of those, but still: a single web page doesn’t cut it. The worth of a library is limited by people’s ability to use it. You need to provide a fast path to victory, or your work will sit unused.

Worse code, better documented, will be chosen over your own.

So a contributor suggested I switch to man pages, then did all the work. Yet another proofread everything. This was quite the eye opener.

Standard man pages have a number of advantages. Once installed on a system, they provide easily accessible bits of documentation. But they also impose some structure, and that ended up highlighting many shortcomings of my original manual: significant parts of the API were scarcely described, many examples were missing, and the whole thing lacked consistency.

Addressing those shortcomings required massive amounts of work, comparable to the library itself. I kid you not, the documentation is now bigger than the source code. Each primitive and each construction has its own man page, complete with a listing of all functions, an informal description, examples, standards involved, and security considerations.

Surprisingly, very little of it was redundant. Users need this, I just didn’t realise.

Then there’s the writing style. Manuals can be Serious Stuff™. They’re not always meant for your average lone hacker, they may be used in settings where one has to look professional. Formal language, impersonal, no joke. Just the information the user needs.

I thought this would suck the life out of my prose, but sticking to this writing style didn’t hurt readability. On the contrary, the additional constraints forced us to think more deeply, and we ended up with something better.

By the way, external editors are a huge asset. If someone wants to review your manual, let them. If they suggest modifications, try to accept them. For Monocypher, I disagreed with a number of their choices, but caved in wherever I felt it didn’t truly matter. Turned out they were right more often than I was.

Distribution

You could just give a URL to your git repository and call it a day. While perfectly acceptable for casual projects and prototypes, this is not ideal.

• The latest commit may not have been as thoroughly tested and reviewed than the latest tag. This could be addressed by having a separate dev branch, but that could confuse prospective contributors

• There might be dependencies one does not want to pass down to users. Monocypher’s test vectors for instance are generated by a fairly recent version of Libsodium. I don’t want to version generated files, but requiring my users to install a competing library just so they can test my own would be a bit ridiculous.

• Your git repository is most likely hosted on a centralised platform such as GitHub or Bitbucket. You probably don’t want to rely on them too much.

You need ready to use releases, a copy of the manual on the web, and an official looking landing page. This generally means building a dedicated web site, with its own domain name. It looks more official than a personal blog, URLs are shorter, you can structure it around your project, and passing the torch to another maintainer is easier.

Then you need a number of bells and whistles:

• TLS (https) support. Especially since Let’s Encrypt, not having it tends to reflect poorly on your project. (This is especially true of crypto libraries such as Monocypher.)

• A nice landing page with a catchy (and accurate!) sales pitch, and pointers to get started. Documentation, download page, FAQ… can be pushed to other pages.

• A way to easily navigate to the main page. A navigation bar at the top works well.

• An RSS/Atom feed, so users can subscribe and keep up with little effort. This is especially important for pushing security updates.

• A “latest news” section in the landing page, so users can see right away if there’s recent activity. If the project becomes mature and doesn’t need further activity, say so.

To look serious, you need to administrate a web site or find someone that will, even if your project has nothing to do with web dev. I find this a bit depressing to be honest.

Marketing

If you build it, they will come.

Yeah, right.

How is your project is ever gonna be used if nobody knows about it? Without awareness, there is no impact.

Even if you don’t care about impact, outreach can get you help. Monocypher would be much worse if it weren’t for the feedback I got. It taught me the hard way how to properly test my code (people found bugs). It showed me validation tools I didn’t know of (ASan, MSan, UBSan), and those tools caught other bugs. It taught me what a real manual looks like. I would never have gotten this if I hadn’t bragged about Monocypher, wrote about it on forums, sought feedback, told a story.

That last one is more important than I initially anticipated. Virtually nobody on Reddit or Hacker News gives a damn about latest changes, or proofs of correctness. The first is only relevant to those who are already following your project, and the second is just a pain to read. (Few projects need proofs of correctness, but many have stuff that is a chore to review.)

Telling the world how I implemented my own crypto however, that is juicy click bait. If you manage to follow through with a good article, you win. Even if you don’t do click baits, do remember that a bad title will effectively hide your article. That’s exactly what happened to this one: it was first titled “Ancillary Does Not Mean Optional”, and nobody read it.

Few click past such an abstruse title, for good reason: they need to assess how interesting the link is likely to be before they click it, or they would waste their time. If at all possible, titles should state the topic, so readers know what they’re clicking up for.

Another way to promote your work is to write something worthy in and of itself, for which your project is relevant. This is how I learned of Left Hand Path, a VR game where you cast spells. The article would have interested me even if there was no game behind it, but it still made me consider purchasing a VR headset just so I can try it.

This is exactly what I’m trying to do here: advertise Monocypher with an article that should be interesting in its own right. While I expect most of you will walk away with some idea of how much ancillary support open source projects require, let’s be honest: my real hope is that some of you notice my crypto library.

I do mean to take over the world, after all.