Documentation

After taking a long break over the winter I’ve started working on neutrino again. When I left off I was working on generating static data in the binary object files: basically, taking neutrino objects and constructing binary data in the generated executables. I know how my code generator should behave for this to work but changing it to do that has been tricky because it is a large amount of relatively complex code (generating binaries is a relatively complex process) and I’ve forgotten how it works exactly. The problem is that I didn’t do a good job of documenting it. So my step one is going to be documenting it. Or, it turns out, that actually will be step two because at this point neutrino doesn’t have a documentation format – so step one will be implementing one.

JavaDoc
I’ve always felt that java’s JavaDoc comments were a good idea in principle but problematic in a lot of the particulars. The good part is that, unlike systems like doxygen and JsDoc, it’s a standard that comes with the language, and it provides all the primitives I want then documenting code. However…

JavaDoc builds on top of block comments. Block comments are for free-form, anything goes text. This means that it’s tricky if the language wants to impose rules on them, even if those rules might be useful. For instance, it’s legal to @link to a class or method that doesn’t exist.

Also, since they it is based on comments the connection between JavaDoc and the code it documents is weak. This, for instance, is meaningless but perfectly legal:

String x = /** Looky here! */ "foo";

Finally, JavaDoc builds directly on HTML which is fine if what you want to generate is HTML but problematic if you want to generate anything else. It is also difficult to read outside of a browser and I almost always read the documentation in the code, not in the generated output.

A small detail that’s always bugged me about them, also, is that they waste two lines of space:

/**
 * Two lines of comment takes up three lines of code
 * because of the start and end markers.
 */

This is a two-line comment, really, and it shouldn’t take up more than two lines in the program.

For all these reasons I’m not keen to base neutrino’s documentation format on JavaDoc and I’m not familiar with any other formats that solve all of these issue. So I’ve ended up starting basically from scratch, with some inspiration from a language I previously worked on, neptune.

Neutrino
I’ll start with the two last points first, the syntax. Here’s one example of what a piece of documentation looks like:

/| Moves the specified number of disks from the @from@
 | peg to the @to@ peg.
 |
 | param(from): Which peg to move from.
 | param(to): Which peg to move to.
\| param(disks): How many disks to move.
def this.move(from:, to:, disks:) {
  ...
}

I’d like the syntax to be such that you don’t need the start and end marker to require a separate line each. In other words, you need to be able to write the documentation on the same lines as those markers. So the way this works is: /| starts a comment and \| marks that that lines is the end of the block, but the whole line is included in the comment. These last markers can also occur alone, without a start marker, in which case it’s a single-line doc comment:

\| Creates a new hanoi instance.
@static def Hanoi.new() => ...

The first character on each line (above that would be the | character) are used to determine how to interpret each line. A | means that it is just text. A @ means that the text is code. For instance:

/| Moves the specified number of disks from the @from@
 | peg to the @to@ peg. For instance, to execute the
 | towers of hanoi with 15 disks do:
 |
 @ def hanoi := new Hanoi();
 @ hanoi.move(from: 0, to: 2, disks: 15);
 |
 | param(from): Which peg to move from.
 | param(to): Which peg to move to.
\| param(disks): How many disks to move.
def this.move(from:, to:, disks:) {
  ...
}

This way it’s as easy to identify and read code examples within the code as it is for a doc generator to recognize.

The second thing that makes these different from JavaDoc is that they’re not comments, they’re annotations and can only occur in the same positions as annotations. Documentation always belongs to a particular element and it is clear from the grammar which element that is. The code above is, in principle, equivalent to:

@doc(...) def this.move(from:, to:, disks:) {
  ...
}

This also means that documentation can stay attached to source elements at runtime and be accessed through reflection, like any other annotations.

Finally, the format used is not HTML but something very close to textile. This is, again, to make it easier to read from source code, and to make it easier to generate documentation in other formats than HTML. I remember we had a doc generator for neptune that generated a PDF file for a module, which worked really well (and looked, if I do say so myself, awesome).

I think this format has a lot of appealing properties and I’m actually eager to get started documenting the code generator.