When documentation becomes a fairytale

It’s time for a pet peeve!

If your technical documentation would sound amazing when narrated by someone like Morgan Freeman, or Judi Dench, it’s not documentation; it’s a fairytale. But this doesn’t mean that you can’t have fun or nuance. Let’s dive into that.

While this article is specifically written with programming and systems-administration-related-tasks in mind; the concepts are likely to be widely applicable in other areas.

• Table of contents
• What is documentation?
• What am I getting at?
• Why is it a liability?
• Heavy maintenance cost
• It’s not skimmable
• When does your audience need to read the documentation?
• For on-boarding
• Day-to-day
• Self improvement
• In an emergency
• Post-mortems
• Bringing the above together
• A fictitious example
• Doing it better
• DB role
• DBPROXY role
• So what’s different?
• An irrelevant tangent
• Key take-aways
• What about this article?
• An extra perk

So I probably better define this so that we’re on the same page:

I’m talking about captured knowledge that is in a state that is suitable to be consumed when it is needed.

It should be fast to navigate to get exactly what you need at the moment that you need it. Yet able to carry the nuance that comes with real-life situations. You shouldn’t need a tool like ChatGPT to summarise it, or to find the relevant points for you.

A few years ago, I was working at a company that had excellent coverage in its documentation. Every time someone joined (including myself), they’d exclaim about how excellent the coverage was and there’d be some virtual high-fiving. Then without fail, after some time had passed, there would be murmurings along the lines of

It’s a bit wordy…

which was putting it nicely. Zooming out, I think an apt way of describing it would be a brain dump. This is an excellent first step on the way to documentation and knowledge retention. But if it is treated as the final step, it’s a liability.

In the tech world, things change rapidly. It’s not just a phrase that gets over-repeated. It’s actually true. This means that the documentation needs to be regularly updated.

The best-written paragraphs are still tangled messes of ideas that need to be re-read to make sure that they still work together. This adds a huge amount of inertia that gets in the way of updating documentation as things change.

Just like a video; finding the information you need, in a reasonable time-frame is not viable in a wall of text.

I think that I should clarify the video remark: A well edited video can be an excellent tool for quickly conveying something that is much easier to digest visually. It must be concise, edited, and time-stamped in external text so that people can quickly get to what they need. But if it’s just someone talking to the camera, the person who will get the most value from it is the person talking. - Few people will invest time in watching any of it, and even fewer will watch enough of it for it to make any sense.

Let’s compare some contexts for when people read the documentation.

They probably have a lot of other things to read. Context and nuance are helpful, but the documentation needs to be concise and well structured. The audience is unlikely to remember much of what they read at this time, if anything. But if it’s well structured, they just need to remember where they found the information and they’ll quickly get back to it in times of need.

Don’t expect anything to have been remembered.

While your audience hopefully isn’t excessively rushed; they certainly don’t have time to spare. Structure helps cash-in on the on-boarding process so they can get right to what they need, and skim around some related areas while they are at it.

Don’t expect everything to have been read.

When your audience finds themselves casually wandering the docs, some context and nuance are amazing for taking their understanding to the next level. They are unlikely to be interested in specific steps at this moment, but rather to simply take in the bigger picture, and gaze into the wonder of possibility.

This is the one time when chatty documentation might be appreciated. It’s also, by far, the most rare and least time-sensitive.

(Thinking production outage.)

F*ck context. Give me the information, right now!

Context and nuance are almost never helpful at this time. We need specific information:

• What to look for in this specific environment/service.
• What to do about it.

Only when the situation gets really desperate, and we have accepted that it is going to be a prolonged outage, is context and nuance helpful. Typically when this happens, each team member is deep diving into a different theory for why something could be broken. They are looking at internal documentation, external documentation for components, maybe some theory. These happen, and it’s important to cater to them, but they very are rare and should not get in the way of getting the primary information quickly.

During the phase of understanding why stuff broke, the context and nuance are incredibly important. They still need to be concise and get to the point quickly, because your audience likely has a lot on their plate, and a lot to take in in a short space of time. But a deeper understanding will go a long way here.

During remediation, your audience is likely going to have to update the documentation. Maybe the context and nuance need to be adjusted. Maybe there are some edge-cases that need to be documented in a quick-reference style. But what ever it is, bloat is the enemy of getting this done.

It’s clear that context and nuance are important. But the times in which they are useful have very little overlap with the times in which they are unhelpful.

Notice that chatty documentation is almost never helpful. And the one time when it might be appreciated, documentation that gets to the point quickly would be at least as good.

The topic of this example is based on the legacy infrastructure of a company that went bust several years ago. And the writing style is from another company. The concepts are dated (intentionally); please don’t get stuck on that.

Here’s an illustration of what I’m talking about:

DB fairytale documentation

Above: A screenshot of the style of documentation that I’m talking about.

This is covering one small portion of the Doing it better documentation (below). This small sample is a realistic representation of documentation at that company.

Good luck trying to find information in a hurry.

Let’s see if we can communication this information better. Here’s the DB documentation:

DB role documentation

Above: A screenshot of the DB example documentation.

DBPROXY role documentation

Above: A screenshot of the DBPROXY example documentation.

• There is a overview.
  • It has a standardised set of information. Not all roles will need the same fields, but most of it will be in common.
  • It’s concise and easy to skim.
• There are headings in a logical, easy to follow structure.
• There is a table of contents generated from those headings.
• Paragraphs are avoided, but used when legitimately needed.
• Repeating steps are abstracted out, and linked.
  • This gives
    • the user the ability to zoom into the information they need, and skim over what they don’t.
    • a single place to keep updated and accurate.
    • a consistent, expected place to find information.
    • an intuitive way to find information as abstraction evolves.
  • There is a balance to this. Let’s play this out to the extreme:
    • If every bullet point gets abstracted out, the user will quickly lose track of where they are at.
    • Conversely, if nothing is abstracted and simply repeated every time it is relevant
      • every instance has to be found and updated every time anything changes.
      • the document gets bloated with repeating information.
  • Use care. But on the whole, it’s beneficial to abstract out a lot.
  • The abstracted sections could be separate documents, or as part of the single document structure as I’ve done in this example.

For those who are curious: We did later solve the replication issue. It was due to user input not being sanitised by the app layer. The data contained characters that were invalid for the field data type that they were stored in in the database. It managed to get past MySQL’s checks during INSERT, but failed during replication.

At the end of the day, it comes down to what your target audience needs. You are writing for your audience, not yourself.

Technical documentation should

• Default to quick reference format.
  • Well structured.
    • Use headings.
    • Have an automatically generated table of contents.
    • Lay it out in a way that compliments how the target audience will actually use it.
  • Prefer bullet points, numbered lists, and tables over paragraphs.
• Don’t be afraid to use a paragraph when you need it.
  • You probably don’t need it.
  • You probably don’t need it.
  • If you find yourself starting to write paragraphs; use that as a trigger to test whether it’s time to take a break.
  • If you do legitimately need it; consider whether it should be separated under another heading like “Background”, “Context”, “Why?”, “Purpose” etc.
• Have context and nuance clearly separated.
  • It should be close to the relevant part of the quick reference information, but clearly not part of it.
    • I like having a separate heading like “Why?”, or “Background”.
• Be as concise as possible regardless of what format it is in.
• Not have unique information in headings. - Headings are for structure and navigation. Don’t rely on the reader reading them.
• Outsource repeating instructions to dedicated documentation.

I’d advise against separating out the different audiences for the documentation into separate documents by default. Instead structure the documentation so that each audience can find what they are looking for quickly, but can also look over the fence when curiosity/necessity dictates. It also makes it much easier to keep everything up-to-date and as complete as possible.

Having said that. It’s important to recognise when the audience is too diverse, or a topic is getting too big, and should be split off into it’s own document that is linked to from the original document.

A brain-dump is an excellent first step to creating documentation. But it is not documentation.

My opinion on this has been miss-construed in the past to be that all documentation should be short. That’s not what I’m saying at all. What I’m saying is to:

Communicate everything that needs to communicated, but keep it concise, and well structured. Don’t add the unnecessary fluff that comes out when we talk.

So is this article documentation?

No. It’s a blog post. Its purpose is for expressing, and hopefully convincing opinions. By nature, this is more chatty.

To see some of my public-facing documentation work, have a look at the handWavey documentation. It’s tiny, but a good example of what I’m talking about.

In my experience; well structured documentation tends to write itself. It creates it’s own to-do list of mini-topics to be covered, which makes it much easier to pick up and put down in an environment where we are juggling priorities. Which, less face it, is the reality of the environments in which we typically write such documentation.