Markdown versus CommonMark: the ambiguity and clarity dilemma

The “standard Markdown” controversy seems to be over and done. I was essentially offline for the weekend, and I came back to find that what was “Standard Markdown” has now become “CommonMark“, respecting Markdown co-creator creator John Gruber’s wishes and licence for the system.

The Standard / Common Markdown project, in accordance with the wishes of @gruber, has been renamed to http://t.co/hdDpdUK5Ug— Jeff Atwood (@codinghorror) September 5, 2014

This is a good thing. Markdown remains Markdown, and this philosophically different fork of it becomes its own thing. It may well come to supplant the original – who remembers now that WordPress was a fork of a different blog platform? – but that’s not now a fait accompli by usurpation of the name.

What’s interesting to me about the whole fracas is that, once you strip away the personal elements, this was a battle of two different visions of how ideas can be useful and spread online.

A world of ambiguity

The original vision of Markdown, as I discussed in my previous post, was something that was easy for the user to pick up and run with, even if that made it hard for developers.

If you wanted to make something more complex for specific use cases, well, that’s why it was open sourced. If your fork stayed true to the original visions while fulfilling the needs of a particular group, you got to call it Markdown still. This is a “let one hundred flowers bloom” approach to tech. Build the minimum viable tool for what you need, and then open source it to let people build their own version.

This made Markdown great for writers but a pain for developers. But the first part was by design, the second a consequence of that.

Shindo Isshin expresses this beautifully:

Gruber expressly didn’t care much about the HTML output. If you want to have control over the resulting HTML, then write in HTML — don’t use a deliberately simplified and limited markup language like Markdown. Markdown was not created for the technocrati, it was created for people who don’t wish, for one reason or another, to directly use HTML for writing.

The net result was, at the very least, dozens of Markdown variants, all suited to different sorts of tasks and writing environments. These seems to have been a very satisfactory outcome for Gruber – as he hasn’t seen the need to fiddle with his initial specification for his original version of Markdown in a decade.

However, that leaves people building “Markdown” tools with problems – which Markdown(s) do they support? And that leads, predictably enough, to the idea that this should all be standardised.

A world of clarity

The camp that led the creation of CommonMark throughly disliked Gruber’s approach, much as they liked his initial work. They do care about HTML output – very much, as it turned out. And to have the consistency and control they feel is needed, Markdown, they argue, needs to be standardised, made more rigorous and easier for developers to implement.

It’s notable that the core people who are part of the project are communities which either entirely or substantially consist of technically-minded folks with a specific set of needs. To write what they write in Markdown without falling back to HTML (which you can do seamlessly in the original version), they needed a more complex version. And that’s what they’ve created – a standardised but more complex version, which suits them very well.

However, the dispute comes because they don’t want it to be a Markdown, they want it to be the Markdown. There is an intellectual appeal to this. It’s neat, tidy and much simpler from certain points of view. You know where you’re standing and what you’re dealing with, particularly if you’re creating sites or software that consume Markdown.

It’s certainly not a coder-specific mentality – look at the number of people who seem to be searching for a single, definable “future of journalism” for example – but it’s not a mindset that is as necessary on the internet as you might think.

For example, think about the idea of site structure on the web. There is one part of the structure of the web that’s designed to be clear and immutable – the URL. That’s a fixed address were a resource is meant to live, and which can be linked to. The rest of the “site’s structure” is actually rendered irrelevant by the ability of the link to put any two pages next to each other. You can create a site structure through navigation and links – but you can also create multiple, contradictory and differentiated site structures from the same set of page, based on created links.

Unlike the physical world, where one object can only be place in one place at a time, the internet allows more complex storage and navigation structures. (See David Weinberger’s truly excellent Everything Is Miscellaneous: The Power of the New Digital Disorder if you want to explore these ideas further.)

So, that “standard, one true version” is a possibility, and an attractive one to some, but not a necessity. The internet is surprisingly good at handling ambiguity.

The price of singular vision

If all of this comes at the cost of user simplicity, well, that was hardly the point was it? Jeff Atwood’s tweets since the final name change make that pretty clear:

@wuerl TL;DR gruber really honestly believes building on sand (ambiguity built in) is a feature. Earthquakes disagree!

— Jeff Atwood (@codinghorror) September 7, 2014

There’s no attempt to hide disdain here – “building on sand” is a pejorative phrase. But it also makes it clear that metaphors are dangerous things to deploy. What’s the Earthquake here? What’s going to come along and destroy all these Markdown variants? Markdown has essentially become dozens of little structures built in different places, if you want to follow the metaphor. CommonMark is an attempt to build one, stronger building. If an “earthquake” came, the diaspora of buildings is more likely to survive than a single system. Ideas are harder to kill then implementations.

At its core, the reason this span into a bigger debate than previous Markdown forks is that the “standardise and define” camp didn’t really want to acknowledge that the other way was a viable alternative that could co-exist with their approach. The original licensing explicitly allowed people to go off and do different things with Markdown – with the proviso that if they were too different, you couldn’t use the name. That’s pretty hard to square with the idea that your agreed way should be the way. But the mindset that seeks standardisation is also the mindset that is most likely to believe that there is one, best way of doing things.

(An aside: I wonder if anyone has ever argued that screwdrivers should be standardised – all these flathead, Philips and Torx screwdrivers are messy, right? Perhaps we could standardise to a single design to cover all needs? And don’t get me started on Allen keys – what a hack they are…)

The “sorry not sorry” apology and renaming to “Common Markdown” was either a shameless attempt to grab the moral high ground without actually giving up anything substantive, or (I hope) just a complete failure to understand what the core problem was in the first place.

Perhaps that’s understandable. If a standardised, single, common version of something is an unambiguous good in your eyes, then it can be very hard to understand why someone appears to be resisting that. They can’t be genuinely resisting this common good – so they must have ulterior motives. And that’s an argument that you can see played out again and again on Twitter, essentially likening Gruber to a child lashing out because his toys are being taken away from him.

The Open Source problem

What strikes me – as an unashamedly lay viewer – is that this seems to go very much against the ethos of Open Source. If you want control of something, keep it closed source and proprietary. Otherwise, people will and can fork what you do. And then other people will create different versions of it. Open Source, at its very best, is about inclusivity and possibilities. Many people can contribute to a single project, and that single project can become many different visions. Diversity of tools and craftspeople is a good thing, even if there’s ambiguity in there.

The behaviour of the CommonMark camp runs the risk of becoming a “pick and choose” approach to Open Source. “We’ll have the ability to fork our own version, thanks very much, but we’re not so keen on those licensing restrictions, and all those messy different forks, thanks.”

Now, it could still be that their vision is so compelling and so useful that it will win in the marketplace of ideas. But this entrance to the marketplace and attempt to hijack the effective ownership of Markdown was not a good sign that they want to compete on quality alone.

For the time being, I’ll be looking for Markdown-supporting tools, not CommonMark supporting ones, until someone can provide me with a compelling reason why this benefits writers, not just developers.

• John Gruber pointed out that he was the sole creator – my apologies to him for not checking my facts around Aaron Swartz:

@adders A niggle: I am Markdown’s sole creator, not co-creator. I assume, you’re thinking of Aaron Swartz, whose feedback was instrumental…

— John Gruber (@gruber) September 8, 2014

The post has been updated.