Standard Markdown is now Common Markdown

Let me open with an apology to John Gruber for my previous blog post.

We’ve been working on the Standard Markdown project for about two years now. We invited John Gruber, the original creator of Markdown, to join the project via email in November 2012…

Let me open with an apology to John Gruber for my previous blog post.

We've been working on the Standard Markdown project for about two years now. We invited John Gruber, the original creator of Markdown, to join the project via email in November 2012, but never heard back. As we got closer to being ready for public feedback, we emailed John on August 19th with a link to the Standard Markdown spec, asking him for his feedback. Since John MacFarlane was the primary author of most of the work, we suggested that he be the one to reach out.

We then waited two weeks for a response.

There was no response, so we assumed that John Gruber was either OK with the project (and its name), or didn't care. So we proceeded.

There was lots of internal discussion about what to name our project. Strict Markdown? XMarkdown? Markdown Pro? Markdown Super Hyper Turbo Pro Alpha Diamond Edition?

As we were finalizing the name, we noticed on this podcast, at 1:15 …

… that John seemed OK with the name "GitHub Flavored Markdown". So I originally wrote the blog post and the homepage using that terminology – "Standard Flavored Markdown" – and even kept that as the title of the blog post to signify our intent. We were building Yet Another Flavor of Markdown, one designed to remove ambiguity by specifying a standard, while preserving as much as possible the spirit of Markdown and compatibility with existing documents.

Before we went live, I asked for feedback internally, and one of the bits of feedback I got was that it was inconsistent to say Standard Flavored Markdown on the homepage and blog when the spec says Standard Markdown throughout. So I changed them to match Standard Markdown, and that's what we launched with.

It was a bit of a surprise to get an email last night, addressed to both me and John MacFarlane, from John Gruber indicating that the name Standard Markdown was "infuriating".

I'm sorry the name is so infuriating. I assure you that we did not choose the name to make you, or anyone else, angry. We were simply trying to pick a name that correctly and accurately reflected our goal – to build an unambiguous flavor of Markdown. If the name we chose made inappropriate overtures about Standard Markdown being anything more than a highly specified flavor of Markdown, I apologize. Standard does have certain particular computer science meanings, as in IETF Standard, ECMA Standard. That was not our intent, it was more of an aspirational element of "what if, together, we could eventually..". What can I say? We're programmers. We name things literally. And naming is hard.

John Gruber was also very upset, and I think rightfully so, that the word Markdown was not capitalized throughout the spec. This was an oversight on our part – and also my fault because I did notice Markdown wasn't capitalized as I copied snippets of the spec to the homepage and blog post, and I definitely thought it was odd, too. You'll note that I took care to manually capitalize Markdown in the parts of the spec I copied to the blog post and home page – but I neglected to mention this to John MacFarlane as I should have. We corrected this immediately when it was brought to our attention.

John then made three requests:

  1. Rename the project.

  2. Shut down the standardmarkdown.com domain, and don't redirect it.

  3. Apologize.

All fair. Happy to do all of those things.

Starting with the name. In his email John graciously indicated that he would "probably" approve a name like "Strict Markdown" or "Pedantic Markdown". Given the very public earlier miscommunication about naming, that consideration is appreciated.

We replied with the following suggestions:

  • Compatible Markdown
  • Regular Markdown
  • Community Markdown
  • Common Markdown
  • Uniform Markdown
  • Vanilla Markdown

We haven't heard back after replying last night, and I'm not sure we ever will, so in the interest of moving ahead and avoiding conflict, we're immediately renaming the project to Common Markdown.

We hope that is an acceptable name; it was independently suggested to us several times in several different feedback areas. The intention is to avoid any unwanted overtones of ownership; we have only ever wanted to be Yet Another Flavor of Markdown.

  1. The project name change is already in progress.

  2. This is our public apology.

  3. I'll shut down the standardmarkdown.com domain as soon as I can, probably by tomorrow.

John, we deeply apologize for the miscommunication. It's our fault, and we want to fix it. But even though we made mistakes, I hope it is clear that everything we've done, we did solely out of a shared love of Markdown (and its simple, unencumbered old-school ASCII origins), and the desire to ensure the success of Markdown as a stable format for future generations.

Edit: after a long and thoughtful email from John Gruber – which is greatly appreciated – he indicated that no form of the word "Markdown" is acceptable to him in this case. We are now using the name CommonMark.

[advertisement] Stack Overflow Careers matches the best developers (you!) with the best employers. You can search our job listings or create a profile and even let employers find you.

Standard Flavored Markdown

In 2009 I lamented the state of Markdown:

Right now we have the worst of both worlds. Lack of leadership from the top, and a bunch of fragmented, poorly coordinated community efforts to advance Markdown, none of which are officially canon. This isn’t merely incovenient for anyone trying to find accurate information about Markdown; it’s actually harming the project’s future.

In late 2012, David Greenspan from Meteor approached me and proposed we move forward, and a project crystallized:

I propose that Stack Exchange, GitHub, Meteor, Reddit, and any other company with lots of traffic and a strategic investment in Markdown, all work together to come up with an official Markdown specification, and standard test suites to validate Markdown implementations. We’ve all been working at cross purposes for too long, accidentally fragmenting Markdown while popularizing it.

We formed a small private working group with key representatives from GitHub, from Reddit, from Stack Exchange, from the open source community. We spent months hashing out the details and agreeing on the necessary changes to turn Markdown into a language you can parse without feeling like you just walked through a sewer – while preserving the simple, clear, ASCII email inspired spirit of Markdown.

We really struggled with this at Discourse, which is also based on Markdown, but an even more complex dialect than the one we built at Stack Overflow. In Discourse, you can mix three forms of markup interchangeably:

  • Markdown
  • HTML (safe subset)
  • BBCode (subset)

Discourse is primarily a JavaScript app, so naturally we needed a nice, compliant implementation of Markdown in JavaScript. Surely such a thing exists, yes? Nope. Even in 2012, we found zero JavaScript implementations of Markdown that could pass the only Markdown test suite I know of, MDTest. It isn’t authoritative, it’s a community created initiative that embodies its own decisions about rendering ambiguities in Markdown, but it’s all we’ve got. We contributed many upstream fixes to markdown.js to make it pass MDTest – but it still only passes in our locally extended version.

As an open source project ourselves, we’re perfectly happy contributing upstream code to improve it for everyone. But it’s an indictment of the state of the Markdown ecosystem that any remotely popular implementation wasn’t already testing itself against a formal spec and test suite. But who can blame them, because it didn’t exist!

Well, now it does.

It took a while, but I’m pleased to announce that Standard Markdown is now finally ready for public review.

standardmarkdown.com

It’s a spec, including embedded examples, and implementations in portable C and JavaScript. We strived mightily to stay true to the spirit of Markdown in writing it. The primary author, John MacFarlane, explains in the introduction to the spec:

Because Gruber’s syntax description leaves many aspects of the syntax undetermined, writing a precise spec requires making a large number of decisions, many of them somewhat arbitrary. In making them, I have appealed to existing conventions and considerations of simplicity, readability, expressive power, and consistency. I have tried to ensure that “normal” documents in the many incompatible existing implementations of markdown will render, as far as possible, as their authors intended. And I have tried to make the rules for different elements work together harmoniously. In places where different decisions could have been made (for example, the rules governing list indentation), I have explained the rationale for my choices. In a few cases, I have departed slightly from the canonical syntax description, in ways that I think further the goals of markdown as stated in that description.

Part of my contribution to the project is to host the discussion / mailing list for Standard Markdown in a Discourse instance.

talk.standardmarkdown.com

Fortunately, Discourse itself just reached version 1.0. If the only thing Standard Markdown does is help save a few users from the continuing horror that is mailing list web UI, we all win.

What I’m most excited about is that we got a massive contribution from the one person who, in my mind, was the most perfect person in the world to work on this project: John MacFarlane. He took our feedback and wrote the entire Standard Markdown spec and both implementations.

A lot of people know of John through his Pandoc project, which is amazing in its own right, but I found out about him because he built Babelmark. I learned to refer to Babelmark extensively while working on Stack Overflow and MarkdownSharp, a C# implementation of Markdown.

Here’s how crazy Markdown is: to decide what the “correct” behavior is, you provide sample Markdown input to 20+ different Markdown parsers … and then pray that some consensus emerges in all their output. That’s what Babelmark does.

Consider this simple Markdown example:

# Hello there

This is a paragraph.

- one
- two
- three
- four

1. pirate
2. ninja
3. zombie

Just for that, I count fifteen different rendered outputs from 22 different Markdown parsers.

In Markdown, we literally built a Tower of Babel.

Have I mentioned that it’s a good idea for a language to have a formal specification and test suites? Maybe now you can see why that is.

Oh, and in his spare time, John is also the chair of the department of philosophy at the University of California, Berkeley. No big deal. While I don’t mean to minimize the contributions of anyone to the Standard Markdown project, we all owe a special thanks to John.

Markdown is indeed everywhere. And that’s a good thing. But it needs to be sane, parseable, and standard. That’s the goal of Standard Markdown — but we need your help to get there. If you use Markdown on a website, ask what it would take for that site to become compatible with Standard Markdown; when you see the word “Markdown” you have the right to expect consistent rendering across all the websites you visit. If you implement Markdown, take a look at the spec, try to make your parser compatible with Standard Markdown, and discuss improvements or refinements to the spec.

Update: The project was renamed CommonMark. See my subsequent blog post.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!

In 2009 I lamented the state of Markdown:

Right now we have the worst of both worlds. Lack of leadership from the top, and a bunch of fragmented, poorly coordinated community efforts to advance Markdown, none of which are officially canon. This isn't merely incovenient for anyone trying to find accurate information about Markdown; it's actually harming the project's future.

In late 2012, David Greenspan from Meteor approached me and proposed we move forward, and a project crystallized:

I propose that Stack Exchange, GitHub, Meteor, Reddit, and any other company with lots of traffic and a strategic investment in Markdown, all work together to come up with an official Markdown specification, and standard test suites to validate Markdown implementations. We've all been working at cross purposes for too long, accidentally fragmenting Markdown while popularizing it.

We formed a small private working group with key representatives from GitHub, from Reddit, from Stack Exchange, from the open source community. We spent months hashing out the details and agreeing on the necessary changes to turn Markdown into a language you can parse without feeling like you just walked through a sewer – while preserving the simple, clear, ASCII email inspired spirit of Markdown.

We really struggled with this at Discourse, which is also based on Markdown, but an even more complex dialect than the one we built at Stack Overflow. In Discourse, you can mix three forms of markup interchangeably:

  • Markdown
  • HTML (safe subset)
  • BBCode (subset)

Discourse is primarily a JavaScript app, so naturally we needed a nice, compliant implementation of Markdown in JavaScript. Surely such a thing exists, yes? Nope. Even in 2012, we found zero JavaScript implementations of Markdown that could pass the only Markdown test suite I know of, MDTest. It isn't authoritative, it's a community created initiative that embodies its own decisions about rendering ambiguities in Markdown, but it's all we've got. We contributed many upstream fixes to markdown.js to make it pass MDTest – but it still only passes in our locally extended version.

As an open source project ourselves, we're perfectly happy contributing upstream code to improve it for everyone. But it's an indictment of the state of the Markdown ecosystem that any remotely popular implementation wasn't already testing itself against a formal spec and test suite. But who can blame them, because it didn't exist!

Well, now it does.

It took a while, but I'm pleased to announce that Standard Markdown is now finally ready for public review.

standardmarkdown.com

It's a spec, including embedded examples, and implementations in portable C and JavaScript. We strived mightily to stay true to the spirit of Markdown in writing it. The primary author, John MacFarlane, explains in the introduction to the spec:

Because Gruber’s syntax description leaves many aspects of the syntax undetermined, writing a precise spec requires making a large number of decisions, many of them somewhat arbitrary. In making them, I have appealed to existing conventions and considerations of simplicity, readability, expressive power, and consistency. I have tried to ensure that “normal” documents in the many incompatible existing implementations of markdown will render, as far as possible, as their authors intended. And I have tried to make the rules for different elements work together harmoniously. In places where different decisions could have been made (for example, the rules governing list indentation), I have explained the rationale for my choices. In a few cases, I have departed slightly from the canonical syntax description, in ways that I think further the goals of markdown as stated in that description.

Part of my contribution to the project is to host the discussion / mailing list for Standard Markdown in a Discourse instance.

talk.standardmarkdown.com

Fortunately, Discourse itself just reached version 1.0. If the only thing Standard Markdown does is help save a few users from the continuing horror that is mailing list web UI, we all win.

What I'm most excited about is that we got a massive contribution from the one person who, in my mind, was the most perfect person in the world to work on this project: John MacFarlane. He took our feedback and wrote the entire Standard Markdown spec and both implementations.

A lot of people know of John through his Pandoc project, which is amazing in its own right, but I found out about him because he built Babelmark. I learned to refer to Babelmark extensively while working on Stack Overflow and MarkdownSharp, a C# implementation of Markdown.

Here's how crazy Markdown is: to decide what the "correct" behavior is, you provide sample Markdown input to 20+ different Markdown parsers … and then pray that some consensus emerges in all their output. That's what Babelmark does.

Consider this simple Markdown example:

# Hello there

This is a paragraph.

- one
- two
- three
- four

1. pirate
2. ninja
3. zombie

Just for that, I count fifteen different rendered outputs from 22 different Markdown parsers.

In Markdown, we literally built a Tower of Babel.

Have I mentioned that it's a good idea for a language to have a formal specification and test suites? Maybe now you can see why that is.

Oh, and in his spare time, John is also the chair of the department of philosophy at the University of California, Berkeley. No big deal. While I don't mean to minimize the contributions of anyone to the Standard Markdown project, we all owe a special thanks to John.

Markdown is indeed everywhere. And that's a good thing. But it needs to be sane, parseable, and standard. That's the goal of Standard Markdown — but we need your help to get there. If you use Markdown on a website, ask what it would take for that site to become compatible with Standard Markdown; when you see the word "Markdown" you have the right to expect consistent rendering across all the websites you visit. If you implement Markdown, take a look at the spec, try to make your parser compatible with Standard Markdown, and discuss improvements or refinements to the spec.

Update: The project was renamed CommonMark. See my subsequent blog post.

[advertisement] How are you showing off your awesome? Create a Stack Overflow Careers profile and show off all of your hard work from Stack Overflow, Github, and virtually every other coding site. Who knows, you might even get recruited for a great new position!