How we made our documentation easier to maintain

We recently released a new version of our documentation site. Along with giving the site a face-lift to bring it in line with our latest design system, we migrated all content from JSX to markdown and used 11ty to generate a static site.

Editing the docs in JSX was tricky. Having to wrap each paragraph, heading or list in components was clunky and made writing documentation a real chore. Our docs are a key resource for users. Maintaining them effectively is vital to user success, so we’re always keen to remove any barriers.

Migrating the content

The first barrier we faced to migrating the content was converting over 100 articles in JSX format to markdown. To do this manually would have been an extremely time consuming and frankly boring task. However our intrepid Product Manager Chris Casey created a Python script to automatically convert a large amount of the content, propelling us to 90% completion in almost no time. The script searched through all of the JSX documentation for repeating code blocks that could then be converted into markdown.

Converting HTML links from JSX to Markdown operated as follows:

python
1regex= re.compile(r'<[Ll]{1}ink(.*?)</[Ll]{1}ink>', re.DOTALL)
2content=block_replacer(regex, content, "link")
3
4def block_replacer(regex, content, mode):
5    for match in regex.finditer(content):
6        text_find=match.group()
7        text_replace=mode_operator(text_find, mode)
8        content=content.replace(text_find, text_replace, 1)
9     return content
10
11def mode_operator(text_find, mode):
12    text_replace=text_find
13    if mode=="link":
14        text_replace=' '.join(text_find.split())
15        text_replace=text_replace.replace('<Link', '<a').replace('</Link', '</a').replace('<link', '<a').replace('</link', '</a')
16        text_replace=html2markdown.convert(text_replace)
17    return text_replace
Copy

The output wasn’t perfect, but with the bulk of the conversion out of the way, it left only a little bit of manual formatting to complete. Having it all in markdown has opened new possibilities to how we generate the output, and gives us a lot more freedom to experiment with our design system for docs.

Choosing a site generator

I’d used 11ty on some personal projects and really like how lightweight it is. I considered other static site generators like Next.js, Gatsby and Hugo. At Pusher we don’t have many React projects and so it felt unnecessary to add one, we do use Go but I am less familiar with it, so 11ty felt like the obvious choice. There’s a great community around it too and I knew it had the features we needed, such as its navigation and table of contents plugins.

Our docs contain a lot of code snippets in various languages and our previous React-based docs sported a great tabbed section which features across our website. I wanted to be able to keep some of these more aesthetically pleasing touchpoints for readers without making the files too heavy.

11ty parses the markdown but also lets you sprinkle in some other templating languages like liquid or Nunjucks. I added some custom Nunjucks paired short codes that we could use to keep the markdown files light but output more complex markup.

bash
1{% snippets ['js', 'swift', 'objc', 'java', 'laravelecho', 'c'] %}
2
3```js
4var pusher = new Pusher("YOUR_APP_KEY", options);
5```
6
7```swift
8let pusher = Pusher(key: "YOUR_APP_KEY")
9```
10
11```objc
12self.pusher = [[Pusher alloc] initWithKey:@"YOUR_APP_KEY"];
13```
14
15```java
16Pusher pusher = new Pusher("YOUR_APP_KEY");
17```
18
19```js
20window.Echo = new Echo({ broadcaster: "pusher", key: "YOUR_APP_KEY" });
21```
22
23```c
24var pusher = new Pusher("YOUR_APP_KEY");
25```
26
27{% endsnippets %}
Copy

11ty supports a wide range of templating languages out of the box. Nunjucks lets you transform your content into rich components with paired short code. Short code blocks can take parameters which control the output of transformation, and the content within the block is also passed through.

javascript
1eleventyConfig.addPairedShortcode(
2    "snippets",
3    (content, languages, method = false) => {
4      const languageMap = {
5        rb: "Ruby",
6        js: "JavaScript",
7        node: "Node.js",
8        php: "PHP",
9        laravel: "Laravel",
10        laravelecho: "Laravel Echo",
11        go: "Go",
12        py: "Python",
13        c: ".NET",
14        java: "Java",
15        bash: "Pusher CLI",
16        swift: "Swift",
17        objc: "Objective-C",
18        http: "http",
19        kotlin: "Kotlin",
20      };
21      return `<div <em>class</em>="bg-snow-light br2 tabbed-snippets" <em>data-method</em>="${method}">
22      <nav <em>class</em>="ph3 bb b--smoke overflow-auto scrollbar--light">
23        <ul <em>class</em>="flex">
24      ${languages
25        .map(
26          (language, i) =>
27            `<li <em>class</em>="mh1"><button <em>class</em>="bn bg-snow-light sans-serif fw6 pt3 pb2 ph2 dragonfruit pointer pa0" <em>data-snippet</em>="language-${language}" <em>data-index</em>="${i}" <em>aria-pressed</em>="${
28              i === 0
29            }">${languageMap[language] || language}</button></li>`
30        )
31        .join("\n")}
32      </ul>
33    </nav>
34    ${content}</div>`;
35    }
36  );
Copy

Here’s how the snippet example renders. With just a bunch of markdown code snippets and a wrapping short code we end up with an interactive tabbed element.

Performance

Our old docs loaded quite a lot of client-side JS, negatively impacting the load times, so we wanted to try and make sure the new docs were as lightweight as possible. We’ve only used client-side JS where necessary to enhance the experience with out more complex components like the tabbed snippets and the search function. Rather than use a module bundler we opted to keep things simple by using JavaScript modules. The new docs is a lot more performant with load times down from 2.5s to 0.9s.

All of Pusher’s website styling comes via a customised Tachyons build. Atomic CSS contains a lot of classes by definition and it’s important to strip out the ones that are not used. We use PurgeCSS for this which reduces the amount of CSS selectors significantly (from 974 to 405). The CSS is then passed through CSSnano to compress it further. All in all our CSS starts with a bundle size of 213kB and is reduced 37kB (9kB transferred) in production.

Lighthouse scores for the old and new docs pages

Algolia powers our search. At build time, we make a big JSON blob of all the content which is uploaded to Algolia for indexing. We use their search API and some custom JS to show the search results. We provide a no-js fallback using Vercel’s serverless functions capabilities to produce the same results on a new page.

No-js fallback search page, generated by a Vercel severless function

Making things open

Our ultimate goal in migrating was to make our docs more logical and collaborative, speeding up updates for users. They are hosted with Vercel and their GitHub integration gives us builds for each commit/PR, making preview sharing really easy.

Inspired by one of the UK Government’s design principles Make things open: it makes things better we have also made our new doc repo public on GitHub so that people can contribute if they spot mistakes. We’ve had some contributions already. Bringing a collaborative ethos to our documentation to include not only more of our team but our users as well, means more eyes on the service, more problem-solving, more efficiency and overall better resource maintenance.