How We Built AtomPair, Our Realtime Collaboration Plugin For Atom.IO

This article is part of

Following the release of AtomPair, our remote pairing plugin for Atom.IO, we thought we’d share how we made it. This blog post is partly to act as a demonstration of how to use presence channels and client events to easily synchronize states for seamless realtime collaboration. Its main purpose, however, is to share what we learnt as we built it, and to encourage you to make it better!

The realtime web is sometimes referred to as ‘the evented web’. Our applications are full of events; events on changing state, events on user activity, and so on. Pusher, built on this paradigm, uses channels in which to distribute events, and requires you to register actions when such events are fired.

Part of what made AtomPair so enjoyable to build was that, like Pusher, Atom’s API is heavily event-driven. For almost any user-enacted event – typing, selections, syntax-changes, copy-pastes, saves, and so on – Atom lets you easily register a callback to execute.

And likewise, for every Websocket event that’s triggered – for instance, when a collaborator types, shares a file, selects a bit of text – Pusher makes it easy to handle that event and change something in your editor.

Starting A Session

Starting a session in AtomPair is fairly straightforward. From your command palette you hit AtomPair: Start new pairing session and it gives you a session ID. You share that with your collaborator, and you can begin pairing.

The purpose of the session ID is to make sure that only those who have the session ID can receive the events fired whilst using AtomPair. We ensure this by using the session ID as the Pusher channel name.

AtomPair uses presence channels to detect when collaborators are connected and client events to broadcast events. In traditional applications a server would be used to authenticate private or presence channels for security reasons; that is, so that channels don’t get spammed with messages by unauthenticated users.

However, given that pairing sessions are within a controlled environment (one’s own Atom app) and between invited collaborators, we decided to stub out authentication using our devangelist Phil Leggetter’s client authentication ‘hack’, which requires entering application secrets on the client.

Please note that you should never use this in a production app, as it exposes your application credentials. We have only used this as Atom is a controlled environment.

So, using this hack, we pass in our app key and secret into the Pusher instantiation. Our session ID consists of the app_key, app_secret, and a random 11-character string (the latter because we supply default keys to get you up and running, and so they wouldn’t alone be enough to ensure controlled access). The session ID becomes the channel name, and both parties can enjoy the benefits of presence channels and know when collaborators connect and disconnect.

Presence channels require each user to have a unique user_id. In AtomPair we use a colour that marks each collaborator’s actions in the session.

coffeescript
1@pusher = new Pusher @app_key,
2    authTransport: 'client'
3    clientAuth:
4        key: @app_key
5        secret: @app_secret
6        user_id: @myMarkerColour # each `member`'s unique identifier
7
8@pairingChannel = @pusher.subscribe("presence-session-#{@sessionId}")
Copy

So, having subscribed to the channel, one has access to all members’ colours with which to identify them.

coffeescript
1@pairingChannel.bind 'pusher:subscription_succeeded', (members) =>
2  colours = Object.keys(members.members)
3  @friendColours = _.without(colours, @myMarkerColour)
4  _.each(@friendColours, (colour) => @addMarker 0, colour)
5  @startPairing()
Copy

This merely gets the colours from the members object, removes my colour, and adds the marker to the view on the first row.

We don’t want to show our own marker so we remove it from the colours retrieved by the members object. Then we add the markers to the view on the first row.

Now, whenever there is a pusher:member_added event, we can do such things as include the member’s marker, sync syntax highlighting, and share the current file.

coffeescript
1@pairingChannel.bind 'pusher:member_added', (member) =>
2  @showAlert("Your pair buddy has joined the session.")
3  @sendGrammar()
4  @shareCurrentFile()
5  @friendColours.push(member.id)
6  @addMarker 0, member.id
Copy

And, when a member disconnects, we can clear their marker:

coffeescript
1@pairingChannel.bind 'pusher:member_removed', (member) =>
2  @showAlert("Your pair buddy has left the session.")
3  @clearMarkers(member.id)
Copy

Now we can start pairing!

Pairing

Triggering Change Events

The main thing that AtomPair does is listen to changes in the text editor and trigger Websocket events accordingly. The listener function that does this essentially hooks into Atom’s Buffer API class (returned by atom.workspace.getActiveEditor().buffer), and binds to the onDidChange event. Whenever the text in the buffer changes, it passes the event.newText, event.oldText, event.newRange and event.oldRange to a callback.

First, we only wanted to process and send the event to other clients if @triggerPush is set to true. That way, we can make sure that we only trigger events if the current user generated them.

To send over a neat payload of the relevant information, we want to decide what the type of event is and the relevant action positions are, so that our collaborators’ editors can easily render the changes.

coffeescript
1@buffer.onDidChange (event) =>
2  return unless @triggerPush
3  if event.newText.length is 0
4    changeType = 'deletion'
5    event = {oldRange: event.oldRange} # we only need the range of the text that has been deleted
6
7  else if event.oldRange.containsRange(event.newRange) #text has been selected over and replaced
8    changeType = 'substitution'
9    event = {oldRange: event.oldRange, newRange: event.newRange, newText: event.newText} #the recipient needs to replace the oldRange with the newText at the newRange
10  else
11    changeType = 'insertion'
12    event  = {newRange: event.newRange, newText: event.newText} # just insert the new text at the new range
13
14  pusher_event = {changeType: changeType, event: event, colour: @myMarkerColour, eventType: 'buffer-change'}
15  @events.push(pusher_event)
Copy

So, in the Pusher event we will send over a buffer-change eventType, the buffer changeType, our marker colour (so that the recipient can show our actions) and the filtered event itself.

You will notice that we are not firing the Pusher event just yet, we are storing them in an @events array. This is so that we can queue events and work within Pusher’s client rate-limit of 10 messages per second, which is set to protect users from huge influxes of client-originating events.

The queue is very simple to implement. Every 120ms the event queue, if not empty, is broadcasted on the @pairingChannel under the event name client-change:

coffeescript
1setInterval(=>
2  if @events.length > 0
3    @pairingChannel.trigger 'client-change', @events
4    @events = []
5, 120)
Copy

Receiving Changes

To react to collaborators’ buffer-change events, we just bind to the client-change event. This event also handles other things such as rendering our partner’s multiline selections, but we’ll just focus on changes to the buffer for now:

coffeescript
1@pairingChannel.bind 'client-change', (events) =>
2  _.each events, (event) =>
3    @changeBuffer(event) if event.eventType is 'buffer-change'
Copy

In turn, the changeBuffer method handles the Pusher payload and transforms the buffer accordingly:

coffeescript
1@clearMarkers(data.colour) # remove the triggerer's marker before rendering it again in a new position
2
3@withoutTrigger =>
4  switch data.changeType
5    when 'deletion'
6      @buffer.delete oldRange #deletes the text in the previous range
7      actionArea = oldRange.start
8    when 'substitution'
9      @buffer.setTextInRange oldRange, newText #substitutes the text in the old range with the new text
10      actionArea = oldRange.start
11    else #i.e. an 'insertion'
12      @buffer.insert newRange.start, newText #inserts at the new range the new text
13      actionArea = newRange.start
14
15@editor.scrollToBufferPosition(actionArea)#auto-scrolls to the action area
16@addMarker(actionArea.toArray()[0], data.colour) #sets the triggerer's marker at the action area
Copy

In order to prevent the changes we’re making from triggering additional events, resulting in an infinite loop, we wrap the process in @withoutTrigger function, which sets @triggerPush to false and then to true again after the callback has been executed.

The rest is made easy by Atom’s API, with methods such as Buffer::delete for deleting at ranges, Buffer::setTextInRange for inserting or substituting text between certain buffer coordinates, and Buffer::insert for plain insertion at a point.

After we’ve changed the buffer, we just call TextEditor::scrollToBufferPosition to autoscroll to the action area, and re-add the agent’s gutter marker on that row.

Now, Over To You!

You can install the plugin by going to your Atom settings page, searching for ‘atom-pair’ and hitting install. Hopefully we have given you an overview of how the plugin works regarding synchronizing buffer changes, but you can check out the source code yourself on the Github repo to learn about the other features.

In the spirit of collaboration, we encourage you to make this project your own. Here are some ideas that might improve collaborators’ experience:

• Coloured cursors à la Google Docs
• Slack integration, to complement the package’s existent HipChat integration.
• Collaborator chat.

No doubt you have ideas of your own on how to make the package better, so feel more than free to send us a pull request!

For more about building realtime applications, check out our