Automatically Build and Publish Node and Electron Applications for Linux

Martin Wimpress speaking at London Node User Group in April, 2017
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

An introduction to and electron-builder, tools that enable you to deliver your Node and Electron applications to millions of Linux users. Packaging for Linux used to be hard, but that has changed: the Snapcraft team have built a platform that makes it simple to build and publish your applications so they run on all the major Linux distributions. It gives you tools to deliver updates at a high frequency without compromising stability. Push a commit and it will automatically build for free in the store for your users.


- Hello, so I'm going to be talking about automatically building and publishing your applications for Linux devices. And I'd just like to point out, it's not a Mac. It's not running Mac OS, see it just all worked. You see how it just all worked without messing about? Right, my name's Martin Wimpress. I'm a software engineer at Communical and I work on Ubuntu and Snapcraft so it's gonna be a two way thing. 'Cause I want to learn some stuff from you here this evening as well so my first easy question is who is familiar with Ubuntu, using Ubuntu? So keep your hands up if that's on the back end for servers and stuff, yeah. And anyone using it on their work stations. Oh, we should get you staffed, that's great. Okay, so this is what we're gonna run through this evening. I'm going to briefly introduce was Snaps and Snapcraft IO are, this isn't a sales pitch. This is some sort of base information you need to understand for the rest of it to make sense. Then we're gonna specifically talk about what's called snapping, node applications using Snapcraft. And we'll cover what Snapcraft and then snapping a lecture on applications using electron-builder. So again, another question, does anyone here working with electron apps and knows what electron-builder is, hands up electron app developers. Okay, there's a good number of you. And those of you, are you familiar with electron-builder? Anyone, you've heard of it, okay, okay. Well, I'll introduce electron-builder as well. Oh, quick, so this is the work flow we're going for here. The idea here is that you commit stuff, it gets automatically built and it gets published to your users. And I'll break down how all of that happens. So fortunately, my colleague, Alan has made a video. - [Narrator] Traditional methods of delivering software on Linux are problematic, packages write all over the file system, they require complex code to manage installation and updates, they're not transactional. If they fail, the end user is left to pick up the pieces. Because this is all so risky, updates have to be a manual operation. At Ubuntu, we build a platform that addressed these issues. What we've called Snaps and Snapcraft. It puts control over publishing in the hands of vendors, giving tools to deliver updates at a high frequency without risking the end user experience. These updates require no end user interaction, even when they fail, if you're building software to run on Ubuntu 1404 or 1604 today, this is the best way to deliver it. They're already well over 1,000 such applications available from hundreds of developers. The format is simple and declarative. Just three stanzas, meta data, confinement and build definition, plug ins exist for popular build systems and languages, it's easily extendable by ending shell commands in the yaml, you can bundle your dependencies of predictable behaviour even for bigger pieces like databases. Build definitions can equally just take pre-built binaries or artefacts. We offer lots of generic foundational components you can build on top of, such as a build farm and Lex D clustering. We also make the tools easy to integrate into your existing build and CI system. Build once, publish once, run on many Linux systems. This list is growing and support for building and testing for Windows and Mac will land soon. Snaps a like container but without the baggage of an OS image included, that means you can even instal Snaps in Lex D or Docker containers. The confinement model can be adjusted. Need internet access or access to hardware? You can just add that to the yaml. You can even package editors and apps that need complete access to the file system by changing one line of the yaml to what we call classic confinement. Health checks let any update validate itself. Users will automatically roll back and you can repoint the stable channel at a previous revision. End users can audit the transactions using the Snap Changes command, channels are a way for the end user to subscribe to a level of risk. You can automatically land you CI builds on edge where your QA and developers can review. You can promote these same changes to more stable channels with a single command. You can set up beta programmes that when enclosed, move users back to stable. No more long tail of releases to support. Tracks let you introduce incompatible changes without breaking existing users. Tracks can be named anything, 2.0, 3.2 or Banana. Each track has it's own stable candidate, beater and edge channels. Systems update automatically, four times a day or on user request, updates are delivered as deltas. We're driving this from the IOT space where you need to update that smart fridge for security fixes but cannot risk breaking it. Four times a day isn't fast enough for all cases. Push updates are coming soon. The Kernel and base operation system are also delivered as snaps, the Kernel and OS roll back on failure just as applications do. The system reboots back to a known good state. Vendor code running on Edge devices is tamper proofed. A great host for Docker containers and just as is the case with other apps on the platform, the Docker Snap is published directly by Docker themselves. Just as the Cloud saves companies from data centre planning, we want update delivery to be solved so people can get on with higher level differentiating tasks. Your feedback is welcome, contact us if you have any questions or thoughts on how we might better support you. It's also open source, you can get Snap D and Snapcraft on GetHub if you want to contribute. Find out more at - So that is the primer, some of that is talking about IOT. We'll gloss over that a little bit today, come back to it a little bit. The important thing here is Snaps are a package format. Snapcraft is the tool to create then. Snap bundled all of their requirements to run that application, so and because it doesn't instal anything onto the host operating system, every snap can have what it needs and it won't contaminate the host OS or other applications so in the context of Node, we can make sure that you ship the right version of Node JS in your application so we're run through this. They're confined, everything you need is inside the snap including specific versions of Node and Node libraries. Transaction of the updated with health checks. Channels that users can subscribe to. If you want long term supported versions, your users can subscribe to 1.0. They won't get bumped 2.0 in this unless you're required. Always moving forwards but version numbers are arbitrary. You can move revisions around. And we'll come to that a bit later on. Currently these are the supported Linux distributions. You have a Snap and it goes in the store. It runs on all of these OSs. It's build once, deploy everywhere. So Snapping Node applications using snapcraft. Snapcraft is our tool to create snaps. It requires a snapcraft.yaml and once you've created one of those, you can just run snapcraft. So this is what a yaml looks like. Now actually I've made this more complicated than it needs to be in order to highlight a couple of points that are relevant to you guys using Node. So the top is all very bland, the name of the application, version number, descriptions and stuff. Grade, stable, we consider this application to be stable. You can choose Beater for example and Devil and you won't be allowed to push Beater and Devil graded applications into the stable channels, for example. And then confinement is strict. So this application is entirely confined from other applications, other snaps and the host operating system. The only access it has is to itself and anything that we specify in interfaces which are down here at the bottom in this plugs bit which is around there, down the bottom somewhere, this application requires access to the network as a network client. The parts make up what is in the snap. So in this case, we're going to use the Node JS plug in because this is a Node application. I've specified Node engine 6.10.2. What that will do is it will go out and grab that version of Node JS and plant that inside the snap, now that was an irrelevant step because that is the default version. If you don't specify a Node engine version, that's what you get but if you've got an old application that requires Node 0.12 for example, you can specify that and build that inside your snap. And because it's gonna run on a common run time, that snap will always run, always continue to work 'cause everything it needs is bundled and isolated together and I've also added the Node package manager stanza. That's brand new, this landed in the new version of Snapcraft that came out yesterday. And in fact, I spoke to some of you at this event last month talking about whether you use MPN or Yarn. So we've landed yarn support, you can use MPN. MPN's the default, yarn is there. The reason we've added yarn or come to when we get to the electron-builder piece of it later on, the source, line there, dot, this snap is inside the source tree so it doesn't need to tell, go to a GetHub repository or a subversion repository or a Tobowl or whatever else. It has all of its source code in this repository. So just get it from here and stage packages, this is a Linux thing, this application needs a Linux application called XL. And what that stage package says is go into the Ubuntu archive and instal that bit of software inside the snap so that means that the host operating system doesn't have to have XL installed. Everything that the snap needs is now self confined. The right version of Node JS, the right version of the application and any dependencies it needs, it relies on. And then the last piece is just basically creating a launcher for it, so our emoji application, it says once it's done the build, the build sits in emoji and once you've installed your snap, you'll just have a command on the system emoji. This is a great thing, you type emoji, I'm feeling very happy today and it displays a string of emoji that mean that. So you can automate this build and publish pipeline. GetHub integration, so every commit will trigger a build and publish in the store so we have a snap store. Just like an app store, it'll publish into edge by default. And that all happens on and it looks like this so if you go there, you click that button, you get a prompt, you fill in some user credentials, it'll list all of your projects. Currently it's in soft launch at the moment, it's a Beater. So it only handles personal projects at the moment. So you'll only see stuff in personal projects. You choose your repository with your snapcraft.yaml in it. That will automatically hook that GetHub repository to our build farm, when you commit code, our build farm will spin up and build your application on all of the support architectures. So this is all the Intel 32 bit, 64 bit, on 32 and 64 bit. And if you've got more exotic requirements like S390 and Power Eight, those architectures are available as well. It means to say that whilst you can, on a Linux machine do all of this locally, you don't have to do that in production, there's facilities there. And this is all open source and free at the point of use. If you do build locally, you can publish to the store manually, you use Snapcraft to register the name of your application, myapp and then you can, after you've built a snap just by running Snapcraft, it will build. You can then push your application and in this case, I'm pushing it to the edge channel. 'Cause it's off the tip of Get. I don't want to push it straight to stable. I'm gonna push it to the edge channel. I can then have QA team do that, run their integration tests from edge, so that's doing it with Node. And now we're gonna come onto electron. - [Narrator] If you can build a website, you can build a desktop application. Electron is a framework for creating native applications with web technologies like JavaScript, HTML and CSS. Electron developers can easily package their applications for Linus, Mac OS and Windows by using electron-builder. Electron applications built with electron-builder for Mac OS and Windows include out of the box auto update support, on Linux, this support comes from Snaps. Enabling Snap builds in electron-builder requires just a few lines of additional configuration to your packages.json. By releasing your electron application in the snap store, you'll gain complete control over publishing and delivering transactional automatic delta updates to your users. Applications published in the snap store will be immediately available as a one click instal by the desktop software centre, installed by default in many Linux distributions. For Tonal junkies, simply Snap instal your app is all you need to add to your instal documentation. Publishing in the snap store improves the discoverability of your application for millions of Linux users. Plus, you'll have access to analytics for your snap application as well, publishing your application to the snap store can be entirely automated by clicking through - So there's currently a portfolio of electron applications in the snap store, these are the logos. You may recognise some of them. There are a few logos I can't put up there because the companies in question are very brand protective. But there are some high profile apps as well. So this isn't like immature, this is production ready stuff. So electron-builder, electron-builder isn't just for snaps. Electron-builder can create installers for all across platform, so it can create DMGs for Mac OS. It can create MSI and XE installers on Windows. And it can create Snaps for Linux. So it's one tool that can create your installers for all the things, we've been working with the electron-builder upstream team to land snap support in electron-builder so it's a first class citizen there. And what electron builder does is it actually wraps up Snapcraft so you don't have to get into the nuts and bolts of creating that yaml file that we looked at earlier. And you don't have to know what libraries you need to pull into via stage packages from the Ubuntu archive. All you do is what you're already doing in electron which is looking after your packages.json file to tell it what stuff you need to run. Electron-builder is a Node application. It's just one of the dependencies in the build chain and it has some stanzas to say how it should work, things like meta data to imbed in the Windows versions, in the Mac OS versions, in the Linux versions. And there's a couple of extra stanzas in there for snap support where you can define the confinement. So it handles all the Linuxy bits. And this is a pull request that I sent to a project recently, the project is Ram Box which is an electron app that basically connects to all of the chat and email things so you have one application with all of your stuff in one place. They're already using electron-builder and that was the extent of what I needed to do in order to land snap support, basically say that there was a snap at target and what the confinement model was. So in this case, classic confinement. So using electron-builder to make a snap. This is the general work flow, you clone the repository where you've got your electron builder in a bold snap. I'm not gonna get into how you set up electron builder in your projects, that's well documented in the electron builder project. Electron builder is part of electron user land tools. It describes how you do that. It has a two packages.json layout, packages.json in the root of the project for build dependencies, sub folder app with the packages.json and that's the runtime requirements for your Node application. That's the basic layout so you clone the repository. You then snap instal yarn and the reason that you instal yarn here is because the electron builder project mandate that you use yarn rather than MPN. And the reason for this is that when you actually use MPN to try and satisfy the build dependencies, it gets in a knot and that there's a bug somewhere that it can't resolve the dependency chain ends up with two versions of one of the libraries. And the electron builder guys have just said everyone should be using yarn for this. It's why we landed yarn support in Snapcraft and why doing this manually, I'm using yarn. So once I've installed yarn, I've just type yarn in my project, all of the requirements get pulled in and when I want to run it, build it, yarn run dist. And on Linux that will create a snap when you run that command, on Windows, you'll end up with a Microsoft Installer and on Mac, DMG. As easy as that, so the snap store has some benefits. And I've come to understand this from talking to electron developers so when you build an electron app on Linux, it doesn't have any auto update support unlike Mac OS and Windows where there's some built in capability to handle automatic updates. So the snap store fixes that. If you build your application and publish it in the snap store, you now have automatic updates. These delta updates, electron applications tend to be over 100 meg so this keeps the amount of data that goes down to the user over the wire on updates small. And it keeps them up to date all the time so it gives so feature parity with Mac OS and Windows users. And it's a way to target Linux as opposed to what you have to do right now is I'm gonna make something for Ubuntu. I'm gonna make something for Fedora, perhaps. You have to make two versions for Ubuntu because the older version has different requirements and all the rest of it, now you don't do that, you just make one version. You publish it once, it runs on all of the Linux platforms. And we have improved discoverability. Alan talked about we have, there's a desktop software centre, that's common among all of the major distributions. Electron applications, when they get published in the store automatically appear in the desktop store. So now your application is discoverable by any Linux user. It has icons, all that meta data that we entered earlier. All of that is what's reelected in that. So in summary, Snaps are packages, Snapcraft creates snaps. gives you access to our build farm. It connects your Get to our build farm to the store. Electron builder creates snaps. The snap store provide deployment parity. And you get simple instal documentation. So now for Linux, your instal documentation can say snap instal my app, none of this MPN instal app to get this curl pipe, that version of, no, none of that. So you can learn more, is where all of the documentation lives. is where you can go and try out the building your personal projects right now. And you can talk to us, we have a discourse forum running. All of the Snapcraft team are in there. That includes developer advocates, the core OS maintainers, the Snapcraft developers, the Snap developers so if you want to start playing and you've got questions, this is the place to find us.