We're hiring
Products

Channels

Beams

Chatkit

DocsTutorialsSupportCareersPusher Blog
Sign InSign Up
Products

Channels

Build scalable, realtime features into your apps

Features Pricing

Beams

Send push notifications programmatically at scale

Pricing

Chatkit

Build chat into your app in hours, not days

Pricing
Developers

Docs

Read the docs to learn how to use our products

Channels Beams Chatkit

Tutorials

Explore our tutorials to build apps with Pusher products

Support

Reach out to our support team for help and advice

Status

Check on the status of any of our products

Products

Channels

Build scalable, realtime features into your apps

Features Pricing

Beams

Send push notifications programmatically at scale

Pricing

Chatkit

Build chat into your app in hours, not days

Pricing
Developers

Docs

Read the docs to learn how to use our products

Channels Beams Chatkit

Tutorials

Explore our tutorials to build apps with Pusher products

Support

Reach out to our support team for help and advice

Status

Check on the status of any of our products

Sign InSign Up

Pusher Channels as an alternative messaging queue

  • Lanre Adelowo
February 4th, 2019
You will need Go 1.9+ and Node 7+ installed on your machine.

Introduction

In this tutorial, we will be building a message queue backed up by Pusher Channels. The application we will build will be a typical login service which upon a successful authentication, an email is sent to the authenticated user informing him of the authentication process and where it originated from. This is quite common with web applications - Twitter, GitHub and Slack do this all the time. We will build the login service in Golang while the email service will be written in NodeJS. The Golang application will publish the data to Pusher channels while the Node.js service will be subscribe to the particular channel and send the email to the user.

Messaging queues are an interesting technique used to improve scalability and a bit of abstraction between the producer and the receiver/consumer as they don’t have to be connected in whatever form. A message queue is nothing much more than a list of messages being sent between two or more applications. A message is basically data produced by an application usually called the producer. That data is then sent into the queue to be picked up by another totally different application - known as the consumer.

Prerequisites

  • Golang ( >= 1.9)
  • Node.js ( >= 7 )
  • A Pusher Channels application. Create one here.

Building the login service

Let’s set up a simple login Golang service. Due to simplicity reasons this application will only handle authentication and will use a memory-mapped list of users.

To get started, we will need to set up our project root directory. We need to create the directory pusher-channels-queue somewhere in $GOPATH. Ideally, this should resolve to $GOPATH/src/github.com/pusher-tutorials/pusher-channels-queue.

After doing the above, we will need to create a go directory since that is where our Golang application will live.

    $ mkdir go

The only external library we will need here are the Channel’s Golang SDK and a library to help us load our Pusher Channels keys. You can fetch that by running the command below:

    $ go get github.com/pusher/pusher-http-go
    $ go get github.com/joho/godotenv

To get started, you will need to create an .env file with the following contents:

    // github.com/pusher-tutorials/pusher-channels-queue/go/.env

    PUSHER_APP_ID="YOUR_APP_ID"
    PUSHER_APP_KEY="YOUR_APP_KEY"
    PUSHER_APP_SECRET="YOUR_APP_SECRET"
    PUSHER_APP_CLUSTER="YOUR_APP_CLUSTER"
    PUSHER_APP_SECURE="1"

Once this has been done, we will need to create a main.go file.

    // github.com/pusher-tutorials/pusher-channels-queue/go/main.go
    package main

    func main() {

            port := flag.Int("http.port", 1400, "Port to run HTTP service on")

            flag.Parse()

            err := godotenv.Load()
            if err != nil {
                    log.Fatal("Error loading .env file")
            }

            appID := os.Getenv("PUSHER_APP_ID")
            appKey := os.Getenv("PUSHER_APP_KEY")
            appSecret := os.Getenv("PUSHER_APP_SECRET")
            appCluster := os.Getenv("PUSHER_APP_CLUSTER")
            appIsSecure := os.Getenv("PUSHER_APP_SECURE")

            var isSecure bool
            if appIsSecure == "1" {
                    isSecure = true
            }

            client := &pusher.Client{
                    AppId:   appID,
                    Key:     appKey,
                    Secret:  appSecret,
                    Cluster: appCluster,
                    Secure:  isSecure,
            }

            mux := http.NewServeMux()

            mux.Handle("/login", http.HandlerFunc(login(client)))

            log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), mux))
    }

In the above, we created an HTTP server that responds to the login route. We will go on to implement the login function subsequently.

Since we will be using a memory mapped list of users to prevent complications that might drive us away from the main focus of the tutorial. We will need to go ahead to create those. Paste the following code in the main.go file.

    // github.com/pusher-tutorials/pusher-channels-queue/go/main.go

    type User struct {
            Email    string
            Password string
    }

    var (
            validUsers = map[string]User{
                    "admin": User{
                            Email:    "youremail@gmail.com",
                            Password: "admin",
                    },
                    "lanre": User{
                            Email:    "youremail@gmail.com",
                            Password: "lanre",
                    },
            }
    )

You should replace youremail@gmail.com with your real email address so as to get the email when we get to the end of the tutorial.

Now back to the login function, you can go ahead to paste the following code in main.go

    // github.com/pusher-tutorials/pusher-channels-queue/go/main.go

    func encode(w io.Writer, v interface{}) {
            json.NewEncoder(w).Encode(v)
    }

    func login(client *pusher.Client) http.HandlerFunc {
            return func(w http.ResponseWriter, r *http.Request) {
                    defer r.Body.Close()

                    var request struct {
                            UserName string `json:"userName"`
                            Password string `json:"password"`
                    }

                    type response struct {
                            Message string `json:"message"`
                            Success bool   `json:"success"`
                    }

                    // Make sure to only respond to the "/login" route
                    // due to limitations in the standard HTTP router
                    if r.URL.Path != "/login" {
                            w.WriteHeader(http.StatusNotFound)
                            return
                    }

                    // Only HTTP posts are accepted
                    if r.Method != http.MethodPost {
                            w.WriteHeader(http.StatusMethodNotAllowed)
                            return
                    }

                    if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
                            w.WriteHeader(http.StatusBadRequest)
                            encode(w, response{"Invalid request body", false})
                            return
                    }

                    // Check if the user exists in our memory mapped list.
                    user, ok := validUsers[request.UserName]
                    if !ok {
                            w.WriteHeader(http.StatusBadRequest)
                            encode(w, response{"User not found", false})
                            return
                    }


                    // Do the passwords match ?
                    if user.Password != request.Password {
                            w.WriteHeader(http.StatusBadRequest)
                            encode(w, response{"Password does not match", false})
                            return
                    }

                    w.WriteHeader(http.StatusOK)
                    encode(w, response{"Login successful", true})

                    host, _, err := net.SplitHostPort(r.RemoteAddr)
                    if err != nil {
                            fmt.Fprintf(w, "userip: %q is not IP:port", r.RemoteAddr)
                            return
                    }

                    var ip = host

                    if host == "::1" {
                            ip = "127.0.0.1"
                    }

                    client.Trigger("auth", "login", &struct {
                            IP    string `json:"ip"`
                            User  string `json:"user"`
                            Email string `json:"email"`
                    }{
                            User:  request.UserName,
                            IP:    ip,
                            Email: user.Email,
                    })
            }
    }

While it is pretty easy to grok through the code above due to the inline comments, I will still like to go through the last few lines. Especially from Line 59.

  • We get the IP of the user from r.RemoteAddr.

    Please note that if you end up running something that does this kind of IP fetching in production, this might not be the right approach if your Go application is behind a proxy.

  • We also check to make sure we have a valid IP address by making use of the net.SplitHostPort utility function.
  • Then we finally publish the data to the auth channel.

At this point, the entire main.go should look like the following:

    // github.com/pusher-tutorials/pusher-channels-queue/go/main.go

    package main

    import (
            "encoding/json"
            "flag"
            "fmt"
            "io"
            "log"
            "net"
            "net/http"
            "os"

            "github.com/joho/godotenv"
            pusher "github.com/pusher/pusher-http-go"
    )

    type User struct {
            Email    string
            Password string
    }

    var (
            validUsers = map[string]User{
                    "admin": User{
                            Email:    "youremail@gmail.com",
                            Password: "admin",
                    },
                    "lanre": User{

                            Email:    "youremail@gmail.com",
                            Password: "lanre",
                    },
            }
    )

    func main() {

            port := flag.Int("http.port", 1400, "Port to run HTTP service on")

            flag.Parse()

            err := godotenv.Load()
            if err != nil {
                    log.Fatal("Error loading .env file")
            }

            appID := os.Getenv("PUSHER_APP_ID")
            appKey := os.Getenv("PUSHER_APP_KEY")
            appSecret := os.Getenv("PUSHER_APP_SECRET")
            appCluster := os.Getenv("PUSHER_APP_CLUSTER")
            appIsSecure := os.Getenv("PUSHER_APP_SECURE")

            var isSecure bool
            if appIsSecure == "1" {
                    isSecure = true
            }

            client := &pusher.Client{
                    AppId:   appID,
                    Key:     appKey,
                    Secret:  appSecret,
                    Cluster: appCluster,
                    Secure:  isSecure,
            }

            mux := http.NewServeMux()

            mux.Handle("/login", http.HandlerFunc(login(client)))

            log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), mux))
    }

    func encode(w io.Writer, v interface{}) {
            json.NewEncoder(w).Encode(v)
    }

    func login(client *pusher.Client) http.HandlerFunc {
            return func(w http.ResponseWriter, r *http.Request) {
                    defer r.Body.Close()

                    var request struct {
                            UserName string `json:"userName"`
                            Password string `json:"password"`
                    }

                    type response struct {
                            Message string `json:"message"`
                            Success bool   `json:"success"`
                    }

                    if r.URL.Path != "/login" {
                            w.WriteHeader(http.StatusNotFound)
                            return
                    }

                    if r.Method != http.MethodPost {
                            w.WriteHeader(http.StatusMethodNotAllowed)
                            return
                    }

                    if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
                            w.WriteHeader(http.StatusBadRequest)
                            encode(w, response{"Invalid request body", false})
                            return
                    }

                    user, ok := validUsers[request.UserName]
                    if !ok {
                            w.WriteHeader(http.StatusBadRequest)
                            encode(w, response{"User not found", false})
                            return
                    }

                    if user.Password != request.Password {
                            w.WriteHeader(http.StatusBadRequest)
                            encode(w, response{"Password does not match", false})
                            return
                    }

                    w.WriteHeader(http.StatusOK)
                    encode(w, response{"Login successful", true})

                    host, _, err := net.SplitHostPort(r.RemoteAddr)
                    if err != nil {
                            fmt.Fprintf(w, "userip: %q is not IP:port", r.RemoteAddr)
                            return
                    }

                    var ip = host

                    if host == "::1" {
                            ip = "127.0.0.1"
                    }

                    client.Trigger("auth", "login", &struct {
                            IP    string `json:"ip"`
                            User  string `json:"user"`
                            Email string `json:"email"`
                    }{
                            User:  request.UserName,
                            IP:    ip,
                            Email: user.Email,
                    })
            }
    }

Run the Go program:

    $ cd $GOPATH/src/github.com/pusher-tutorials/pusher-channels-queue/go
    $ go run main.go

You can try to send requests to the service with cURL by:

    $ curl  -X POST localhost:1400/login -d '{"username" : "admin", "password"  :"admin"}'

This will produce a response such as:

    {"message":"Login successful","success":true}

Building the Node.js email service

We have made progress by publishing the events to Pusher Channels. You can verify that the events are published by looking at the Debug Console of the dashboard.

To build our Node.js email service, we will need to go back to the root directory, pusher-channels-queue. After which we will create the node directory as it will house our Node.js application.

    $ mkdir node

We will need a couple libraries for the application;

  • pusher-js - the NodeJS SDK for Pusher Channels.
  • nodemailer - We need this to send emails.
  • dotenv - We need this to load environment variables from a file.
  • handlebars - We need to dynamically replace contents of the email before sending it. Things like username and IP address come to mind here.
  • fs - We need to be able to read the content of the email template from the filesystem. You can have a look at the email template here.

To install the above, you will need to create a package.json file that contains the following:

    // github.com/pusher-tutorials/pusher-channels-queue/node/package.json
    {
      "dependencies": {
        "dotenv": "^6.2.0",
        "fs": "^0.0.1-security",
        "handlebars": "^4.0.12",
        "nodemailer": "^4.7.0",
        "pusher-js": "^4.3.1"
      }
    }

You will need to run npm install to get install those dependencies.

Since we need to subscribe to Pusher Channels, we need to first include the required values in .env.

    // github.com/pusher-tutorials/pusher-channels-queue/node/.env
    PUSHER_APP_CLUSTER="YOUR_APP_CLUSTER"
    PUSHER_APP_SECURE="1"
    PUSHER_APP_KEY="YOUR_APP_KEY"
    MAILER_EMAIL="you@gmail.com"
    MAILER_PASSWORD="Password"

Then create an index.js file

    // github.com/pusher-tutorials/pusher-channels-queue/node/index.js

    require('dotenv').config();
    const Pusher = require('pusher-js');
    const nodemailer = require('nodemailer');
    const handlebars = require('handlebars');
    const fs = require('fs');

    const pusherSocket = new Pusher(process.env.PUSHER_APP_KEY, {
      forceTLS: process.env.PUSHER_APP_SECURE === '1' ? true : false,
      cluster: process.env.PUSHER_APP_CLUSTER,
    });

    const transporter = nodemailer.createTransport({
      service: 'gmail',
      auth: {
        user: process.env.MAILER_EMAIL,
        pass: process.env.MAILER_PASSWORD,
      },
    });

    const channel = pusherSocket.subscribe('auth');

    channel.bind('login', data => {

      fs.readFile('./index.html', { encoding: 'utf-8' }, function(err, html) {
        if (err) {
          throw err;
        }

        const template = handlebars.compile(html);
        const replacements = {
          username: data.user,
          ip: data.ip,
        };

        let mailOptions = {
          from: '"Pusher Tutorial demo" <foo@example.com>',
          to: data.email,
          subject: 'New login into Pusher tutorials demo app',
          html: template(replacements),
        };

        transporter.sendMail(mailOptions, function(error, response) {
          if (error) {
            console.log(error);
            callback(error);
          }
        });
      });

      console.log(data);
    });

In the above code, we read the contents of index.html and process it like a handlebars template with handlebars.compile(html). This is because we are dynamically replacing {{ username }} and {{ ip }}.

So far, we have not created the index.html . You will need to create the aforementioned file and paste the following contents:

    // github.com/pusher-tutorials/pusher-channels-queue/node/index.html

    <!doctype html>
    <html>
      <head>
        <meta name="viewport" content="width=device-width" />
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
        <title>Simple Transactional Email</title>
        <style>
          /* -------------------------------------
              GLOBAL RESETS
          ------------------------------------- */

          /*All the styling goes here*/

          img {
            border: none;
            -ms-interpolation-mode: bicubic;
            max-width: 100%;
          }

          body {
            background-color: #f6f6f6;
            font-family: sans-serif;
            -webkit-font-smoothing: antialiased;
            font-size: 14px;
            line-height: 1.4;
            margin: 0;
            padding: 0;
            -ms-text-size-adjust: 100%;
            -webkit-text-size-adjust: 100%;
          }

          table {
            border-collapse: separate;
            mso-table-lspace: 0pt;
            mso-table-rspace: 0pt;
            width: 100%; }
            table td {
              font-family: sans-serif;
              font-size: 14px;
              vertical-align: top;
          }

          /* -------------------------------------
              BODY & CONTAINER
          ------------------------------------- */

          .body {
            background-color: #f6f6f6;
            width: 100%;
          }

          /* Set a max-width, and make it display as block so it will automatically stretch to that width, but will also shrink down on a phone or something */
          .container {
            display: block;
            Margin: 0 auto !important;
            /* makes it centered */
            max-width: 580px;
            padding: 10px;
            width: 580px;
          }

          /* This should also be a block element, so that it will fill 100% of the .container */
          .content {
            box-sizing: border-box;
            display: block;
            Margin: 0 auto;
            max-width: 580px;
            padding: 10px;
          }

          /* -------------------------------------
              HEADER, FOOTER, MAIN
          ------------------------------------- */
          .main {
            background: #ffffff;
            border-radius: 3px;
            width: 100%;
          }

          .wrapper {
            box-sizing: border-box;
            padding: 20px;
          }

          .content-block {
            padding-bottom: 10px;
            padding-top: 10px;
          }

          .footer {
            clear: both;
            Margin-top: 10px;
            text-align: center;
            width: 100%;
          }
            .footer td,
            .footer p,
            .footer span,
            .footer a {
              color: #999999;
              font-size: 12px;
              text-align: center;
          }

          /* -------------------------------------
              TYPOGRAPHY
          ------------------------------------- */
          h1,
          h2,
          h3,
          h4 {
            color: #000000;
            font-family: sans-serif;
            font-weight: 400;
            line-height: 1.4;
            margin: 0;
            margin-bottom: 30px;
          }

          h1 {
            font-size: 35px;
            font-weight: 300;
            text-align: center;
            text-transform: capitalize;
          }

          p,
          ul,
          ol {
            font-family: sans-serif;
            font-size: 14px;
            font-weight: normal;
            margin: 0;
            margin-bottom: 15px;
          }
            p li,
            ul li,
            ol li {
              list-style-position: inside;
              margin-left: 5px;
          }

          a {
            color: #3498db;
            text-decoration: underline;
          }

          /* -------------------------------------
              BUTTONS
          ------------------------------------- */
          .btn {
            box-sizing: border-box;
            width: 100%; }
            .btn > tbody > tr > td {
              padding-bottom: 15px; }
            .btn table {
              width: auto;
          }
            .btn table td {
              background-color: #ffffff;
              border-radius: 5px;
              text-align: center;
          }
            .btn a {
              background-color: #ffffff;
              border: solid 1px #3498db;
              border-radius: 5px;
              box-sizing: border-box;
              color: #3498db;
              cursor: pointer;
              display: inline-block;
              font-size: 14px;
              font-weight: bold;
              margin: 0;
              padding: 12px 25px;
              text-decoration: none;
              text-transform: capitalize;
          }

          .btn-primary table td {
            background-color: #3498db;
          }

          .btn-primary a {
            background-color: #3498db;
            border-color: #3498db;
            color: #ffffff;
          }

          /* -------------------------------------
              OTHER STYLES THAT MIGHT BE USEFUL
          ------------------------------------- */
          .last {
            margin-bottom: 0;
          }

          .first {
            margin-top: 0;
          }

          .align-center {
            text-align: center;
          }

          .align-right {
            text-align: right;
          }

          .align-left {
            text-align: left;
          }

          .clear {
            clear: both;
          }

          .mt0 {
            margin-top: 0;
          }

          .mb0 {
            margin-bottom: 0;
          }

          .preheader {
            color: transparent;
            display: none;
            height: 0;
            max-height: 0;
            max-width: 0;
            opacity: 0;
            overflow: hidden;
            mso-hide: all;
            visibility: hidden;
            width: 0;
          }

          .powered-by a {
            text-decoration: none;
          }

          hr {
            border: 0;
            border-bottom: 1px solid #f6f6f6;
            Margin: 20px 0;
          }

          /* -------------------------------------
              RESPONSIVE AND MOBILE FRIENDLY STYLES
          ------------------------------------- */
          @media only screen and (max-width: 620px) {
            table[class=body] h1 {
              font-size: 28px !important;
              margin-bottom: 10px !important;
            }
            table[class=body] p,
            table[class=body] ul,
            table[class=body] ol,
            table[class=body] td,
            table[class=body] span,
            table[class=body] a {
              font-size: 16px !important;
            }
            table[class=body] .wrapper,
            table[class=body] .article {
              padding: 10px !important;
            }
            table[class=body] .content {
              padding: 0 !important;
            }
            table[class=body] .container {
              padding: 0 !important;
              width: 100% !important;
            }
            table[class=body] .main {
              border-left-width: 0 !important;
              border-radius: 0 !important;
              border-right-width: 0 !important;
            }
            table[class=body] .btn table {
              width: 100% !important;
            }
            table[class=body] .btn a {
              width: 100% !important;
            }
            table[class=body] .img-responsive {
              height: auto !important;
              max-width: 100% !important;
              width: auto !important;
            }
          }

          /* -------------------------------------
              PRESERVE THESE STYLES IN THE HEAD
          ------------------------------------- */
          @media all {
            .ExternalClass {
              width: 100%;
            }
            .ExternalClass,
            .ExternalClass p,
            .ExternalClass span,
            .ExternalClass font,
            .ExternalClass td,
            .ExternalClass div {
              line-height: 100%;
            }
            .apple-link a {
              color: inherit !important;
              font-family: inherit !important;
              font-size: inherit !important;
              font-weight: inherit !important;
              line-height: inherit !important;
              text-decoration: none !important;
            }
            .btn-primary table td:hover {
              background-color: #34495e !important;
            }
            .btn-primary a:hover {
              background-color: #34495e !important;
              border-color: #34495e !important;
            }
          }

        </style>
      </head>
      <body class="">
        <table role="presentation" border="0" cellpadding="0" cellspacing="0" class="body">
          <tr>
            <td>&nbsp;</td>
            <td class="container">
              <div class="content">

                <!-- START CENTERED WHITE CONTAINER -->
                <table role="presentation" class="main">

                  <!-- START MAIN CONTENT AREA -->
                  <tr>
                    <td class="wrapper">
                      <table role="presentation" border="0" cellpadding="0" cellspacing="0">
                        <tr>
                          <td>
                            <p>Hi {{ username }},</p>
                            <p>You’ve successfully signed into the demo app.</p>
                            <p>You signed in from the IP address, {{ ip }}</p>
                            <table role="presentation" border="0" cellpadding="0" cellspacing="0" class="btn btn-primary">
                              <tbody>
                                <tr>
                                  <td align="left">
                                    <table role="presentation" border="0" cellpadding="0" cellspacing="0">
                                      <tbody>
                                        <tr>
                                          <td> <a href="https://pusher.com"
                                                          target="_blank">Visit
                                                          Pusher</a> </td>
                                        </tr>
                                      </tbody>
                                    </table>
                                  </td>
                                </tr>
                              </tbody>
                            </table>
                          </td>
                        </tr>
                      </table>
                    </td>
                  </tr>

                <!-- END MAIN CONTENT AREA -->
                </table>


              <!-- END CENTERED WHITE CONTAINER -->
              </div>
            </td>
            <td>&nbsp;</td>
          </tr>
        </table>
      </body>
    </html>

We listen for the login event and pick out the important data from there. In this case, the user’s name and IP address from which they logged in. After which we send the email to the user.

You will need to start the Node.js service by running node index.js. After doing that, you can send login requests to the Golang service again.

You should check your email:

Please note that you might need to allow “Insecure apps”. Please visit https://support.google.com/accounts/answer/6010255?hl=en

Conclusion

In this tutorial, we have leveraged Pusher Channels as a messaging queue between two different applications. While we used this to send email notifications, we can use this for much more interesting patterns depending on your application’s needs.

The entire source code of this tutorial can be found on GitHub.

Clone the project repository
  • Go
  • JavaScript
  • Node.js
  • Channels

Products

  • Channels
  • Beams
  • Chatkit

© 2019 Pusher Ltd. All rights reserved.

Pusher Limited is a company registered in England and Wales (No. 07489873) whose registered office is at 160 Old Street, London, EC1V 9BW.