Building a realtime feed with Node.js and AMP

Something we can’t deny about Accelerated Mobile Pages (AMP) is that they are fast.

On mobile devices, web pages can take several seconds to load. However, AMPs are web pages designed to load near instantaneously due to the limited set of allowed functionality they provide and a distribution system built around caching.

AMP also uses custom elements called AMP components to implement features with complex interactions such as video players or embeddable content.

In this tutorial, we’re going to implement a realtime feed in AMP using two of these components:

• amp-live-list, which provides a way to display and update content live so you can easily implement a live blog.
• amp-youtube, which displays YouTube video.

In the backend, we’ll use a Node.js/Express web server with EJS as the template engine.

This is how the final application will look:

The amp-live-list component will regularly poll the server for updated content. When a new item is added, a button will be shown, which when clicked, will update the page to show the latest videos (with a simple pagination).

You can find the entire source code of the application on this GitHub repository. Also, you need to take into account that realtime features on AMP pages are limited and you can do a lot more using Pusher in your website or app.

Setting up the project

Let’s start by creating a package.json file for our project:

bash
1npm init
Copy

Or if you want to accept all the defaults:

bash
1npm init -y
Copy

Next, install body-parser, ejs, express, and moment as dependencies:

bash
1npm install body-parser ejs express moment
Copy

The project will have the following structure:

bash
1|— views
2| |— index.ejs
3|- package.json
4|- server.js
Copy

• The view directory contains the EJS template for the main page of the app.
• In the root directory, we can find the package.json file, as well as the file for the Express server (server.js).

Once you have created the necessary files and directory, you’ll be ready to start coding.

Building the Express server

The web server is a standard Express app with an array to hold the information of the videos we’re going to show.

We will have two endpoints:

• /, with an optional page query parameter, which will render the template with the list of videos (paginated).
• /new, which will register a new video.

In the server.js file, let’s start by requiring the modules we’re going to use:

javascript
1const express = require('express');
2    const bodyParser = require('body-parser');
3    const moment = require('moment');
Copy

Next, define the array for the videos and a constant for the number of posts that will be shown on a page:

javascript
1const posts = [];
2    const pageSize = 5;
Copy

Now let’s configure the Express server to use EJS as the view engine and the JSON parsing:

javascript
1const app = express();
2    app.set('view engine', 'ejs');
3    app.use(bodyParser.json());
4    app.use(bodyParser.urlencoded({ extended: true }));
Copy

This is the definition of the route that will present the videos:

javascript
1app.get('/', (req, res) => {
2      let currentPage = 1;
3      const totalPosts = posts.length;
4      const pageCount = Math.ceil(totalPosts / pageSize);
5
6      if(req.query.page) {
7        currentPage = parseInt(req.query.page, 10);
8      }
9
10      const start = (currentPage - 1) * pageSize;
11      const end = currentPage * pageSize;
12
13      res.render('index', 
14        {
15              posts: posts.slice(start, end),
16              pageSize: pageSize,
17              pageCount: pageCount,
18              currentPage: currentPage,
19        }
20      );
21    });
Copy

As you can see, we are implementing simple pagination, calculating the page count, and the start and end elements of the post array that will be shown based on the optional page query parameter.

Then, in the last lines, the index template is rendered, passing an object with the properties we’ll need.

This is the definition of the route to registering a new video:

javascript
1app.post('/new', (req, res) => {
2      const post = {
3              id: posts.length + 1,
4              videoID: req.body.videoID,
5              timestamp: moment().unix(),
6              timestampStr: moment().format(),
7      };
8
9      posts.unshift(post);
10
11      res.status(200).json(post);
12    });
Copy

We create a new object with an ID based on the length of the array of posts, the ID of the video to show (req.body.videoID), and a timestamp in Unix and ISO 8601 format (for example, 2017-06-09T08:02:17-05:00). Then we add this object at the beginning of the array with unshift and return a success response.

Finally, let’s start the server with:

javascript
1app.listen(3000, () => {
2      console.log('Server listening on port 3000');
3    });
Copy

Building the live blog template

An AMP HTML page will act as the content of the template. A good starting point is the boilerplate code you can find on the Create Your First AMP Page official tutorial.

For our app, we’ll use a modified version of that code. Open the file views/index.ejs and copy the following code:

html
1<!doctype html>
2    <html amp lang="en">
3      <head>
4        <meta charset="utf-8">
5        <script async src="https://cdn.ampproject.org/v0.js"></script>
6        <title>AMP Live Video Blog</title>
7        <link rel="canonical" href="http://localhost/" />
8        <meta name="viewport" content="width=device-width,minimum-scale=1,initial-scale=1">
9
10        <link href='https://fonts.googleapis.com/css?family=Roboto' rel='stylesheet'>
11
12        <script async custom-element="amp-youtube" src="https://cdn.ampproject.org/v0/amp-youtube-0.1.js"></script>
13        <script async custom-element="amp-live-list" src="https://cdn.ampproject.org/v0/amp-live-list-0.1.js"></script>
14
15        <style amp-boilerplate>body{-webkit-animation:-amp-start 8s steps(1,end) 0s 1 normal both;-moz-animation:-amp-start 8s steps(1,end) 0s 1 normal both;-ms-animation:-amp-start 8s steps(1,end) 0s 1 normal both;animation:-amp-start 8s steps(1,end) 0s 1 normal both}@-webkit-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-moz-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-ms-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-o-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}</style><noscript><style amp-boilerplate>body{-webkit-animation:none;-moz-animation:none;-ms-animation:none;animation:none}</style></noscript>
16
17      </head>
18      <body>
19
20      <body>
21    </html>
Copy

For this example, we’re not going to need the JSON metadata from Schema.org. It’s optional but required to make your content eligible to appear in the Google News carousel.
However, we need to keep the canonical link reference, as it is required for the page to be valid. This link must point to itself (the AMP version of the page), so it’s just changed to:

html
1<link rel="canonical" href="http://localhost:3000/" />
Copy

Refer to this page to know more about both options.

AMP allows the use of custom fonts with some restrictions. This page uses the Roboto font:

html
1<link href='https://fonts.googleapis.com/css?family=Roboto' rel='stylesheet'>
Copy

All AMP components used must be imported in the header. The following are the scripts for the YouTube and live list components we’re going to use:

html
1<script async custom-element="amp-youtube" src="https://cdn.ampproject.org/v0/amp-youtube-0.1.js"></script>
2    <script async custom-element="amp-live-list" src="https://cdn.ampproject.org/v0/amp-live-list-0.1.js"></script>
Copy

In addition to the boilerplate style, we’re going to use some custom CSS styles. Since AMP can’t reference external stylesheets, all styles must live in the head of the document with some restrictions, so let’s add our custom CSS inside a <style amp-custom> tag in the head of the document:

html
1<!doctype html>
2    <html amp lang="en">
3      <head>
4        ...
5
6        <style amp-custom>
7          body {
8            font-family: 'Roboto', sans-serif;
9            background: #EDEDED;
10          }
11          .header {
12            color: #fff;
13            padding: 20px  20px;
14            top: 0px;
15            width: 100%;
16            background: #E53935;
17            text-align:center;
18          }
19          #amp-live-list-videos {
20            margin-bottom: 50px;
21          }
22          #live-list-update-button {
23            color: #E53935;
24            border: 1px solid #E53935;
25            padding: .7em .8em;
26            cursor: pointer;
27            background-color: #fff;
28            text-transform: uppercase;
29            letter-spacing: .2em;
30          }
31          .post {
32            margin-top: 20px;
33          }
34          #live-list-update-button {
35            position: fixed;
36            top: 10px;
37            right: 16px;
38          }
39          .pagination {
40            display: inline-block;
41            padding: 0;
42          }
43          amp-live-list [pagination] nav {
44            display: flex;
45            align-items: center;
46            justify-content: center;
47          }
48          .pagination li {
49            display: inline;
50            padding: 10px;
51            list-style-type: none;
52            color: #000;
53          }
54        </style>
55
56      </head>
57      ...
58    </html>
Copy

Now, in the body section, let’s add the page header:

html
1<!doctype html>
2    <html amp lang="en">
3      <head>
4        ...
5      </head>
6      <body>
7        <div class="header">
8          <h1>AMP Live Video Blog</h1>
9        </div>
10      </body>
11    </html>
Copy

And the amp-live-list component:

html
1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            id="amp-live-list-videos"
8            data-poll-interval="15000"
9            data-max-items-per-page="<%= pageSize %>"
10            <% if (currentPage > 1) { %>  disabled  <% } %>
11        >
12
13        </amp-live-list>
14      </body>
15    </html>
Copy

The id attribute identifies the amp-live-list component since multiple live lists are allowed on a single page.
The data-poll-interval attribute specifies how frequently the component will poll new data (in milliseconds). The minimum interval is fifteen seconds.
The data-max-items-per-page specifies the maximum number of items displayed on the page.
If we’re not on the first page, we’re going to add the disable attribute so the component stops polling for new data. (Since new posts will be added to the first page due to how the pagination mechanism works, there’s no need to poll the server if the user isn’t on that page.)

In this example, the amp-live-list component contains three elements as direct children.

The first element is a button with the event on="tap:[ID_OF_THE_LIVE_LIST_COMPONENT].update" to load the latest changes from the server. This button is required and it will appear when new changes are discovered. These changes won’t be shown to the users until they click the button:

html
1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            ...
8        >
9          <button
10              id ="live-list-update-button" 
11              update on="tap:amp-live-list-videos.update" >
12            You have updates
13          </button>
14
15        </amp-live-list>
16      </body>
17    </html>
Copy

The second element is <div items>, the list of items to show. These items are required to have a unique id and a data-sort-time attribute (a timestamp used for sorting entries. Higher timestamps will be inserted before older entries):

html
1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            ...
8        >
9          ...
10
11          <div items>
12          <% posts.forEach((post) => { %>
13
14            <div class="post" id="<%= post.id %>" data-sort-time="<%= post.timestamp %>">
15              <amp-youtube
16                width="480"
17                height="270"
18                layout="responsive"
19                data-videoid="<%= post.videoID %>"
20              />
21            </div>
22
23          <% }) %>
24          </div>
25
26        </amp-live-list>
27      </body>
28    </html>
Copy

For each post, we’re using an amp-youtube component to render the video. You can find out more about this component here.

Finally, the third element is <div pagination></div> element. It’s our responsibility to insert within this element any markup needed for pagination (which in this case is only shown if there’s more than one page):

html
1<!doctype html>
2    <html amp lang="en">
3      ...
4      <body>
5        ...
6        <amp-live-list 
7            ...
8        >
9          ...
10
11          <div pagination>
12            <nav aria-label="amp live list pagination">
13
14              <% if (pageCount > 1) { %>
15              <ul class="pagination">
16
17                <% for (var i = 1; i <= pageCount; i++) { %>
18                  <% if (currentPage == i) { %>
19                    <li><%= i %></li>
20                  <% } else { %>
21                    <li><a href="/?page=<%= i %>"><%= i %></a></li>
22                  <% } %>
23                <% } %>
24
25              </ul>
26              <% } %>  
27            </nav>
28          </div>
29
30        </amp-live-list>
31      </body>
32    </html>
Copy

And we’re done. You can find out more about how the amp-live-list component works on this page.

Testing the application

Execute the web server with:

bash
1node server.js
Copy

If you go to http://localhost:3000, you’ll see a blank page:

It’s recommended you use Chrome because you can validate an AMP page just by adding #development=1 to the URL, for example, http://localhost:3000/?page=1#development=1 and view the result of the validation in the Chrome DevTools console:

To register videos, you can use a tool like Postman to hit the http://localhost:3000/new endpoint with a JSON payload like the following:

javascript
1{
2      "videoID": "[YOUTUBE_VIDEO_ID]"
3    }
Copy

Or curl:

javascript
1# POST
2    curl -H "Content-Type: application/json" -X POST -d '{"videoID":"[YOUTUBE_VIDEO_ID]"}' http://localhost:3000/new
3
4    # In Windows, change single quotes to quotation marks and escape the ones inside curly brackets
5    curl -H "Content-Type: application/json" -X POST -d "{\"videoID\":\"[YOUTUBE_VIDEO_ID]\"}" http://localhost:3000/new
6
7    # Or use file, for example data.json
8    curl -H "Content-Type: application/json" -X POST --data @data.json http://localhost:3000/new
Copy

After a few seconds (fifteen at most), the button to load the new videos should be visible:

And after adding more than five videos, the pagination links will be shown:

In addition, if you look at the network tab of the console, you’ll see that a request to the server is made every fifteen seconds:

Conclusion

You’ve now learned how to implement a realtime feed with pagination on AMP using the amp-live-list component. We have covered insertions, but updates and deletions are also supported by this component. Here’s an example of how to update existing list items.

Remember that you can find the source code of this app in this GitHub repository.