Build a realtime table with DataTables and Nest.js

If you have been around the web for sometime now, you will agree with me that extensive data from the database can mostly be perfectly rendered to users by using tables. This is where tools like DataTables really shines, when initialized, it will immediately add more features like searching, ordering and paging to tables.

It drives further than just a preview of data by users. It also brings about an excellent user experience when you add a realtime functionality in order to make changes available to consumers in realtime.

This tutorial will guide you through the process of implementing a realtime feature in tables. The knowledge gained here, will amongst other things help you implement realtime functionality in any web application. Pusher provides awesome tools to make implementing realtime functionality easy on any platform.

The backend of the application will be powered by Nest.js. A progressive Node.js framework for building efficient and scalable server-side applications, Nest.js leverages TypeScript to create reliable and well structured server-side applications.

A quick look at what we'll be building:

Prerequisites

A basic understanding of TypeScript and Node.js will help you get the best out of this tutorial. It is assumed that you already have Node and npm installed. Kindly check Node.js and npm for further instructions and installation steps.

Setting up the application

As at the time of writing, there was no CLI to setup a Nest.js application. The simplest and most efficient way is to clone the starter repository made available on their website.

Now let’s run a command that will create a new project folder named realtime-table-nest-pusher on your machine. Open your terminal or command prompt and run this command :

    $ git clone https://github.com/nestjs/typescript-starter.git realtime-table-nest-pusher

Go ahead and change directory into the newly created folder and install all the dependencies for the project.

bash
1// change directory
2    cd realtime-table-nest-pusher
3    
4    // install dependencies
5    npm install
Copy

See it working

Run the application with :

    npm start

The command above will start the application on the default port used by Nest.js. Open your browser and navigate to [http://localhost:3000]. You should see a welcome message like what we have below

Server dependencies

Run the command below to install the server dependencies required for this project.

    npm install ejs body-parser pusher

• ejs: this is a simple templating language for generating HTML markup with plain JavaScript.

• Body-parser: a middleware used for extracting the entire body portion of an incoming request stream and expose it on req.body .

• Pusher: a Node.js client to interact with the Pusher REST API

Pusher account setup

Head over to Pusher and sign up for a free account. This is important as it is required before you can have access to an API key and easily access all the awesome features offered by Pusher.

Create a new app by selecting Channels apps on the sidebar and clicking Create Channels app button on the bottom of the sidebar:

Configure an app by providing basic information requested in the form presented. You can also choose the environment you intend to integrate Pusher with for a better setup experience:

You can retrieve your keys from the App Keys tab:

Bootstrap application

Under the hood, Nest uses the Express library and therefore, favors the popular MVC pattern.

To set this up, open up main.ts file and update it with the content below:

typescript
1// ./src/main.ts
2    
3    import { NestFactory } from '@nestjs/core';
4    import { AppModule } from './app.module';
5    
6    import * as bodyParser from 'body-parser';
7    import * as express from 'express';
8    import * as path from 'path';
9    
10    async function bootstrap() {
11      const app = await NestFactory.create(AppModule);
12      app.use(express.static(path.join(__dirname, 'public')));
13       app.set('views', __dirname + '/views');
14       // set ejs as the view engine
15       app.set('view engine', 'ejs');
16      await app.listen(3000);
17    }
18    bootstrap();
Copy

This is the entry point of the application and necessary for bootstrapping Nest.js apps. I have included the Express module, path and set up ejs as the view engine for the application.

Datatables setup

Generally, getting DataTable running on any website or app require little or less configuration as it is easy to set up and quite straight forward. All that is required is to include the necessary links on our HTML page.

For this we will create a folder called views within the src folder. Now go ahead and create a new file named index.ejs within it. In the newly created file, paste in the following code:

html
1<!DOCTYPE html>
2    <html lang="en">
3    <head>
4        <meta charset="UTF-8">
5        <meta name="viewport" content="width=device-width, initial-scale=1.0">
6        <meta http-equiv="X-UA-Compatible" content="ie=edge">
7        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
8        <link rel="stylesheet" href="https://cdn.datatables.net/1.10.15/css/jquery.dataTables.min.css">
9        <link rel="stylesheet" href="/style.css">
10        <title> Datatable Realtime </title>
11    </head>
12    <body>
13        <div class="container">
14            <div class="row">
15                <h2 class="text-center">
16                    Realtime Data Table
17                </h2>
18                <div class="col-md-7">
19                    <table id="realtime" class="display" width="100%"></table>
20                </div>
21                
22                <div class="col-md-4 col-md-offset-1">
23                    <h3 class="text-center">Create New Employee</h3>
24                    
25                    <div class="form-group">
26                        <label for="name">Name</label>
27                        <input type="text" name="name" id="name" placeholder="Name" class="form-control">
28                    </div>
29                    <div class="form-group">
30                        <label for="position">Position</label>
31                        <select name="position" id="position" class="form-control">
32                            <option value="">--Select Position--</option>
33                            <option value="Frontend Developer">Frontend Developer</option>
34                            <option value="UI/UX Engineer">UI/UX Engineer</option>
35                            <option value="iOS Engineer">iOS Engineer</option>
36                            <option value="Android Developer">Android Developer</option>
37                        </select>
38                    </div>
39                    <div class="form-group">
40                        <label for="office">Office</label>
41                        <select name="office" id="office" class="form-control">
42                            <option value="">--Select Office--</option>
43                            <option value="Lagos">Lagos</option>
44                            <option value="London">London</option>
45                            <option value="New York">New York</option>
46                            <option value="Berlin">Berlin</option>
47                        </select>
48                    </div>
49                    <div class="form-group">
50                        <label for="extn">Extn</label>
51                        <input type="number" name="extn" id="extn" placeholder="Extn" class="form-control">
52                    </div>
53                    <div class="form-group">
54                        <label for="startDate">Start Date</label>
55                        <input type="date" name="startDate" id="startDate" placeholder="Start Date" class="form-control">
56                    </div>
57                    
58                    <div class="form-group">
59                        <button class="btn btn-info" id="add-employee">Add Employee</button>
60                    </div>
61                </div>
62            </div>
63        </div>
64        
65        <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
66        <script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.16.2/axios.js"></script>
67        <script src="https://js.pusher.com/4.1/pusher.min.js"></script>
68        <script src="https://cdn.datatables.net/1.10.15/js/jquery.dataTables.min.js"></script>
69        <script src="https://cdn.datatables.net/plug-ins/1.10.15/api/row().show().js"></script>
70        <script src="/data.js"></script>
71        <script src="/main.js"></script>
72    </body>
73    </html>
Copy

This will serve as the homepage for this application. In order to make this page look presentable, we included a CDN CSS file each for Bootstrap and DataTable. Further, we’ve included a custom CSS file named style.css. You can download it here from the repository. Once you are done, create a new folder named public within the src folder and save the stylesheet file in it.

In addition, we included a table element identified by an id of #realtime, this element will hold the DataTable. We have also defined relevant input fields that will be used to add values to a dataset that we’ll revisit later in the tutorial.

Also included is a CDN file each for jQuery, Axios, Pusher, and DataTable. And finally, just before the closing tag of the <body> element on the page, we’ve included two script file named data.js and main.js. data.js will contain sample dataset that will be used to initialize our DataTable, while main.js will hold the custom script for the application. Now, go ahead and create this two files and save them in the public folder as we’ll need them later in the tutorial.

Initialize DataTable

Earlier, we inserted a table element with an id #realtime on our page. In order to initialize the DataTable and create a blueprint for our table, we’ll use jQuery to append the DataTable method to the element and passed an object as argument. The object has a data property which takes a dataset . You can download the dataset here and save the content in the data.js that was created earlier. We also include another property called columns and set its value to an array of objects with each object’s value serving as a column header for our table.

javascript
1//   ./src/public/main.js
2      
3    
4    const app = {
5        ...
6        start() {
7          const dataTable = $('#realtime').DataTable({
8            data: dataSet,
9            columns: [
10              { title: 'Name' },
11              { title: 'Position' },
12              { title: 'Office' },
13              { title: 'Extn.' },
14              { title: 'Start date' }
15            ]
16          });
17          ...
18        }
19      };
20      
21      $(document).ready(() => app.start());
Copy

Handling routes

The controller layer in Nest.js is responsible for receiving an incoming request and returning the appropriate response to the client. Nest uses a controller metadata @Controller to map routes to a specific controller. The starter project already contains a controller by default. We will make use of this in order to render the home for this app. Open ./src/app.controller.ts and edit as shown below:

typescript
1// ./src/app.controller.ts
2    
3    import { Get, Controller, Res } from '@nestjs/common';
4    
5    @Controller()
6    export class AppController {
7      @Get()
8      root(@Res() res) {
9        res.render('index');
10      }
11    }
Copy

This controller will ensure that Nest maps every / route to index.ejs file.

Adding new records to the table

We’ve already included the relevant input fields required to add more records to the dataset in the DataTable. To make this work, we’ll use a custom function called buildForm() to retrieve all the values of the input fields. Open ./src/public/main.js and add the function

javascript
1const app = {
2        buildForm() {
3          return [
4            $('#name').val(),
5            $('#position').val(),
6            $('#office').val(),
7            $('#extn').val(),
8            $('#startDate').val().replace(new RegExp('-', 'g'), '/')
9          ];
10        },
11        
12        start() {
13        ...
14        }
15      };
16      
17      $(document).ready(() => app.start());
Copy

In addition we created two more methods processForm() and addRow() . While the former is responsible for processing and passing the formData returned by buildForm() to the server, the latter takes in two arguments and handles the addition of realtime data to DataTable.

javascript
1// ./src/public/main.js
2    
3    const app = {
4        buildForm() {
5          ...
6        },
7        
8        processForm() {
9          const formData = this.buildForm();
10          const baseURL = 'http://localhost:3000';
11          axios.post(`${baseURL}/record`, formData)
12            .then(response => console.log(response));
13        },
14        
15        start() {
16          ...
17        }
18      };
19      
20      $(document).ready(() => app.start());
Copy

Passed the formData to the server, we will set this up in a bit:

javascript
1// ./src/public/main.js
2    
3    
4    const app = {
5        buildForm() {
6         ...
7        },
8        
9        processForm() {
10          ...
11        },
12    
13        addRow(dataTable, data) {
14          const addedRow = dataTable.row.add(data).draw();
15          addedRow.show().draw(false);
16      
17          const addedRowNode = addedRow.node();
18          $(addedRowNode).addClass('highlight');
19        },
20    
21        start() {
22        ...      
23        }
24      };
25      
26      $(document).ready(() => app.start());
Copy

As stated earlier, this method takes in dataTable instance and the newly added data as arguments.

The methods row.add() and .draw() are inbuilt DataTables API methods, other DataTables methods implemented in addRow() are .show(), .draw(false) and .node():

• row.add() adds a new row to the table using the given data.
• .draw() redraws and updates the table in the current context.
• .show() displays a field in our table. This is useful for cases when you want to have extra form fields available, but only show them under certain conditions.
• .draw(false) adds a new row without resetting or distorting the current page.
• .node() serves as an event listener, it returns the DOM element for the requested field thus enabling DOM manipulation of the field.

We then take our processForm() method which we built and bind it to a button using jQuery’s .click() method. When the button is clicked, addRow() automatically executes its functions on our table.

javascript
1const app = {
2        buildForm() {
3          ...
4        },
5        
6        processForm() {
7          ...
8        },
9    
10        addRow(dataTable, data) {
11         ...
12        },
13    
14        start() {
15          ...
16          });
17          
18          // bind the processForm() method to a button
19          $('#add-employee').on('click', this.processForm.bind(this));
20        }
21      };
22      
23      $(document).ready(() => app.start());
Copy

Find the complete custom script here.

Create a controller

Earlier, we already configure the app.controller.ts to render the homepage and display the form for consumers. The next thing we need to do is build the controller that will handle the data posted to the server. Create a new folder named table in the src folder and create a new file called table.controller.ts within it.

typescript
1import { Controller, Post, Res, Body, HttpStatus } from '@nestjs/common';
2    import { TableService } from './table.service';
3    
4    
5    @Controller('record')
6    export class TableController {
7        constructor(private tableService:TableService){}
8    
9        @Post()
10        addNewRecord(@Res() res, @Body() data: String) {
11            this.tableService.add(data);
12            res.status(HttpStatus.OK).send('Pushed');
13        }
14    
15    }
Copy

One of the most important modules imported here is the TableService . It was injected into the controller through the constructor. As recommended by Nest a controller should handle only HTTP requests and abstract any complex logic to a service.

Realtime service with Pusher

As required within the TableController above, lets create a component as a service. This will basically receive the formData and publish it to a designated channel for the client side to listen and subscribe to. So create a new file within table folder named table.service.ts

typescript
1// ./src/table/table.service.ts
2    
3    import { Component } from '@nestjs/common';
4    
5    @Component()
6    export class TableService {
7        add(newEmployee) {
8            const Pusher = require('pusher');
9    
10            var pusher = new Pusher({
11                appId: 'YOUR_APP_ID',
12                key: 'YOUR_API_KEY',
13                secret: 'YOUR_SECRET_KEY',
14                cluster: 'YOUR_CLUSTER',
15                encrypted: true
16              });
17    
18              pusher.trigger('employees', 'new-employee', newEmployee);
19        }
20    }
Copy

Here we have initialized Pusher with the required credentials in order to be able to trigger an event named new-employee through a channel named employees.

Connecting the dots

To make this work, both the TableController and TableService needs to be registered within the app.module.ts file.

typescript
1// ./src/app.module.ts
2    
3    import { TableService } from './table/table.service';
4    import { TableController } from './table/table.controller';
5    import { Module } from '@nestjs/common';
6    import { AppController } from './app.controller';
7    
8    @Module({
9      imports: [],
10      controllers: [AppController, TableController], // add controller
11      components: [TableService], // add service
12    })
13    export class AppModule {}
Copy

Updating the UI

To update the table once a form is submitted, in our main.js file (client) we used the subscribe() method from Pusher to subscribe to the created employees channel.

javascript
1const app = {
2        buildForm() {
3          ...
4        },
5        
6        processForm() {
7         ...
8        },
9    
10        addRow(dataTable, data) {
11          ...
12        },
13    
14        start() {
15          ...
16          // subscribe to a channel
17          var pusher = new Pusher('YOUR_API_KEY', {
18            cluster: 'CLUSTER',
19            encrypted: true
20          });
21      
22          var channel = pusher.subscribe('employees');
23          channel.bind('new-employee', (data) => {
24            this.addRow(dataTable, data);
25          });
26        }
27      };
28      
29      $(document).ready(() => app.start());
Copy

Bringing it all together

Restart the development server if it is currently running. Check your page on [http://localhost:3000.](http://localhost:3000.)

Conclusion

So far, we have learnt how to leverage on the realtime functionality provided by Pusher to add more records to existing data. Feel free to explore more by visiting Pusher’s documentation. And lastly, the complete source code of this demo application can be found here on github.