Realtime Tweet Statistics with Pusher and Node.js

This article is part of the Building Realtime Apps Tutorials series, updated on a regular basis.

Every so often at Pusher we hold a hack event to give everyone a chance to try something new; to take a risk they might not otherwise take, or simply to learn something they’ve wanted to learn.

In our latest hack event I decided to build something that’s been sitting in my notebook for a couple of years now—I wanted to build a realtime dashboard for monitoring and comparing keywords on Twitter.

It took a little longer than expected (as does everything) but it works great and has already proven incredibly useful in monitoring the trends of keywords throughout the day.

In true Pusher fashion, I’ve released both the API server and graph demo for you to take and use as you wish. We’d love to hear what you make with it!

• Demo: <http://tweet-stats.herokuapp.com\>
• API code: <https://github.com/pusher/tweet-stats-api\>
• Graph demo code: <https://github.com/pusher/tweet-stats-demo\>

Technology used

The realtime tweet statistics API and graphs use a variety of technologies. Here’s a selection of the most important:

Node.js is my go-to technology when quickly putting something together. Both the API server and front-end demo use Node.js to act as a data processor and Web server.

The Pusher Node.js library is used to hook Pusher’s realtime API into the Node.js server.

The Node.js server itself is powered by Express, allowing you to quickly build up a REST API without any hassle.

Tweets are retrieved from the Twitter Streaming API using the Twitter Node.js package.

Bower is used to manage the front-end libraries that need to be installed. It can be quite an odd experience if you’re not used to it, though it’s worth trying to see if you like it.

Realtime graphs are produced using the Epoch library. It’s great for throwing in large quantities of realtime data and not having to worry about building a performant method of visualisation.

Finally, front-end magic is performed using the CSS flexbox to allow a dynamic number of graphs to fit perfectly in any size window. Nifty!

Getting started – API

The first thing you need to do is set up the API server so you can start scraping and counting tweets. There are a few steps but it shouldn’t take too long:

• Create a free Pusher account
• Create a Twitter application
• Generate access tokens on the “Keys and Access Tokens” tab
• Create a copy of config.example.js and name it config.js
• Fill the configuation options with the values from Pusher and Twitter
• Add the keywords you want to track as an array of strings in config.js
• Install the dependencies by running npm install
• Test the API locally by running node index.js and checking an API endpoint
• Upload the API somewhere public (this is already set up for Heroku)

Getting started – Demo

The final step is set up the demo so you can see the results of your hard work:

• Install the server dependencies by running npm install
• Install the front-end dependencies by running bower install
• Change the Pusher application key in /public/js/dashboard.js
• Change the API endpoint in /public/js/dashboard.js
• Test the demo locally by running node index.js and checking the graphs
• Upload the demo somewhere public

Bask in the glory of your realtime tweet dashboard

At this point you should already be enjoying monitoring your favourite keywords on Twitter. However this is just the beginning; there are countless ways to take this to the next level.

Have you used this? Or perhaps you’ve extended it to do something new? Let us know about it!

For more about building realtime applications, check out our tutorials library, updated on a regular basis.