Building a realtime analytics dashboard for a Laravel application

In today’s world, it’s important for website administrators and webmasters to have useful data regarding issues such as the performance of their sites. This helps them to be proactive in tackling issues with their sites. In this tutorial, we’ll build a middleware that logs all requests made to our application and pushes updated analytics on those in realtime to a dashboard. Here’s a preview of our app in action:

Prerequisites

• PHP 7.2 or higher, with the MongoDB driver installed. You can find installation instructions here.
• Composer
• MongoDB (version 3.4 or higher). Get it here.
• A Pusher account. Create one here.

Setting up the app

Laravel by default uses SQL databases as the backend for its Eloquent models, but we’re using MongoDB in this project, so we’ll start off with a Laravel installation configured to use MongoDB. Clone the repo by running:

    git clone https://github.com/shalvah/laravel-mongodb-starter.git

You can also download the source directly from this link.

Then cd into the project folder and install dependencies:

    composer install

Lastly, copy the .env.example to a new file called .env. Run the following command to generate an application encryption key:

    php artisan key:generate

Note: If your MongoDB server requires a username and password, add those in your .env file as the DB_USERNAME and DB_PASSWORD respectively.

Logging all requests

We’ll create a middleware that logs every request to our database. Our middleware will be an "after” middleware, which means it will run after the request has been processed but just before sending the response. We’ll store the following details:

• The relative URL (for instance, /users)
• The HTTP method (for instance, “GET”)
• The time it took to respond to the request
• The day of the week
• The hour of day,

Let’s create our RequestLog model. Create the file app/Models/RequestLog.php with the following content:

php
1<?php
2    
3    namespace App\Models;
4    
5    use Jenssegers\Mongodb\Eloquent\Model;
6    
7    class RequestLog extends Model
8    {
9        protected $guarded = [];
10    }
Copy

Then create the file app/Http/Middleware/RequestLogger.php with the following content:

php
1<?php
2    
3    namespace App\Http\Middleware;
4    
5    use App\Models\RequestLog;
6    use Carbon\Carbon;
7    use Closure;
8    
9    class RequestLogger
10    {
11        public function handle(\Illuminate\Http\Request $request, Closure $next)
12        {
13            $response = $next($request);
14    
15            if ($request->routeIs('analytics.dashboard')) {
16                return $response;
17            }
18    
19            $requestTime = Carbon::createFromTimestamp($_SERVER['REQUEST_TIME']);
20            $request = RequestLog::create([
21                'url' => $request->getPathInfo(),
22                'method' => $request->method(),
23                'response_time' => time() - $requestTime->timestamp,
24                'day' => date('l', $requestTime->timestamp),
25                'hour' => $requestTime->hour,
26            ]);
27    
28            return $response;
29        }
30    }
Copy

The if condition above prevents us from logging anything if the request is to view the analytics dashboard.

Now, let’s attach the middleware to all our routes. In your app/Http/Kernel.php, add the middleware class to the $middleware array:

php
1protected $middleware = [
2        // ...
3        // ...
4        \App\Http\Middleware\RequestLogger::class,
5    ];
Copy

We won’t be using any authentication in our app, but we need some of the frontend scaffolding Laravel provides, so we’ll run this command:

    php artisan make:auth

Displaying our analytics

Let’s add the route for the analytics dashboard. Add the following to the end of your routes/web.php:

    Route::get('/analytics', 'AnalyticsController@index')->name('analytics.dashboard');

Next, we’ll create a class that retrieves our analytics using MongoDB aggregations. Create the file app/Services/AnalyticsService.php with the following content:

php
1<?php
2    
3    namespace App\Services;
4    
5    
6    use App\Models\RequestLog;
7    use Jenssegers\Mongodb\Collection;
8    
9    class AnalyticsService
10    {
11    
12        public function getAnalytics()
13        {
14    
15            $perRoute = RequestLog::raw(function (Collection $collection) {
16                return $collection->aggregate([
17                    [
18                        '$group' => [
19                            '_id' => ['url' => '$url', 'method' => '$method'],
20                            'responseTime' => ['$avg' => '$response_time'],
21                            'numberOfRequests' => ['$sum' => 1],
22                        ]
23                    ]
24                ]);
25            });
26            $requestsPerDay = RequestLog::raw(function (Collection $collection) {
27                return $collection->aggregate([
28                    [
29                        '$group' => [
30                            '_id' => '$day',
31                            'numberOfRequests' => ['$sum' => 1]
32                        ]
33                    ],
34                    ['$sort' => ['numberOfRequests' => 1]]
35                ]);
36            });
37            $requestsPerHour = RequestLog::raw(function (Collection $collection) {
38                return $collection->aggregate([
39                    [
40                        '$group' => [
41                            '_id' => '$hour',
42                            'numberOfRequests' => ['$sum' => 1]
43                        ]
44                    ],
45                    ['$sort' => ['numberOfRequests' => 1]]
46                ]);
47            });
48            return [
49                'averageResponseTime' => RequestLog::avg('response_time'),
50                'statsPerRoute' => $perRoute,
51                'busiestDays' => $requestsPerDay,
52                'busiestHours' => $requestsPerHour,
53                'totalRequests' => RequestLog::count(),
54            ];
55        }
56    }
Copy

Here are the analytics we’re gathering:

• averageResponseTime ****is the average time taken by our routes to return a response.
• statsPerRoute contains information specific to each route, such as the average response time and number of requests.
• busiestDays contains a list of all the days, ordered by the number of requests per day.
• busiestHours contains a list of all the hours, ordered by the number of requests per hour.
• totalRequests is the total number of requests we’ve gotten.

Then create the controller, app/Http/Controllers/AnalyticsController.php with the following content:

php
1<?php
2    
3    namespace App\Http\Controllers;
4    
5    use App\Services\AnalyticsService;
6    
7    class AnalyticsController extends Controller
8    {
9        public function index(AnalyticsService $analyticsService)
10        {
11            $analytics = $analyticsService->getAnalytics();
12            return view('analytics', ['analytics' => $analytics]);
13        }
14    }
Copy

Now let’s create the markup. Create the file resources/views/analytics.blade.php with the following content:

php
1@extends('layouts.app')
2    
3    @section('content')
4        <div class="container" id="app">
5            <div class="row">
6                <div class="col-md-5">
7                    <div class="card">
8                    <div class="card-body">
9                        <h5 class="card-title">Total requests</h5>
10                        <div class="card-text">
11                            <h3>@{{ totalRequests }}</h3>
12                        </div>
13                    </div>
14                    </div>
15                </div>
16                <div class="col-md-5">
17                    <div class="card">
18                    <div class="card-body">
19                        <h5 class="card-title">Average response time</h5>
20                        <div class="card-text">
21                            <h3>@{{ averageResponseTime.toFixed(4) }} seconds</h3>
22                        </div>
23                    </div>
24                    </div>
25                </div>
26            </div>
27    
28            <div class="row">
29                <div class="col-md-5">
30                    <div class="card">
31                    <div class="card-body">
32                        <h5 class="card-title">Busiest days of the week</h5>
33                        <div class="card-text" style="width: 18rem;" v-for="day in busiestDays">
34                            <ul class="list-group list-group-flush">
35                                <li class="list-group-item">
36                                    @{{ day._id }} (@{{ day.numberOfRequests }} requests)
37                                </li>
38                            </ul>
39                        </div>
40                    </div>
41                </div>
42                </div>
43                <div class="col-md-5">
44                    <div class="card">
45                    <div class="card-body">
46                        <h5 class="card-title">Busiest hours of day</h5>
47                        <div class="card-text" style="width: 18rem;" v-for="hour in busiestHours">
48                            <ul class="list-group list-group-flush">
49                                <li class="list-group-item">
50                                    @{{ hour._id }} (@{{ hour.numberOfRequests }} requests)
51                                </li>
52                            </ul>
53                        </div>
54                    </div>
55                    </div>
56                </div>
57            </div>
58    
59            <div class="row">
60                <div class="col-md-5">
61                    <div class="card">
62                    <div class="card-body">
63                        <h5 class="card-title">Most visited routes</h5>
64                        <div class="card-text" style="width: 18rem;" v-for="route in statsPerRoute">
65                            <ul class="list-group list-group-flush">
66                                <li class="list-group-item">
67                                    @{{ route._id.method }} @{{ route._id.url }} (@{{ route.numberOfRequests }} requests)
68                                </li>
69                            </ul>
70                        </div>
71                    </div>
72                    </div>
73                </div>
74                <div class="col-md-5">
75                    <div class="card">
76                    <div class="card-body">
77                        <h5 class="card-title">Slowest routes</h5>
78                        <div class="card-text" style="width: 18rem;" v-for="route in statsPerRoute">
79                            <ul class="list-group list-group-flush">
80                                @{{ route._id.method }} @{{ route._id.url }} (@{{ route.responseTime || 0}} s)
81                            </ul>
82                        </div>
83                    </div>
84                    </div>
85                </div>
86            </div>
87        </div>
88    @endsection
89    
90    <script>
91        window.analytics = @json($analytics);
92    </script>
Copy

We’ll be using Vue.js to automatically bind data on the view, so the @{{…}} around expressions above signify to Laravel’s templating engine that these are JavaScript, not PHP, expressions. Let’s write the Vue code. Replace the code in your resources/assets/js/app.js with the following:

javascript
1require('./bootstrap');
2    
3    window.Vue = require('vue');
4    const app = new Vue({
5        el: '#app',
6    
7        data: window.analytics
8    });
Copy

Updating the dashboard in realtime

Now let’s update our request logging middleware so that it broadcasts the updated analytics to the frontend whenever a new request is made. We’ll be making use of Laravel’s event broadcasting for this.

First, we’ll create an AnalyticsUpdated event. Create the file app/Events/AnalyticsUpdated.php with the following content:

php
1<?php
2    
3    namespace App\Events;
4    
5    use App\Services\AnalyticsService;
6    use Illuminate\Broadcasting\Channel;
7    use Illuminate\Contracts\Broadcasting\ShouldBroadcastNow;
8    use Illuminate\Queue\SerializesModels;
9    use Illuminate\Foundation\Events\Dispatchable;
10    use Illuminate\Broadcasting\InteractsWithSockets;
11    
12    class AnalyticsUpdated implements ShouldBroadcastNow
13    {
14        use Dispatchable, InteractsWithSockets, SerializesModels;
15    
16        // The updated analytics
17        public $analytics;
18    
19        public function __construct()
20        {
21            $this->analytics = (new AnalyticsService())->getAnalytics();
22        }
23        public function broadcastOn()
24        {
25            return new Channel('analytics');
26        }
27    }
Copy

Now, update your app/Http/Middleware/RequestLogger.php so it looks like this:

php
1<?php
2    
3    namespace App\Http\Middleware;
4    
5    use App\Events\AnalyticsUpdated;
6    use App\Models\RequestLog;
7    use Carbon\Carbon;
8    use Closure;
9    
10    class RequestLogger
11    {
12        public function handle(\Illuminate\Http\Request $request, Closure $next)
13        {
14            $response = $next($request);
15    
16            if ($request->routeIs('analytics.dashboard')) {
17                return $response;
18            }
19    
20            $requestTime = Carbon::createFromTimestamp($_SERVER['REQUEST_TIME']);
21            $request = RequestLog::create([
22                'url' => $request->getPathInfo(),
23                'method' => $request->method(),
24                'response_time' => time() - $requestTime->timestamp,
25                'day' => date('l', $requestTime->timestamp),
26                'hour' => $requestTime->hour,
27            ]);
28    
29            // we broadcast the event
30            broadcast(new AnalyticsUpdated());
31            return $response;
32        }
33    }
Copy

We need to do a few things to configure event broadcasting in your app. First, to enable event broadcasting, open up your config/app.php and uncomment the line in the providers array that contains the BroadcastServiceProvider:

php
1'providers' => [
2        ...
3        // uncomment the line below
4        // App\Providers\BroadcastServiceProvider::class,
5        ...
6    ],
Copy

Then sign in to your Pusher dashboard and create a new app. Copy your app credentials from the App Keys section and add them to your .env file:

bash
1PUSHER_APP_ID=your-app-id
2    PUSHER_APP_KEY=your-app-key
3    PUSHER_APP_SECRET=your-app-secret
4    PUSHER_APP_CLUSTER=your-app-cluster
Copy

Then change the value of BROADCAST_DRIVER in your .env file to pusher:

    BROADCAST_DRIVER=pusher

Open up the file config/broadcasting.php. Within the pusher key of the the connections array, set the value of the encrypted option to false:

php
1return [
2        // ...
3        'connections' => [
4            'pusher' => [
5                'driver' => 'pusher',
6                'key' => env('PUSHER_APP_KEY'),
7                'secret' => env('PUSHER_APP_SECRET'),
8                'app_id' => env('PUSHER_APP_ID'),
9                'options' => [
10                    'cluster' => env('PUSHER_APP_CLUSTER'),
11                    'encrypted' => false, 
12                 ],
13             ],
14        ]
15    ];
Copy

Note: Laravel sometimes caches old configuration, so for the project to see your new configuration values, you might need to run the command php artisan config:clear

Then install the Pusher PHP library by running:

    composer require pusher/pusher-php-server "~3.0"

On the frontend, we need to install Laravel Echo and the Pusher JavaScript library. Do this by running:

bash
1# install existing dependencies first
2    npm install
3    npm install pusher-js laravel-echo --save
Copy

Next, uncomment the following lines in your resources/assets/js/bootstrap.js:

javascript
1// import Echo from 'laravel-echo'
2    // 
3    // window.Pusher = require('pusher-js');
4    //
5    // window.Echo = new Echo({
6    //     broadcaster: 'pusher',
7    //     key: process.env.MIX_PUSHER_APP_KEY,
8    //     cluster: process.env.MIX_PUSHER_APP_CLUSTER,
9    //     encrypted: true
10    // });
Copy

Now open up your resources/assets/js/app.js and add the following code to the end:

javascript
1Echo.channel('analytics')
2        .listen('AnalyticsUpdated', (event) => {
3            Object.keys(event.analytics).forEach(stat => {
4                window.analytics[stat] = event.analytics[stat];
5            })
6        });
Copy

Here we listen for the AnalyticsUpdated and update each statistic accordingly. Since we’ve bound the data item of the Vue instance to window.analytics earlier, by changing a value in window.analytics, we can be sure that Vue will automatically re-render with the updated values.

Now run npm run dev to compile and build our assets.

For us to test our app, we need some routes to visit. These routes should take different amounts of time to load, so we can see the effect on our statistics. Let’s add a dummy route that waits for how many seconds we tell it to. Visiting /wait/3 will wait for three seconds, /wait/1 for one second and so on. Add this to the end of your routes/web.php:

php
1Route::get('/wait/{seconds}', function ($seconds) {
2        sleep($seconds);
3        return "Here ya go! Waited for $seconds seconds";
4    });
Copy

Let’s see the app in action. Start your MongoDB server by running mongod. (On Linux/macOS, you might need to run it as sudo).

Then start your app by running:

    php artisan serve

Visit your analytics dashboard at http://localhost:8000/analytics. Then play around with the app by visiting a few pages (the wait URL with different values for the number of seconds) and watch the stats displayed on the dashboard change in realtime.

Conclusion

In this article, we’ve built a middleware that tracks every request, a service that computes analytics for us based on these tracks, and a dashboard that displays them. Thanks to Pusher, we’ve been able to make the dashboard update in realtime as requests come in. The full source code is available on GitHub.