I've recently gotten a Digital Ocean Droplet (aka virtual machine, just like AWS EC2) to mess around with doing all the magic stuff that usual hosting providers such as Heroku do for you. Automatic deploys, web servers, etc.

Right now I'm at a point where I'm hosting this blog as a Rails app, deployed automatically with Capybara. I also host jonandjess.studio as a static website. It's a Nextjs app (I wanted to test their export function) which gets deployed with a simple yarn deploy which runs next export then scp the build folder in the correct folder on my Droplet (/var/www/jonandjess/html).

It has an nginx config that allows to route calls to jonandjess.studio to its index.html file.

To be honest, just doing that was pretty eye opening to me on how things work.

I get amazed by this pretty quick, which is fun. For example, I'm working on a V2 for jonandjess.studio. I wanted Jess to be able to access the progress so she can give feedback, but it gets annoying when she randomly bugs me to see what I have so far. So now I'm just deploying the V2 at /var/www/jonandjess/v2 and added an nginx config to route there when the url is jonandjess.studio/v2. Brilliant! (I know, I know...).

Anyways, now that we're at this point, I wanted to look into fun, simple monitoring solutions, because now that I can play around with the Droplet and nginx configs, I felt like understanding thing like Elastic Search would be easier (I'm not there yet, but soon!).

So the first solution I just installed, which was amazingly easy to do, is goaccess, an open source log visualizer.

By default Nginx logs every calls it gets at /var/log/nginx/access.log (you can also override this path, and the format in the nginx config). I wanted to use GoAccess to have a visualisation of calls for jonandjess.studio.

setup goaccess on Ubuntu

First, you need to install goaccess.

sudo apt-get install goaccess

will do what it says (it is always recommended to run sudo apt-get update first to make sure you install the latest version of the package).

Once you have that, goaccess has an easy CLI with a lot of options to run it (it can also have a config file, my case was simple enough for me not to use one for now).

To run the CLI version, which looks great, you can run

goaccess access.log -c

note that access.log is the path for your logs, so if you wanted to run this for your nginx logs from any directory, you'd pass it /var/log/nginx/access.log.

That is nice, but this would imply that I have to ssh into my Droplet each time. I really wanted a web-based dashboard, which GoAccess offers, at no additional cost! (FREE!).

Web-based goaccess dashboard

You can pass a -o /var/www/html/report.html option to the command line, which will output the dashboard as an html file.

Combine this with --real-time-html option and your html file will get automatically updated + goaccess will use websockets to connect your browser and offer live reloads.

Speaking of the websocket, it uses port 7890, so you'll have to open this port with your firewall. Opening a port on Ubuntu is done with

sudo ufw allow 7890

Now that your port is open, you can run the goaccess command with all previously stated options.

I wanted to report to be accessed at jonandjess.studio/report.html (go check it out!) so my final command is

goaccess /var/log/nginx/access.log -o /var/www/jonandjess/html/report.html --log-format=COMBINED --real-time-html

This is nice, but then do I just CMD+Q my terminal hoping for this to keep running always? It doesn't seem too stable, so I decided to make it a service controlled by systemctl so it can restart automatically etc.

Add goaccess to systemd

systemd allows you transform scripts into services, which it will then take care of, according to a list of rules you set (restart when it fails, start when the server boots, etc). You can also easily control your service with commands such as systemctl start myservice, instead of using things such as lsof to find which process it is running on and killing it, etc.

So first, we need to put our command into a file so it can be ran by systemd. I created my script in /usr/local/bin/goaccessreport:

#!/bin/bash
goaccess /var/log/nginx/access.log -o /var/www/jonandjess/html/report.html --log-format=COMBINED --real-time-html

The script will get executed, so you need to make this file executable with

chmod +x /usr/local/bin/goaccessreport

Ok, now we have an executable which starts our goaccess live web dashboard (you can test by typing /usr/local/bin/goaccessreport and it should start the web dashboard.

Last thing we need to do is create a service file for systemd, which tells systemd what to do with it. We can create the file at /etc/systemd/system/goaccessreport.service:

[Unit]
Description=Goaccess Web log Report For Site Example.com 
After=network.target

[Service]
Type=simple 
User=root 
Group=root 
Restart=always 
ExecStart=/usr/local/bin/goaccessreport 
StandardOutput=null 
StandardError=null 
[Install]
WantedBy=multi-user.target

I'd invite you to learn more about systemd - it's a little confusing how it works, but when you get a hold of it, it's great and very useful!

And... that's it! Now we can run

systemctl enable goaccessreport.service
systemctl start goaccessreport.service

to enable and start our service. jonandjess.studio/report.html is up and running. The Restart=always directive in the service file will ensure that systemd will always keep the service up (either when the Droplet reboots, or the script fails, etc).