I wanted to share a couple of tools we have been using lately that have been helpful in keeping us notified when we are having issues with our production software or hardware.
The first tool is Zabbix.
Zabbix is a powerful, open-source monitoring tool designed to track the performance and availability of various resources, including networks, servers, applications, and cloud services. It provides real-time monitoring and alerting, allowing you to quickly identify and resolve issues.
We use it to track the status of our production machines – hard drive space, memory used, etc., but we also use it to track the online status of software like Cronicle, Bitfocus Companion, etc. These tools are vital to the success of our services, so I want to know the instant they go down. It’s rare that they do, but I still need to know as soon as possible, and preferably before something fails because it’s down.
At my church, we have a server set up to run virtual servers through a hypervisor, and our IT department set up a simple linux virtual server for me. On that server, I run several Docker containers for software like Cronicle, Companion, Tally Arbiter ;), and more. I manage all of our containers through Portainer which is a GUI for Docker.
So – using Zabbix, I can create a host for any of my computers and set up things to monitor within those computers. For most of these, it’s just simple web checks. I can have Zabbix go check to see, for example, if the Companion web interface loads once a minute. If it doesn’t load, I can assume that Companion is no longer running, for whatever reason – the computer is off, the software crashed or was closed, etc.
Zabbix will then automatically report a problem with the service within the user interface. That’s great – but what I really need is to be notified on my phone. This is where the Ntfy service comes in.
Ntfy is an open-source notification service that lets you send real-time alerts. You can get notifications on your phone, computer, or browser. We use it with Cronicle whenever a job fails, error chaining on the job will automatically send out a Ntfy alert to my phone. I also use it for things like letting me know a recorder has started – anything that I just want to know about but don’t need or want to go specifically look at a device to know the status. Ntfy has a Companion module (I wrote it) and I also made a Cronicle plugin for it.
Zabbix can be configured to send alerts to a variety of media types – email, Discord, Slack, SMS, you name it. It comes automatically with several types. Someone in the community created a Ntfy type. I forked it and added additional support options, available here: https://github.com/josephdadams/zabbix-ntfy
So with these two tools, I am able to know instantly when a service has gone down, by receiving a push notification to my phone.
Here is a video that explains how to set up a web scenario in Zabbix and get pinged by Ntfy:
Maybe this will help you monitor your equipment as well! Leave me a comment with what you might use it for.