Recently I faced the issue of providing a workplace for a duty shift as an administration decided to provide a 24/7 support. We employed a necessary amount of administrators who were in charge to do shifts (including nights) and watch an operability of systems.
While my colleague was describing procedures for people on duty for cases of emergency failures, I started to develop their information support. To begin with, I considered keeping an open monitoring at theirs and reload a page manually to get actual data.
Such solution was refused because a shift worker could forget about Zabbix or stay keen on a CS game and miss a moment of emergency. So we decided to buy 2 displays, hang them on the wall and transfer a monitoring picture directly on them. A common computer was connected to the displays, which had two opened browser windows with an auto reloading page. To do this, we installed a Chrome application named Easy Auto Refresh: https://chrome.google.com/webstore/detail/easy-auto-refresh.
This tool is extremely simple: you set an auto reloading timer and click «Start».
One display had a standard Zabbix dashboard and a temperature diagram, another one was agreed to display a network map with NetPing sensors.
A draft is below:
As well as a NetPing device is my usual equipment in Zabbix, it checks its availability only. Therefore I displayed a picture named "trigger" independently, which receives and information from "data source". Here you can see the settings of a trigger:
Adding a trigger to a map:
At the screen I see a linkage of NetPing with their sensors and sensors statuses (OK, not OK). The second was a network map with equipment:
Therefore I have two screens:
- Display 1: a dashboard with alerts and a temperature diagram;
- Display 2: a network map with NetPing, a network map with equipment
There still was an issue of notifications. Sending SMS messages and emails did not work out because a shiftman could just fall asleep. Hence we decided to use a sound notification as well. I would like to start off saying that Zabbix does not have a default sound notification, but it does have a script to start. This gave us a great opportunity to reinvent the wheel. To begin with, I will say that a computer running opened browsers with Zabbix screens is running on Linux, which simplifies solving my issue of audio alarming. I wrote a number of scripts on a Zabbix server, which fulfill a very simple task: they log onto a computer via ssh and start there another simple script: play_alarm.sh
Script draft is audio_notify.sh
#!/bin/bash -l
ssh user@monitorpc
/path_to_script/play_alarm.sh
Script draft is play_alarm.sh
#!/bin/bash –l
Audioplay /path_to_alarm_file/alarm.wav
Configuring a new way of alarming:
I linked this way of alarming to must-react events and checked its work – when a problem appeared, notifications and emails were sent and horrible peep, previously downloaded from the Internet, sounded from speakers that were connected to a computer. It did work out, but this was not the end of my adventures, because an administration decided that this peep is not enough and the problem that happened must be said by voice.
Such task was more difficult to solve: I asked a pretty girl from Human Resources to read aloud to the microphone the texts similar to the following: «Attention! It is an alarm because of a high temperature in a server room!», «Attention! A connection with network equipment is lost! Please see Zabbix network map for further details!», etc.
For each type of problem a separate wav file was created as well as an individual script for both machines. Unique scripts were linked to a concrete event type and a tuneful woman voice (instead of a regular peep) notified a shift on duty that something was not working properly.
Conclusion
I needed the following to provide a workplace for a duty shift:
- A computer with Linux OS
- Speakers
- 2 LCD screens 35"
- A voice of a tuneful HR
A computer displayed 4 browser windows with a dashboard, a temperature diagram, a network map and a map of NetPing sensors. To do all these things I spent 3 hours from the moment of purchasing a necessary equipment.
P.S. BTW, for the IT departments that are close to a server room, you can install an alarm, connected to a NetPing. For example this one: http://www.netpingdevice.com/products/buzzer-ac10
