I recently ran into an issue where my I had Xymon clients on a network which was unpingable from the Xymon server. The clients could could send data to the server, but they were hidden behind network address translation and only on a local network. I had one box (the gateway for the unreachable clients), that saw both networks. I played with the idea of running bbproxy on it, but didn’t really want an entire Xymon server and it really was not necessary since the other clients could get their data out. Looking at hobbitlaunch.cfg I noticed it was really just a larger clientlaunch.cfg, which gave me the idea of just copying over bbnet and using on a client install – which worked perfectly. Read more…
After a recent job switch I’ve had the opportunity to setup Xymon from scratch and start developing even more scripts for new pieces of software and work flows. One of my first tasks was to setup a new “cluster” using all the software I felt most comfortable with as a show case in order to determine if my preferred tools worked as well or better than the currently used ones – or not. After a week or so of setting up a full Cobbler installation, Xymon and my own Glovebox, I presented it to my new employers with positive responses. After all that work I wanted to make sure that the people using the system were as happy as possible with the monitoring needs – one thing was mentioned more than others, which was the ability to easily see the status of Sun’s Grid Engine running on the cluster. I immediately set to work and came up with a quick solution for them using Xymon and a script that parsed the output of ‘qstat -f’ As with my Xen monitoring script, it runs in one place and sends in data for all the associated machines. Meaning for each execution node you’ll have a column with just its information, and a combined column for the qmaster. Read more…
Over the years I’ve had to write plenty of Hobbit / Xymon scripts to monitor various different things within my employers systems. Since most all of our applications are custom there are not always built in tests that will work for us. For example, we use Xen for our development virtual machines and being able to track what was going on with those virtual machines is important and being able to identify a VM within Xymon at a moments glance is important to us, so we created a test that does just that. We have created in house scripts for MySQL Status, MySQL Running Queries, our in house distributed services, Lighttpd (as discussed earlier on this blog), Apache, Memcached, etc. This doesnt include the hundreds of different snmp tests we’ve added to Devmon for monitoring our network equiptment.
Read more…
My employer starting using lighttpd on one layer of our architecture about a year or so ago, until now that layer has kind of been a black box to the majority of the technical staff due to not having mod_status enabled. In preparation for it being turned on (I requested it be so after using it on my own servers), I have created a Xymon Monitor (formally know as Hobbit) script which hits the /server-status page on the localhost and reports that data back to Xymon. The data it reports includes requests per second and “amount increase since last script run” for the “Total KBytes” and “Total Accesses” numbers. I also created a graph for the requests per seconds stat.
The Graph definition is as follows:
[lighttpd]
TITLE lighttpd Requests/Second
YAXIS # reqs/sec
DEF:RPS=lighttpd.rrd:reqpersec:AVERAGE
LINE2:RPS#0000CC:reqs/sec
COMMENT:
GPRINT:RPS:LAST:Requests per Second : %5.1lf (cur)
GPRINT:RPS:MAX: : %5.1lf (max)
GPRINT:RPS:MIN: : %5.1lf (min)
GPRINT:RPS:AVERAGE: : %5.1lf (avg)
Read more…