What I do: Interviewing for Linux Engineers

Now and then I’m called on to help interview candidates for linux admin/engineer slots and as I’ve been doing some of that lately I thought I’d share the way I go about doing a technical interview. This approach seems to work equally well over the phone or in person.

What I do: Voice Integration With Nagios and Asterisk

I was called on to provide a method of alerting from within nagios that was more active and direct than the usual use of email or SMS messages.  So I came up with a simple way to have a nagios notification place a phone call to our off hours tier3 support line to report certain very rare but serious problems.

Dynamic Rack Face Diagrams With PHP and GD

Anyone who has to manage servers and other equipment in remote datacenters can appreciate the need for good documentation.  One reason you appreciate good documentation so much is because it’s so rare. People are lazy and forgetful and when changes are made by lots of different people in lots of different locations it’s easy for reference docs to get out of date and unreliable. So from those two concepts was born a need to create dynamic rack face diagrams and end the dependance on the manual task of updating and distributing static visio diagrams. Attached are templates for most common rack sizes and example php code on how to make it work.

Don't be a Slave to Your Vendors

One of the reasons I like and support the use of open source software is that you can avoid most of the drama that comes from relying on 3rd party vendors.  By this I mean.. you must pay exorbitant sums for ongoing maintenance, you are locked into their product upgrade treadmill, you have little say in the direction of their products, you have a single source for support, and if your vendor gets acquired there is a very good chance the product you depend on will go away or change in ways that force you to abandon it with even more pain.  I’ve seen this play out from both sides of the table having spent time in both enterprise environments and working for software companies.

What I do: Practical Data Visualization

I’m often ask what it is I do for a living… and being lazy I usually just say ‘computer stuff’.   In an effort to provide a little more context to anyone who may be interested this is one in a series of postings where I’ll cover some aspect of what it is I do.

In my current role I spend part of the time doing development projects. (aka programming) I’m not a hard core developer though.. it’s not my full time occupation nor do I want it to be.  I work mostly with perl and php when necessary, mysql and occasionally PostgreSQL or Oracle all under various flavors of linux. (debian is my favorite). Usually these development tasks are related to some sort of management automation for a global VoIP network but sometimes they involve making complex things easier to understand.  Part of that involves automating the collection of large amounts of data and then presenting in a meaningful way so that problems and long term trends can be identified.  What follows are some examples of the sorts of things I mean.

What I do: Power DNS Real World Results

We have had a Power DNS recursing cacher deployed at one of our busiest sites for a few months now and I thought others might benefit from some real world performance info.  This is running on some older hardware.. dual Xenon 2.8Ghz system with 4G of ram and the only job it’s doing is running this recursor. These three graphs tell the tale.  The first shows that the system is handling peaks of about 3800 queries per second and that about 99% of those are being answered in a fraction of a millisecond.  The second shows that cache hits are averaging about 70-75% and the third shows that it’s doing this work while using at most one quarter of the CPU.  Add to those impressive performance levels that I’ve had zero issues since putting it in production six months ago.

What I do: Another Day in the Bunker

I thought some might find this interesting. Most who work outside of tech (and many inside tech) never see data centers like this. To get to our company servers in this hosting facility I have to pass through two man traps and a total of 5 doors with hand scanners. In addition, each rack of equipment is locked with a unique 6 digit code. There are hundreds of cameras inside the main datacenter and guards constantly watching from a secure room.

System Rescue CD to the.. rescue!

Here’s the scenario.. It’s 1 am and I have to shut down a critical linux server to relocate it in a rack to make room for new equipment. It should have been a 5 minute job.. but on powering up the server it refused to boot past printing the word ‘Grub‘ on the screen. This isn’t good.. this server is needed by a couple hundred thousand customers and rebuilding it wasn’t planned or scheduled.

Putting the UBEE Cable Modem from Time Warner in Bridge Mode

Time Warner has been offering faster cable modem speeds lately for much the same price as slower speeds a couple years ago. When I got the new modem I noticed there didn’t seem to be a way to put it in bridge mode to work with the linux router I already had. After a little digging I found that the management page for doing this was ‘branded out’ and the link to it was removed from the interface.

What I do: Mysql Reporting Date Range Cheat Sheet

If you have to do any sort of reporting using mysql or other databases you will probably need to use relative date ranges that don’t depend on what day the report is being run. That is if you need a report for ‘last month’ that runs from the first to last day of the previous month.. or previous 3 months etc but not include any days in the current month.   Here is a small collection of queries…