As a devops doing lots of R&D, the daily usual ops stuff can seem a little bit boring. Manually looking at each nightly alert, digging through the log files looking for an unusual pattern, checking your collectd graph are things you’d easily forget about to work on the latest trending technologies.
When part of your job is made of repeating, not that interesting task, there are 2 things to do.
First, build tools that take care of those things for you as much as possible. Fine tuning Nagios thresholds to reflect usual productivity peaks helps to get rid of false positives. Using stream processors like Rieman makes finding unusual patterns and generating alerts easy. Until you only have the real uncommon patterns left to check.
Now you’re done with automation, turn the remaining boring part of the job into a motivating way to start your day.
I believe in routines a lot as a way to increase productivity. Preparing the kids breakfast when I get up, having an outdoor break at 4PM, writing down what I just learnt are a very important part of my day. They give it a rhythm, a frame I can fill with everything else, starting with all the unexpected ops things.
Because routines are important, you need to pick them up carefully. Some of them are useless, the Monday morning 10:00 AM meeting is toxic. Stack your routines and you won’t let anything left for the unexpected anymore.
Since I’ve stopped working 24/7, I have setup a routine to start my workday. Starting your workday means switching your brains and thoughts from your personal, daily self to you professional self. You evacuate home related problems to focus on work related ones, collect information, and prioritize tasks.
This is where the daily checks – and fixes – go. They are a very subtle, data driven way to start my workday.
Data driven because after a night off, there are lots of questions I need to get answers to. What’s the state of my platform? Of my weekly planning? What nightly batch did fail (this happens too)? Why?
Subtle because the data collection turns into actions. As I get the best possible picture of what the night looked like, and therefor what I can expect my day to be, things get fixed, issues get open, a report mail leave, tasks get prioritize… Like the dawn turns the night into day rises a new productive workday.