"A Day in the Life" is our most requested blog topic currently, so here's a first installment. Featured today is one of our Systems Administrators, Jonathan.
Here's the play-by-play:
00:00
Finally finished the migration of a Plesk-based domain for a Managed Server customer that has had me stumped for 2 days. The Plesk Migration Manager is picky and opaque in many ways. It faulted on one large domain (11GB) out of many, without any real indication of the problem. Since this domain was migrating from a Plesk 8.2 instance to a Plesk 9.2 instance, the alternative migration strategies were somewhat convoluted.
00:30
Finished documentation of the migration adventure of that Plesk-based domain.
Checked the Modwest Support queue (we use Request Tracker for our ticketing system) for any late-night urgent requests before turning in for the night.
04:30
Z, the 2.5-week old newborn, is fussy and needs a diaper change.
05:30
Z is upset. After a quick verification of diaper situation and other potential causes of this state, took her downstairs and we napped on the couch to give my wife C a much needed break.
07:30
Z and I are awake again. Took her back upstairs for a diaper change and a reunion with C.
08:30
Working with one of our Customer Support Specialists (L1) on a shared SSL certificate issue for one of our shared hosting environment customers.
09:00
Checking hardware upgrade options for one of our Managed Server customers. He wants more disk space and there is the option of adding more drives to his server, not just replacing the existing drives. Adding drives is a lot less work on our part and gives the customer even more disk space, while utilizing all of the drive bays in his server.
09:20
Researching filesystem options for our Cyrus mail servers in our shared hosting environment. ZFS is an option, but we'd only really want to do that with Solaris at this point and we've had recent bad experiences with our Solaris implementation. So, really, that's not an option right now. That leaves the two main contenders as ext3 and reiserfs. Finding dialog and debate on the merits of each type going back years. There's a lot of old information out there -- beware.
Our new Systems Administrator, who has expertise in storage systems, is working on some great new projects for our storage environment. Interesting stuff in the works.
10:10
Helped L1 look at an unusual account information discrepancy for one of our customers, which turned into a handful of customers. All with similar account IDs. Seems to be LDAP-related, so maybe a typo in a script recently. It doesn't seem to be affecting performance for these customers, but shows some odd information internally when we view their account/billing information.
10:15
Packing up to head to the office. Departure delayed while watching over Z while C grabs a shower. Mentally thanking Modwest (again) for allowing me flexibility in my work schedule.
11:15
Arrived at the Modwest offices. It's about a 10 minute walk from home to the office; along the bike path, across the Clark Fork River via the Orange Street Bridge, and along the bike path again to the Modwest Offices in the historic Wilma Building.
Along the way, thought about scheduling hardware and software maintenance later this month for one of our VPS servers.
Ah! Forgot that today is monthly clean-up day. It's the day that our Operations Team puts other projects aside to clean up the things that have been fluttering around begging for attention. I'm focusing on some Managed Server clean-up tasks -- preparing for decommissioning of some hardware that former customers are no longer using.
13:00
Digging out of monthly task clean-up.
Need food.
Also need to stop by a specific local restaurant/bar to ask about catering next month.
13:15
Generated list of Managed Server hardware that needs to be decommissioned over in our data center a few blocks away.
Spent a good while looking for traces of disappearing email deliveries for a Managed Server customer.
Corresponded with L1 regarding a database restore for another Managed Server customer.
Still need food.
14:30
Of course, I needed to talk to the appropriate catering person who is only available in the morning. Will try again on Friday or Monday.
Stopped by the data center and synchronized the data from a to-be-decomissioned Managed Server machine to an external drive. We usually keep that data around for a month after a Managed Server customer leaves Modwest, because it is not infrequent that they come back to us a week or two later because they forgot something important.
15:00
Figured out the best options for the Managed Server disk upgrade that I was researching earlier in the day. Sent a note to the customer detailing the different options and my recommendation. This particular customer likes details.
15:30
Started decommissioning resources, IPs in particular, of a former Managed Server customer.
16:00
Familiarized myself with older Postfix/Mailscanner implementation on an older Red Hat Managed Server. Ended up throwing a brick at the problem and configured Mailscanner to not scan any mail from the domain of our customer's client. Not an ideal solution, but since it was quite difficult to get detailed information from the remote client on the types and content of the emails that he was sending, we decided to just let it all through.
16:30
Working on a database restoration for a Managed Server customer. Made a note to self to complete our internal documentation on how to correspond with customers with regards to restore requests, specifically with Managed Server customers. Some customers want their live data to be just like it was a day or two ago. Others want the recovered data placed in a specific location that they can view. Either way, it saves time for the Operations Team if we know what the customer wants when the task to restore gets passed along to us.
17:15
Big Boss says that I have too much on my plate and to delegate a specific Wusage migration task to him. Yay!
17:30
Still working on that database restore for the Managed Server customer.
- While reading more about Cyrus email server filesystem recommendations
- While discussing, with another Systems Administrator, upstream bandwidth providers (who we shall not call out by name)
- While checking on available cabin lodging at Glacier next weekend
- While... uh oh. We blew out the keg in the Modwest Kegerator
Ought to head home soon -- where soon was 2 hours ago.
18:00
Finished the database restore for that Managed Server customer.
Helped our Systems Engineer track down a problem with PostgreSQL backups for an internal VPS container.
18:07
Headed out the door for home.
18:30
Got home to another Z-related hospital bill. Sigh. So,...
Took a short nap with Z on the couch.
19:30
Prepared for short grocery store trip to the Good Food Store and debating whether I should take Z solo or not. Seems a bit risky having a 10 minute separation between a possibly frantic hungry baby and her food source.
21:00
Back from the store, which wasn't such a short trip. We all ended up going, which was probably not the most efficient use of our time, but that's okay.
21:30
Yay! Heard from a friend on Facebook that I haven't heard from since 1997. I was wary of my late-night FB signup 6 months ago, but I admit it has been fun to catch up with 'certain' old friends. She's actually doing what she thought she'd end up doing -- scientific illustrations.
Making dinner and listening to KCRW streamed from Santa Monica College in Santa Monica, CA. KCRW is streaming at our house most of the day.
23:45
Sent out party announcement to Modwest staff. C and I eloped last summer (a week before our 16th anniversary) and are planning a party for friends and family in August.
Changed Z's diaper.
Went to bed.
So as requested, there's 24 hours of diaper-changing and systems administration at Modwest. We have two more volunteers for this blog project (so far), so stay tuned and we'll post those soon.
Got other ideas for the Modwest Blog? Let us know in the Modwest Community.
-JM
Subscribe to this blog's feed
You need to sleep more.
Posted by: Avatar | July 07, 2009 at 03:44 PM
Did you ever get to eat?
Posted by: Ednor Therriault | July 07, 2009 at 10:53 PM
hmmm...
slightly different from the way I remember your days but interesting
Posted by: Dan G | July 09, 2009 at 06:17 PM
Loved reading this, John. Always put family first - even when customers like me are asking endless questions.
Posted by: Gloria O'Rourke | July 16, 2009 at 04:05 PM