Sysadmin Rituals: a poll

[yesthattom]

About once a week I get an idea for a book. One that I've been mulling over would be called "System Administration Rituals". The power of ritual is undervalued in technology fields, but I find a lot of power in having rituals: there is a ritual I do every morning when I get in, a ritual I do when changing backup tapes, a ritual for bringing new users into the system, a ritual for everything. The benefit is that I'm not re-inventing the wheel every time I do something, and what little brain-power I have :-) can be used for other things.

I've been painfully aware of co-workers that could really benefit from adopting rituals in their system administration practices.

There are good rituals and bad rituals. I've made improvements in my life by recognizing bad rituals, upgrading old rituals (which surely began as good rituals but degrated or became obsolte). I've been painfully aware of co-works with bad rituals, and I've blantantly stolen (i.e. bad authors copy, great authors steal) rituals that I've seen worked well for others.

What are your system administration rituals? Is there a litmus test for detecting bad rituals? What do all good rituals have in common? What makes a good ritual good?

Comments

[pyrrhus.livejournal.com]

I call them "scripts". :)

[fings.livejournal.com]

For non-deterministic problems, we have an actual rubber-chicken here at work, which we shake over problem equipment, and in extreme cases, leave in place on top of the box.

Laugh all you want, but I'm not about to get rid of it. :)

[confessionalbox.livejournal.com]

Ohhh I like that! I may steal it if I ever have a gig again. It reminds me of how we would always go to an all you can eat brazilian steakhouse every time we resurected one of the SQL or exchange servers. It was partially in celebration, partially so we could offer a sacrifice of many cows to the gods (you should have seen the reaction when the Sr. Network admin tried to expense the Fogo de Cho dinners as ritual sacrifices)

[jsbillings]

I saw your talk at LISA03 about time management, where you mentioned sysadmin rituals, and I tried to think of a couple.

We have some, for example: creating new accounts. It involves creating a kerberos id, AFS space, creating a maildrop account and updating the charging information. Unfortunately, it is poorly documented, so every time a new person is hired, they have to relearn it.

[ximinez]

I have lots of rituals. I tend to encode them into shell scripts, aliases, or batch files. ;)

[beowabbit]

sync; sync; halt isn't anything but a ritual any more, but I still type it. Don't want the gods to think I'm getting cocky.

I was taught

[chrisla.livejournal.com]

sync; sync; sync; halt

I was told "there's no place like home; there's no place like home; there's no place like home" Clicking your heels together helps.

I type it too, my co-workers laugh every time.

[airshipjones.livejournal.com]

delete tmp files, bak files, and other junk.

[tcb.livejournal.com]

ping 198.6.1.1

when I need to test outside connectivity, UUNET's caching nameserver comes out of my fingertips without thinking..

I like the book idea. not sure who the market really is, but I like it.. maybe one of the shorter O'Reilly books like the smileys one?

[xtat.livejournal.com]

I suppose checking your logs regularly is good practice. Also, if you have an external AIDE or Tripwire database, check it against the installed copy now and then to catch the somewhat brighter system crackers.

These are really the things I wish I did more often :>

Wuffo you wanna jump outta a perfectly good airplane?

[mushmook.livejournal.com]

The story goes that military parachutists are required to actually reach for and touch their auxillary chute pulls during their pre-jump routine. Civillian jumpers typically don't do this. Stats show that when the sh!t hits the fan, military jumpers are more than twice as likely to reach around and pull their aux cord during a chute deployment failure... due to the ingrained body memory from their pre-jump routines.

I have a cursing ritual. Er, cozening ritual. Whatever. It's quite detailed. Starts with 'baby' (as in, "come on, baby, boot up nice and get that www server running this time, I need more than just FTP and telnet out of you, you can do it") and deteriorates eventually - in very precise steps - to 'whore' (as in, "you f**k!ng whore piece of sh!t!!!"). This ritual works on servers, hardware, automobiles, and even recalcitrant blenders.

If the natural order of cursing isn't followed, though, it will take twice as long to get the thing working. ;-)