Skip to content

Root Cause?


I have yet again engaged in an argument on Twitter with people who think Root Cause Analysis is a terrible, bad and misguided thing, and I've decided to put this very simplistic explanation of why I don't agree with that perspective here so I can just point people at it the next time I need to have this discussion.

Continue reading "Root Cause?"

A developer, a sysadmin, and a DBA walk into a conference room…


…and that's not a joke, it's how your design meetings should be starting.

A few days ago on Twitter someone asked a seemingly innocent question: "I'm writing a post on the failure of Stored Procedures as a platform.  What, in your view, were the reasons they didn't work out?"

A lot of reasons were given: "They're hard to test" (They're not - your unit tests should be testing your database.),  "They're not in git" (They should be - If they arent your revision control process is fucked because your database isn't controlled), "They're fundamentally unreadable and require exponentially more tacit knowledge aka are awful for new devs to understand" (They're not "fundamentally" anything, and if they're documented well any competent developer should be able to understand them), "They encourage silos where DBAs say no." (This is a people problem: Your process doesn't facilitate understanding between your DBAs and the rest of your team).
Some folks even came up with what I would argue are good reasons, like "Badly written stored procedures don't scale well" (which is true: If your stored procedures involve lots of processing overhead the DB server becomes a processing bottleneck, which is a Bad Thing), and "It's an additional moving part in the system" (generally something to avoid, unless that moving part is the simplest solution to a problem).

I was all set to have a friendly difference of opinion on this issue until I saw this blog post, which starts out great and quickly goes off the rails into the weeds and starts eating slugs with the DevOps "we don't need no stinkin' sysadmin/DBA" children.

So now you get a rant about why you still need a Developer, a Sysadmin, and a DBA.

Continue reading "A developer, a sysadmin, and a DBA walk into a conference room…"

The Problem of the 9-to-5 Sysadmin


Tom Limoncelli (yes, that Tom) recently wrote a blog post that came to my attention by way of Twitter in which he lamented his bank's scheduled downtime and the implications of routine "weekend work" in terms of an organization's respect for the time and work-life balance of its sysadmin staff.

This was posted the "Rants" section of his blog and is obvisouly ment to be taken as slightly tongue-in-cheek alongside the idea that every sysadmin in geekdom's creation would really rather be watching the Star Wars movie, but it's broadly representative of an attitude I've seen emerging more and more in our profession: That sysadmin work should be viewed as a 9-to-5 gig. I in turn ranted a little bit about that on Twitter, but I think it merits following up with a longer form discussion, so let's have a blog post before the end of the year!

Continue reading "The Problem of the 9-to-5 Sysadmin"

IP Wars: Revenge of the $sth

So as some of you may or may not have heard me bitching, Invision's IP management scheme is roughly one step below "Write it all on a clipboard". It's electronic (which is good), but also brain-damaged (it things that is a valid netmask, and doesn't care if you define overlapping networks). Why do I care? Well, our CTO is about to go before the allocation gods in a very Oliver Twist way. Small and humble, he shall go before the great robed IP gods of ARIN with his rWhois in his hands, genuflect, and utter the fateful words "Please sir, I want some more?". Invision's rWhois is managed by the aforementioned brain-damanged software. It believes we are using about 50% of our IP space. We believe we are using roughly 83%. ARIN says you must efficently utilize (their words) 80% of your IP space before they give you more. If our CTO goes before the numbering gods, they will strike him down for the sin of having a broken rwhois server. And they would be right to do so. We are technically in violation of our reporting requirements, as the aforementioned broken software feeds the rwhois server, and the rwhois server does not like the way our data tastes. Continue reading "IP Wars: Revenge of the $sth"