Back in April some jackass promised that there would be a follow-up post about migrating bsd-box.net to use the LetsEncrypt CA.
Oh right, that was me.
As usual life interfered, but here's the followup: The bsd-box.net server has been replaced with a shiny new machine, LetsEncrypt is still my CA of choice for this system, and a bunch of other things have changed. More below the jump.
Most if not all readers of my blog are probably aware of the Lets Encrypt project, which officially exited Beta this month.
For those of you not familiar with it, the basic premise is "It's the goddamn 21st century, and there is no reason every website shouldn't be available over HTTPS. We're giving away certificates for free, and giving you an automated tool to acquire and renew them. You have no more excuses!"
Most of you also know I was originally quite skeptical of this project: I'm not a huge fan of trusting third-party programs with my cryptography, and I like to ensure that I'm maintaining control of the impotant bits (like private keys) at all times. The final implementation however appears to be well-designed and reasonably secure, enough so that I have used it for this year's certificate renewal on bsd-box.net.
What follows is a brief description of the Lets Encrypt process on FreeBSD: Its successes, its failures, and some thiings I may be submitting patches for in the near future.
Tom Limoncelli (yes, that Tom) recently wrote a blog post that came to my attention by way of Twitter in which he lamented his bank's scheduled downtime and the implications of routine "weekend work" in terms of an organization's respect for the time and work-life balance of its sysadmin staff.
This was posted the "Rants" section of his blog and is obvisouly ment to be taken as slightly tongue-in-cheek alongside the idea that every sysadmin in geekdom's creation would really rather be watching the Star Wars movie, but it's broadly representative of an attitude I've seen emerging more and more in our profession: That sysadmin work should be viewed as a 9-to-5 gig. I in turn ranted a little bit about that on Twitter, but I think it merits following up with a longer form discussion, so let's have a blog post before the end of the year!
OK, for the record: I DESPISE Linux. I Hate, Loathe, Abhor and Revile it. I am a BSD-Bigot and proud of it, and if all the BSDs suddenly evaporated I would eschew Linux in favor of a commercial Unix (probably AIX).
Why do I hate Linux so much? Simply put, it's shoddy code written by shoddy coders. In my experience shit mysteriously breaks for no reason, standards and conventions are arbitrarily ignored, critical components of the system are perpetually at version zero-dot-something, regression testing seems to be a myth, and the average Linux developer seems to make no effort to ensure their code will work on anything except their particular favorite distribution (to say nothing about porting it to <GASP> a BSD system, or <HORRORS> Commercial Unix).
This particular tirade was kicked off by the Ubuntu update-notifier program suddenly and mysteriously no longer popping up update notification balloons. As some of you know my company ships a Linux-based appliance (built around a very stripped-down Ubuntu plus our commercial packages), and one of the things that made me go with Ubuntu was that they had gotten the update-notifier thing working beautifully and it had been stable for several versions.
Lo and Behold about a month ago our support guys came to visit me and asked "Hey, is the update notifier bubble broken?" I looked upon it and saw that it indeed appeared to be non-functional, but as all good (lazy) admins are wont to do I demanded they test and verify the breakage.
The breakage came back to me verified earlier this week, and as I really couldn't be assed to figure out why the update-notifier is happy to display the "you must reboot!" dialog box but refuses to display the "Yo, bitch! You have updates!" notifier icon & bubble I took the easy way out and re-implemented update-notifier in Python.
For those of you wondering where bsd-box.net went this weekend, here's the scoop:
A while back I stopped being a data protection douche and implemented backups. I didn't blog about the implementation, but it was bacula writing to rsync.net over sshfs (fuse for the win).
These backups worked fine for a good while, but on Friday (my first full backup since upgrading to 8.0) the damn thing blew up and took the server down with it. Near as I can tell without being on the console something caused the FUSE kernel module to go insane - The system goes unresponsive, starts taking forever to respond to pings and eventually falls over entirely.
Troubleshooting that mess is on my list as soon as I can reproduce the problem in a test lab, but until then bsd-box.net is running a ghetto-ass tar-then-SCP backup routine (so I'm not a data protection douche since I've still got backups, but my backups are decidedly more ghetto).
Those of you who sent me emails, panic not: I have a very fine secondary MX which will eventually despool whatever you sent me. If it's urgent go ahead and resend.