Skip to content

Managing Multiple FreeBSD Machines with radmind -- Part Two

UNIX This is part Two of an N-part series (I'm thinking 4 parts) discussing the investigation of radmin as a patch/deployment tool for FreeBSD. It will be filled in over the course of Q2/2009 as we test (and possibly deploy) radmin at Premier Heart. This part deals with some more basics: Creating an radmind "Base Load" to distribute to your machines. The radmind Base Load is the template that will be deployed to all servers and consists of Three major parts:
  • A "Positive Transcript" (often just called a transcript), which lists files and directories to be included in radmind deployments
  • A "Negative Transcript", which lists files and directories to be created or adjusted, but not replaced by radmind deployments
  • A "Defaults Transcript", which is a special case of a negative transcript.
  • A "Command File", which lists the transcripts to be applied, and the order in which to apply them. These transcripts are used to create "Load Sets", which are what radmind actually distributes to the clients.
  • Although you can use any machine to create your radmind Base Load, I suggest dedicating a build machine for each architecture to this purpose to avoid accidentally picking up local cruft. These machines only need enough room to store the userland, source, ports, local source & binaries and enough room to rebuild it all. Since you'll only be using them when it's time to build a patched version of your system virtual machines are uniquely suited to playing build box. I've outlined the process step by step below -- Steps 1 through 4 happen on the build machine, while steps 5 & 6 happen on your radmind server.
    Step Zero: Clean Up
    Before you start creating your transcripts, clean up the build server. Get rid of world-build remnants (/usr/obj), ports tree junk (/usr/ports/distfiles/* ; do a make clean in the ports tree too), and anything else you don't want to push out to all your machines. This avoids having to make a really big Negative transcript (or pushing unnecessary files to your clients), and also makes transcript generation faster since there are fewer files to walk.
    Step One: Create a basic Negative transcript
    All FreeBSD systems have a few things that you don't want to propagate out to the universe. Since we know what some of them are we can exclude them right at the beginning. To exclude items from the base load you use the fsdiff command to generate transcript entries. fsdiff -1 -c sha1 Path The -1 option causes fsdiff to produce a single entry, rather than traversing an entire tree and explicitly listing its children. Placing a directory in the Negative transcript prevents fsdiff from messing with its children. Some stuff you probably want to exclude: /dev /etc/rc.conf /mnt /tmp /usr/local/src /usr/tmp /var Store your negative transcript in /var/radmin/client/negative.T and create a new Command file in /var/radmin/client/command.K which contains a negative entry for this transcript (n negative.T).
    Step Two: Create your Defaults transcript
    As mentioned earlier, the Defaults transcript is a special case of a Negative transcript, and is created the same way. If you have any files you want to push out ONCE AND ONLY ONCE, you will add them to this file using fsdiff -C -c sha1 Path. For example, if you deploy /etc/master.passwd to every machine and then add local users, that file would be added to the Defaults transcript. Note that here we are using -C instead of -1 -- My assumption is that you may want to propagate entire directories of "defaults" (for example, template Apache configurations), which means you want fsdiff to traverse any directories you specify. Store your defaults transcript in /var/radmin/client/defaults.T and add it to the /var/radmind/client/command.k file as you did with the negative transcript.
    -- You now have a command.K file with two negative transcript entries -- one for negative.T (the things we don't want to mess with ever) and one for defaults.T, which we'll only push if it doesn't exist already. The items listed in those transcripts won't be part of the Base transcript, which prevents storing stuff we're not going to distribute on the radmind server. Note that the ordering of items in the Negative and Defaults transcripts is important -- They need to be in depth-first lexical order (/etc comes before /usr; but /etc/rc comes before /etc.old). If you get the ordering wrong fsdiff will complain, so you can go back and fix it (and yes, you MUST fix it - this warning is a fatal error). -- Step Three: Create the Positive Base Transcript
    Armed with your Negative and Defaults transcripts (and the command.K file containing them) you are now ready to create a Positive Transcript for your deployment.
    To do this run fsdiff -C -c sha1 -o base.T / -o /var/radmind/client/base.T This command will walk your entire filesystem, ignoring the stuff in your Negative and Default transcripts, and create your Base transcript. When it finishes examine base.T to ensure it doesn't contain any unwanted files (either stuff that varies client-to-client (you'll want to add this to your negative or default transcript) or local junk you don't want to distribute (remove it from base.T, and from the filesystem. If you update your negative or default transcript you should re-run the fsdiff command to rebuild the base transcript to ensure you don't have any unexpected results from your change (like added or missing files).
    Step Four: Create and Upload the Load Sets
    The transcripts describe what a filesystem should look like, but you need to centralize the files somewhere in order to distribute it to clients. The lcreate command does this. For the Base transcript, run: lcreate -h radmind_server /var/radmind/base.T For the Defaults transcript, run: lcreate -h radmind_server /var/radmind/defaults.T For the Negative transcript, run: lcreate -h radmind_server -N /var/radmind/client/negative.T The Base and Defaults transcripts upload a copy of each file in the transcript to the radmind server. The -N flag for the Negative transcript makes lcreate upload all the files as "zero-size" (Since we're excluding these files we don't need to upload them) If any of the files have changed since you ran fsdiff you will get a warning about checksum mismatches. This is usually a good indication that (a) the file shouldn't be distributed to clients because it changes locally, or (b) You did something on the server between running fsdiff and lcreate that changed the file, and you probably need to re-run fsdiff.
    Step Five: Verify and Publish the Load Sets
    Once uploaded the radmind server keeps the load sets and transcripts in a temporary directory (/var/radmind/tmp/file and /var/radmind/tmp/transcripts). Before making them available you should check to ensure that they were not corrupted in transmission. To do this you use the lcksum utility. From /var/radmind/tmp/transcripts run lcksum -c sha1 -n on base.T and defaults.T -- If lcksum reports any checksum errors you either missed a message from fsdiff back in Step 4 or the file was corrupted in transmission -- You should delete the affected temporary transcript and files directory, rebuild the load set and re-send it. negative.T is treated differently -- Run lcksum -c sha1 negative.T to update the checksums in negative.T to reflect the zero-length data that is uploaded. Once you are satisfied with the results of lcksum, move your transcripts from /var/radmind/tmp/transcript/ to /var/radmind/transcript and the load set directories from /var/radmind/tmp/file/ to /var/radmind/file . You can now create Command files under /var/radmind/command that refer to these transcripts.
    Step Six: Creating command (.K) files on the server
    Command files are lists of transcripts to apply to a given client. Every entry in the command file begins with p (indicating a Positive transcript), or n (indicating either a Negative or Defaults transcript). When creating your Command files, you should list transcripts in order of precedence -- Base first (as a positive transcript), then Negative transcripts (obviously as negative), then Defaults (also as negative. I'll talk about patch sets in part Four, but they are Positive transcripts that get added to the Command file after everything else.

    As promised, this part comes with EXAMPLE CODE -- YAAAAAY Examples! The tarball below contains a script that creates Base, Negative and Defaults transcripts semi-automagically (Steps 2-4). As a bonus it also creates a command.K file that you can copy up to your server (Step 6). All you need to do is populate the negative.list and defaults.list files and the script does the rest. For your downloading pleasure: radmind-tools.tbz
    In the next part I'll talk about deploying the radmind base load to an unsuspecting target machine.


    mikeg's blog on : Managing Multiple FreeBSD Machines with radmind -- Part Three

    Show preview
    This is part Three of an N-part series (I'm thinking 4 parts, plus an epilogue) discussing the investigation of radmin as a patch/deployment tool for FreeBSD. This part deals with initial deployment of machines using radmind, and it has some prerequisi

    mikeg's blog on : Managing Multiple FreeBSD Machines with radmind -- Part Four

    Show preview
    This is part Four of an N-part series (Definitely 4 parts, plus an epilogue) discussing the investigation of radmin as a patch/deployment tool for FreeBSD. This part deals with creating and deploying patch sets using radmind. I'm assuming you already b


    Display comments as Linear | Threaded

    No comments

    Add Comment

    E-Mail addresses will not be displayed and will only be used for E-Mail notifications.
    To leave a comment you must approve it via e-mail, which will be sent to your address after submission.

    To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.

    Form options