[ Prev ][ Table of Contents ][ Front Page ][ Talkback ][ FAQ ][ Next ]
LINUX GAZETTE
...making Linux just a little more fun!
Replicating a Linux System - Yet Another Method
By Ben Okopnik

Why Replicate?

Replication - sometimes referred to as cloning - is the process of copying an installed filesystem from one machine to another (usually to a new machine) instead of going though the entire installation process again; a good thing indeed. Given the speed at which computers improve, by the time you have your system tweaked and tuned exactly the way you like it - usually at the cost of dozens if not hundreds of hours of playing with it - the hardware is outdated. Replication allows you to conserve all that effort by simply transferring the entire configured system to new hardware.

What makes replication a uniquely wonderful experience under Linux is the large number of options for doing so, which implies flexibility and the ability to adapt the method to your exact situation - just as I have here. Other OSes are generally confined to one rather restrictive method (in effect, "dd"ing the hard drive contents to an identical drive.) With Linux, you can pick whatever method suits your needs. However, there is a cost - the same one that applies to just about anything else in Linux. As the XFree folks say, "don't set policy - provide a mechanism." What you have to do, rather than blindly running a provided menu-driven application (what is that thing doing behind the scenes? Can you troubleshoot the process if it goes wrong?) is a) understand the process, and b) choose the tools that will make it happen. In the world of Unix, where "small sharp tools" is a major tenet of the underlying philosophy, this is the default way to work.

A Little Insanity Goes A Long Way

Before you read any further, A WARNING:
This is not the standard way to replicate a Linux system. What is described below was an experiment just to see if it could be done. Yes, you could probably repeat my results - you might even find it as convenient as I did to do it this way - but if you mess it up and end up causing clogged toilets in New Guinea, earthquakes in Abu Dhabi, and war between Earth and the intelligent cephalopods of Beta Cygni 4, don't blame me!
It seems that for a number of people, myself most emphatically included, the best way to learn how something works is to take it apart and then put it back together (usually turning a five-dollar toy into a three-day-long project and seven gallons of sweat, but - oh well.) So... about a week ago, I was given a computer, somebody's "outdated" box. (Wind*ws is great. No, really. There's no way I'd run it, but I love the effect it has: it makes people think that a P3-400/64MB machine is "outdated". There are always going to be a lot of uninformed folks out there, so I can see a lifetime of free computers ahead of me! Somebody else here wants to give me a really nice 486 Gateway laptop, too...)

Of course, as far as I'm concerned, new computer = Linux installation. That's like ham and eggs, bread and butter, agua caliente para chocolate (yes, I'm getting a little hungry here, but you get the idea.) It's just natural. However, my install CDs are old enough that I have to wipe off the Tyrannosaurus footprints - I mean, they're almost a year old! - so I wasn't going to do that. Hmm. My Net access these days consists of walking over to the marina office (I live on a sailboat) and plugging into a phone jack - dragging over a desktop PC and leaving it plugged in for a week wasn't going to be looked on favorably. Scrap that scenario. I could order the latest CD set from CheapBytes.com, but I didn't feel like waiting for several days: I had a time slot now. This was going to take a little planning.

My laptop has a well-polished install of Debian on it - I've had quite a while to get it all set up exactly the way I want it. I could just "dd" everything over, and - oh. The laptop drive is 12GB; the drive I dug up was an old 6GB that had a 2GB partition with some historical data on it and a really ancient Debian install. I didn't want to just scrap the data, didn't feel like messing around with my backups and tape drives, and 12 won't fit into 6 anyway. Let's see. A fair chunk of that 12 is my music collection; heck, I'll just leave it on the 'top. Next, there's my "Data Dump": an FTP archive backup, a mirror of the Comprehensive Perl Archive Network (CPAN)... yeah, all that could stay there too. A little decision-making, and my entire kit of programs turned out to fit into about 2GB - just what the doctor ordered! All except one thing, that is - how would I get it from here to there, keeping all the permissions, etc. correct? And what other problems would I run into?

There are a number of standard methods for installing Linux: CD or other media, an archive or a loop-mounted image via a network, raw copying between same-sized drives with "dd", and - closest to what I wanted - a filesystem dump between identically laid-out systems (see Jim Dennis' tip in LG#68, "Bulk File Transfers from Wind*ws to ???".) However, I wasn't doing any of those, not even the last; instead, I was going to transfer bits and pieces to a different FS layout. Would it work? In a move strongly fueled by way too much machismo, entirely too little sense, and Mike Orr's column in this very magazine (see his "The Foolish Things We Do With Our Computers" [look in the Index of All Issues to find it] -- perhaps this whole article belongs in that category!), I said to myself: "Self... let's do it anyway. We can always restart from scratch."

Diving Head-First Into The Dark Waters

One of my main tools for this project was the Bootable Business Card from the LNX-BBC project. I booted the new machine with it and used "mke2fs" with the "-j" option (this creates a journal on the new partition, something I needed since I use ext3 on my laptop) to reinitialize "/dev/hda2" - this is where Linux would live. Next, I mounted "/dev/hda2" on "/mnt/0", activated the network card by entering "insmod dmfe" (the new desktop came with an on-board Davicom), assigned it an IP address ("ifconfig eth0 192.168.0.5"), and I could now "see" the laptop - which was ready and waiting at 192.168.0.3, courtesy of a NetGear PCMCIA card and a crossover cable.

I don't run any network services on my laptop by default - one of the first things I do in almost any install is disable "inetd" - but here, I wanted to use "sshd" for a specific reason. Therefore, Baldur (the laptop) got a command of "su -c '/etc/init.d/ssh start'", and Ymir (the desktop) got "rsync -e ssh -avz --exclude-from file.lst root@192.168.0.3:/ /". The exclude list, a file called "file.lst" (files and directories that "rsync" should ignore) was a fairly short one: "/proc/", not being a "real" filesystem, didn't need to be copied (in fact, doing so would be a bad idea); however, I did make an empty "/proc" directory on the new partition. I skipped "/lost+found/" as well - one was automatically created on Ymir when I ran "mke2fs" (besides, it's not just a simple directory; see "mklost+found" for more info). I also most carefully did not copy the journal-related files ("/.journal" and "/.memdump") from Baldur; they would have no relation to the disk state on Ymir, and could probably screw things up pretty badly. Last, I added the directories that I didn't want to copy to the new system.

Once done with the copying - which took a while, even at 100Mb/s - I did a "chroot /mnt/0 /bin/bash" on Ymir, which made this new filesystem the root ("/") and ran a shell in it. Success! I thought of this as the "three-quarter-way point"; most of the moving was done, and the only remaining critical thing was making the system bootable. Of course, I had to do a bit of system-specific setup: modifying "/etc/fstab" to reflect the correct partitions, fixing "/etc/modules" to load the right ones, adjusting "/etc/lilo.conf" and running "/sbin/lilo" to boot the right image... oh, whoops. Baldur boots via "initrd"; the "initrd" image includes information about which partition gets "pivoted to" as root, and that was different between Baldur and Ymir. OK, then - I jumped over to Baldur and ran "mkinitrd" with the "-r /dev/hda2" option, which built me a new "initrd" image with the correct partition as the root. I "rsync"ed it to Ymir, and life was lookin' good.

As a last step before rebooting, I ran "/sbin/lilo -v3", and... aaarrgh. I got a message about the boot sector being made by a previous version of LILO (remember that ancient Debian distro that had been on there?), and the new one simply refused to rewrite it! <sigh> Clearly, this would not normally be an issue, but this time, it was a pain. Ah - LILO's manual mentions the "QuickInst" script that is supposed to be used for initial installs only - perfect. (See LILO's "doc" directory.) "QuickInst" walked me through setting up some basics, and... <laugh> it didn't work - the "lilo.conf" that it wrote was too simple, and didn't match my setup. That was OK, because it did rewrite the boot sector; I simply copied the correct version of "lilo.conf" back into "/etc", re-ran "lilo -v3", and All Was Cool.

I then rebooted, swapped the BIOS settings back so that Ymir would boot from the HD rather than the CDROM, watched a few errors fly by... and was greeted with a login prompt. Hurrah! That was 90% of the way there; the rest was just more tweaking. For the record, the things that produced the errors were: "hdparm" disk tuning parameters had to be changed, and one old module was still loading from "/etc/default/hotplug.usb". Other than that, all that remained was much like setting up a new system, although a lot less work. Let's see:

That was all of it. It's been more than a month now and I haven't had any trouble at all. Sure, I've left a lot of stuff out of this description in order to make myself look smarter - I probably rebooted, switching between the LNX-BBC CD and the HD, about a dozen times because I'd forgotten "just one little thing" - but the above was the critical path, the stuff that had to happen to make it work. About two hours of work, and I had an up-to-date Linux system that's all tuned to my preferences - setting it up from scratch and getting it all adjusted would have taken much, much longer.

After-Action Report

As a result of doing this, I learned a number of important things: a little more knowledge about the "initrd" boot process, a little more experience at fixing LILO problems, what exactly is affected when you move from one system to another - and most important of all, how to do this process and its associated pitfalls. As well, it's another tool in my "knowledge toolbox" - and that's always a good thing. In fact, the laptop on which I'm typing this - a brand-new Dell Inspiron - had Wind*ws XP on it just a short while ago... :) It is now a very smoothly operating Linux 'top. The actual time that I spent on the cloning, excluding the transfer time (12GB takes a while to move across), was about 10-15 minutes; experience is a wonderful thing.

Happy Linuxing to all!

References

Copyright © 2002, Ben Okopnik. Copying license http://www.linuxgazette.com/copying.html
Published in Issue 83 of Linux Gazette, October 2002
[ Prev ][ Table of Contents ][ Front Page ][ Talkback ][ FAQ ][ Next ]