The usual disclaimers apply…this is what worked for me – if you try it and it goes wrong and you haven’t taken the normal sensible precautions before embarking on an endeavour like this then please don’t come crying to me.

Step 1 – Partition the new hard disk

I found it easiest to do this from Windows. I connected the new hard disk to the PC and created the partitions using the normal Disk Management tool. You will save yourself some trouble if you “match up” the partition numbers on the old and new disks (i.e. if your Windows partition is on partition #1 on the old disk, make sure it is on partition #1 on the new disk also). On some of the PCs there was a Dell diagnostics partition present (so the Windows partition was actually the second partition). More about this below.

Step 2 – Delete (or rename) the HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices key in the registry

Doing this now will save you a world of pain later on. The problem is that Windows maintains a mapping of volume IDs to drive letters under this registry key

Here, my new drive was mounted as drive N: when I partitioned it. If I do nothing now, Windows will remount it as drive N: (instead of C:) when I later boot from it, which won’t end well. If I just zap this registry key, Windows will recreate it when it next boots (although it will have forgotten about what drive letters you liked for your USB keys etc.) I guess you could just delete the entries for C: and N:…I didn’t try this.

Step 3 – Reboot into Linux

I variously used Fedora Core 8 and Knoppix 5.1.1…both contained the required NTFS tools and both worked perfectly. At this stage, I have both the old and the new drives attached to the PC. I did this with various combinations to attachment from direct PATA to USB-to-PATA or USB-to-SATA converters and they all worked perfectly.

Step 4 – Dell Diagnostics Partition

One of the candidates was a Dell PC and I wanted to preserve the Dell Diagnostics partition, partly because it is a good thing to have and partly – as I mentioned above – to keep my partition numbering consistent between the old and the new drives (which saves a bunch of trouble). I did this by using the gparted utility to create a new partition on the new disk, similar in size to the original (but rounded up to an even cylinder boundary) with a system ID of ‘de’ (copied from the original). Then I simply did a “dump” sector-by-sector copy of the diagnostics partition from the old to the new:-

# dd if=/dev/sda1 of=/dev/sdb1 bs=32768

This worked a treat. I could then recreate my NTFS partition (again using gparted) as the second partition on the drive. It may seem that this rather wastes the effort of partitioning the disk in Step 1 above, but by doing it this way the standard Microsoft boot code is installed in the MBR. Of course, there are lots of other ways the same thing could be achieved, such as copying the MBR from the original disk to the new disk using the command

# dd if=/dev/sda of=/dev/sdb bs=512 count=1

…and then using gparted to resize the partitions accordingly. Either way, be very careful. If you manage to zap the partition table on your original disk at this stage you are going to have a seriously bad day !

The fun isn’t over yet…see below !

Step 5 – Clone the NTFS parition

The next step is to clone the contents of the old NTFS partition into the new one. This is done using the aptly-named ntflsclone command:-

# ntfsclone -O /dev/sdb2 /dev/sda2

In this form, the command clones /dev/sda2 to /dev/sdb2. Once again, be careful. This command doesn’t ask for any confirmation and if you get your partitions mixed up (e.g. specify them the wrong way around) it will hurt.

Step 6 – Resize the partition

At this point, all of the data is copied from the old disk to the new disk, but the filesystem on the new disk still “thinks” it is the same size as it was on the old disk. If you just reboot at this point you will find – to your dismay – that the new filesystem is no bigger than the old one, even though the partition containing the filesystem is bigger.

To fix this, we run ntfsresize on the new partition

# ntfsresize /dev/sdb2

Step 7 – Fix the number of “hidden sectors” in the NTFS Boot Sector

If the new NTFS partition doesn’t start on the same sector number as the old one, you will need to adjust the number of “hidden sectors” recorded at offset 0x1C in the NTFS boot sector.

TODO: Describe using “hexedit” to complete this job

I did it a different way (kind of by accident). I wound up booting from the Windows Vista installation DVD and selecting “Repair”. This located and repaired the incorrect boot sector for me automatically.

Step 8 – Insert the new disk and reboot

All going well, it will boot up exactly as before – except possibly faster – and you will miraculously have lots of available space

Step 9 – Defragment

If your original disk was heavily fragmented (which it probably was if you were running close to capacity for a good while), then the copy is still heavily fragmented. Now would be an excellent time to defragment the new partition.

Other Alternatives

In the course of my research I came across a promising tool called XXCLONE. This works a different way, being file-oriented rather than sector/block-oriented. Apparently, it copies individual files from the old to the new. As an experiment, I cloned one of the PCs using this tool rather than using the procedure described above. Initially all seemed well…I could boot from it and everything appeared to work OK. However, a few days later I was trying to install Microsoft.NET Framework 3.5 and found that it simply would not install. I can’t be certain that the problem was with the method used to clone the disk, but when I went back and recloned using the Linux tools I could then install the same .NET Framework without any problems. If the problem was caused by XXCLONE it is a pity…it is very elegant and simple to use. I have sent some feedback to the developers…hopefully it will be helpful to them.

Acknowledgements

I can’t claim to have worked all of this out from scratch by myself. The most useful online guide I found was Michael Dominok’s at http://www.dominok.net/en/it/en.it.clonexp.html.

Anyway, here’s something which I hope will be useful.

I find it awkward to pick out my Mac’s IP addresses at a glance from the output of “ifconfig -a”.  Here’s a command-line that makes it very easy…

[eamonn@stig ~]$ ifconfig -a | perl -ne '$interface=$1 if (/^([a-z]+[0-9]+:)/); print "$interface $1\n" if (/inet (\d+\.\d+\.\d+\.\d+)/);'
lo0: 127.0.0.1
en0: 192.168.78.112

Now even the most commited command-line disciple is likely to baulk at typing all that, so I have an alias set up in ~/.bash_login.  The quotes are a little fiddly to get right, so here’s the alias…

alias ipa='ifconfig -a | perl -ne '\''$interface=$1 if (/^([a-z]+[0-9]+:)/); print "$interface $1\n" if (/inet (\d+\.\d+\.\d+\.\d+)/);'\'
 
A slight variation of the same thing works on Linux also (with adjustments to compensate for the differences  in output format)…
 
alias ipa='ifconfig -a | perl -ne '\''$interface=$1 if (/^([a-z]+[0-9]+) /); print "$interface $1\n" if (/inet addr:(\d+\.\d+\.\d+\.\d+)/);'\'

I hope its useful…

(click on the photo above for a bigger version) 

The circuit diagram of the master socket is:-

There are two sets of three connectors.  The connectors are labelled L1, L2 and R.  Each connector is directly connected to its counterpart on the other side via a breakable link on the circuit board.  The breakable links are tracks on the circuit board which run between pairs of oblong holes (visible in the photo above on the far right-hand side).  I think the idea is that two lines could be delivered via the same socket by breaking these links (although I have never seen or heard of this being done).  The (two-wire) eircom line is connected to L1 and L2.  L1 from the left-hand side connects directly to pin 6 on the RJ11 socket and via the breakable link to pin 4 in the RJ11 socket.  Similarly, L2 from the left-hand side connects directly to pin 1 on the RJ11 socket and via another breakable link to ping 3 in the RJ11 socket.  Since pretty much all modern telephones only connect to pins 3 and 4 of the RJ11 socket, this is really all you need.  For historical reasons, there is a 1.8uF capacitor connecting L1 on the left-hand side to the R pin on the left-hand side.  This is connected to pin 2 directly (and pin 5 via another breakable link) on the RJ11 socket.  This provides a separate ringing signal for (old) phones that need it.

Apart from the capacitor, the only other component on the circuit board is a 470K resistor.  Between R and L2.  This (in series with the capacitor) provides a load which eircom can use to test the line remotely when there is nothing else connected (even at a few tens of Hertz, the impedance of the resistor will dominate that of the capacitor).

 I had expected to find some sort of surge arrestor or something like that in there but there isn’t one (the equivalent BT master socket has a surge arrestor, but is otherwise identical internally – although obviously the socket is physically different).

 Practically speaking, you can ignore the R pin completely…I think pretty much all modern telephones will derive their own ringing signal from the line and don’t rely on it being delivered separately from the master socket.  So the only significant connections in the master socket are these:-

The idea is that the eircom line will connect to L1 and L2 on the left-hand side (I think these are labelled S1 and S2 in newer sockets) and you will connect your own internal wiring to L1 and L2 on the right-hand side.  Then, in the event of a fault, it is easy to isolate your internal wiring from the bit eircom are responsible for (thus avoiding a costly call-out charge if you haul out and eircom engineer to a fault which turns out to be with your internal wiring !). 

There are stories told of old modems which require you to snip everything except pings 3 and 4.  I have only come across it twice in my long and distinguished career fiddling with such things.  I can’t claim to have a clear understanding of exactly why this is (sometimes) necessary…I have been offered several conflicting explanations.  I think it is to do with some (cheap and nasty) modems shorting some pins together internally.  If anyone would care to volunteer an explanation I will update this article accordingly.

[This post is a slight departure from my stated policy of trying not to increase the average level of inane wittering on the Internet any further by keeping my opinions to myself in this blog, but this particular insight is just too penetrating not to share it with the world. Like all of my opinions, it is - of course - entirely correct ]

I have had an epiphany. I now know what causes pretty much all network problems: GUIs. At first blush, this may sound like a slightly sweeping statement but I have come to believe in it very deeply. Stick with me here and I will explain why.

Take today for example: Myself and one of my esteemed colleagues squandered an hour of our lives dealing with a guy in a secondary school where we support the Internet router. His Internet access was broken. I’ll spare you the long and painful details of the hour…suffice it to say that by the end of it we determined that there was a server sitting between his 140 snotty, insolent teenagers and our router. It took a startlingly large fraction of that hour to glean from this barely-adequate specimen of humanity that this server even existed. We also figured out that – somehow – the server was at the core of the problem. After a conversation reminiscent of having teeth pulled, our hero volunteered that – infact – there had been a change to the server that morning: he had uninstalled Microsoft ISA server off it !! Somehow – and it completely eludes me how anyone can be quite this gormless – it never entered his head that perhaps uninstalling the proxy/firewall software off the server separating the hormonal masses from the Internet router might be somehow related to his current predicament (140 horny teenagers separated from their porn supply and becoming increasingly antsy about it).

So, how do I extrapolate from this to my theory about GUIs being the root of all evil ? Well, if that server had been a Linux server, there is no way on earth that this guy would have taken it upon himself to touch it. The slightly arcane (yes…I admit it) Linux command-line has a way of scaring off people like this who really need most of their brain power just so they remember to breathe regularly and are taking huge risks by trying to apply their limited stock of intelligence to anything else. In short, command-lines have a way of making things appear a little harder to do than they actually are and therefore act as a built-in safety-net, preventing “special” people from trying to do things they are simply not equipped to do. GUIs, have exactly the opposite effect: they allow the dimmest of knuckle-dragging troglodytes to poke and prod at things they don’t really understand until eventually they manage to break it.

I formulated a more limited form of this theory some years ago when I formed the opinion that Checkpoint Firewall-1 was the source of all security problems on the Internet. When I first started working in the field of data security I could never really understand how hackers seemed to be able to waltz past the best of access lists and firewalls as if they weren’t there. How could it be that the hackers were all so clever and the developers of firewalls were all apparently dribbling idiots ? Then, one day, I was on-site with a large multinational customer watching the guys in there trying to get an application working through a Checkpoint firewall. So, they fired up Checkpoint’s very lovely GUI and they added the rule they thought should do the trick. It didn’t, so the relaxed the rule a little further. It still didn’t, so they relaxed it a little further again, and so the cycle continued through several iterations until eventually the application did work and the “firewall” was reduced to the functional equivalent of a piece of wire. At that moment I understood for the first time that security holes were rarely caused by weaknesses in firewalls and far more often caused by mental deficiencies in those charged with configuring them. I also understood that the GUI was at fault: Checkpoint’s (lovely) GUI makes it very easy to set up rules without the bothersome inconvenience of having to have the remotest understanding of what the hell you are doing. If they had a PIX rather than a Checkpoint (these were the halcyon days before PIX Device Manager, when all was right with the world), this would not have happened. The only thing I didn’t grasp at the time was exactly how generally-applicable the GUI theory was.

Most of the CPU utilisation appears to be Kernel-space:-

The obvious place to start is to find out which process(es) are doing the damage:-

Pretty consistently the culprit seems to be svchost with a process ID of 1468…it is running steadily at around 50%.

So what is it doing ? Microsoft’s Knowledgebase article 314056 says:-

The Svchost.exe file is located in the %SystemRoot%\System32 folder. At startup, Svchost.exe checks the services part of the registry to construct a list of services that it must load. Multiple instances of Svchost.exe can run at the same time. Each Svchost.exe session can contain a grouping of services. Therefore, separate services can run, depending on how and where Svchost.exe is started. This grouping of services permits better control and easier debugging.

Easier debugging my ass. Anyway, it goes on to give some information about how to find out what this particular instance of svchost is doing. The command:-

tasklist /svc /fi "pid eq 1468"

shows exactly which services are running under this particular process ID:-

So it only remains to figure out which of these services has gone awry. A little bit of guesswork comes in here, matching the abbreviated names from the output above to services. This isn’t too difficult. All I had to do was stop the services listed one-by-one until the CPU utilisation dropped to zero. At it turned out, the culprit was the “TapiSrv”, the Telephony service. This wouldn’t stop when I asked politely (which was a bad sign all by itself) and when it was the only one remaining the CPU utilisation was still at 50%. When I forcibly killed the svchost process from Task Manager the CPU utilisation finally dropped to 0%.

Hindsight is wonderful. It turned out that the reason the Telephony service wouldn’t stop was because it depended on the Fax service and it was the Fax service that was really stuck. Once I had figured that out it dawned on me that the problem began around the time I was (unsuccessfully) trying to send a Fax to my bank to berate them for their stupidity (but that’s a whole other blog entry). I have worked around the problem for the moment by disabling the Fax service and all is well again (by Windows standards, at any rate). I haven’t yet got around to working out what is wrong with the Fax service (the urgency has passed…I found another way to beat up the bank !)

SCART pinout (female connector seen from the front). Image is taken from Wikipedia.

After a visit to Wikipedia (source of all wisdom) and a bit of poking around with a multimeter, I found a 2.5v signal from the DVD player to the TV on SCART pin 16, even when the DVD player was turned off. This should tell the TV to expect RGB (rather than composite video) from the DVD player but appears to have a negative side-effect. Snipping pin 16 in the SCART cable has cured the problem.

I have spent the better part of the day trying to diagnose an IPSec connectivity problem between a Cisco PIX (version 6.3.5) and a WatchGuard Firebox of some kind (I have no visibility of it). The problem turns out to be rather subtle, so I thought I would share it here.

The configuration on the PIX is fairly standard (IP addresses have been changed to protect the innocent !):-

crypto ipsec transform-set Esp3DesMD5 esp-3des esp-md5-hmac
:
:
access-list VPNToFirebox permit 192.168.1.0 255.255.255.0 192.168.2.0 255.255.255.0
:
:
crypto map VPN 60 ipsec-isakmp
crypto map VPN 60 match address VPNToFirebox
crypto map VPN 60 set pfs
crypto map VPN 60 set peer 2.2.2.2
crypto map VPN 60 set transform-set Esp3DesMD5
crypto map VPN 60 set security-association lifetime seconds 86400 kilobytes 8192
:
:
isakmp key ******** address 2.2.2.2 netmask 255.255.255.255 no-xauth no-config-mode


The ISAKMP (Phase 1) SA establishes just fine, but the IPSEC (Phase 2) SA never comes up. Watching the debug information on the PIX, here’s what happens:-


ISAKMP : Checking IPSec proposal 1
ISAKMP: transform 1, ESP_3DES
ISAKMP: attributes in transform:
ISAKMP: SA life type in seconds
ISAKMP: SA life duration (VPI) of 0x0 0x1 0x51 0x80
ISAKMP: SA life type in kilobytes
ISAKMP: SA life duration (basic) of 8192
ISAKMP: group is 2
ISAKMP: encaps is 61433
ISAKMP: authenticator is HMAC-MD5
ISAKMP (0): atts not acceptable. Next payload is 0
ISAKMP (0): SA not acceptable!
ISAKMP (0): sending NOTIFY message 14 protocol 0
return status is IKMP_ERR_NO_RETRANS


The key is the “encaps is 61433″ line. What this should say is “encaps is 61443″ which is the (old, pre-RFC3947) encapsulation ID for ESP via NAT Traversal. As it is, the PIX has no idea what “61433″ is supposed to be and the SA negotiation fails.

Here’s what the debug output looks like when talking to another PIX (which sends the correct ID):-


ISAKMP (0): processing SA payload. message ID = 177759204
ISAKMP : Checking IPSec proposal 1
ISAKMP: transform 1, ESP_3DES
ISAKMP: attributes in transform:
ISAKMP: encaps is 61443
ISAKMP: SA life type in seconds
ISAKMP: SA life duration (VPI) of 0x0 0x1 0x51 0x80
ISAKMP: SA life type in kilobytes
ISAKMP: SA life duration (basic) of 8192
ISAKMP: authenticator is HMAC-MD5
ISAKMP: group is 1
ISAKMP (0): atts are acceptable.IPSEC(validate_proposal_request): proposal part #1,

Here the encapsulation ID is correct and the tunnel comes up.

The problem only arises when NAT is detected on the path between the Firebox and the PIX…otherwise there is no need to encapsulate the ESP inside UDP.

References

• Negotiation of NAT-Traversal in the IKE – http://tools.ietf.org/html/draft-ietf-ipsec-nat-t-ike-04

I have a good excuse. I figure a blog is a viable substitute for a memory and that is how I intend to use it. I have little intention of banging on about Iraq (and the activities of the evil warmongers there), politics (even though yesterday was election day in Ireland), religion (which I am generally in favour of up to the point where it calls upon me to actually be good) or sex (which I am generally in favour of up to the point where it calls upon me to actually be good). That’s not to say that I definitely won’t impose my view about all of those things on the world, but it isn’t the main idea. Rather, the plan is to use this as a way of writing down lots of stuff that I figured out once and will probably need to figure out again at some point long after I have forgotten how I did it the first time.

You can thank/blame (depending on your perspective) John Dunn for inspiring me to actually do this since it was the arrival of his blog (to which I will link if he ever makes it public ) that finally pushed me over the edge. I feel certain that his comment will be found below before too much longer.

If, in a spirit of benevolence and generosity, you feel compelled to follow me down the path towards being a blogger, I highly recommend WordPress. I found it very easy to set up (it really did take only 10 minutes) and it is real a pleasure to use.