Ubuntu 12.04 on 3TB disks

With the purchase of my new system, I wanted to install Ubuntu 12.04 lts server edition. By then, I was not properly aware of the technical limits of disk size, when installing.

One of the main problems is that at installation, all disks < 2TB can be installed as usual. Bigger disks, however, require a different partition table layout and hence a different label: GPT

So how to deal with bigger disks?

My first installation attempt failed fully. At the ‘install grub’ all kinds of fatal error messages. No clue on why, I rebooted, removed the partition tables and tried again. When that failed also, you start wondering why, so I started to dig around on the interwebs. That’s where GPT came in my sight and the redlight indicating not to use fdisk anymore.

First, I tried to use the systems raid. As motherboards nowadays run Raid 1, I did select the raid option, build a raid set of the two 3TB disks and started the installation. A flunky side effect was the system not being able to recognize the full 2.7TB (which is the 3TB, however set in a different format). Instead, it just recognized 801 GB of raid1 set. Weird, but ok. This installation failed, as mentioned.

On the web I found an article which explained that the raid used by motherboards not necessarily is interchangeable. Having 3TB disks, I do not want to end up not using them when a new system comes in mind. Software raid was at least as good as harware raid – and I’m using software raid for the past 7 years without a glitch. (yes, I know, ZFS is better, but not available during install and I don’t have time for the additional fuzz around replacing. It’s on my todo list)

Converted back to software raid, using the setup below. And, using the rescue option of the installation disk, I used parted to execute the code below to ensure the disks were to be seen as gpt:
select /dev/sda
mklabel gpt
select /dev/sdb
mklabel gpt

Lots of links indicated I had to leave space for GPT bios. In the install menu from Ubuntu, there is an option to select ‘ reserved by Bios’, which is the GPT size. Some said 1M, others said 1G, so I used 100M as the ‘let us try’ option.

This host is to be a KVM server, so small footprint on install. It has 32GB memory, so I need to consider swap. And the optimum performance settings.

Using partman, as part of the instal, I set the disk layout as below:

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048           10239   4.0 MiB     EF02
   2           10240         1964031   954.0 MiB   FD00
   3         1964032        64464895   29.8 GiB    FD00
   4        64464896       650401791   279.4 GiB   FD00
   5       650401792      2516867071   890.0 GiB   FD00  Linux RAID
   6      2516867072      5860533134   1.6 TiB     FD00  Linux RAID

Partition 1 is the EFI partition which is needed for GPT, 2 is the boot partition, as an MD device, 3 is the MD device is swap, and 4 is the remainder of the KVM-host system.

Partition 5 and 6 are 2 separate MD drives, with their own physical volume. This allows me to use the smaller footprinted VMs in partition 5 and the bigger ones in partition 6, leaving sufficient space to use lvm snapshots to make backups.

In each PV, I created a LV per VM and mounted this on a certain directory. Using virt-manager, select ‘edit’ then ‘connection details’ and use the tab storage, to add these directories as volumes for virt-manager. This will allow to select this directory – as a LV within a PV – to be used as storage. (thus allowing the usage of lvm snapshots)

Using a bash script, I execute per VM the creation of the snapshot, the mount on a working direcoty, create a tar file containing the image and then unmount and lvremove the snapshot. Be very careful as using the wrong commands renders your disk unusable…


 

Comments are closed.