linux

Fixing Cloud Servers stuck at gPXE Boot

Derek Morton

Jul 22, 2014 • 2 min read

With Rackspace Cloud Servers there's currently an issue with PVHVM resizing images smaller that causes them to lose their MBR; when this happens you'll see the following in the Java console:

gPXE Boot

If you see this your MBR is gone and you'll need to get the server in Rescue Mode (Actions menu -> Enter Rescue Mode) to re-install GRUB to the MBR.

After your server has booted up into Rescue Mode your original hard disk is at /dev/xvdb; you'll need to chroot into your old OS:

# mount /dev/xvdb1 /mnt/
# mount -t proc none /mnt/proc
# mount -o bind /dev /mnt/dev
# mount -t sysfs sys /mnt/sys
# chroot /mnt/

Once you get into the chroot you'll then need to drop to the grub prompt to re-install GRUB to the MBR (the grub-install command doesn't seem to work with Xen guests):

# grub
grub> device (hd1) /dev/xvdb
grub> root (hd1,0)
grub> setup (hd1)
grub> quit

After GRUB is re-installed to the MBR exit Rescue Mode and your server should boot up normally again; if you're seeing this issue after building a server from an image (extreme edge-case) you've still got a bit more work to do.

While still in the Rescue Mode chroot you'll want to reset the root password (it didn't get set on the initial boot since the server didn't boot):

# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully

Once the password has been updated you can now exit rescue mode (yay!!!).

After the server boots up normally you'll still need to do some work via the Java console (Actions menu -> Connect via Console) to reconfigure the networking stack since that also wasn't done on the initial boot:

# uuid=$(uuidgen)
# xenstore-write data/host/$uuid '{"name":"resetnetwork","value":""}'

Wait a few seconds and run the following to ensure things went through successfully (should get a return code 0 as below):

# xenstore-read data/guest/$uuid
{"message": "", "returncode": "0"}

The networking reset will re-read the networking configuration from the hypervisor and get the server network accessible.

Sign up for more like this.