Increasing the disk size on a vCenter Server Appliance in vSphere 6.0

As VMware continues to push in the direction of unix-based appliances for their vSphere management components, those without a Unix background (like myself) are having to come to grips with the Unix versions of common administrative tasks. Increasing the disk size on a vCenter Server appliance (VCSA) is one such task.  In vCenter 6.0 VMware has introduced Logical Volume Management (LVM) which really simplifies the process of increasing the size of a disk, and allows it to be done while the appliance is online.  VMware KB 2126276 covers all the steps required to increase the size of a disk, but this guide will cover it in slightly more detail.

Step 1: identify which disk (if any) has a problem with free space.

To do this, I connect to the appliance via SSH or the console, enable and enter the shell, and use the df -h command.

For more information on using command line tools for working with disk space can be found in my post Useful Unix commands for managing disk space on VMware appliances.

I can see that both /storage/core and /storage/log are 100% used.  I’m guessing that /storage/core is full with vpxd crashdumps that are being generated because vCenter is crashing after being unable to generate logs on /storage/log.  Based on this guess, I’ll increase the size of /storage/log and then manually delete the crashdumps on /storage/core and monitor the situation.  I won’t cover the steps involved in deleting the vpxd crashdumps in this post, but it basically involves deleting core.vpxd.* and *.tgz files in the /storage/core directory.

Step 2: Increase the size of the affected disk using the vSphere Web Client

Looking at the table in VMware KB 2126276, it tells me that the disk mounted to /storage/log is VMDK5.  The way this is presented is a bit confusing in my opinion, because the disk we’re looking for is listed as hard disk 5 in the web client, but the filename of the disk is vmname_4.vmdk (the numbering of virtual disks is thrown out in this way because hard disk 1 is vmname.vmdk, and hard disk 2 is vmname_1.vmdk).  Where the KB article says “VMDK5”, it really just means “the fifth VMDK file”.

The reason my /storage/logs disk filled up is because I’ve increased the logging levels on my vCenter appliance to try to catch an issue that had been occuring.  Because of the increased amount of logs being generated, I’m going to increase the size of this VMDK to 25GB.  I don’t want to go overboard because the disks are thick provisioned by default.

Step 3: expand the logical drive and confirm that it has grown successfully

Return to the SSH session and expand the logical drive(s) that have been resized.  The following command will expand any disks that have had their vmdk files resized.

1vpxd_servicecfg storage lvm autogrow

If the operation is successful, you should see a message similar to the following.

1VC_CFG_RESULT=0

In my case, I did get that message eventually,  but I also got a bunch of the following errors:

1/usr/lib/vmware-vpx/disk_utils.sh: line 3: echo: write error: No space left on device

The reason I saw that error is because my /storage/core disk is 100% used.  As mentioned I’m going to free up space on that drive manually, so I’ll ignore that error for now.

If I run df -h again, I can see that /storage/log is now 25GB in total size.  Job done!

Note: In the vCenter 5.x appliance, increasing disk sizes was a bit of a pain. The operation had to be performed while vCenter was offline, and involved adding a brand new disk, copying files from the old disk to the new one, and editing mount points.  For anyone who is working with a vCenter 5.x appliance, the steps are in KB 2056764.