Fixing file system corruption in the vCenter Server Appliance

Share on:

I recently had a storage network outage in my lab environment, and after powering back on my vCenter Server Appliance (VCSA) I was rudely greeted with the following information at the console:

failed to start file system check

Ouch!  I’ve never dealt with file system corruption in a VCSA before, and the internet doesn’t seem to contain much information on what to do next.  This post is my effort towards changing that.

The first thing I did was type journalctl to view the system logs, as suggested.  That displayed some additional information on the issue.

journalctl file system check

It suggests running fsck manually, without the -a or -p options.  My background is heavily windows-focused, and I’ve never used fsck before, but I eventually figured out the syntax required to fix the problem, which is to run fsck against the damaged partition listed in the top line of the previous screenshot, i.e. fsck /dev/disk/by-parttitionuuid/79d76ed0-0297-4e33-a6bd-252099f2c613

Thankfully, I was able to use tab completion to save me from typing (and probably mis-typing) the entire partition UUID myself.  Running that command kicked off the fsck process and prompted me to fix probably 10-20 different errors:

fsck output

After completing all that, I got a success message.  It might be a good idea to run the whole thing twice, just to be sure.

success

After that, I rebooted my vCenter appliance, and thankfully it booted normally!  I hope this post helps someone out there who is facing this issue.


comments powered by Disqus