Today our VDP appliance suddenly power off without any information. I know this because i didn’t get daily report from its appliance. I connect into vCenter and see VDP log which is contain several backup have failed to executed. I wondering why this could be happen? Um.. after doing quick research i found that this because inconsistency when VDP creating snapshot of one of our server’s VMDK. Worse that error also make our VDP now cannot powered on. *sigh*
So i open the KB page of VMware here and get similiar symptom problem.
Cannot open the disk ‘/vmfs/volumes/519dee4c-3f94……./vm_disk.vmdk’ or one of the snapshot disk it depends on.
Here is quick keyword for the solution: “Remove VMDK which is has problem from VDP disk setting”
Then start powered on VDP again.
Horay.. now it can powered on!
But wait.. another problem is coming up. Now from my vSphere Client Virtual Machine show warning notification: Virtual Machine Disks Consolidation is Needed in one of my VM. So, i need to do consolidation where can found in VM > Snapshot > Consolidate.
The question is coming up. Why do we need consolidation?
The simple answer is to unite the several snapshot disk that previously broken. You’ll see at every VM only should have one VMDK (if you not created the snapshot). But if you already have snapshot before so now you should have several VMDK, naming precisely with (NameOfYourDisk-XXXXX.vmdk); XXXX usually represent with number, this called as Delta Disk. If you do not consolidate those broken VMDK it can growing up and make your storage insufficient.
So, now i’ll do the consolidate proses.
[Consolidate process is starting…..]
Another error (again) comes up.
Unable to access file since it is locked
Oh, dear God, please give me an angel to cover this streak problem 🙁
I thought that error related to VMDK which currently use by VM or even VDP. But when i powered off both of VM & VDP those error still appear. Ok, let’s we kill the process of VMDK it self using the steps below:
- Login into ESXI via ssh (you should enable ssh service on your ESXI before)
- Open Datastore consist problem VMDK using “cd /vmfs/volumes/<datastore>/<VMDIR>”
- Locate vmware.log then open with text viewer. I use cat vmware.log to view the log.
- See error regarding locked VMDK with note “Failed to lock the file” (please see picture below)
- View the process ID by using “# lsof | grep <name_of_locked_file>” (please see picture below)
- Kill the process ID using “kill <PID>”
- Restart your consolidation process on selected VM which has still message need consolidation
Picture for step number 4.
Hope you could to do those step i have explained above.