Archive for the ‘Hancock’s VMware Half Hour’ Category

HOW NOT TO: Compromise your VMware vSphere Hypervisor ESXi 5.1, 5.5, 6.0, 6.5, 6.7, 7.0, 8.0 by adding to Microsoft Active Directory

Thursday, August 22nd, 2024

In this video presentation which is part of the Hancock’s VMware Half Hour I will show you HOW NOT TO: Compromise your VMware vSphere Hypervisor ESXi 5.1, 5.5, 6.0, 6.5, 6.7, 7.0, 8.0 by adding to Microsoft Active Directory. I will demonstrate the exposure, and discuss how to avoid it.

In this video demonstration the ESXi servers are ESXi ARM 7.0, but the same functionality has been built into ESXi since 5.1.

On the 29th July 2024, Microsoft  discovered a vulnerability in ESXi hypervisors being exploited by several ransomware operators to obtain full administrative permissions on domain-joined ESXi hypervisors.

this publication is here – https://www.microsoft.com/en-us/security/blog/2024/07/29/ransomware-operators-exploit-esxi-hypervisor-vulnerability-for-mass-encryption/

VMware vExperts – Christian Mohn wrote about it here – VMware vSphere CVE-2024-37085 – A Nothing Burger

and Bob Plankers goes into more detail here – Thoughts on CVE-2024-37085 & VMSA-2024-0013

Please have a read of these publications.

Broadcom have issued updates and fixes to vSphere 7.0 and 8.0, and VCF 4.x and 5.x only. There is no security update for 6.7.

https://support.broadcom.com/web/ecx/support-content-notification/-/external/content/SecurityAdvisories/0/24505

HOW TO: Fix Synchronous Exception at 0x00000000XXXXXXX on VMware vSphere Hypervisor 7.0 (ESXi 7.0 ARM) on a Raspberry Pi 4

Tuesday, August 20th, 2024

In this video presentation which is part of the Hancock’s VMware Half Hour I will show you HOW TO: Fix Synchronous Exception at 0x00000000XXXXXXX on VMware vSphere Hypervisor 7.0 (ESXi 7.0 ARM) on a Raspberry Pi 4.

It has been well documented that the Raspberry Pi 4 UEFI Firmware Image can cause this fault which renders the UEFI boot image corrupt. See here https://github.com/pftf/RPi4/issues/97

The UEFI firmware imaged used in the lab in this video is v1.37, it is debated as too whether this has been fixed in later releases v1.37, some suggest rolling back to v1.33 !

For the sake of continuity I’ve included previous EE Videos and Articles I’ve created here

Part 51. HOW TO: Update the VMware vSphere Hypervisor 7.0 ARM Edition (ESXi 7.0 ARM edition) from v1.12 to v1.15 on a Raspberry Pi 4

Part 20: HOW TO: Install and Configure VMware vSphere Hypervisor 7.0 (ESXi 7.0 ARM) on a Raspberry Pi 4

Part 23: HOW TO: BOOT VMware vSphere Hypervisor 7.0 (ESXi 7.0 ARM) from an iSCSI LUN for the Raspberry Pi 4

Workaround and Fix – VMware vRealize Log Insight 8.14.1.0-22806512 to 8.16.0-23264422 upgrade failure

Wednesday, August 14th, 2024

These are my memory dump notes from working with a Failed upgrade, and Snapshot revert failed!

I’ve been meaning to upgrade my VMware vRealize Log Insight 8.14.1.0-22806512 appliance in the #homelab for a while, so I was surprised at first when trying to simply upgrade the PAK file it failed with not enough storage in /tmp, so I tried both these updates

  • VMware-vRealize-Log-Insight-8.16.0-23264422.pak
  • VMware-vRealize-Log-Insight-8.16.0-23364779.pak

but failed, so after SSHing into the appliance and checking all the storage, and removing older log files, I noticed that /tmp is defined as a “ram drive”.

So I increased the memory in the Appliance by 2GB, shutdown, changed the memory, and powered on. SSHed back to the appliance

SSH loginisght storage space

SSH log insight storage space

Using the command 

mount -o remount,size=5G /tmp/

Increased the size to 5GB, so at least the upgrade could complete correctly, I also used 

tail -f /storage/var/loginsight/upgrade.log to check the upgrade status, but trying to apply the Scheme upgrade to Cassandra it failed, this is a stock VMware vRealize Log Insight 8.14.1.0-22806512, so not sure why it failed, and there is not much info on the Broadcom site about VMware vRealize Log Insight.

The GUI stated the Upgrade had failed, and although it stated reverting to 8.14.1.0-22806512 it failed.

So I reverted to the snapshot, I had taken before applying the upgrade, but to my surprise, reverting to a snapshot ended up with a completely non-working appliance, the GUI stopped responding. So reaching out for the backups to restore the appliance, quickly found this VM was missing from the jobs!

Duh! Oh shite, got to fix the appliance now, and this is how I fixed it.

1.SSH to appliance

2. service loginsight stop (be prepared to wait a long time!)

3. /usr/lib/loginsight/application/sbin/li-cassandra.sh –startnow –force

it will respond with

Running Operations for Logs stop……..done
Starting Cassandra…..done

WARNING: Be sure to stop Cassandra before attempting to start Operations for Logs!
In worst case, restart the virtual appliance.

4. nodetool-no-pass flush

5. nodetool-no-pass repair –full

6. /usr/lib/loginsight/application/sbin/li-cassandra.sh –stopnow –force

7. service loginsight start

At this point I still did not have a working VMware vRealize Log Insight 8.14.1.0-22806512.

So I then applied the PAK manually.

8. /usr/lib/loginsight/application/sbin/loginsight-pak-upgrade.py /tmp/VMware-vRealize-Log-Insight-8.16.0-23364779.pak (this was already uploaded via WinSCP to /tmp)

and wait…Log Insight Upgrade

There’s alot of changes in the appliance from 8.14 to 8.16 as you can see above!

and now

Log Insight 8.16

Log Insight 8.16

 

 

 

 

 

Log Insight 8.16

Log Insight 8.16

So I hope my notes helps you upgrade your Appliance if you get stuck!

Oh, and I’ve added the VM to a backup job just in case for future, so snapshots don’t always save you!