andysworld!

This blog is specific to the #intel #optane demo units received as part of the vExpert Program. but later you will observe that this applies to all storage devices connected to ESXi 7.0 or ESXi 8.0.

and again my special thanks to fellow #vExperts –  Mr vSAN, Matt Mancini, and vCommunity Guy for arranging this fantastic opportunity to work with #intel #optane demo units for free in our #homelabs .

These demo units received may have been previously used as part of the #intel loan program.

I received 10 (ten) #intel #optane – Intel® Optane™ SSD DC P4800X Series 375GB, 2.5in PCIe x4, 3D XPoint™

the form factor I selected for my #homelab was U.2 15mm, rather than a PCIe slot in card, because I want to use them, in the storage/disk slot of a server, I could connect them to a U.2 to PCIe card, but I would rather use them as “intended”.  More on the complications of that later with my #homelab.

For ease, I did quickly connect them all to a recommended (thanks Mr vSAN) – StarTech.com U.2 to PCIe Adapter – x4 PCIe – For 2.5″ U.2 NVMe SSD – SFF-8639 PCIe Adapter – U.2 SSD – PCIe SSD – U.2 drive (PEX4SFF8639) for testing and formatting in my test bench. In fact I’ve now got a bucket full of these cards, I’ve tried and tested, how difficult can it be to connect a U.2. NVMe interface to a PCIe slot, when some cards are £50 GBP, and other cards are available from that well known China website for £1.99 GBP ! and some are described as Low Profile – NOT! – but more on that later!

You may notice if you look through the above photos, there is one U.2 #intel #optane unit with a RED DOT! Read on.

This because it was faulty, ah, or so I thought! I must admit, it was very odd, because it worked in Windows 10, and on checking in ESXi and Ubuntu, the devices were present.

But when trying to create a datastore, no device was available to create a datastore.

BUT BUT BUT After discussions with Mr vSAN,  Mr Todd (MrVSAN) suggested checking that the #intel #optane SSD how not been formatted to/with LBA 4K sectors! Because ESXi 7.0 and 8.0 does not support LBA 4K. I was surprised that it would not list the SSD device!

Interestingly on Twitter at the same time, another vExpert was also having similar issues!

Checking with an Ubuntu Live “CDROM” USB flash drive

 

 

Using the nvme-cli, which you can pull using sudo apt-get install nvem-cli, and use the command sudo nvme list to list NVMe devices

to check the LBA format – sudo nvme id-ns -H /dev/nvmeXnY | grep “LBA format” you can see in the screenshot below I have two NVMe devices /dev/nvme0n1
/dev/nvme1n1 and both show

[3:0] : 03x Current LBA Format Selected

if you look at LBA Format 3: it states Data Size – 4096 bytes !

 

Argh! 4K ! lets just reformat with – sudo nvme format -l 0 /dev/nvme0n1, it does not display any progress, as there is no -v verbose option, but eventually it will respond with Success Formatting Namespace:1

for #shitsandgiggles I’ve left /dev/nvme1n1 formatted as 4k, but now above you can see /dev/nvme0n1 is 512 sectors, so now back and restart ESXi. I’ll do a quick video on /dev/nvme1n1 for Hancock’s VMware Half Hour

If I now check the storage devices in ESXi, there is a Local NVMe SSD available

and if you now try to create a datastore – Viola!

You will notice from the above storage device list the 4K formatted NVMe device is still missing. Q.E.D

Anyway kudos and my sincere Thanks to Simon Todd aka Mr vSAN !

So onward with my #intel #optane #homelab journey more later!

I’ve been desperately searching for inexpensive 10GBe SFP+ network interface cards for use with my new #intel #optane #homelab #vSAN which support both VMware vSphere 7.0 (ESXi 7.0) and 8.0 (ESXi 8.0), the major reason is so I can use the same lab for vSphere 7.0 and then upgrade to vSphere 8.0.

You can use the VMware Hardware compatibility list to find them, but it does take some searching and testing, so I’ve settled on the following network interface cards

Dell 0Y40PH Broadcom 57810S Dual Port 10GbE SFP+ Network Card Low Profile Dell P/N: 0Y40PH – this uses the qfle3i driver in ESXi 7.x and 8.x.

Dell 0XYT17 Intel X520-DA2 Dual Port 10GB SFP+ NIC with SFP this uses the ixgben driver for ESXi 7.x and 8.x

which I’ve tested both with ESXi 7.0 and ESXi 8.0.

What does surprise me, in it’s taken many months to find the first card above, which is present on the HCL, and also functions with ESXi 7. and 8.0, without adding any additional drivers, e.g. it works out of the box!

We were scrapping many Dell R910 from the datacentre, and I was surprised after testing that the Dell 0XYT17 Intel X520-DA2 Dual Port 10GB SFP+ NIC with SFP still functions with ESXi 7.0 and ESXi 8.0, and was manufactured 10 years ago, and is still present on the HCL today! Well done #intel for stopping these cards going to landfill!

This is a bargain card and firmware is still available and can be updated from Dell ! From memory 21.x !

So ideal for an inexpensive #homelab

Argh, I’ve got one issue, more later on that!