Understanding HP Flex 1. Mappings with VMware ESXv. Esx Server 3.5 Serial' title='Esx Server 3.5 Serial' />Sphere Virtual. Kenneths Blog hq. Virtual hire quality. Virtual. Kenneths Blog hq. Virtual. Ive written this blog as an add on to Frank Dennemans blog about Flex 1. Goal of this blog is to get a clear vision about the Flex 1. HP uses to facilitate their blades with NICs, with the special focus towards VMware ESXv. Sphere. If you are looking for the HP Flex. Fabric mappings click here. First we start of looking at the NIC to Interconnect mappings. These are pretty straight forward and should be known to all HP c Class Administrators. In our example we use HP BL4. G6 blades with 4 Flex 1. NICs two onboard and two provided via a Dual Port Mezzanine CardPlease note that the connections that are drawn below are hardwired connections on the Backplane of the HP c. Enclosure. The reason that we use Mezzanine Slot 2 instead of Slot 1 is due to the fact that we have other servers in the enclosure as well that already have a connection via Mezzanine Slot 1So, our VMware v. Sphere Host is physically equipped with 4 1. E19045-01/blade.x6220/820-0045-16/figures/chap_vm-5.gif' alt='Esx Server 3.5 Serial' title='Esx Server 3.5 Serial' />Sanjeev Naldurgkar is a Technical Marketing Engineer at Cisco Systems with Server Access Virtualization Business Unit SAVBU. With over 12 years of experience in. Design and Deployment of Cisco HyperFlex System for a Hyperconverged Virtual Server Infrastructure with HX Data platform 1. VMWare vSphere 6. U2. I have just gone to add a serial connection from my APC UPS to my management server which is running under on a VM on ESXi. I powered down the VM and then. GB NICs so you would expect to see 4 vmnics in ESX right. Wrong The HP Virtual Connect Domain virtualizes each 1. GB NIC and creates 4 Flex. Nics for it. After doing some math  we can conclude that we will get  1. ESX Host. The image below shows us that we get 4 Flex. Nics per Port and how these Flex. Esx Server 3.5 Serial' title='Esx Server 3.5 Serial' />Nics correspond to a vmnic from within ESX. So in the image above we see that for example Port 1 from the Onboard Adapter is divided into 4 Flex. Nics 1. A, 1. B, 1. Kansas Out Of State Hunting License Cost. C and 1. D. PCI numbering and thus the order in which the vmnics are numbered within ESX is based on 1. A onboard, 2. A onboard, 1. B onboard, 2. B onboard, 1. C Onboard, 2. C Onboard etc. Notice that the first 8 vmnics are from the Onboard Card and the second 8 vmnics are from the Mezzanine Card. From within the HP Virtual Connect Manager we can divide the available 1. GB speed over those 4 Flex. Nics, for example we can give 1. A vmnic. 0 1. GB, 1. B vmnic. 2 7. GB, 1. C vmnic. 4 1. GB which will leave us with 1. GB to give out for 1. D vmnic. 6. Since v. Sphere has much better i. Secrets Of The Millionaire Mind Declarations Pdf more. SCSI performance than ESX 3. GB bandwidth to connect the Left. Hand i. SCSI storage. Technically this means that we give 1 Flex. Nic 1. 0GB which leaves us with 0. GB to share among the other 3 Flex. Nics remaining per port. The image below shows how the technical design looks now From a Virtual Connect Manager perspective we used the following settings in the attached Server Profile see image belowPleaste note that we defined all 1. NICs and left 6 of them Unassigned. The Unassigned ones are the Flex. Nics from Mezzanine Slot 2 which didnt got any bandwidth assigned to them as you can see in the Allocated Bandwidth column. So for i. SCSI we selected MZ2 1 A en MZ2 2 A as the 2 links with 1. GB allocated, leaving 0. GB for MZ2 1 B, MZ2 2 B etc etc. The final picture from v. Switch perspective looks like this, where we separated Service Console 1. GB v. Switch. 0 VMotion 7. GB v. Switch. 1 Fault Tolerance 1. GB v. Switch. 2 VM Networks 1. GB v. Switch. 3And gave the full 1. GB to the i. SCSI Storage. Switch. 4Please note that the above design contains two single point of failures, whenever the Onboard NIC fails my whole front end fails same story whenever the Mezzanine Card fails, in that case my whole storage will be lost. Customer constraints however kept me from doing it the way displayed in the image below which obviously is technically the best way. In the image below we also cover hardware failure from either the Onboard or Mezzanine Card. So now that Ive explained the mappings from Virtual Connect Flex. Nics towards ESX vmnics lets take a look at the rest of the Virtual Connect Domain configuration. There are two Shared Uplink Sets SUS created          FRONTEND which controls the COS, VMotion, VM Networks and Fault Tolerance          STORAGE which controls the physically separated i. SCSI Storage LAN. The FRONTEND SUS is connected via 4 1. GB connections towards two Cisco 6. GB Active2. 0GB PassiveThe STORAGE SUS is connected via 4 1. GB connections towards two Cisco Nexus 5. GB Active2. 0GB PassiveWord of advice Its recommended to use Portfast settings on the endpoints of the Shared Uplink Set connections. While doing failover tests we noticed that our networking department didnt turned on Portfast as we had requested which resulted in spanning tree kicking in whenever we powered on a Virtual Connect Module. Word of advice Next issue we ran into where some CRC errors in the Virtual Connect Statistics while the Ciscos didnt register any CRC errors. These errors disappeared when we defined the Shared Uplink Sets as 1. GB static speed instead of auto. Last word of advice while implementing a technical environment like this its crucial to test every possible failure, from single ESX Host to all the separate components. Ive wrote very detailed documents about it and it helped us discover a very strange technical problem which Im currently investigating. Update 0. 4 1. 2 2. Since this post is one of my top articles I decided to write more about the extensive testing, read all about it here. For those who are interested Ill explain the strange technical problem to close off of this blog Whenever a Virtual Connect Module fails, the downlinks towards ESX will fail as well since these are hardwired via c. Backplane. So for example, whenever the module from Interconnect Bay 1 fails, vmnic. ESX whenever configured correctly obviously When the module is powered on again the v. Switch uplinks will be restored. This is the case for Interconnect Bay 1 and 2 but this isnt the case for Interconnect Bay 5 and 6. Whenever we simulate a device failure on Interconnect Bay 5 or 6 we obviously lose our corresponding vmnics connections but they wont come back online when we power on the Interconnect unit again. Currently the only way to get our connection back is to reboot the whole ESX Host. Im currently working on firmwares as it seems that this will resolve my issue. Ill keep you guys posted. Problem update 0. For the last weeks Ive been swapping e mails with VMware Support since updating the HP firmwares Virtual Connect and all the other components  didnt solve the problem. First of all VMware gave me an alternate Broadcom bnx. Next step was to unload the bnx. NIC Link is Down entries. So Ive enabled debugging mode on the bnx. Interconnect Module again and remarkably my vmnic connection got restored this time I did the test again without the enable debugging mode and my connection never gets restored. Very odd I passed out the new details towards VMware and currently awaiting there response.