Transition to ESXi

With the release of vSphere 4.1, VMware has publicly stated that the "fat" install of vSphere will slowly transition into the abyss. Quote from VMware:

"VMware ESX. VMware vSphere 4.1 and its subsequent update and patch releases are the last releases to include both ESX and ESXi hypervisor architectures. Future major releases of VMware vSphere will include only the VMware ESXi architecture.

• VMware recommends that customers start transitioning to the ESXi architecture when deploying VMware vSphere 4.1."

Before getting into my personal comments on ESXi, let's look at some architectural differences of ESX and ESXi (from VMware):
Within ESX, we have the Service Console which manages any connection points within the VMkernel (SSH, CIM, vCenter agents, etc.). As many of you know, ESX is about a 2GB install.


Within ESXi, the Service Console is no longer used for management - integrated VMkernel agents are utilized (VMware API such as the Management Framework and CIM). Contrary to competing hypervisors, ESXi and ESX have full operability within the VMware environment - each of these can utilize the same exact tools and advanced features. Compared to ESX's 2GB install, ESXi runs around 70MB.

Majority of our customers are comfortable with ESX - this is due to advanced troubleshooting within the Service Console. Previous versions of ESXi had limited functionality within their "Tech Support Mode" (emulated, stripped down *nix shell). So what's new within 4.1 for advanced logging and troubleshooting? (from VMware)

"- vCLI Enhancements. vCLI adds options for SCSI, VAAI, network, and virtual machine control, including the ability to terminate an unresponsive virtual machine. In addition, vSphere 4.1 provides controls that allow you to log vCLI activity. See the vSphere Command-Line Interface Installation and Scripting Guide and the vSphere Command-Line Interface Reference.


- Lockdown Mode Enhancements. VMware ESXi 4.1 lockdown mode allows the administrator to tightly restrict access to the ESXi Direct Console User Interface (DCUI) and Tech Support Mode (TSM). When lockdown mode is enabled, DCUI access is restricted to the root user, while access to Tech Support Mode is completely disabled for all users. With lockdown mode enabled, access to the host for management or monitoring using CIM is possible only through vCenter Server. Direct access to the host using the vSphere Client is not permitted. See the ESXi Configuration Guide.


- Tech Support Mode Enhancements. In ESXi 4.1, Tech Support Mode is fully supported, and is enhanced in several ways. In addition to being available on the local console of a host, it can also be accessed remotely through SSH. Access to Tech Support Mode is controlled in the following ways:

• Both local and remote Tech Support Mode can be enabled and disabled separately in both the DCUI as well as vCenter Server.
• Tech Support Mode may be used by any authorized user, not just root. Users become authorized when they are granted the Administrator role on a host (including through AD membership in a privileged group).
• All commands issued in Tech Support Mode are logged, allowing for a full audit trail. If a syslog server is configured, then this audit trail is automatically included in the remote logging.
• A timeout can be configured for Tech Support Mode (both local and remote), so that after being enabled, it will automatically be disabled after the configured time."

In my opinion, the biggest feature here is the advancement within Tech Support Mode - we are now able to remotely log in via SSH and perform advanced features. Granted the command-set will not be as robust as ESX, core functionality is still there.

Due to this transition, I would suggest everyone start baking ESXi in their environments. One other suggestion is start getting used to the vCLI/vMA tools from VMware. Each of these tools will make your life a little easier for advanced diagnostics and configurations within ESXi. Some other thoughts:

- Ease of install: No need to layout disk slices within the Service Console. I've seen potential issues due to mis-configurations of either the /var or /home directories when Administrators have used this to store VM files or patches. Furthermore, ESXi installs rather fast compared to ESX.

- Security: I think this will be a driving factor around the security community. Without the Service Console, there will be less entry points within the hypervisor. ESX is still secure, but it's a distro of Linux - people can exploit certain vulnerabilities to gain unauthorized entry.

- No need for RAID 1 local SC Install: Due to ESXi's small footprint, why not use an internal USB key? Most blade systems and servers are now coming with internal USB dongle functionality. Rather than shell out funding for a pair of 36 or 72GB drives with a RAID controller, let's use something small and efficient.