Hi folks,
It’s been a really long time since I’ve posted. So long in fact that I started wondering if I was ever going to get back to it. But here we are and I’m happy say that I just renewed the site and domain name. Better yet, I’m back in a position where I can actually post things. So why am I railing against ESX in my first post back? Is it because I’m anti-VMware or because I’m a Red Hat “fanboy”?
Anyone that knows me, knows the answer to both of those questions is “no”. I’m a huge fan of Red Hat and most of the products (I am employed there, but that doesn’t mean I believe in every product..). If you know me, you know I’m practical and believe in the right tool for the job. And while I can’t always make things “easy”, I can at least keep things from being more complicated than they need to be. And that’s where we are today. Plugging ESX into Red Hat’s distribution of OpenStack makes thing more complicated than they need to be and has the potential to really hamstring you at the same time.
Lets start with the official diagram from the upstream OpenStack documentation (Picture found at http://docs.openstack.org/trunk/config-reference/content/vmware.html):
ESX in OpenStack
In a nutshell, here’s how it works in the diagram.. vCenter in this example has 3 DRS clusters. Each DRS cluster is presented by vCenter to Nova as a single node. And those “single nodes” are scheduled just like any other KVM Nova node. So a request for a new VM comes in, sent to the scheduler, then to one of the Nova compute nodes, then to vCenter. vCenter and DRS decide the specific node within the DRS cluster within there.
This all seems ok, right? Right? So why do I have a problem with this.. Let’s start:
- Introducing non-standard file formats. We have to use VMDK files, even though RHEL-OSP (OpenStack in general) already uses a standard format. Oh, and we have to now sync glance with the VMDK store too. Yay for adding in more processes/procedures/formats. So I’m being melodramatic, but even VMware states that large images will take an ungodly amount of time unless you’re using 5.1.
- Wasting resources. Why would I want to set up, maintain, power, and cool a Nova compute node, just to use it to forward requests? “Hi, I’d like to add latency to my environment please? Here, here, and here would be great.”
- Spend more money. If I’m using upstream OpenStack for free, why would I want to spend money on Vmware? If I’m paying for subscriptions for RHEL-OSP, why would I want to spend money on Vmware? The only “benefit” would be the features that come with DRS – automatic scheduling of resources, and some HA, but DRS itself is an add on to ESX, it doesn’t come with it for free. Part of the point of OpenStack (even with paid subscription the RHEL-OSP) is to save money, not spend more, and spending money on Vmware (with or without DRS) goes against this. There is a reason why the major hosting providers don’t use Vmware – it’s not because it isn’t a good product, it’s because the economics don’t work.
- Different use cases. Virtualization and Cloud service different use cases right now; pets & cattle, if you will. Sure, cloud depends heavily on virtualization, but they’re not the same. RHEV handles HA for VMs, Live Migration, and all of the other things that support your more critical virtualized workloads. RHEL-OSP handles all of the other mass workloads that depend more on scale and economics. And while I definitely see things converging in the next few years, I’d rather save money/resources by using RHEV & RHEL-OSP or even use VMware and RHEL-OSP rather than spend money I don’t need to by putting VMware into the OpenStack infrastructure.
- Ephemeral disks are not currently supported by Vmware. This is a major hit as it is a major feature of RHEL-OSP and OpenStack in general.
- You cannot inject SSH keys into VMs running within vCenter. Again, this is a major hit on a major feature.
- Networking – officially, Vmware supports Nova networking and OpenStack networking. Unfortunately, neither one really scale beyond a proof-of-concept. To get anywhere, you really need to use…. Nicira (NSX)!!! Another add on that will cost you more money. Vmware doesn’t officially support OpenVswitch and results with it are currently spurious. And to top it off, the last update to NSX (Aug 20?) was only for RHEL 6.4, which means it’s only good for RHEL-OSP 4. Nothing for RHEL 6.5 or RHEL 7, which means nothing for RHEL-OSP 5. Not good. (services only engagement..
- Networking pt2 – The last I checked, setting up Nicira was a services only engagement requiring at least 8 nodes. This means the level of complexity for your environment just went up exponentially, not to mention the investment in software, hardware, and support. And it doesn’t support RHEL-OSP 5, or RHEL 7, or RHEL 6.5 (as of this writing).
- I could keep going.. really..
I realize and agree that VMware will likely update the network drivers to include RHEL 7 and RHEL-OSP 5, but do you really want to be dependent on VMware for your network drivers? What happens when you’re ready to go the RHEL-OSP 6 but NSX isn’t? They’re not even a networking company! I’m sorry, but it’s one thing if you’re waiting on Red Hat or Canonical or HP to provide an update to their OpenStack distro – but if you’re waiting on a third party to update a product that isn’t even their core competency, before you can update your infrastructure, that is not optimal.
Lets be clear – I have nothing against VMware. I’ve said nothing bad about what vSphere was designed for. But in the case of OpenStack, it is a shoe-horned approach. VMware’s long term strategy isn’t even to integrate with OpenStack – it’s to simply interact with the API’s.. but that it is another post…
If you’re trying to do OpenStack with ESX because you already know ESX and think that will help you, stop. You’re wrong. Go back and learn OpenStack the right way, using KVM, then figure out other ways of doing things. ESX is a great hypervisor for what it was designed for. But if you want to learn OpenStack, then learn it for real. Don’t hamstring yourself or paint yourself into a corner because you think you can save yourself some time. Every time you reach out to the community for help, you’ll have to preface your statement with “I use ESX”. And you’ll get some help, but at some point you’ll realize that you have to go back and learn the “regular way” anyway, so it might as well be now.
Hope this helps,
Captain KVM
— Many thanks to Art & Steve
