VMware Partners Aren't Putting Much Stock In AWS, Azure Cloud Benchmarking Comparisons

VMware published results of a commissioned benchmarking study last week as evidence that its public cloud Infrastructure-as-a-Service is better than Amazon Web Services and Microsoft Azure from a price-performance standpoint.

Durham, N.C.-based market research firm Principled Technologies, which conducted the study, found that VMware's vCloud Hybrid Service offers twice the CPU power of Microsoft Azure and three times the storage I/O performance of AWS.

VMware, armed with the benchmarking results, said AWS and Microsoft customers would have to pay significantly more to get the same levels of storage and compute power that's available on vCHS.

[Related: VMware: Our Public Cloud Performs Better Than Amazon and Microsoft, And It's Much Cheaper]

The study has generated lots of debate in the cloud industry. Some experts have questioned whether Principled Technologies made an accurate like-for-like comparison between the different clouds, while others have dismissed the study as a VMware marketing stunt.

One VMware partner told CRN he doesn't put much stock in the study because VMware's comparisons don't reflect the way AWS and Microsoft are positioning their public cloud IaaS offerings.

"VMware built their cloud to make old legacy workloads and architectures scream. Amazon -- and likely Microsoft -- would say you are an idiot to transplant old applications into the cloud. Instead, you should rethink your app architecture and design it to take advantage of what cloud has to offer," said the partner, who asked not to be named to avoid damaging his relationship with the vendor.

Mathew Lodge, vice president of cloud services at VMware, said last week that a virtual machine running on vCloud Hybrid Service, with 140GB of storage, was able to maintain a minimum of 3,300 IOPS during the benchmarking. This, he said, is three times what AWS can offer in terms of storage performance.

However, the VMware partner told CRN he doesn't consider this a meaningful comparison because AWS customers don't use its public cloud this way.

"Amazon's take would be that a customer would never need 3300 IOPS out of a single VM, because you should design your application to be distributed across multiple data centers and geographies," the partner said.

AWS has also shown it can scale its cloud to handle huge workloads, such as Netflix streaming, and VMware's cloud doesn't yet have this kind of marquee use case, the partner said.

"There's no way you are doing that in vCHS, because VMware's monolithic architecture wasn’t built to scale out in that manner," the partner said.

NEXT: Why Pricing Is Such A Big Deal In The Cloud Market

Pricing has been an issue for all cloud vendors, as many customers have found out the hard way that a monthly cloud bill can easily go higher than budgeted when expectations aren't managed properly.

Earlier this month, several VMware partners told CRN they're not seeing much interest in vCHS because it's too expensive and doesn’t include unique, must-have features.

"The response to VMware’s public cloud offering has been tepid at best," Gil Lalo, practice director for enterprise architecture at C/D/H, a Grand Rapids, Mich.-based VMware partner, told CRN Monday.

"Amazon paved the way with EC2 and now Microsoft’s Azure is broadly seen as a mature product that is ready for business. VMware’s public cloud offering is seen as a too-little-too-late knee jerk offering," Lalo said.

Unlike AWS, Microsoft, Google and others, VMware doesn't let customers swipe a credit card and get access to vCHS. VMware does publish its MSRP pricing for vCHS, but customers must work with a partner to get details about volume discounts, packaging options and configurations.

For this reason, it's tough to compare vCHS' pricing and performance to other vendors' clouds, Jason Monden, president and co-founder of Dallas-based cloud startup XOcur, told CRN.

XOcur sells a SaaS service that does price-performance comparisons between different vendors' clouds. But it doesn't work with vCHS because VMware doesn't make enough information about the service publicly available, according to Monden.

"With Amazon, you can sign up with a credit card, use their APIs to deploy anything and everything, and determine all of the options -- and the performance and pricing that goes along with it. You cannot do this with VMware vCHS," Monden told CRN.

While Monden has business reasons for wanting VMware to be more transparent, he believes VMware could benefit from a more open approach.

"VMware would raise the bar against the other providers if they made their cloud open and direct to the public," Monden said. "It would then be a more level playing field for them, but it would also expose their pricing, packaging and services to the general public."

VMware and Microsoft declined comment. AWS didn't respond to a request for comment.

PUBLISHED AUG. 11, 2014