Comment from: Gregg Dickson [Visitor] Email
Gregg DicksonGreat article Tony!
One more benefit of NFS that we learned the hard way when we lost 50+ production servers in a single instant.

No SCSI reservation conflicts!

Because of a bug in a well known vendor's storage monitoring software, we encountered a SCSI reservation conflict on our FC SAN which caused all of our vSphere Enterprise Plus hosts to lose connectivity to the SAN.

With help from VMware support we were able to recover the primary ~2TB VMFS volume that contained all of the VM images within about 2 hours.

As for the ~4TB of RDM's that contained all of our unstructured file shares, we had to recover from backup, a 48 hour outage in all. Not a pleasant experience!

Needless to say, we are planning to move to NFS. As you mentioned, many of the storage vendors aren't even aware of the issues with VMware and SAN storage architectures. They have that “deer in the headlights” look when we ask for an NFS storage solution for VMware.
07/16/10 @ 19:58
Comment from: Tony Asaro [Member] Email
Great point! I can't believe I missed SCSI reservation conflicts - which I heard from a number of IT professionals as an issue. Thanks for the feedback!

It would be interesting to see how you progress on the NFS front. Obviously NetApp provides a solution and I know that BlueArc and Isilon are focusing heavily on VMware as well. Which vendors are you looking at?


07/18/10 @ 20:43
Comment from: Gregg Dickson [Visitor] Email
Gregg DicksonWe are hoping to find a single platform to provide both NFS and iSCSI in the 10TB capacity range. We also don't want to spend a lot of time managing, tweaking and tuning so that narrows the field quite a bit.

The leading candidates at this point are Compellent and NetApp. We are beating the bushes to find other players. We'll take a look at BlueArc and Isilon.

07/19/10 @ 11:48
Comment from: Tony Asaro [Member] Email
My first question is why do you need both? Yes - there are a limited number of vendors that support both. But if you are using VMware why use iSCSI given all of the challenges? I am curious to know - not in a challenging way.

07/19/10 @ 15:15
Comment from: Gregg Dickson [Visitor]
Gregg DicksonFair question. We currently still have Exchange and SQL Server on separate physical servers rather than VMware. And Exchange is clustered using MSCS.

We have been debating migrating the entire enterprise onto VMware but are undecided at this point. We are probably the right size to pull it off with less than 400 employees, moderate email traffic and no heavy OLTP. However, we had performance issues trying to run Citrix on VMware 3.5 supporting ~120 users a few years ago so there is some hesitation within the organization toward moving resource intensive apps/servers back into that environment.

Part of the issue back then (2006) was our lack of understanding of the horizontal scalability paradigm of VMware. We were only running a pair of VM's on each of a pair of ESX hosts. This config resulted in 25-30 concurrent users per VM during peak periods. That wasn't a problem except when a rogue process hung one of the VM's or consumed massive amounts of memory, which happened too frequently on that version of Citrix with the application mix that we were running. Knowing what we know now, we should have been running 4 or even 8 VM's on each server and it probably would have been a fantastic combination.
07/19/10 @ 16:50
Comment from: Tony Asaro [Member] Email
And VMware has improved performance a ton since then as well. Not only in the overall storage stack but specifically with IP networks. They improved both the iSCSI and NFS stack.

07/20/10 @ 00:32

Leave a comment

Your email address will not be revealed on this site.
(Line breaks become <br />)
(For my next comment on this site)
(Allow users to contact me through a message form -- Your email will not be revealed!)
« Dell and 3PAR = No BrainerThe Emperor Has Too Many Files »