My first urge was to blog about Dell acquiring EMC the same day it was announced. But I decided to give it some time and do what my blog site says - contemplate.
The world of IT is drastically changing in obvious and subtle ways. Interestingly, Dell is blazing new trails making stunning and unpredictable moves. Dell should change its name to "Balls". First they went private and now they buy EMC. Pretty ballsy.
The Good Stuff
1. EMC has a lot of really smart people.
2. EMC has the biggest, baddest and arguably best Enterprise IT sales force in the world.
3. Dell is the king of supply chain, operational efficiency and managing costs. Additionally, the buying power of the combined companies is pretty staggering.
4. The combination of Dell and EMC has created a mega-private company enabling them to work with a bit more autonomy than a publicly held one.
5. EMC is one of the best in performing high tech M&A - if they retain this capability, it will play a major role in their success going forward.
6. Michael Dell will be the CEO. He has proven that he is bold, smart and ambitious. And let's face it - he is a bit of a celebrity in the high tech world and that does make a difference internally (employees) and externally (customers, partners, investment community and media).
7. Dell and EMC are now a total solutions provider and just did a leapfrog over HP and IBM.
The Bad Stuff
1. Dell and EMC have very different cultures. We have seen the damage that culture clashes create (HP+Compaq and Sun+StorageTek). I think Dell can avoid this but they shouldn't ignore it.
2. EMC and VMware are still leaders in the IT world but they are not as strong as they once were and are arguably trending downwards. If you don't agree that EMC and VMware are becoming less relevant, I offer this acquisition as proof - it is the only reason why a deal like this could even happen. Dell, EMC and VMware are all valuable to IT but they are not strategic and they never will be if they just focus on plumbing/infrastructure.
3. As I said - the world is changing. Cloud is here and is a real threat to the status quo. More and more applications are cloud native. Certainly, the complete replacement of on-premises IT infrastructure is far from imminent but it is on the decline and is far less important than it once was.
4. IT departments are transforming from being break/fix support organizations into service providers that are partners with the business. Additionally, there is a digital evolution in progress within the Enterprise. Dell, EMC and VMware are not really a part of this strategic transformation of IT that is already underway.
Dell just became a powerhouse of traditional IT solutions. But traditional IT is on the wane. So while they do all of the necessary heavy-lifting - (streamline the company, lower costs, retain their customers and employees, pay down debt, increase revenue, etc.) - they must waste no time and re-invent themselves from a leader in traditional IT - (which they just became) - and tranform into a leader of the new world of IT.
There has been a great deal of focus on high performance storage. A part of this focus and even hype has been around IOPS with some vendors claiming millions upon millions. At some point all of the IOPS claims become just useless noise.
While IOPS numbers may provide some value, keep in mind that what you test in the lab may have very little applicability in the real world. And what the vendors often claim will most likely be even further afield from the realities of your environment.
Keep in mind that the number of IOPS you get is based on the block size of your applications. Therefore, since block sizes are variable it is difficult to predict what your real world IOPS requirements will be. For example one application can have a 4k block size, another 8k blocks and another 64k blocks, etc.
You are probably already aware that most of the IOPS numbers that vendors tout are based on small block sizes. IOPS testing with a 4k block size will result in twice the number of IOPS as 8k blocks and 16 times the number of IOPS as 64k blocks and so on. One vendor that I was aware of that claimed millions of IOPS was using a 1k block size to achieve really high numbers even though it was not reflectiive of the real world.
I often get the question - is doing a head-to-head IOPS comparison between storage systems using the same workloads valuable to determine which one performs better? And the answer is of course - not necessarily. Some storage systems may work really well with one workload but may act differently when multiple mix-workloads are thrown at it. Often contention of resources can greatly impact performance for one or all applications accessing the storage system. Some architectures react better than others in mixed workload enviroments. It is also essential to understand that most performance tests are done in optimal conditions. I suggest running a performance test and then fail a drive. How well does the storage system perform when it needs to do a RAID rebuild at the same time it is doing read/write operations? How well does it perform when it is also running a replication process? Remember, the backend can also impact primary I/O performance and again, some storage systems are better architected than others.
Testing latency can often be a more accurate predictor of what you will experience in real world environments. IOPS is dependent on block sizes, which is variable and therefore ulitmiately unpredictable. However, latency is more of a constant and therefore more predictable. Storage latency simply means the amount of time it takes for the storage system to respond back to the application. Getting sub-ms response times can result in query times being reduced signficantly.
Analogy: IOPS, Latency and Throughput
I've created a simple analogy that explains IOPS, Latency and Throughput. You ordered something online and it takes two days for that box to be delivered to you. Inside that box is one large item. You then order something else online and it also takes two days for that box to arrive at your door. It is the same size box but within it are 100 small items. In this analogy the Latency is 2 days, which is the same for both boxes. The IOPS is the number of items that were delivered to you. In the first box it was just one item and in the second box it was 100 items. So the latency was exactly the same but the IOPS were drastically different!
In both cases you got your boxes within the same amount of time but the IOPS varied based on what was being sent to you.
This brings up throughput. Using the same box analogy, throughput is the number of boxes being delivered that fit in the truck. Assuming that all of the boxes are the same size, the truck can carry 200 boxes at full capacity. Therefore your throughput, in this analogy, is 200 boxes. Again, the number of items (IOPS) within those boxes will vary greatly based on the size of those items (block size).
Latency is 2 days, maximum throughput is 200 boxes and IOPS is an unpredictable variable.
Caching and Storage Performance
Caching is also important for many storage systems in order to provide good performance. However, it will depend on how cache friendly your application workloads are. Industry-wide the consensus is that you should generally get between a 60-70% cache hit ratio - so the chances are you will benefit from caching.
I believe that partitioning is a good feature for caching because it can address the mixed workload contention issue that I discussed above. I also like caches that allow you to pin data into the cache so that you can ensure high performance of a hungry application. Additionally, the larger the cache memory the more likely you will get cache hits because it will keep more data in the cache pool.
Some storage systems support only read cache and typically reads are the majority of your I/Os. However, you will obviously be creating writes as well (otherwise there would be nothing to read) and a write-back cache can be quite beneficial. Write-back caching is a bit more complicated to implement for the vendors and not everyone has this feature so be sure to ask whether their storage systems support it or not.
Storage performance testing with a cache should be done mindfully. If you are getting 100% cache hits during your testing then it is giving you a skewed view of that storage system's performance. It is improbable that you will get 100% cache hits in the real world. I do think it is important to do performance testing with caching turned on and not just to the disk drives - since caching will play a role in improving performance. Assume between 60-70% cache hits with about a 60-70% of the I/Os being from reads - as a general rule of thumb. You may have specific types of workloads that go outside of these parameters but my recommendations are pretty reflective of the broad market.
The Importance of Storage Performance
Storage performance is not an exact science since there are so many variables. Interestingly, all of the current attention on performance will help to solve many problems for the entire market...eventually.
The goal is to remove performance off the table as an issue by creating architectures with enough head room so that the storage system will never be the bottleneck. We will get to a point where you won't have to worry about storage performance testing because the storage systems will be such speed demons that it will handle anything you throw at it. Such systems do exist today but you typically have to pay a premium for them. Performance is important but what are you willing to pay for it?
In the end it is the combination of price and performance that will alter the landscape forever and not just performance alone. The good news is...this shift is occuring even as we speak.
The storage landscape has gone through a number of transformative stages in the last two decades. EMC and NetApp emerged in the first stage, challenging the status quo and essentially creating the storage networking industry. They made storage that was less expensive and relatively easier to manage, and delivered intelligence to protect and manage data. The second stage saw innovators such as 3Par, Compellent, Data Domain, EqualLogic, and Isilon, all with unique architectures and technologies that, among this very diverse group, delivered far easier management, incredible scalability, capacity optimization, and intelligent tiering.
These two stages were good for customers (i.e., the end-users of these products and solutions). Innovation and competition drove down pricing and created a wide range of features for greater performance, ease of management, scalability, reliability, recoverability, infrastructure optimization, and more.
We are now at the threshold of a third transformative stage of the evolution of storage, one that is focusing very heavily on performance.
Performance Isn’t Everything!
While this next stage of storage is focused on providing transparent and cost effective performance, all of the other capabilities introduced in the previous transformative stages are also requisite. NAS and SAN support, data protection, replication, capacity optimization, highly virtualized storage, and a bevy of data management software features and capabilties combined with performance also define this next stage because each stage is progressive and all technological advances must be integrated if end-users are to achieve maximum value.
Performance For Everyone
You can debate the merits of performance, but the bottom line is that worrying about it should just be taken off the table. Once upon a time, storage was complex, unreliable, inefficient, hard to scale, and expensive—and all of those issues have been addressed. It is now time to remove performance as a challenge and add it to that list of storage issues that no one ever has to think about again.
There are studies that show that only between five and ten percent of all I/Os require high performance using some flash-based technology. But which five to ten percent? When do they need that higher performance? And if that small percentage of I/Os is vying for resources that other less performance-driven I/Os are consuming, will that impact the business? This is why performance should be a requisite just like ease of use, scalability, reliability, and other storage system features. This is why performance is so critical.
If you have an unlimited budget, than performance has always been in your grasp. But, in this next transformative stage of storage, it is essential that performance is commoditized, which can be defined as “the movement of a market from differentiated to undifferentiated price competition and from monopolistic to perfect competition." - Wikipedia
This is essentially what is happening in storage: performance is being addressed within different layers and offerings in the marketplace. And, over time, the preferred methods and solutions will become the mainstays in every data center. We are at the point in the process where the market is just beginning to shake out the winners and the losers. What will eventually emerge are performance solutions priced for consumption by nearly everyone, regardless of their IT budgets.
The reason there is so much attention focused on storage performance right now is that we are at the threshold of delivering the technology at a cost point that enables us to remove performance as an issue for everyone, in every environment, and for every application to ensure that no one ever has to think about performance again. This will impact profitability, user satisfaction, and operational efficiency.
Here is what is going on with Microsoft and Hyper-V:
- From a technology perspective Microsoft Hyper-V is finally a viable alternative to VMware ESX. It has analogous functionality as well has some competitive advantages. It is being used for mission-critical applications in a large number of customers and this will continue to grow.
- VMware finally has a serious challenger and they don't get much bigger and more powerful than Microsoft. This is good for the users - competition raises the the bar on every level including pricing, service and technology. Microsoft is a real market threat to VMware and have the resources, brand and muscle in the Enterprise to change hearts and minds at the C-level. Further, they have leverage that no other company has in terms of everything they offer.
- Interestingly, Microsoft is getting something right that VMware has failed to do and that is delivering a compelling virtualized storage stack. SMB 3.0 provides unique capability and VMware has nothing that even comes close.
- MS still has some significant challenges - most notably Systems Center. vCenter is very easy to use and there are hundreds of thousands of IT folks trained on it. Systems Center is a bit of a bear and rather complex.
Incumbency is hard to overcome in part because the customers have invested time and money becoming experts as well as optimizing and tuning their environments. Ignorance is another challenge - there is a large IT population out there Microsoft has to educate and if they aren't willing to listen...
VMware is a sales and marketing machine and they are focused on one thing - virtualization. Microsoft has a huge portfolio of applications and solutions and therefore far less focused.
VMware has stumbled a bit over the last few years falling short of coming out with the NEXT BIG THING in the virtual data center. However, software defined networking and the NSX is very compelling and will give something else for Microsoft to address going forward. NSX isn't a slam dunk…yet… and VMware has a long road ahead driving it into the market - but it is compelling and has the potential to change the landscape.
- I am associated with the Virtualization Technology Users Group (VTUG.com), which consists of over 5,000 virtualization professionals and we are seeing some real traction with Hyper-V. We are talking to customers considering a multi-hypervisor strategy and some are actually sweeping the floor replacing VMware with Hyper-V. The motivation for this has largely been pricing with huge savings. Of course pricing isn't everything but if all other things are equal (or close enough to equal) naturally cost will be a major factor. Another very important factor is the channel. System integrators, VARs and resellers all are selling VMware - making it difficult to compete. Bringing Hyper-V to their customers creates another conversation and opportunity for the channel. I am seeing some channel partners really embrace Hyper-V as a way to differentiate themselves.
So what happens in the virtualization wars going forward? A large segment of VMware's customer base will be very loyal to them. However, Microsoft will continue to gain in market share getting both greenfield opportunities, as part of a multi-hypervisor strategy and completely replacing VMware in some cases. Microsoft is shooting for full-blown domination of the market re: NT crushing Novell. I am skeptical that this will happen - certainly not any time soon. However, I do believe Microsoft could eventually have the most installations as VMware continues to generate the most revenue. VMware therefore needs to focus more on how to get more revenue from their loyal base. EMC has done a great job doing this with their storage customers and I am certain VMware will take a page from their book.
Additionally, there will be large pockets of KVM (e.g. Sears with 10,000 and eBay with over 40,000 KVM installations) within the data center and Amazon EC2 outside the data center.
Microsoft is on fire. I have rarely seen them so aggressive, focused and productive. In terms of virtualization - they are acting more like a startup (in some ways) than a massive software company. They are developing real intellectual property in record time. It is actually a bit inspiring to see such a large company driving so hard on all cylinders to be a market leader. And talk about a war worth fighting - the virtual data center and the cloud are the future of IT.
I have been working with the Virtualization Technology Users Group (VTUG) . VTUG is an independent virtualization users group with over 4,000 end user members that focuses on all platforms including ESX, Hyper-V, KVM and EC2. The 10th Annual VTUG Summer Slam event is taking place in Brunswick Maine and will have more than 1,000 IT professionals and over 50 vendors. Keynotes include technical evangelists from VMware and Microsoft. This is an excellent event and they end the day with a huge lobster bake that also includes ribs and chicken! And the live music is cool too.
VTUG is doing a pre-event boot camp the day before Summer Slam in Maine. The focus is on Hyper-V for the VMware expert. It is a hands-on lab and is about six hours long - so it will go in-depth. This is a first come, first serve opportunity. The boot camp is free so take advantage of it.
It is interesting how many storage vendors participate at Summer Slam. VTUG is running a program called "Bring Your Storage Peer". And if you do there are four chances to win some cool prizes. Learn more about the "Bring Your Storage Peer" program.
VTUG is branching out its events to other parts of the country with Silicon Valley Slam on August 1. It will take place at the DoubleTree in San Jose. VMware, Microsoft and Amazon will be speaking at this event.
VTUG is now conducting webcast events as well. The first upcoming webcast is Hyper-V for the VMware Expert. Let's face it, Hyper-V is becoming a contender in the virtualization arena. The more you know about it the better it is for you.
All of these events are free to IT end users.
I'll be at all them - I hope to see you there.