I've been pondering Cisco and their Unified Computing System (UCS) and decided to finally blog on it. I participated in a InfoSmack podcast that included a brief discussion on UCS and recently talked to Cisco's Jesse Freund - who manages analysts relations for the Cisco UCS. My first reaction when Cisco announced getting into the server business was - well let's just say I was a bit skeptical. First , Cisco UCS upsets the IT ecosystem, which is not necessarily a bad thing but it does have ramifications that need to be considered in terms of go-to-market, the impact to Cisco's channels and alliances, and of course it creates natural enemies that are big and powerful enough that even Cisco needs to be mindful of them - including Dell, HP and IBM. Certainly all three are already competitors on various levels but they also have alliances and must live together in the data center. But now Cisco is going after their bread and butter server business, which is a whole new ball game. The second major reason to be skeptical is that the core competencies needed to sell and support servers is different than supporting networking devices. Cisco doesn't have the operations, sales, marketing, support, on and on and on. Third, server margins are not nearly as healthy as typical Cisco products. Do you want to focus your time and resources making 60 or 70 cents per dollar or 20 or 30 cents per dollar? The math is pretty simple. Related to that the server market is boring - it is declining and not growing. Now this is a big deal to Cisco. Clearly a lot of thought went into this. And there are some really smart people that work there. So what is the business case that makes UCS compelling and strategic? Now maybe everyone reading this is already way ahead of me and get why this is important but I haven't read or seen anything concerning UCS that really focuses on my hypothesis and why this matters: Cisco isn't really going after the server market per se but rather it wants to own the core of the data center. It wants to own the infrastructure that runs applications that run the business. At the heart - at the very core of the data center are compute resources - CPU and memory. "Owning" the core of the data center - can make you strategic, valuable and essential. But it will take a lot more than just providing commodity CPUs and memory to own the core of the data center. Server virtualization by its very nature enables you to move from one CPU to another transparently. That is why if you want to own the core of the data center you need to really consider the value add provided in addition to the compute resources. And that value add has to be very, very sticky. Additionally, it should be something you are already really, really good at. For Cisco the first step in the value add strategy is a natural and obvious - go to where you are strongest - and with Cisco that is networking. Therefore the other part of the UCS strategy is focused on converged networks, everything over Ethernet including the last big hold out, Fibre Channel. Fibre Channel over Ethernet (FCoE) becomes very important to them especially since networked storage is a critical part of the data center. A converged network using Cisco infrastructure including LAN, backup, management, VOIP and all storage protocols including iSCS, CIFS, NFS and FCoE results in lower cost, less support, and the reduction of cable spaghetti. The Cisco vision is to have all roads lead to Cisco UCS via a converged Cisco network. By binding the core of the data center to the network - the arena that Cisco owns - gives them a major advantage and allows them to leverage their core competencies. And they can add more value - easier management, consolidated infrastructure, more efficient controls between the core of the data center and all the roads that lead to it. Additionally - you need the network in order to have a more efficient virtual server environment for mobility, availability, reliability and disaster recovery. Cisco doesn't own the server virtualization software, the network storage, the applications - but they provide a platform to bring all of these things together. And they have the leverage - based on economics - to have everyone want to be their partner, even some of their major competitors - co-opetetion - because again of the economics involved. That is another essential part of Cisco's value add strategy, get all the partners to integrate and optimize around the UCS platform as evidenced by the VMware, EMC, NetApp, Cisco alliances. This intelligent and important move brought them closer together with the biggest players that provide the other essential aspects of the virtual data center - server virtualization and storage. I am not convinced that Cisco is going to rule the universe with this strategy but it certainly poises them for owning more strategic footprint within their customer base (which is pretty much everyone). And I am just talking about strategy and vision - it is the go-to-market execution that will matter here and there are still a number of challenges facing them.. Dell, HP and IBM will not sit idly by and watch Cisco run rampant over them. Keep in mind that while Dell, HP and IBM servers are in a lot of places, they do not enjoy the same ubiquity as Cisco networking solutions. On the other hand these leading server companies are ready and willing to move money around to make deals. Cisco will have to perform a tough balancing act. They cannot win this game by matching hard dollar savings versus soft dollar esoterica and cost saving rhetoric. At the same time they can't allow themselves to lose sight of the bigger vision and driving customers viscerally towards it. Integrating compute resources with networking solutions makes a ton of sense if you are Cisco. UCS enables Cisco to be the core of the data center with all roads leading to that core provided by Cisco's converged networking solutions. It is a strategy of Core + Access. For years Cisco has struggled to be more than IT "plumbing" - to be more strategic to IT, let alone the actual business units. By running business applications on Cisco equipment they are far better positioned to achieve this objective. By elevating themselves through providing solutions where business applications "live" is strategic and could be landscape changing.
We have all collectively just made it through the toughest economic year and good riddance! To quote the comedian Lily Tomlin - "No matter how cynical you become, it's never enough to keep up". We certainly have enough cynicism in the IT world but at the beginning of every new year I think most of us feel a moment of optimism. So I am going to go with that for right now. Here are some things to think about for Storage and IT going forward in 2010: - Intelligent tiered storage will be provided by a number of storage vendors. The key will be that the tiering will be at a sub-LUN level and will be based on policies. I cannot emphasis the value of intelligent tiering enough - I write about it in more detail in these two blogs here and here. - File services will become more important as file data continues to consume more and more of our storage capacity. Included in this will be NAS and CAS solutions as well as software that manages this data. It is important that we efficiently store file data and we must make sure that all of this unstructured content becomes useful to our businesses. Right now unstructured data is pretty much synonymous with useless data. This dynamic must change - the enormous amount of unstructured content is the elephant in the data center and it is just sitting there taking up lots of space and needing to be constantly fed (I will end my analogy there so I don't offend anyone). - Private IT clouds will be top of mind for many IT professionals. They want to leverage virtualization, management, policies and reporting to implement utility models for their IT environments. Sure, we've been talking about this for 20 years and it still hasn't really happened en masse but we now have more tools at our disposal than ever before to enable it. - Public IT clouds will continue to be used for new web-based businesses, and lower tiers of applications and data. However, the big impact to the brick and mortar IT world is the use of applications in the cloud. The more customers use these services, the more IT infrastructure that normally would have been used in-house will not be required. - Green IT will be more of a priority (or is this just wishful thinking on my part?). Data Centers produce 6% of green house gas emissions worldwide - more than the airline industry. It is essential that IT finally embrace Green as an initiative. And by doing so they will reduce cost and feel like they are actually doing something positive for the world. I quote the comedian Lily Tomlin once again - "I always wondered why somebody doesn't do something about that. Then I realized - I was somebody." - More disk-to-disk backup with dedupe and less tape in the data center is inevitable. The majority of the world still uses tape as their primary backup target but the shift to disk with dedupe is well on its way and will become more pervasive in 2010. - Automation is essential in a world that consists of PBs of storage capacity. How can you hope to efficiently manage all that infrastructure without letting smart machines make decisions for you? Yes, that is a scary thought but it is the only way to manage. That is why visibility is required for automation to work. IT professionals need the reporting tools to assure them that things are running efficiently. - Data dedupe or rather data compression in primary storage will become a priority - if not a reality - in 2010. The problem with deduping data in primary storage is that if it is done in real-time it interjects latency. And if you want to access the data after deduping than you have to re-hydrate that data. However, with intelligent tiering - as described above - you can actually create a policy that moves unused data to a lower tier and then compress it. Intelligent tiering at the sub-LUN level enables other efficiencies - which is why it is so important to storage optimization. This solves the latency problem because it is a post process and rehydration isn't an issue because it is data that no one is accessing. And let's face it - 60-80% of your content is never accessed 90 days after its creation. That means that with 1 PB of data, 600 to 800 TB can be deduped or compressed. Even at a ratio of 2-to-1 you can lower capacity in that scenario by 300 to 400 TB. Sounds pretty good to me. - Along those same lines, intelligent sub-LUN tiering will make SSD / EFD much more valuable to customers allowing them to have small amounts of this technology being used to maximum effect. As a result less FC drives and more SATA will be found in Enterprise-class storage. - Cisco, Dell, HP and IBM will offer the IT world "unified" platforms. Some will be more unified than others. In the case of Cisco's Unified Computing System (UCS) they don't own much of the intellectual property but instead rely on VMware and EMC for the virtualization and storage systems. Dell, HP and IBM have the storage piece in in the case of HP and IBM can also use Xen or perhaps even KVM for the virtualization. But in the end, these solutions are integration of existing technologies - some of it good; some of it mediocre and some of it crap. And the TCO will be at best pretty good and at worst unquantifiable. Coming to a town near you in 2010. - Desktop virtualization will rise in visibility, offerings and implementation in 2010. You don't virtualize desktops for consolidation but for management and security. IT professionals can upgrade and replace these systems much more easily. If the data is stored on network storage then the company controls it and not the user. It also reduces infrastructure costs by removing capacity from each desktop. - PC power management is becoming a big item and will be a priority in 2010. There is software out there that manages the PC power down process and saves between 30% and 50% of the electricity bill of these systems. It is kind of a no-brainer to implement especially since many power companies actually give you an incentive to implement this software. - Server virtualization continues to thrive. Most of the customers I've spoken with this year have implemented server virtualization and there is a growing number that have a "Virtual First" policy. In other words, applications are going on virtualized servers by default and if the business unit doesn't want this they need to justify why they need a dedicated physical server.
Gluster is a startup that is providing an open source file system that is EASY to use and manage, has fast and versatile performance, is good for people who have 2 TB of storage or 20 PB or more of storage (and everything in between), provides scale up and scale out and will also work with existing files systems like ext3, ext4 and ZFS (and I also suggest they support Btrfs soon). But given all of that - the first time I heard about these guys my first reaction was - NOT ANOTHER FILE SYSTEM!!! Haven't we heard it all before? Yes and no. The most important thing to me about Gluster and why it matters is their go-to-market strategy. They basically have the Red Hat model giving the software away for free and charging for support. Assuming that the software is as good as they say it is - this model could be landscape changing. Consider people who are implementing test / dev environments that want a low cost networked storage solution that is easy to manage and cost effective. Consider companies that want to build huge archives of content that don't have a huge infrastructure budget. Consider the unbridled growth of content and the fact that a new model is required to contain costs and manage scale. And that is just the early adopters - consider how a solution like this can begin to nibble away at the bread and butter storage environment as it gains credibility and traction over time. Certainly, the traditional storage market will continue to thrive - just like Unix has not been obsoleted by Linux - but I believe that the timing may be right for someone like Gluster. Think about it - thousands and thousands of users that build their own scalable storage system using commodity hardware with a free open source solution that is EASY, FAST, INTELLIGENT and RELIABLE. All four of these attributes have to be true for the masses to use it to the scale that Gluster I am sure is hoping for. And of course, Gluster's support has to be stellar and worth the annual pricing - which I believe is modest. The Red Hat of storage? Not such a crazy idea and the timing may just be perfect given all of the market dynamics. * * * * * * * * * The following is a very informative email discussion I had with Jack O'Brian and Kamal Varma of Gluster: Tony Asaro: If GlusterFS is a file system - why do you need another file system? Gluster Folks: GlusterFS is a complete storage stack that includes volume management, software RAID, I/O scheduling, cache management, distributed locking, etc. For storing the data persistently on disk, Gluster (like many others) chose to use a standard disk format, rather than introduce a new one. Gluster features and performance are not dependent on the disk file system, but we benefit from leveraging a proven existing technology that is more convenient for the customer. Other file systems (from Lustre to Hadoop) also leverage the disk file system for low level management of block devices, however they do so with proprietary data formats. Gluster stores the data just as file and folders. Tony Asaro: GlusterFS is in user space - why did you take the approach? Gluster Folks: The two primary reasons are simplicity and flexibility, and we knew this could be achieved without sacrificing performance. Kernel implementations of file systems are by nature complex with many dependencies. Installation in user space is as simple as installing other common applications - without any need for unique kernel modifications or patches. User space implementation also enabled superior configuration flexibility. File system functionality is implemented in modules that can be stacked in user space to match the configuration to a given workload. User space modules also accelerate time to market and rapid maturity of new features - new modules can be quickly developed and integrated without the complexity of a monolithic code base with kernel dependencies. Tony Asaro: The first conclusion will be that if it runs in user space it will not perform all that well. Gluster Folks: We understood scalability from our experience building the first cluster that scaled to over 1000 nodes. We also had extensive OS design experience including microkernel design where OS functionality is implemented in user space. To validate our approach, we tested it. Network latency and disk latency are much higher than context switching. You can actually be faster by being in user space with techniques like eliminating the metadata server and making system calls very efficient. Sophisticated optimization algorithms are much simpler to implement in user space. If being in user space was the problem that people perceive it to be, VMware wouldn't exist. Tony Asaro: You claim you have a great volume manager - explain why you believe your volume manager provides value to users. Gluster Folks: Gluster Storage Platform creates/manages volumes across multiple machines into one global namespace. This gives seamless access for clients to the entire storage pool in the cluster from one mount point. The distribution of data (for load balancing) is done automatically. Use cases that require high availability can set up mirrored configurations across nodes (synchronous replication) - so our volume management functionality is quite flexible. Unlike others we do not have a metadata server - neither centralized nor distributed metadata server - Gluster clients use a deterministic elastic hashing algorithm to compute location of files thereby eliminating the need for the metadata server. In addition to removing this choke point, this enables linear scalability and better reliability. From an architecture standpoint, the power of Gluster volume management comes from the stackable design (unique to Gluster) which could be viewed as a programmable file system. Tony Asaro: You work with ZFS - which also has a volume manager - how do you integrate with ZFS and do you have overlapping functionality? Gluster Folks: ZFS is limited to a single system and doesn't have the capability to scale across multiple machines under a unified namespace. Gluster works in conjunction with ZFS, managing the task of unifying individual ZFS volumes across the cluster in the global namespace. ZFS manages the physical disks within each system as described above. Tony Asaro: You claim that your performance is versatile and is good with large files, small files, streaming and transactional - how do you achieve this? Gluster Folks: This is again a benefit of the modular, stackable design. Different optimization techniques are written in individual modules, and the stackable design allows them to be combined in clever ways to match a given workload. For example, if you have many files less than 64k the quick-read module fetches the file in one network operation. For streaming large files the read-ahead, I/O caching, and DMA capabilities provide optimization. If you have an application with a lot of stat calls, Gluster uses the stat pre-fetch module to recognizes this and fetch the information in bulk to serve from a local stat table, freeing the file system for other work. Tony Asaro: What are your File OPS? Based on what configuration? Are you going to participate in SpecFS? Gluster Folks: Here's an interesting test result. We tested a read workload using 128k blocks vs. the more common use of small block size such as 4k (it was a throughput test and we were not optimizing for IOPS). We achieved 131,000 IOPS across 8 storage nodes (16,375 per node). The configuration used 32 clients running IOzone. Each server had 3 RAID controllers and 18TB of storage (142TB total capacity). Interconnect was InfiniBand QDR (one card per server). We are planning to run additional tests to get some eye-opening IOPS numbers, we'll keep you posted. We do plan to participate in SpecFS down the road. It hasn't been a focus since it is very NFS and CIFS dependent and most of our customers prefer the Gluster native protocol with other benchmarks or their own application tests. Tony Asaro: You mentioned using your own agent above, which to me isn't nearly as a big of a deal in user space as it is with host operating systems. But others have tried and failed with this model. How can you justify an agent? Gluster Folks: As you note the big difference is Gluster is not kernel-dependent. Those that are kernel-dependent suffer from being difficult to install, require custom patches, and are tied to a specific version of the kernel. For those in userspace, they required a proprietary API so you had to change your application (e.g. Hadoop, Maxiscale). Our FUSE model is portable (and POSIX compliant) and installs with one command. Of course we also give the choice of NFS, CIFS, WebDav, etc, most customers choose Gluster native protocol. Tony Asaro: What is your throughput performance? Based on what configuration? With the configuration above, we achieved 16 GB/sec for read and 12 GB/sec write. The test was designed such that data was served of the disk vs. cache. Tony Asaro: How many downloads of your file system has there been? Gluster Folks: Over 60,000 from the Gluster download site. We also have several mirror download site as well as being included in Debian, Ubuntu, Fedora, FreeBSD, and other distributions. That's cumulative since Jan 2007 (1.0 release). Current download rate is ~4500 per month with steady mo./mo. increases.
I had a discussion with the blogger at i365. The three questions they asked me: 1. In a recent blog post, you discuss the hype around Information Lifecycle Management a few years back and how that hype has declined. You also point out, however, that many of ILM's ideas--matching data to the right storage environment in order to balance cost, performance, and security--have been realized to some extent. How does cloud storage fit into this as a new way by which to store data? 2. Following up on that, do you see the hype around "cloud computing" going the same way of ILM and only having a moderate effect on businesses? Or is cloud computing/cloud storage a realization of something bigger? 3. Your post "Your Data Needs Diet and Exercise!" discusses the pressing need for the storage ecosystem - both vendors and customers - to work on new ways to make the exponential increase in worldwide data manageable. Are there any major innovations you see happening in the next few years that will make this happen? To find out my answers - go the check out their blog: Experts Corner: Q&A with Tony Asaro at ContemplatingIT
I believe that intelligent tiered storage (aka automated tiered storage) is one of the most valuable capabilities that can be provided by a storage system especially as capacities continue to sky rocket. What is intelligent tiered storage? It is the ability to move data - based on policy and behavior - from high cost storage to lower cost storage. The goal is to increase utilization and optimization of storage assets while driving down cost. This needs to be a transparent process with no downtime or impact to users or applications. Companies of all sizes can benefit reducing their capital costs by significant percentages. And those environments that have massive storage capacities can literally save millions of dollars. Based on various research and talking to hundreds of customers over the years network storage utilization is typically 50% or less. It is also common that 60-80% of all data stored on tier 1 and 2 storage is dormant 90 days after its creation. In fact, much of it is dormant two weeks after it is created. These statistics represent a huge amount of waste. However, very few companies have actually implemented intelligent tiered storage. One of the main reasons for this is that most of the storage systems that can move data do so at the volume level and this is not granular enough. For example, if a policy to move data is based on how many I/Os hit that volume within a 90 day period then that volume may never be moved. A volume could have 10% of its data that is in constant use and requires optimal performance but the other 90% of the data on that volume is never accessed. However, the active 10% dictates the policy for the other 90% and that volume would not be moved. Therefore, it is important to be able to "stretch" a volume across different storage media and RAID levels in smaller chunks. Compellent is the leader and innovator for intelligent tiered storage and based on policy they move data at the block level - a feature they call Data Progression. Compellent has provided this functionality for over four or five years. They have a large number of customers using Data Progression and it has been extremely successful for them. I actually wrote a report on this back in 2006. Now that EMC is validating intelligent tiering, Compellent would be wise to leverage the momentum that is sure to follow around this capability (EqualLogic experienced a real increase in sales when EMC announced iSCSI). EMC announced general availability of their intelligent tiering solution called FAST. There are a number of articles that provide you with the details of this announcement - but one of the best is from StorageNerve. For me, the details of this announcement are less important than the implications. What I think is important about the EMC FAST announcement is they are validating intelligent tiered storage. EMC has the clout and the resources to educate the market on what customers should be thinking about, investing in and implementing. Additionally, they will mobilize their competitors to put their shoulders to the wheel and provide competitive functionality. And the result is that customers will be able to take advantage of this capability by multiple vendors in 2010 and 2011. For example, 3PAR just announced Policy Advisor - this is a software tool that analyzes data usage on their InServe storage system and based on policy and behavior will move data to different tiers. It is no coincidence that 3PAR announced general availability of Policy Advisor right before EMC announced FAST. At the moment EMC FAST is able to move data at the LUN level - which isn't granular enough - in my view - to provide the value customers are really looking for with this technology. EMC's stated vision is to be able to move data at a "sub-LUN" level later in 2010. I believe that once they achieve this much more value will be derived from FAST. 2010 promises to be the year of intelligent tiered storage with a number of storage vendors coming to market with their version of this capability. I am looking forward to hearing the announcements, general availability and customer implementations. We are on the threshold of intelligent tiered storage becoming a reality in the data center and I believe it will significantly change the economics of storage.