Musings on storage, tuning, backups and more – delivered with a no-nonsense approach. By Dimitris Krekoukias
There are a lot of myths and misinformation, plus more than a modicum of misunderstanding, regarding how storage systems can use available bandwidth, especially with certain newer kinds of media.
I wanted to explain some of the harsh facts of storage system design in the real world, and why one shouldn’t just add up drive speeds to estimate performance.
Continue reading “The Harsh Realities of PCIe Lane Shortage in Storage Systems”
I posit that we now have a whole new class of consumer that is completely oblivious to certain hitherto fundamental concepts – and this can lead to poor business decisions and overall sub-optimal execution and results.
I got the idea after a discussion with an ex colleague (that’s now working for a cloud vendor) where he proudly proclaimed that infrastructure is unimportant and uninteresting.
I’ll start generically and shift to IT. The generic aspect of this problem is very interesting, since it’s lowering quality in all sorts of fields.
And never forget: Just because something is widely and easily available doesn’t mean it’s better. It simply means that more people have access to it.
Continue reading “The Loss of Important Knowledge and Acumen Through Perceived Commoditization”
I got the idea for this post after seeing certain vendors claim they were the first and only with certain data reduction technologies (I’m not talking about dedupe and compression). I thought – how come Nimble never made a big deal about this? After all, what those vendors were claiming didn’t seem to be very interesting compared to how Nimble systems efficiently write data… yet those vendors were acting as if they’d cured a particularly virulent disease.
Nimble systems naturally avoid any wasted space when writing. This is an inherent part of the design and not something that was added later.
Continue reading “How Nimble Storage Systems do Block Folding”
Just a quick post to address something many people either get wrong or just live with due to convenience.
In summary: Please, let’s stop using average I/O sizes to characterize storage system performance. It’s wrong and doesn’t describe how the real world works. Using an average number is as bad as using small block 100% read numbers shown in vanity benchmarks. Neither is representative of real life.
Using a single I/O size for benchmarking became a practice for a vanity benchmark and to provide a level playing field to compare multiple products.
But, ultimately, even though the goal of comparing different systems is desirable, using a single I/O size is fundamentally flawed.
In past articles I have covered topics about how Nimble Storage lowers customer risk with technologies internal to the array such as:
Now, it’s time to write a more in-depth article about the immense customer benefits of InfoSight, Nimble’s Predictive Analytics platform.
Continue reading “InfoSight: How Nimble Customers Benefit from AI and Big Data Predictive Analytics”
I got the idea for this stream-of-consciousness (and possibly too obvious) post after reading some comments regarding new ways to do high speed I/O. Not something boring like faster media or protocols, but rather far more exotic approaches that require drastically rewriting applications to get the maximum benefits of radically different architectures.
The comments, in essence, stated that such advancements would be an utter waste of time since 99% of customers have absolutely no need for such exotica, but rather just need cheap, easy, reliable stuff, and can barely afford to buy infrastructure to begin with, let alone custom-code applications to get every last microsecond of latency out of gear.
Before we begin, something needs to be clear: Although dual-ported NVMe drives are not yet cost effective, the architecture of Nimble Storage is NVMe-ready today. And always remember that in order to get good benefits from NVMe, one needs to implement it all the way from the client. Doing NVMe only at the array isn’t as effective.
Continue reading “Practical Considerations for Implementing NVMe Storage”
The cardinal rule for enterprise storage systems is to never compromise when it comes to data integrity and resiliency. Everything else, while important, is secondary.
Many storage consumers are not aware of what data integrity mechanisms are available or which ones are necessary to meet their protection expectations and requirements. It doesn’t help that a lot of the technologies and the errors they prevent are rather esoteric. However, if you want a storage system that safely stores your data and always returns it correctly, no measure is too extreme.
I wanted to bring this crucial issue to light since I’m noticing several storage vendors being either cavalier about this or simply unaware.
I will explain why solutions that don’t offer some sort of automated, live SSD firmware update mechanism are potentially extremely risky propositions. Yes, this is another “vendor hat off, common sense hat on” type of post.
Continue reading “The Importance of SSD (and other) Firmware Updates”
This topic is very near and dear to me, and is one of the big reasons I came over to Nimble Storage.
I’ve always believed that storage systems should behave gracefully and predictably under pressure. Automatically. Even under complex and difficult situations.
It sounds like a simple request and it makes a whole lot of sense, but very few storage systems out there actually behave this way. This creates business challenges and increases risk and OpEx.
Continue reading “The Well-Behaved Storage System: Automatic Noisy Neighbor Avoidance”
I normally deal with different aspects of storage (arguably far more exciting) but I thought I would write something to provide some common sense perspective on the current state of 7-Mode to cDOT adoption.
I will tackle the following topics:
Recently, many vendors announced the availability of large SSDs. It’s not extremely exciting – it’s just a larger storage medium. Sure, it’s really advanced 3D NAND, it’s fast and ultra-reliable, and will allow some nicely dense configurations at a reduced $/GB. Another day in Enterprise Storage Land.
But, ultimately, that’s how drives roll – they get bigger. And in the case of SSD, the roadmaps seem extremely aggressive regarding capacities, with 100TB per device coming.
Then I realized that several vendors don’t have large SSD capacities available.
But why? Why ignore such a seemingly easy and hugely cost-effective way to increase density?
In this post I will attempt to explain why certain architectural decisions may lead to inflexible design constructs that can have long-term flexibility and scalability ramifications.
Continue reading “Architecture has long term scalability implications for All Flash Appliances”
The idea for this article came from seeing various people attempt product testing. Though I thought about storage when writing this, the ideas apply to most industries.
There are really three different kinds of testing.
The first is the incomplete, improper, useless in almost any situation testing. Typically done by people that have little training on the subject. Almost always ends in misleading results and is arguably dangerous, especially if used to make purchasing decisions.
The second is what’s affectionately and romantically called “Real World Testing”. Typically done by people that will try to simulate some kind of workload they believe they encounter in their environment, or use part of their environment to do the testing. Much more accurate than the first kind, if done right. Usually the workload is decided arbitrarily 🙂
The third and last kind is what I term “Proper Testing”. This is done by professionals (that usually do this type of testing for a living) that understand how complex testing for a broad range of conditions needs to be done. It’s really hard to do, but pays amazing dividends if done thoroughly.
Let’s go over the three kinds in more details, with some examples.
I really resisted using the “flash in the pan” phrase in the title… first, because the term is overused and second, because I don’t believe solid state is of limited value. On the contrary.
However, I am noticing an interesting trend among some newcomers in the array business, desperate to find a flash niche to compete in:
Writing their storage OS around very specific NAND flash technologies. Almost as bad as writing an entire storage OS to support a single hypervisor technology, but that’s a story for another day.
Continue reading “Are some flash storage vendors optimizing too heavily for short-lived NAND flash?”
(Edited: My bad, it was 2TB/s, up from 1.3TB/s, the solution has been getting bigger and upgraded, plus the post talks about the E5400, the newer E5600 is much faster).
What do you do when you need so much I/O performance that no one single storage system can deliver it, no matter how large?
Continue reading “NetApp delivers 2TB/s performance to giant supercomputer for big data”