NetApp E-Series posts top ten price performance SPC-2 result

NetApp published a Top Ten Price/Performance SPC-2 result for the E5660 platform (the all-SSD EF560 variant of the platform also comfortably placed in the Top Ten Price/Performance for the SPC-1 test). In this post I will explain why the E-Series platform is an ideal choice for workloads requiring high performance density while being cost effective.

Executive Summary

The NetApp E5660 offers a unique combination of high performance, low cost, high reliability and extreme performance density, making it perfectly suited for applications that require high capacity, very high speed sequential I/O and a dense form factor. In EF560 form with all SSD, it offers that same high sequential speed shown here coupled with some of the lowest latencies for extremely hard OLTP workloads as demonstrated by the SPC-1 results.

The SPC-2 test

Whereas the SPC-1 test deals with an extremely busy OLTP system (high percentage of random I/O, over 60%writes, with SPC-1 IOPS vs latency being a key metric), SPC-2 instead focuses on throughput, with SPC-2 MBPS (MegaBytes Per Second) being a key metric.

SPC-2 tries to simulate applications that need to move a lot of data around very rapidly. It is divided into three workloads:

  1. Large File Processing: Applications that need to do operations with large files, typically found in analytics, scientific computing and financial processing.
  2. Large Database Queries: Data mining, large table scans, business intelligence…
  3. Video on Demand: Applications that deal with providing multiple simultaneous video streams (even of the same file) by drawing from a large library.
The SPC-2 test measures each workload and then provides an average SPC-2 MBPS across the three workloads. Here’s the link to the SPC-2 NetApp E5660 submission.

Metrics that matter

As I’ve done in previous SPC analyses, I like using the published data to show business value with additional metrics that may not be readily spelled out in the reports. Certainly $/SPC-2 MBPS is a valuable metric, as is the sheer SPC-2 MBPS metric that shows how fast the array was in the benchmark.

However… think of what kind of workloads represent the type of I/O shown in SPC-2. Analytics, streaming media, data mining. 

Then think of the kind of applications driving such workloads.

Such applications typically treat storage like lego bricks and can use multiple separate arrays in parallel (there’s no need or even advantage in having a single array). Those deployments invariably need high capacities, high speeds and at the same time don’t need the storage to do anything too fancy beyond being dense, fast and reliable.

In addition, such solutions often run in their own dedicated silo, and don’t share their arrays with the rest of the applications in a datacenter. Specialized arrays are common targets in these situations.

It follows that a few additional metrics are of paramount importance for these types of applications and workloads to make a system relevant in the real world:

  • I/O density – how fast does this system go per rack unit of space?
  • Price per used (not usable) GB – can I use most of my storage space to drive this performance, or just a small fraction of the available space?
  • Used capacity per Rack Unit – can I get a decent amount of application space per Rack Unit and maintain this high performance?

Terms used

Some definition of the terms used in the chart will be necessary. The first four are standard metrics found in all SPC-2 reports:

  • $/SPC-2 MBPS – the standard SPC-2 reported metric of price/performance
  • SPC-2 MBPS – the standard SPC-2 reported metric of the sheer performance attained for the test. It’s important to look at this number in the context of the application used to generate it – SPC-2. It will invariably be a lot less than marketing throughput numbers… Don’t compare marketing numbers for systems that don’t have SPC-2 results to official, audited SPC-2 numbers. Ask the vendor that doesn’t have SPC-2 results to submit their systems and get audited numbers.
  • Price – the price (in USD) submitted for the system, after all discounts etc.
  • Used Capacity – this is the space actually consumed by the benchmark, in GB. “ASU Capacity” in SPC parlance. This is especially important since many vendors will use a relatively small percentage of their overall capacity (Oracle for example uses around 28% of the total space in their SPC-2 submissions, NetApp uses over 79%). Performance is often a big reason to use a small percentage of the capacity, especially with spinning disks.

This type of table is found in all submissions, the sections before it explain how capacity was calculated. Page 8 of the E5660 submission:

E5660 Utilization

The next four metrics are very easily derived from existing SPC-2 metrics in each report:

  • $/ASU GB: How much do I pay for each GB actually consumed by the benchmark? Since that’s what the test actually used to get the reported performance… metrics such as cost per raw GB are immaterial. To calculate, just divide $/ASU Capacity.
  • Rack Units: Simply the rack space consumed by each system based on the list of components. How much physical space does the system consume?
  • SPC-2 MBPS/Rack Unit: How much performance is achieved by each Rack Unit of space? Shows performance density. Simply divide the posted MBPS/total Rack Units.
  • ASU GB/Rack Unit: Of the capacity consumed by the benchmark, how much of it fits in a single Rack Unit of space? Shows capacity density. To calculate, divide ASU Capacity/total Rack Units.
A balanced solution needs to look at a combination of the above metrics.

Analysis of Results

Here are the Top Ten Price/Performance SPC-2 submissions as of December 4th, 2015:

SPC 2 Analysis

Notice that while the NetApp E5660 doesn’t dominate in sheer SPC-2 MBPS in a single system, it utterly annihilates the competition when it comes to performance and capacity density and cost/used GB. The next most impressive system is the SGI InfiniteStorage 5600, which is the SGI OEM version of the NetApp E5600 🙂 (tested with different drives and cost structure).

Notice that the NetApp E5660 with spinning disks is faster per Rack Unit than the fastest all-SSD system HP has to offer, the 20850… this is not contrived, it’s simple math. Even after a rather hefty HP discount, the HP system is over 20x more expensive per used GB than the NetApp E5600.

If you were buiding a real system for large-scale heavy-duty sequential I/O, what would you rather have, assuming non-infinite budget and rack space?

Another way to look at it: One could buy 10x the NetApp systems for the cost of a single 3Par system, and achieve, while running them easily in parallel:

  • 30% faster performance than the 3Par system
  • 19.7x the ASU Capacity of the 3Par system
These are real business metrics – same cost, almost 20x the capacity and 30% more performance.

This is where hero numbers and Lab Queens fail to match up to real world requirements at a reasonable cost. You can do similar calculations for the rest of the systems.

A real-life application: The Lawrence Livermore National Laboratory. They run multiple E-Series systems in parallel and can achieve over 1TB/s throughput. See here.

Closing Thoughts

With Top Ten Price/Performance rankings for both the SPC-1 and SPC-2 benchmarks, the NetApp E-Series platform has proven that it can be the basis for a highly performant, reliable yet cost-efficient and dense solution. In addition, it’s the only platform that is in the Top Ten Price/Performance for both SPC-1 and SPC-2.

A combination of high IOPS, extremely low latency and very high throughput at low cost is possible with this system. 

When comparing solutions, look beyond the big numbers and think how a system would provide business value. Quite frequently, the big numbers come with big caveats…

D

Technorati Tags: , , , ,

4 Replies to “NetApp E-Series posts top ten price performance SPC-2 result”

  1. Hey D, thanks for the opportunity to add some extra color to your comparison!

    On the surface, there is nothing wrong with making the types of simplistic statistical comparisons you’ve made above using the SPC-2 results data. All is fair in love and war!

    However – I do feel making a simple comparison between the E5600 and the 3PAR 20850 is like comparing a Ford Fusion and a Ford F-450. I’m positive the Ford salesperson would want you to understand they are two different products for two different sets of requirements – and therefore have two very different price points.

    The 3PAR 20850 and the E5600 really just do not compare for any of the capability aspects which a customer would be wanting when interested in a product in the High End segment. a few for example:
    – a single-system array scaling to 8 active-active controllers versus 2 for E5600
    – > 3,900 TB Raw versus 360TB for E5600
    – 1,920 max drives versus 360 for E5600
    – 1,024 max SSD versus 120 for E5600
    – 62,844 SPC-2 MBPS versus 8,236 SPC-2 MBPS
    – and so on….

    It is a very different product in a different segment.

    Happy to take this thought path further if you wish !

    Paul Haverfield
    Storage CTO
    HPE Storage APJ region

    1. Hi Paul, hope you’re well.

      I’m focusing on valuable business metrics like capacity/RU and performance/RU. Those aren’t simplistic comparisons… 🙂

      Read the section of my article after the “However… think of what kind of workloads” phrase.

      I fully accept the new 3Par box is fast (if anything, I picked it for the comparison because it was the fastest), but for 3rd Platform applications that typically need high throughput and use storage like lego bricks, I’d argue the 3Par system (and even ONTAP) simply isn’t a great fit.

      One of the reasons massively parallel systems like the one in the Lawrence Livermore example use E-Series. High performance and capacity density. To get 1.3TB/s and the capacity density that organization requires would be impossible with the 3Par system. Different design center.

      You guys still got an awesome number, all I’m trying to do is inject some business sense in the comparison. Way too many people just look at the top-line metric and ignore the rest.

      Thx

      D

      1. All Good D,

        If we had an SPC-2 on the 3PAR 8200 – which is comparable to the E5600 on form factor / scaling / price point / Lego block type storage system…. –I’m very positive our MBPS/RU and so on would be very favorable!

        The benefit here of course is one architecture between the low end ‘Lego block’ and the very High End ‘Monster Truck’ 🙂

        If your’e interested I have a blog on the SPC-2 result for 3PAR 20850 versus the VMAX here:
        http://hpe.to/6003BPlIJ

        Best regards,

        Paul.

        1. Ah, but that’s hugely hypothetical unless you submit SPC results for such a system.

          Stick to the audited numbers please 🙂

          Now that you mention it… Would the 3Par 8200 really be 20x less expensive than the 20850 per ASU GB yet maintain the sequential speeds of a NetApp E5600? 🙂 (because that’s the delta we’re talking about here to make it equal to the E5600).

          If so that also means the 20850 is a really bad deal for HP customers and they’re better off buying many 8200 boxes instead and running them in parallel.

          Nothing wrong with hero numbers. As long as they also make business sense for the applications concerned.

          Thx

          D

Leave a comment for posterity...