Finally, some postmark results for OSX! And how does it do versus Windows?

My colleague Ian (last name withheld to save him from the Mac zealots) compiled the postmark code on his beloved Mac and ran it with the same settings I use in general (see older blog posts, just search for postmark).

I’ve been curious for the longest time to see how OSX performs in this test, since most UNIX and -alike systems work great with it. I wanted to see if OSX would be appropriate for a high IOPS-type environment (my belief being that due to the choice of kernel and filesystem it would suck – Mach and HFS+ not being exactly ideally suited to such tasks).

This is obviously not the most scientific test but I think it is good enough to get a rough gauge.

I’m still waiting for the specifics on the Mac but it’s an older Intel-based 17″ Macbok Pro with a 2.16GHz CPU, 5400 RPM HD and 2GB RAM.

The horrendous result (I think my rusty abacus did better once):

Time:
1259 seconds total
1186 seconds of transactions (16 per second)

Files:
20092 created (15 per second)
Creation alone: 10000 files (163 per second)
Mixed with transactions: 10092 files (8 per second)
9935 read (8 per second)
10064 appended (8 per second)
20092 deleted (15 per second)
Deletion alone: 10184 files (848 per second)
Mixed with transactions: 9908 files (8 per second)

Data:
548.25 megabytes read (445.92 kilobytes per second)
1158.00 megabytes written (941.85 kilobytes per second)

To compare and contrast (and save you from searching the older posts):

On a similar-spec Thinkpad T60 running XP (1.8GHz Core Duo, 2GB RAM, 60GB 5400 RPM HD):

Time:
174 seconds total
91 seconds of transactions (219 per second)

Files:
20092 created (115 per second)
Creation alone: 10000 files (222 per second)
Mixed with transactions: 10092 files (110 per second)
9935 read (109 per second)
10064 appended (110 per second)
20092 deleted (115 per second)
Deletion alone: 10184 files (268 per second)
Mixed with transactions: 9908 files (108 per second)

Data:
548.25 megabytes read (3.15 megabytes per second)
1158.00 megabytes written (6.66 megabytes per second)

 

And on the spankin’ T61p running 2008 Server (2.6GHz Core 2 Duo, 4GB RAM, 200GB 7200 RPM HD):

Time:
110 seconds total
39 seconds of transactions (512 per second)

Files:
20092 created (182 per second)
Creation alone: 10000 files (454 per second)
Mixed with transactions: 10092 files (258 per second)
9935 read (254 per second)
10064 appended (258 per second)
20092 deleted (182 per second)
Deletion alone: 10184 files (207 per second)
Mixed with transactions: 9908 files (254 per second)

Data:
548.25 megabytes read (4.98 megabytes per second)
1158.00 megabytes written (10.53 megabytes per second)

 

The drive and CPU speed is important but postmark results are largely a function of filesystem and cache efficiency. It’s also worth noting that postmark is in no way optimized for windows since it is just standard C code and indeed was meant to be run on Unix boxes. Typically, good Unix filesystems beat Windows in postmark (my record run time was like under 10s on a Solaris box and DMX).

Unless something is wrong, HFS+ and/or the OSX cache are execrable for this kind of workload, which is a pity. Maybe there are better mount options? Some tuning options?

This is huge! Even if there’s some issue (disk fragmentation, for instance) the difference in sheer IOPS performance between OSX and pretty much anything else is staggering.

Any Mac users out there that want to chime in and save the day please let me know, I’ll send you the source and you can compile it whichever which way. I truly hope there is some serious error here.

D

9 Replies to “Finally, some postmark results for OSX! And how does it do versus Windows?”

  1. Blame it on the HFS. It’s been a while since I did postal benchmarking but what was the file size you used? If it’s more than the block size I’d point the finger at HFSs auto defrag “feature”. Every time OSX reads a

  2. (Apologies, I think wordpress input validation was cranky and I was sleepy)

    So, Blame it on HFS. It’s been a while since I did postal benchmarking but what was the file size you used? If it’s more than the block size I’d point the finger at HFSs auto defrag “feature”.

    Every time OSX fopens a file that’s smaller then 20MB it will defrag it. As I recall it reads all blocks into memory and then attempts to write them to contiguous space before returning the fopen.

    I’d be curious how postmark does on a ZFS, or even UFS, disk.

  3. I’m willing to put up my own Mac as a test platform if you want to send me the code/executable. My home computer is a 24″ iMac 2.8Ghz Core 2 Duo Extreme with Mac OS X 10.5.2.

    HFS+ is a journaled filesystem and always seems a bit slower to me. I wonder how Xsan filesystems would compare. Mac’s aren’t slow machines by any means, and I’ve seen large amounts of IO pushed through them without skipping a beat. I’m sure the way the test works isn’t optimal making it seem worse than it actually is.

  4. I’ve sent the code to Daniel, let’s see what his result will be – it will of course be better since his box is much faster than the laptop we tested with but let’s see.

    In any case, large amounts of large-file sequential I/O, for instance, are not the same as large amounts of randomized I/O with thousands of files with different sizes (which is what postmark does). The idea is to emulate a very, very busy server doing tons of IOPS.

    Most other OSes do more or less OK, in Linux the edge goes sometimes to ReiserFS vs ext3 but the difference is not 1000x, it’s relatively small.

    If HFS+ has the defragmenting feature “HFS Fan” describes, maybe the lesson is to turn it off if possible and re-run the benchmark?

    I’m relying on help from the Mac fanbase here since I don’t yet have one, I’m just posting the results.

    BTW, to Daniel: Xsan is Quantum’s Stornext product and it’s also not meant for small-file random I/O.

    See ZFS for something that would do great in this benchmark. I just don’t see Apple moving to it yet (stupid resource forks).

    This may seem only relevant to servers but I’ve noticed that any machine that does great on postmark is always extremely responsive when it comes to I/O.

    D

  5. I can confirm that Xsan really truly sucks with small file read/writes. The basic problem is that it makes some very basic assumptions about your file sizes and access patterns. As I recall small files are pooled into single transactions which can really kill you on latency, especially serial access to lots of small random files. It’s not Xsans fault, it’s just not good with small random io, like you said.

    With regards to the defrag feature I don’t believe it can be activly disabled. However, down the page at http://www.kernelthread.com/mac/apme/fragmentation/ Amit lists a couple of conditions that may function as work arounds. Setting the test files as read only seems like a possible solution.

    10.5 currently has limited r/w ZFS support if you don’t mind prerelease software. I’m not sure what the official plans are over there, but look for some significant ZFS opportunities whenever 10.6 comes down the pipe. I, for one, am looking forward to large scale ZFS adoption with 10.6 and 10.7.

  6. There’s some speculation as to whether Apple will allow ZFS to be the default FS or not. I think not, there’s still too much dependency on HFS+. Which is a pity.

Leave a comment for posterity...