Concurrent file read approaches?

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

I question whether this is something worth aiming for. That's to say, I see no
reason why you shouldn't serialise all access to the file at the Java level.
Unless you know something about the filesystem and disk hardware that I don't,
you aren't really going to have "concurrent" access to the file anyway at the
hardware/OS level.

But, if you do want something closer to genuinely concurrent access, then split
the file up, and put each part onto a different physical disk.

The interaction between Java, OS, controllers, disks, various buffers
are much more complex than that.

I tried making a small test program.

Here are some results.

Reading 1KB chunks at random locations:

One synchronized RAF: 18969
Multiple RAF: 9094
One synchronized RAF: 17656
Multiple RAF: 9609
One synchronized RAF: 18219
Multiple RAF: 8922

Reading 10KB chunks at random locations:

One synchronized RAF: 37984
Multiple RAF: 26438
One synchronized RAF: 36562
Multiple RAF: 25938
One synchronized RAF: 36047
Multiple RAF: 26171

Reading 100KB chunks at random locations:

One synchronized RAF: 224016
Multiple RAF: 378406
One synchronized RAF: 220344
Multiple RAF: 357063
One synchronized RAF: 217187
Multiple RAF: 349047

It appears that multiple is an advantage when reading
small chunks and a disadvantage when reading bigger chunks
on my config (SUN Java 1.5.0, Windows XP SP2,
2 WD Raptor disks in RAID-0).

I will neither say that I can explain it or suggest that it will
be so for all configs.

I can post the program if there is interest.

Arne

 

[Chris Uppal]

Arne Vajhøj wrote:

[me:]

The interaction between Java, OS, controllers, disks, various buffers
are much more complex than that.

Indeed, I was only recommending a starting point for thinking about these
issues, not offering a final analysis.

-- chris