Pin the buffer on the output chain

[Tom Anderson]

Greets yalls,

Given:

OutputStream out; // this is not buffered

Which is better:

Writer w = new BufferedWriter(new OutputStreamWriter(out));
Writer w = new OutputStreamWriter(new BufferedOutputStream(out));

?

I suppose i'm wondering about (a) whether encoding characters to bytes is
more efficient in bulk than in drips and drabs (and i suppose it must be,
from cache effects alone, but how much so?) and (b) what happens when a
lot of translated characters get written to an unbuffered stream - do the
bytes go in character-by-character (IYSWIM), or in bulk?

The thing that niggles me is that BufferedWriter has a newLine() method,
so i tend to use that for its convenience. But if it's sitting on top of
an OutputStream, then i might need buffering there as well.

I know i could go and read the source code, or do some measurements, and
give me a day or so and i probably will, but this is something i've been
wondering about, so i thought i'd share!

tom

 

[Roedy Green]

Writer w = new BufferedWriter(new OutputStreamWriter(out));
Writer w = new OutputStreamWriter(new BufferedOutputStream(out));

This can be most easily answered by experiment. While you are at it
figure it out for InputStreamReader too. You must run them many times
to make sure you are not getting fooled by JVM startup time/caching
artifacts.

Why would Sun invent BufferedWriters if it is more efficient not to
use them?

When you find out, let us know. I will teach the FileIO Amanuensis to
to it the preferred way.

see http://mindprod.com/applet/fileio.html

If you don't want to do the work, I will. You have revived an ancient
curiosity itch even if the answer may make no practical difference.

The benchmark could create 500 MB file encoded in UTF-8 with cyclic
16-bit chars (to give the encoder something to do), then read it and
average the read/write time for the alternative ways of coding it over
5 trials.

It is a gene programmers have. Whenever there are two or more ways of
doing something, they HAVE to know which is the best way.
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

Writer w = new BufferedWriter(new OutputStreamWriter(out));
Writer w = new OutputStreamWriter(new BufferedOutputStream(out));

It turns out it makes a huge difference. I will post code shortly.
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Lew]

Tom Anderson wrote, quoted or indirectly quoted someone who said :

It turns out it makes a huge difference. I will post code shortly.

Please also show the environment and which one won.

I wonder if things like RAM, CPU, disk subsystem performance and such
influence the outcome.

 

[Roedy Green]

Writer w = new BufferedWriter(new OutputStreamWriter(out));
Writer w = new OutputStreamWriter(new BufferedOutputStream(out));

one thing to be aware of is BufferedWriters measure their buffers in
chars and BufferedOutputStreams measure them in bytes.
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

Writer w = new BufferedWriter(new OutputStreamWriter(out));
Writer w = new OutputStreamWriter(new BufferedOutputStream(out));

Turns out neither is best. The best in splitting your buffer space in
half and using half for BufferedOutputStream and half for
BufferedWriter. (the optimal split has yet to be determined.)
The differences are substantial.

// Speeds are in seconds averaged over 2 trials using
104,857,600 character files and 65,536 character buffers.
// BufferedWriter 5.53
// BufferedOutputStream 25.64 W O R S T
// DoubleBufferedWriter 3.67 B E S T

// BufferedReader 5.16
// BufferedInputStream 17.18 W O R S T
// DoubleBufferedReader 4.17 B E S T

Code is posted at
https://wush.net/websvn/mindprod/fi...&path=/com/mindprod/example/TestBuffered.java
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

Tom Anderson wrote, quoted or indirectly quoted someone who said :



Please also show the environment and which one won.

I wonder if things like RAM, CPU, disk subsystem performance and such
influence the outcome.

Full description of my machine is at
http://mindprod.com/contact/equipment.html

The crucial features are

2 GHz AMD Athlon 64 X2 3800+ dual core

3 GIG ram.

Windows 7 64-bit JDK 1.6.0_18 64 bit.

SATA-3 7200 RPM disk

It is a simple Java program you can run without any fuss.

I still have to do a few experiments to see how best to allocate RAM
between the BufferedOutStream and the BufferedWriter.

--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

Code is posted at
https://wush.net/websvn/mindprod/fi...&path=/com/mindprod/example/TestBuffered.java

Turns out giving 25% of the space to BufferedOutputStream and 50% of
space to BufferedInputStream is optimal.

Speeds are in seconds averaged over 3 trials using 104,857,600
character files and 65,536 character buffers.

BufferedWriter 4.37 seconds
BufferedOutputStream 24.12 seconds W O R S T
DoubleBufferedWriter 10% to BufferedOutputStream 4.00 seconds
DoubleBufferedWriter 25% to BufferedOutputStream 3.75 seconds B E S T
DoubleBufferedWriter 50% to BufferedOutputStream 3.86 seconds
DoubleBufferedWriter 75% to BufferedOutputStream 3.87 seconds

BufferedReader 4.10 seconds
BufferedInputStream 14.81 seconds
DoubleBufferedReader 10% to BufferedInputStream 3.96 seconds
DoubleBufferedReader 25% to BufferedInputStream 4.03 seconds
DoubleBufferedReader 50% to BufferedInputStream 3.88 seconds B E S T
DoubleBufferedReader 75% to BufferedInputStream 3.94 seconds
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

Code is posted at
https://wush.net/websvn/mindprod/fi...&path=/com/mindprod/example/TestBuffered.java

Sun and Jet agree that using a BufferedOutputStream is worst, but
disagree on what is best.

SUN JVM 1.6.0_18 RESULTS
Speeds are in seconds averaged over 3 trials using 104,857,600
character files and 65,536 character buffers.

BufferedWriter 4.31 seconds BufferedOutputStream 22.18 seconds W O R S
T DoubleBufferedWriter 10% to BufferedOutputStream 4.03 seconds
DoubleBufferedWriter 25% to BufferedOutputStream 3.65 seconds B E S T
DoubleBufferedWriter 50% to BufferedOutputStream 3.82 seconds
DoubleBufferedWriter 75% to BufferedOutputStream 3.68 seconds
BufferedReader 3.94 seconds BufferedInputStream 14.11 seconds W O R S
T DoubleBufferedReader 10% to BufferedInputStream 4.02 seconds
DoubleBufferedReader 25% to BufferedInputStream 3.82 seconds
DoubleBufferedReader 50% to BufferedInputStream 3.77 seconds B E S T
DoubleBufferedReader 75% to BufferedInputStream 3.81 seconds
JET RESULTS
Speeds are in seconds averaged over 3 trials using 104,857,600
character files and 65,536 character buffers.

BufferedWriter 4.76 seconds
BufferedOutputStream 129.08 seconds W O R S T
DoubleBufferedWriter 10% to BufferedOutputStream 7.10 seconds
DoubleBufferedWriter 25% to BufferedOutputStream 4.91 seconds
DoubleBufferedWriter 50% to BufferedOutputStream 4.64 seconds
DoubleBufferedWriter 75% to BufferedOutputStream 4.36 seconds B E S T

BufferedReader 5.08 seconds
BufferedInputStream 109.91 seconds W O R S T
DoubleBufferedReader 10% to BufferedInputStream 16.01 seconds
DoubleBufferedReader 25% to BufferedInputStream 5.11 seconds
DoubleBufferedReader 50% to BufferedInputStream 5.09 seconds
DoubleBufferedReader 75% to BufferedInputStream 5.01 seconds B E S T
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

Code is posted at
https://wush.net/websvn/mindprod/fi...&path=/com/mindprod/example/TestBuffered.java

For future reference this BufferedWriter vs BufferedOutputStream stuff
is documented at
http://mindprod.com/jgloss/buffer.html#WRITERVSOUTPUTSTREAM
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html

 

[Roedy Green]

For future reference this BufferedWriter vs BufferedOutputStream stuff
is documented at
http://mindprod.com/jgloss/buffer.html#WRITERVSOUTPUTSTREAM

also the file I/O amanuensis will generate you Java code using the
technique.
See http://mindprod.com/applet/fileio.html
--
Roedy Green Canadian Mind Products
http://mindprod.com

When a newbie asks for help tracking a bug in a code snippet, the problem is usually is the code he did not post, hence the value of an SSCCE.
see http://mindprod.com/jgloss/sscce.html