performance question

[Jon Harrop]

I find reproducible results that show Java is comparable
to C++, you say, "Oh, but those benchmarks aren't valid." Well, the fact
that they are reproducible means that people can decide for themselves.

Reread what I wrote above: I tested the benchmark that you cited claiming
that it showed Java to be fast and found the exact opposite to be true.
Indeed, I did this for every single testable benchmark and they all showed
the same results.

The fact remains that Java is not the slowpoke you claim.

That is a triumph of hope over reality. Most of the benchmarks we've looked
at in this thread prove that to be completely wrong.

The fact is that it is, for many tasks Java is comparable, and for most,
in the same ballpark as C++.

For some tasks, yes.

The fact is that you have not countered with any reasonable evidence
for *your* claims,

Nonsense. I only just posted my results for the benchmark that _you_ cited
and they showed, again, that Java is much slower.

while the evidence just mounts and mounts and mounts for the "Java is near
enough to C++" side.

What drivel. Look at the benchmark the OP posted. A dozen Java
implementations later and 20x as much code and it is still 2x slower than
the OPs C code.

I read the articles with the same eye to reproducibility and comparability
you claim to use.

I doubt that: I have a PhD in computational physics from the University of
Cambridge.

I'm convinced by the hard evidence, enough to discount your flamebait
rants.

No, you and Steve subscribe to the religious faith that Java is performant
in the face of overwhelming evidence to the contrary.

At this point I am aware that Jon is only going to continue his polemical
approach not grounded in tests and facts, for what agenda I can only
speculate. My intent is to counteract the misrepresentations and provide
evidence for the objective-minded to decide for themselves.

If that were true you would have optimized the Java code in any of these
benchmarks (e.g. the OPs) to run as fast as the other languages.

Instead you have simply restated your position of blind faith several times.

Go ahead, optimize them. I dare you. You can start with the symbolic
simplifier I just posted.

 

[Daniel Pitts]

It's an interesting point - with closed source 3rd party code, for
example, each advance in HotSpot buys you better performance, but advances
in your C/C++ compiler have no effect unless the developer does a
recompile. With open-source code, presumably you could 'keep up' yourself,
and it's of course possible a closed-source developer has access to a
better compiler than you do [*maybe!*].

Again, this is pure speculation.

How so? Its been a consistent pattern with Java. As new JVMs come out,
legacy code gains the benefits of it immediately. This has been true,
and it is most likely a continuing trend.

 

[Steve Wampler]

Reread what I wrote above: I tested the benchmark that you cited claiming
that it showed Java to be fast and found the exact opposite to be true.
Indeed, I did this for every single testable benchmark and they all showed
the same results.

For your environment. Not for mine, using the code provided in
this thread. What is your environment, incidently? Mine is (for the recent
set of tests):

CentOS4.5, gcc 4.1.1, Sun JDK 1.6u3 on a dual-Opteron (2GHz) with 2GB ram

With runs of 10,000,000 lines (since the OP has stated the real case may
involve billions of lines), the 'optimized' java is 1/3 again *faster* than
the C version - on the above hardware.

 

[Steve Wampler]

It's an interesting point - with closed source 3rd party code, for
example, each advance in HotSpot buys you better performance, but advances
in your C/C++ compiler have no effect unless the developer does a
recompile. With open-source code, presumably you could 'keep up' yourself,
and it's of course possible a closed-source developer has access to a
better compiler than you do [*maybe!*].

Again, this is pure speculation.

Oh, absolutely - we agree! I was giving my interpretation of what was said.

 

[Jon Harrop]

What is your environment, incidently?

Athlon64 X2 4400+
Debian Lenny
GCC 4.2.3
Sun JDK 1.6

 

[Steve Wampler]

Athlon64 X2 4400+
Debian Lenny
GCC 4.2.3
Sun JDK 1.6

Now that is interesting. I was hoping it was a single cpu
system as I could see that making a significant difference
on the Java side.

Is Lenny a 64-bit kernel? CentOS4.5 has a 32-bit kernel on the
Opteron.

I'd be surprised in going from 4.1.1 to 4.2.3 made that
much of an improvement on the C++ side, but I can't see
the difference between the Opteron and Athlon64 resulting
in as much a swing as we're seeing between your machine
and mine.

I'd be interested in seeing what others are getting with
the two programs, also.

 

[Lew]

I doubt that: I have a PhD in computational physics from the University of
Cambridge.

Oh, I bow to you, Great One!

You accuse me of being "religious" when I clearly state that I find Java to be
slower than C++ on many of the benchmarks. This to me is evidence that you
are a (highly educated) troll.

 

[Steve Wampler]

Missing optimizations.

I'm confused by this. Earlier you said there was no evidence of Hotspot
helping long running programs. Are you saying that Hotspot helps even
short running programs - so as I shorten the loop it will continue to
help? (I'm uncertain of how to test this outside of a profiler, as
it may have to get quite short to see a change - and measurement noise
may dominate when down to a few loop instances. I can run many repeats
of the shorter loops, I suppose.)

 

[George Neuner]

All functional programming languages (including F#) have runtimes but none
have startup times anything like as slow as Java.

Most language runtimes are not doing JIT compilation.

As for F#, current versions of Windows make use of .NET internally so
the VM is preloaded by the OS and always running (.NET 1.1 is a
required update for 2K/XP/2003 and .NET 2.0 is required for Vista).
This eliminates much of the startup time for a .NET application.

Mono and Java applications have similar startup overhead on Linux
where neither is normally running.

George

 

[Lew]

Most language runtimes are not doing JIT compilation.

As for F#, current versions of Windows make use of .NET internally so
the VM is preloaded by the OS and always running (.NET 1.1 is a
required update for 2K/XP/2003 and .NET 2.0 is required for Vista).
This eliminates much of the startup time for a .NET application.

Mono and Java applications have similar startup overhead on Linux
where neither is normally running.

Thank you for debunking that disingenuous claim, George. It came from the
same person who also said:


Don't you love it when people argue both sides of a point just so they can argue?

 

[George Neuner]

Having just recently completed a round trip on Amtrak from Tacoma, WA,
to Beaumont, TX, and back, I'm not sure the train would beat the plane
on coal hauling, even if there was only one plane.

The train is about 40 hours each way, if there are no delays, but trains
seem really good at finding ways to be dealyed. A plane is about 4.5 hours.
So that one plane could take about 10 trips for every one train trip. A
quick check on Wikipedia shows that a C-5A (the first cargo plane that
came to mind) can carry about 1/8th of the coal that Joshua Cranmer's
example used...so it looks like the plane could actually win this!

In 4.5 hours you *should* be able to fly from Tacoma to D.C. Just
shows the airlines are farting around as usual. I don't know how fast
a C-5A is though - I doubt it can go .9 Mach like a typical passenger
jet.

Did anybody else see in the news a few weeks ago that somebody had set
a new speed record for the northern Gumball Rally route? NYC to Long
Beach, CA in 31 hours by car! Unfortunately I didn't save the article
when I saw it. I've tried googling for it but I can't seem to find it
again.

George

 

[RedGrittyBrick]

Java is OK. I have a number of issues with it including syntax,
organization, and the fact that it religiously enforces OO which IMO
is not an optimal programming model for many problems. While I am
perfectly comfortable working in OO (pure or not), I prefer languages
that let me mix different programming styles more freely.

Interesting. When I started learning Java, my biggest problem was that I
felt I was writing purely procedural code in Java. I tended to have
classes with only static methods.

 

[Jon Harrop]

I'm confused by this. Earlier you said there was no evidence of Hotspot
helping long running programs. Are you saying that Hotspot helps even
short running programs - so as I shorten the loop it will continue to
help? (I'm uncertain of how to test this outside of a profiler, as
it may have to get quite short to see a change - and measurement noise
may dominate when down to a few loop instances. I can run many repeats
of the shorter loops, I suppose.)

Exactly. I see no evidence that HotSpot is doing anything different to a
normal static optimizer, i.e. it does not appear to incrementally improve
the performance of the Java code as it runs.

 

[Jon Harrop]

Now that is interesting. I was hoping it was a single cpu
system as I could see that making a significant difference
on the Java side.

Is Lenny a 64-bit kernel? CentOS4.5 has a 32-bit kernel on the
Opteron.

Everything is 64-bit here.

I'd be surprised in going from 4.1.1 to 4.2.3 made that
much of an improvement on the C++ side, but I can't see
the difference between the Opteron and Athlon64 resulting
in as much a swing as we're seeing between your machine
and mine.

Agreed.

 

[Jon Harrop]

You accuse me of being "religious" when I clearly state that I find Java
to be slower than C++ on many of the benchmarks.

If those are your findings, you should not have stated "The fact remains
that Java is not the slowpoke you claim.".

 

[Andreas Leitgeb]

How so? Its been a consistent pattern with Java. As new JVMs come out,
legacy code gains the benefits of it immediately.

But then, next big step to expect would be thorough 64bit-capability, which
is likely to take a while till the jvm will be able to handle 64bit-values
*in one go*.

This has been true, and it is most likely a continuing trend.

It is true, that in java optimization happens on the runtime-side,
and is improved every once in a while, but the amount of this effect
in future improvements is pure speculation.
The same could have been expected from C-compilers, and well, it didn't
seem to work for that little sample program. Why assume that java would
optimize that piece of code better in future?

 

[Steve Wampler]

Exactly. I see no evidence that HotSpot is doing anything different to a
normal static optimizer, i.e. it does not appear to incrementally improve
the performance of the Java code as it runs.

I'm not sure how to tell, either. For example, in the following run,
*something* is causing a performance bump somewhere between 10 and 20
millions lines of output - which is well after the effect you were
attributing to cache warming earlier, which would have been taking
place between 0 and 1 millions lines output. Is this the
result of HotSpot kicking in again after 2-4 seconds of running?
I have no simple way of telling but something is happening and
it happens consistently (I've seen it every repeat of this run).

->time java -server T1 10000000 20 >/dev/null
Took 2.067 seconds
Took 2.131 seconds
Took 1.634 seconds
Took 1.631 seconds
Took 1.625 seconds
Took 1.622 seconds
Took 1.632 seconds
Took 1.628 seconds
Took 1.621 seconds
Took 1.625 seconds
Took 1.626 seconds
Took 1.615 seconds
Took 1.616 seconds
Took 1.621 seconds
Took 1.63 seconds
Took 1.627 seconds
Took 1.625 seconds
Took 1.621 seconds
Took 1.63 seconds
Took 1.613 seconds
java -server T1 10000000 20 > /dev/null 33.45s user 0.09s system 99% cpu 33.581 total

 

[Lew]

If those are your findings, you should not have stated "The fact remains
that Java is not the slowpoke you claim.".

Yes, I should have, because you were claiming 10-100x slower, and I found only
2x slower. Try being honest for a change.

 

[Lew]

I'm not sure how to tell, either. For example, in the following run,
*something* is causing a performance bump somewhere between 10 and 20
millions lines of output - which is well after the effect you were
attributing to cache warming earlier, which would have been taking
place between 0 and 1 millions lines output. Is this the
result of HotSpot kicking in again after 2-4 seconds of running?
I have no simple way of telling but something is happening and
it happens consistently (I've seen it every repeat of this run).

->time java -server T1 10000000 20 >/dev/null
Took 2.067 seconds
Took 2.131 seconds
Took 1.634 seconds
Took 1.631 seconds
Took 1.625 seconds
Took 1.622 seconds
Took 1.632 seconds
Took 1.628 seconds
Took 1.621 seconds
Took 1.625 seconds
Took 1.626 seconds
Took 1.615 seconds
Took 1.616 seconds
Took 1.621 seconds
Took 1.63 seconds
Took 1.627 seconds
Took 1.625 seconds
Took 1.621 seconds
Took 1.63 seconds
Took 1.613 seconds
java -server T1 10000000 20 > /dev/null 33.45s user 0.09s system 99%
cpu 33.581 total

It doesn't do much good to present facts before the intellectually dishonest,
as our trollish Cambridge God, er, Don, Mr. Harrop is.

 

[Lew]

Interesting. When I started learning Java, my biggest problem was that I
felt I was writing purely procedural code in Java. I tended to have
classes with only static methods.

Interesting to me, too, as I have found that one of the singular strengths of
Java is its insistence on OO programming.

I've noticed that people often prefer to take shortcuts when writing code -
breaking the OO style being one such. The problem is the concomitant
reduction in maintainability and stability of the code. As a good part of my
career has been to pick up after absent programmers, I am quite sensitive to
the true lifetime cost of such shortcuts.

What seems reasonable to an author, especially a highly-skilled one as,
presumably, Mr. Neuner is, will often cause trouble for those who follow, as
teams hire a range of skills and experience.