Getter performance

[Aéris]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello everybody.

I work on an application where performances are important.

To optimize it, I thought a direct access to a variable (foo.bar) would
be more efficient than a getter call (foo.getBar()).
I thought by avoiding the call to a method and all that goes with it
(context switching, stacking, return value…), I can save time, but this
code sample prove the contrary :
http://pastebin.com/bP1nqxce
Direct variable access : 1041 ms
Getter call : 556 ms

The difference is even more important if I don't modify the variable
value (lines 29 and 36 commented) :
Direct variable access : 95 ms
Getter call : 4 ms

How can we explain this not obvious huge difference ( 50 and 95% ) ?

- --
Aeris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOmfUwAAoJEK8zQvxDY4P97f4H/2qXiwLnDwuKxsHEkGYtk0va
HSASqi5Dj5g0RxD6h4G65UveBI1ALLFZAJmjrsklOl2OgN5YSvmudxpLBzcdk53S
4ans2g5lLzc21Mw5UtdW5oz3TeZZ6Z0Cbna9MloDopw6wJM06VzblMPbNr+XQfwt
S49rnd0RckKToR8lzveIJXNWuPmxKmh+JsV85ZFWepSLXB4xhsGCxuNdWq/Opjq5
s1nPvCUL5R6vKEbspyV4BRqrDEXwNKE4EPTHqgTNkklc9IMNTDjM0bPM2PhJwKlW
iX8xFqMeyizSAOPmNdaRq907LLSZhFo5jSc2JsKBj7AYb4oy8ewP0ozYnDMaFt8=
=AztU
-----END PGP SIGNATURE-----

 

[Arne Vajhøj]

I work on an application where performances are important.

To optimize it, I thought a direct access to a variable (foo.bar) would
be more efficient than a getter call (foo.getBar()).

Wrong approach. Write nice clean code. If it is fast enough then
fine. If not then measure where the bottleneck is. It seems highly
unlikely to be in getters.

I thought by avoiding the call to a method and all that goes with it
(context switching, stacking, return value…), I can save time, but this
code sample prove the contrary :
http://pastebin.com/bP1nqxce
Direct variable access : 1041 ms
Getter call : 556 ms

The difference is even more important if I don't modify the variable
value (lines 29 and 36 commented) :
Direct variable access : 95 ms
Getter call : 4 ms

How can we explain this not obvious huge difference ( 50 and 95% ) ?

First thing would be to run a lot more than 100000 times. Such
small intervals will be very random on a multi tasking OS.

Arne

 

[Arne Vajhøj]

Wrong approach. Write nice clean code. If it is fast enough then
fine. If not then measure where the bottleneck is. It seems highly
unlikely to be in getters.


First thing would be to run a lot more than 100000 times. Such
small intervals will be very random on a multi tasking OS.

With 1000000 and -server I get:

1763
1608

which is rather close.

Arne

 

[Aéris]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Le 15/10/2011 23:36, Arne Vajhøj a écrit :

Wrong approach. Write nice clean code. If it is fast enough then
fine. If not then measure where the bottleneck is. It seems highly
unlikely to be in getters.

I agree with you, my code is clean and fast enough for the moment but
this question of getter is more general and nag me since few time

First thing would be to run a lot more than 100000 times. Such
small intervals will be very random on a multi tasking OS.

Tested on Integer.MAX_VALUE, same result.

But I notice that if I read « tmp » value, results are differents.
Adding
if (tmp.isEmpty()) {
System.out.println("Empty");
}
leads to more expected result :
Direct access : 106 ms
Getter call : 2223 ms
Seems Java compiler is very efficient and don't really call the getter
if the return value is not read and the getter has no side-effect.

- --
Aeris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOmgJOAAoJEK8zQvxDY4P9wqIIAJnfxjQgsoUMmKr7CxIamTBr
KeQNH7gCGX709/d2ZAb3PCRQ9UTKiuteEU8JHal6735ziyifIBtwxHBtQTB6jduC
74uW1aMzMUYJbRpBL0wViL7SYW7Ob57a1Oz8/bYl1+htDEVb2FWMMmfgbhDf9HeT
MXMilvcCVlDs4OxG3NnFL14Y7Nv1He6YzV3BaMOIz1serH9pZy1/NCgdazfhwrZH
6bGwtYET2hbzabda9o2CrX8jRjenyLWz1ad9QPanLvHDWJRf+Y316lNXLmvhbR++
+cQzQJ8hmj9nlyFGJuQ1tG0VzTE/E/qgcdN/sCOWiApC4Nhsp3tUD+68H02bFTU=
=UnUI
-----END PGP SIGNATURE-----

 

[BGB]

With 1000000 and -server I get:

1763
1608

which is rather close.

this is an interesting result...

I was aware that the JVM optimized getters, but unclear is why they are
slightly faster than direct field access, rather than exactly the same
speed (or sometimes slightly faster and sometimes slightly slower).

 

[markspace]

this is an interesting result...

I was aware that the JVM optimized getters, but unclear is why they are
slightly faster than direct field access, rather than exactly the same
speed (or sometimes slightly faster and sometimes slightly slower).

I don't think the result indicates that getters are slightly faster, or
vice versa. It's just to close too call, and there's isn't really
anyway to say one is universally better, speed-wise.

Software engineering-wise, and best practice-wise, the getter is
undoubtedly the better option.

 

[Arne Vajhøj]

Le 15/10/2011 23:36, Arne Vajhøj a écrit :

I agree with you, my code is clean and fast enough for the moment but
this question of getter is more general and nag me since few time

If this is for fun, then it is OK. 25 years ago I spent a lot
of time trying to optimize Sieve of Eratosthenes in VAX
assembler. Fun.

But this is work, then stop wasting your employers money and
focus on real problems.

Tested on Integer.MAX_VALUE, same result.

But I notice that if I read « tmp » value, results are differents.
Adding
if (tmp.isEmpty()) {
System.out.println("Empty");
}
leads to more expected result :
Direct access : 106 ms
Getter call : 2223 ms
Seems Java compiler is very efficient and don't really call the getter
if the return value is not read and the getter has no side-effect.

The JIT compiler can do many things.

Another problem with microbenchmarks is that they typical
measure scenario X1 vs Y1 and there are no guarantee that
X1 faster than Y1 implies that sum(Wi*Xi) is faster
that sum(Wi*Yi) for i=1..n, which is the real world.

Small differences can mean a lot of difference for a
single scenario, which is not relevant for the real world
with hundreds or thousands of almost identical but still
different scenarios.

Arne

 

[Aéris]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Le 16/10/2011 01:44, Arne Vajhøj a écrit :

But this is work, then stop wasting your employers money and
focus on real problems.

This is not my employer's money, but my free time =)

- --
Aeris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOmrx0AAoJEK8zQvxDY4P9zscIAL+ERs3ds9+Qc5dbsq/pGduN
zAu7LmCOU37+pVuPKZuiwV7YQOwM4TDHutVLX3vfocYsTkxCZSz6LyqBrWBxMAOx
NIc6c608ndRjre44aHX3m2Fk5lZFqJ+8pIRzDVIPCKImBMgsdO/EuYZdn9a+BiD7
q9vcGquiL/zMVUJNuKas8ahiqsquL1l1t0Mi0s1ooeqwsBp0Thw1Xg6qNoAQZ2/c
1u2/mo80Ixb2976BlkQir0enjYQV0D3lHobc4SKZ+Dc8vhS7i+MkjcHBqifPxYMK
b6rERWKaJXVdCerE/QKyAtnmEG9Pc5q7dQNSjgSLvg6EJ4deVKuU/2iUtpzE8Wo=
=WQdS
-----END PGP SIGNATURE-----

 

[Eric Sosman]

I work on an application where performances are important.

To optimize it, I thought a direct access to a variable (foo.bar) would
be more efficient than a getter call (foo.getBar()).
I thought by avoiding the call to a method and all that goes with it
(context switching, stacking, return value…), I can save time, but this
code sample prove the contrary :
http://pastebin.com/bP1nqxce
Direct variable access : 1041 ms
Getter call : 556 ms

The difference is even more important if I don't modify the variable
value (lines 29 and 36 commented) :
Direct variable access : 95 ms
Getter call : 4 ms

How can we explain this not obvious huge difference ( 50 and 95% ) ?

Insufficiently precise measurement. You are trying to tell
whether two nickels are heavier than four pennies by piling each
stack of coins in turn atop a bulldozer and weighing the bulldozer.
If you want to measure tiny effects, seek a measurement framework
that doesn't involve huge effects.

Fundamentally, though, you're going about the larger task in
the wrong way: You are simply guessing that the speed of access to
a variable is important to the performance of your application. Do
you have even the slightest shred of evidence that this is so? Or
are you trying to improve your car's fuel economy by scraping off
all the paint to reduce the vehicle weight?

1: Decide what performance criteria the application must satisfy.
Make this quantifiable: "Throughput of at least X doodads per eyeblink"
or "95% of responses in no more than Y milliquivers," not undecidable
like "As fast as possible."

2: (Optional) Make back-of-the-envelope calculations about data
rates, data set sizes, and so on, just to estimate the resources
that will be required. Is the load within reach of a laptop, or do
you need a clustered network of supercomputers? Can you handle the
traffic with WiFi, or do you need sixteen 10Gb/s fibers in parallel?

3: Write the application, as cleanly and simply as you can in
light of [1] and [2]. During this stage, bear in mind that even the
most expert of programmers is usually surprised about where his code
spends the majority of its time (this is not opinion; it's been found
over and over again in actual experiment). In other words, favor
"clean and simple" over "clever and reputedly fast."

4: Measure the performance, and compare to the thresholds of [1].
If you satisfy [1], stop!

5: (Only if [1] isn't satisfied) Apply profilers, debuggers, and
other tools to discover which parts of the program are too slow. Apply
your efforts to speeding up those parts only; pay no attention to the
others. Then return to [4]. In very rare cases (usually when [2] was
skipped or when some external agency forces a change in [1]), return
to [3] and start over.

Do not begin by trying to micro-optimize; that way lies madness.
Trust me; I know: Been there, done that, bear the scars.

 

[Lars Enderin]

2011-10-16 13:14, Aéris skrev:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Le 16/10/2011 01:44, Arne Vajhøj a écrit :

This is not my employer's money, but my free time =)

- --
Aeris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOmrx0AAoJEK8zQvxDY4P9zscIAL+ERs3ds9+Qc5dbsq/pGduN
zAu7LmCOU37+pVuPKZuiwV7YQOwM4TDHutVLX3vfocYsTkxCZSz6LyqBrWBxMAOx
NIc6c608ndRjre44aHX3m2Fk5lZFqJ+8pIRzDVIPCKImBMgsdO/EuYZdn9a+BiD7
q9vcGquiL/zMVUJNuKas8ahiqsquL1l1t0Mi0s1ooeqwsBp0Thw1Xg6qNoAQZ2/c
1u2/mo80Ixb2976BlkQir0enjYQV0D3lHobc4SKZ+Dc8vhS7i+MkjcHBqifPxYMK
b6rERWKaJXVdCerE/QKyAtnmEG9Pc5q7dQNSjgSLvg6EJ4deVKuU/2iUtpzE8Wo=
=WQdS
-----END PGP SIGNATURE-----

Why are you wasting bandwidth by including PGP signatures? Your
contributions are not all that important.

 

[Jaap Droogers]

Hello everybody.

I work on an application where performances are important.

To optimize it, I thought a direct access to a variable (foo.bar) would
be more efficient than a getter call (foo.getBar()).
I thought by avoiding the call to a method and all that goes with it
(context switching, stacking, return value…), I can save time, but this
code sample prove the contrary :
http://pastebin.com/bP1nqxce
Direct variable access : 1041 ms
Getter call : 556 ms

The difference is even more important if I don't modify the variable
value (lines 29 and 36 commented) :
Direct variable access : 95 ms
Getter call : 4 ms

How can we explain this not obvious huge difference ( 50 and 95% ) ?

You forgot the JIT compiler. All the nice clean code you have written is
optimized by the compiler. You don't write C code, remember.
Why is Java so fast: it uses the JIT compiler which optimizes the code
for the machine the program is running on.
if you write
int a = myObject.getA();

the compiler compiles it probably as:
int a = myObject.a;

There is also a change that you try to optimize your code and the JIT
compiler does not now how to optimize it, so your code runs slower.

Jaap

 

[BGB]

You forgot the JIT compiler. All the nice clean code you have written is
optimized by the compiler. You don't write C code, remember.

note:
many C and C++ compilers will perform the same optimization in many
cases. some compilers will require both to be present in the same
compilation unit, but some others (such as GCC and LLVM/Clang) can make
use of link-time optimization.

these sorts of optimizations are nothing specific to the JVM.

Why is Java so fast: it uses the JIT compiler which optimizes the code
for the machine the program is running on.
if you write
int a = myObject.getA();

the compiler compiles it probably as:
int a = myObject.a;

There is also a change that you try to optimize your code and the JIT
compiler does not now how to optimize it, so your code runs slower.

yep.

at the CPU level the reference may potentially compile down to a single
"mov" instruction.

if the getter is called but its value isn't used, then there is a chance
that it may be discarded altogether (so no CPU instructions are produced
in this case).

if trying to micro-benchmark or micro-optimize things (yes... including
in C), one will usually spot this case by the running time dropping to a
very small number (IOW: the code runs impossibly fast).


generally though, it is not really productive to worry about
micro-optimizing things. better IMO to leave optimizing for when and
where it actually matters.

 

[David Lamb]

I don't think the result indicates that getters are slightly faster, or
vice versa. It's just to close too call, and there's isn't really anyway
to say one is universally better, speed-wise.

A really thorough investigation would have to take into account a lot of
performance factors; on multitasking systems (that is, most systems) a
single expensive process swap that happened on one loop and not the
other would make a huge difference. You'd need to run the whole
experiment several times (not just lots of times in one program run).
But markspace' comment is almost certainly correct: too close to call.

Software engineering-wise, and best practice-wise, the getter is
undoubtedly the better option.

Exactly.

 

[David Lamb]

You forgot the JIT compiler. All the nice clean code you have written is
optimized by the compiler. You don't write C code, remember.
Why is Java so fast: it uses the JIT compiler which optimizes the code
for the machine the program is running on.
if you write
int a = myObject.getA();

the compiler compiles it probably as:
int a = myObject.a;

Yes, and... thats why you need to do the loop thingy already mentioned:
you have to amortize the JIT expense over many calls.

Which also, if I understand things correctly, means that the first time
you do almost anything in a Java program, it's much slower than the Nth
time (for some value of N). I sometimes see programs where bringing up a
menu the first time seems to take forever, while the 2nd is faster; is
that due to the JIT having to run each time I invoke the program?

 

[Roedy Green]

To optimize it, I thought a direct access to a variable (foo.bar) would
be more efficient than a getter call (foo.getBar()).

Hotspot inlines getters, but not right away. It interprets the code
for a while to monitor it. If you measure a very short running program
you will not see any optimisation effects.

Jet also inlines getters.
--
Roedy Green Canadian Mind Products
http://mindprod.com
It should not be considered an error when the user starts something
already started or stops something already stopped. This applies
to browsers, services, editors... It is inexcusable to
punish the user by requiring some elaborate sequence to atone,
e.g. open the task editor, find and kill some processes.

 

[Paul Cager]

Yes, and... thats why you need to do the loop thingy already mentioned:
you have to amortize the JIT expense over many calls.

Yes, the benchmark doesn't let the code "warm up" before measuring.

It is also worth pointing out that within the loop being measured,
there is a call to rand() which is:

public static String rand() {
return new BigInteger(130, random).toString(32);
}

I expect what we are really measuring here is the performance of
SecureRandom and BigInteger - the previous comment about pennies and
bulldozers was good.

http://www.ibm.com/developerworks/java/library/j-jtp02225/index.html
is a really useful page when writing micro-benchmarks.

 

[Roedy Green]

You'd need to run the whole
experiment several times (not just lots of times in one program run).
But markspace' comment is almost certainly correct: too close to call

One thing that bedevils such testing is garbage collection which
happens at "random" times and makes things look slower than they
really are. That is why you need very long tests to average them out.
See http://mindprod.com/jgloss/benchmark.html
for some hints.

You to avoid the initial class load time or the Hotspot time to
optimise if you are after just the speed of the final code.

Knuth has frightened people from even investigating speed out of
curiosity. He was trying to stop people from writing needlessly
optimised unreadable code.

If there are two ways of doing things, both equally readable and
maintainable, it strikes me as silly to blind yourself to knowing
which is faster. You might as well do it that way whether it is
necessary or not.

The approach I see advocated could be parodied by saying you should
select a random sort algorithm including insertion sort, and bubble
sort and only putting in a clever algorithm after benchmarking and
analysis prove it is necessary.

I think you should know without benchmarking each individual case
which Map or Collection would be best for a given task.
--
Roedy Green Canadian Mind Products
http://mindprod.com
It should not be considered an error when the user starts something
already started or stops something already stopped. This applies
to browsers, services, editors... It is inexcusable to
punish the user by requiring some elaborate sequence to atone,
e.g. open the task editor, find and kill some processes.

 

[Daniel Pitts]

I think you should know without benchmarking each individual case
which Map or Collection would be best for a given task.

I nearly agree.

An experienced programmer should know which Map or Collection would work
sufficiently well for a given task. "Best", by some definitions, would
almost always be a custom class (perhaps not even implementing Map or
Collection). The thing is that best may be much better than needed.

 

[Eric Sosman]

[...]
Knuth has frightened people from even investigating speed out of
curiosity.

Oh, yeah, right. As in "Ri-i-i-i-ght."

That's why he didn't write three entire volumes of TAOCP (and
more in the works) in an effort to reach beyond O() notation to the
constant factors that lie behind. That's why he didn't devote a
whole lot of text and a whole lot of subtle mathematics to winkling
out differences between O(n log n) Quicksort and O(n log n) Mergesort
and O(n log n) Heapsort and O(n log n(?)) Shellsort. That's why he
never explored threaded trees (they improve some operations by a
factor of only O(log n) so who cares?). That's why he never bothered
with Marsaglia's rectangle-wedge-tail method for generating normally-
distributed random numbers; it's just a trifling micro-optimization.

Oh, sorry, wait: By "frightened" you don't mean "dissuaded," but
literally "scared." Well, I confess that some of Knuth's math goes
over my head. But I feel at the same time that someone who shrinks
from even trying to understand it does not merit the title of
"engineer."

 

[Arved Sandstrom]

I nearly agree.

An experienced programmer should know which Map or Collection would work
sufficiently well for a given task. "Best", by some definitions, would
almost always be a custom class (perhaps not even implementing Map or
Collection). The thing is that best may be much better than needed.

Roedy may have meant "best" in this sense. It's a word with lots of
definitions, one of which is "most suitable". For a software
developer/engineer I'd think that "most suitable" would always be the
reigning definition of "best"; another definition (like the one you
posit as a possibility, leading perhaps to a custom class) might be more
of a computer science thing.

Roedy's right, though, with his overall comments, although I wouldn't
blame anything on The Donald [1]. For starters, way too many people -
having never read anything by Knuth except various quotes, have only
ever seen "premature optimization is the root of all evil" in isolation.
They've never read the important sentences surrounding that one, that
supply essential context.

I've seen it myself, that folks seize on this one sentence and disparage
reasonable design-time efforts to think about performance. And I've
heard often enough from other developers who should know better that we
should "write it first and we'll profile it later if it's too slow". Or
a memory hog. These folks forget that Knuth talked about _premature_
optimization: there's a lot of design and implementation-time
optimization that ain't.

c2.com on their page about "UniformlySlowCode"
(http://c2.com/cgi/wiki?UniformlySlowCode) discusses a situation that
can happen a lot if you don't practice experience-seasoned non-premature
optimization. I've seen this happen more than once, that an app is
sluggish and a memory beast, and yet when you analyze it you find that
you have to fix, well, everything.

AHS

1. Man, how can Trump be The Donald? It's just not right. I've set
matters straight here.