M
Martin Schoeberl
As I'm working an a Java processor I'm interested to benchmark embedded
Java systems. However, the usual SPECJvm98 is far too large for many
embedded system.
After searching around on the net to find embeddded benchmarks in Java
I've decided to write a new benchmark suit.
It contains several micro-benchmarks to evaluate CPI for single bytecodes
or short sequences of bytecodes, one synthetic benchmark (the Sieve of
Eratosthenes) and two application benchmarks.
To provide a realistic workload for embedded systems a real-time
application was adapted to create the first application benchmark (Kfl).
The application is from one node of a distributed motor control system. A
simulation of the environment (sensors and actors) and the communication
system (commands from the master station) forms part of the benchmark for
simulating the real-world workload.
The second application benchmark is an adaption of a tiny TCP/IP stack
(Ejip) for embedded Java. The benchmark contains two UDP server/clients
exchanging messages via a loopback device.
There is a great variation in processing power on different embedded
systems. To handle these variation all benchmarks are `self adjusting'.
Each benchmarks consists of an aspect that is benchmarked in a loop and
an `overhead' loop that contains any overhead from the benchmark that
should be subtracted from the result (this feature is for the micro
benchmarks). The loop count adapts until the benchmark runs for more than
a second. The number of iterations per second is calculated which means
that higher values indicate better performance.
The benchmark framework needs only two system functions: One to measure
time in millisecond resolution and one to print the results. These
functions are encapsulated in LowLevel.java and can be adapted to
environments where the full Java library is not available. For example,
the leJOS system has very limited output capabilities and and a special
LowLevel.java exists in the lejos subdirectory. The benchmark is
straight-forward to use. To compile and run all benchmarks on a standard
JVM simple type:
javac jbe/DoAll.java
java jbe.DoAll
I've put the results of JOP, leJOS, JStamp (aJ80), EJC and, for a
reference only, different JVM versions (Suns JVM and gcj) running on a
Pentium MMX 266 at my website:
http://www.jopdesign.com/perf.jsp
If someone has access to other embedded Java solutions it would be nice
to add benchmark results to the table. You can find the source of the
embedded benchmark at the website above.
I would be especially interested in following platforms: the aJ100, TINI,
Cjip, Lightfood in an FPGA, Moon and the PSC1000 (now called Ignite).
Martin
Java systems. However, the usual SPECJvm98 is far too large for many
embedded system.
After searching around on the net to find embeddded benchmarks in Java
I've decided to write a new benchmark suit.
It contains several micro-benchmarks to evaluate CPI for single bytecodes
or short sequences of bytecodes, one synthetic benchmark (the Sieve of
Eratosthenes) and two application benchmarks.
To provide a realistic workload for embedded systems a real-time
application was adapted to create the first application benchmark (Kfl).
The application is from one node of a distributed motor control system. A
simulation of the environment (sensors and actors) and the communication
system (commands from the master station) forms part of the benchmark for
simulating the real-world workload.
The second application benchmark is an adaption of a tiny TCP/IP stack
(Ejip) for embedded Java. The benchmark contains two UDP server/clients
exchanging messages via a loopback device.
There is a great variation in processing power on different embedded
systems. To handle these variation all benchmarks are `self adjusting'.
Each benchmarks consists of an aspect that is benchmarked in a loop and
an `overhead' loop that contains any overhead from the benchmark that
should be subtracted from the result (this feature is for the micro
benchmarks). The loop count adapts until the benchmark runs for more than
a second. The number of iterations per second is calculated which means
that higher values indicate better performance.
The benchmark framework needs only two system functions: One to measure
time in millisecond resolution and one to print the results. These
functions are encapsulated in LowLevel.java and can be adapted to
environments where the full Java library is not available. For example,
the leJOS system has very limited output capabilities and and a special
LowLevel.java exists in the lejos subdirectory. The benchmark is
straight-forward to use. To compile and run all benchmarks on a standard
JVM simple type:
javac jbe/DoAll.java
java jbe.DoAll
I've put the results of JOP, leJOS, JStamp (aJ80), EJC and, for a
reference only, different JVM versions (Suns JVM and gcj) running on a
Pentium MMX 266 at my website:
http://www.jopdesign.com/perf.jsp
If someone has access to other embedded Java solutions it would be nice
to add benchmark results to the table. You can find the source of the
embedded benchmark at the website above.
I would be especially interested in following platforms: the aJ100, TINI,
Cjip, Lightfood in an FPGA, Moon and the PSC1000 (now called Ignite).
Martin