How big is the 1.5 JRE download?

[zoopy]

Though if you rename the 1.5 rt.jar to
a .zip and open it, WinZip pops up a 'comment'
saying simply 'PACK200'.. It still lists
the 'File compression ratio' as a fat 0% though.

Yep, see that comment too.

Of all the rt.jar files still hanging around on my system (1.3.0/1, 1.4.0/1/2, 1.5.0), none of them
is compressed. I've always assumed this was to speed up class-loading.

In 1.5/5.0 they additionally speed up class-loading by something called class data sharing
(<http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html>).
In the jre1.5*/bin/client directory you'll find a "classes.jsa" file (Java Shared Archive) which
apparently contains an internal representation of the core classes.

 

[Andrew Thompson]

There was at least a three times size reductuion
with standard Zip compression, a saving of some
20 Meg of disk space.

It just occured to me that the '9 Meg' I
quoted for the compressed file is wrong..

I compressed the *entire* zip file, rather
than each entry inside it. Given what you
were saying about how each file has it's own
self contained expansion instructions/patterns
(words to that effect), compressing a single
big file structure represented in a zip *should*
be more efficient than compressing the same files
individually.

Bottom line, I would expect that 'each entry
compressed individually' would result in a
significantly larger final archive file.

OK ..stuff expectaions, here is my test..

The file in question is the 1.5 rt.jar

Form Size
---- ----
Disk (extracted): 29,873,493
Zip Uncompressed: 31,971,302
Zip Each entry: 15,279,773
Zip Entire file: 9,693,680

Or, to put that another, more visual way..
<http://www.physci.org/test/graph/in...isk+None+Each+All&val=299+320+153+97&width=70>

 

[Andrew Thompson]

.

In 1.5/5.0 they additionally speed up class-loading by something called class data sharing
(<http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html>).
In the jre1.5*/bin/client directory you'll find a "classes.jsa" file (Java Shared Archive) which
apparently contains an internal representation of the core classes.

Holy Crap! At 12,058,624 bytes, it is ironic that
the file to 'speed up' 1.5 is larger than the entire
1.3.1 core classes (11,646,454)..

 

[Christophe Vanfleteren]

Holy Crap! At 12,058,624 bytes, it is ironic that
the file to 'speed up' 1.5 is larger than the entire
1.3.1 core classes (11,646,454)..

That's probably not so surprising since the core library has grown quite a
lot since the 1.3 days.
And the file only gets created during installation (or via a switch in the
java command) anyway.

 

[Andrew Thompson]

That's probably not so surprising since the core library has grown quite a
lot since the 1.3 days.

The problem here, Christophe, is that
your letting the truth get in the way
of a good story. ;-)

 

[Michael Borgwardt]

If an obfuscator replaces a class name such as "SignInJustInTimePanel" with
an obfuscated name such as "qw" does that destroy anything of importance to
the user?

If the user just wants tu run the code as is, no. But it's information (important,
for example, if the user wanted to call those classes from his own code), and it's
gone. That's why I compared it to lossy compression.

 

[Hamilcar Barca]

An UNCOMPRESSED Zip archive.

I forgot about that possibility, and you're of course correct.

 

[Mickey Segal]

Only if the class is never used as a component in another application.
Obfuscators should only be applied to end-user applications.

Obfuscators allow you to exclude particular class names from obfuscation.
It would be good to tie obfuscation to the public / private designations
within Java. I don't know whether existing obfuscators do this.

 

[Chris Uppal]

There was at least a three times size reductuion
with standard Zip compression, a saving of some
20 Meg of disk space. If compressed files were
actually faster as well, I cannot believe Sun
would be so silly as to not do it.

Oh, my boy, my boy... It is time you left this world of make-believe behind.

/Of course/ Sun could be that silly...

(I'm not saying they /are/, mind, but the certainly could.)

Cynicism aside, another reason might be to avoid having to duplicate the
relatively complicated code for JAR file reading in the initial class loader
(which couldn't use the "normal" java.util.zip stuff during bootstrapping).

-- chris

 

[Chris Uppal]

The file in question is the 1.5 rt.jar

Form Size
---- ----
Disk (extracted): 29,873,493
Zip Uncompressed: 31,971,302
Zip Each entry: 15,279,773
Zip Entire file: 9,693,680

I tried the test that I mentioned yesterday: compressing each .class files with
a compressor that had been pre-trained with "typical" class file contents.
That should allow the compressor to work better since its dictionary is already
populated with common sub-strings, and so avoid much of the "warm-up" overhead
of separate compression.

I'm using the rt.jar from the Windows 1.4.2 JDK.

Size without compression:
26,380,124

Size compressing all and only the .class files with maximum compression:
13,532,241

Size compressing the .class files pre-trained with the data from
java/lang/String.class
12,703,747

(Using java/lang/Object.class and java/lang/Class.class as training instead:
12,772,088
12,535,712
so the choice of trainer doesn't seem to make much difference.)

Which suggests that the loss caused by independent compression of the .class
files is not really all that great. Certainly, /I/ wouldn't bother adding the
complexity of training to the JAR file format on the basis of these figures.

-- chris

 

[Andrew Thompson]

It is time you left this world of make-believe behind.

...ehh (shrugs), it has pretty colors.

 

[Andrew Thompson]

Cynicism aside, another reason might be to avoid having to duplicate the
relatively complicated code for JAR file reading in the initial class loader
(which couldn't use the "normal" java.util.zip stuff during bootstrapping).

Not so quick!

...I recalled something Zoopy mentioned
(couldn't be bothered Googling, so I'll quote)

"In 1.5/5.0 they additionally speed up class-loading by something called
class data sharing
(<http://java.sun.com/j2se/1.5.0/docs/guide/vm/class-data-sharing.html>).
In the jre1.5*/bin/client directory you'll find a "classes.jsa" file (Java
Shared Archive) which
apparently contains an internal representation of the core classes."

And guess which two packages are extensively mentioned
in the classes.jsa file? (I just dumped the entire
12 Meg into trusty little TextPad and did a 'find'..)

java.util.zip and java.util.jar are mentioned
extensively, whereas java.beans.XMLEncoder*
is not, nor is any of the CORBA* stuff (*just
to check Sun did not also store a representation
of the ZipEntries of the main rt.jar in it,
and since I couln't be bothered trying to decipher
Sun specs at this time in my temporal day..).

So.. your other reason was...

/Of course/ Sun could be that silly.

....hmmmm. Yes they *could* be that silly,
but it is a ..flimsy premise without further
information to support it.

[ So there!
Oh and ..loving these *colors*,
wish you were here! ;-) ]

 

[zoopy]

On Thu, 19 Aug 2004 14:53:01 GMT, Andrew Thompson wrote:
Form Size
---- ----
Disk (extracted): 29,873,493
Zip Uncompressed: 31,971,302
Zip Each entry: 15,279,773
Zip Entire file: 9,693,680

Here are some figures for the various compression utilities/techniques applied to rt.jar (C:\Program
Files\Java\jre1.5.0\lib\rt.jar).

I've used Cygwin's zip, gzip and bzip2 using their option for the best compression, and compared
with pack200 of J2SE 5.0. Pack200 by default compresses output using gzip, the option -g prevents this.

command file size (bytes) relative
------- ---- ------------ --------
rt.jar 33,348,378 100 %
zip -9 rt.jar.zip 10,063,673 30.18 %
gzip --best rt.jar.gz 10,063,554 30.18 %
pack200 -g rt.jar.pack 9,875,459 29.61 %
bzip2 --best rt.jar.bz2 8,234,905 24.69 %
pack200 rt.jar.pack.gz 4,397,016 13.19 %


pack200 on its own (no gzip) does slightly better than zip and gzip. This was also the case when I
tried it on rt.jar of jre1.4.2 (also similar relative size figures).


I've also compressed an *uncompressed* zip file containing text files (source of Unix file(1) utility):

command file size (bytes) relative
------- ---- ------------ --------
file-4.10.zip 1,479,766 100 %
pack200 -g file-4.10.zip.pack 1,443,237 97.53 %
zip -9 file-4.10.zip.zip 370,410 25.03 %
gzip --best file-4.10.zip.gz 370,284 25.02 %
pack200 file-4.10.zip.pack.gz 358,737 24.24 %
bzip2 --best file-4.10.zip.bz2 317,013 21.42 %

Pack200 -g gives a poor result on this zip file (with text files), but it seems to do quite a good
job on compressing class files.

 

[Andrew Thompson]

I've used Cygwin's zip, gzip and bzip2 using their option for the best compression, and compared
with pack200 of J2SE 5.0. Pack200 by default compresses output using gzip, the option -g prevents this.

command file size (bytes) relative
------- ---- ------------ --------

....

graphed as..

I've also compressed an *uncompressed* zip file containing text files (source of Unix file(1) utility):

command file size (bytes) relative

...and,
<http://www.physci.org/test/graph/in...+bzip2-best+&val=148+144+37+37+36+32&width=50>

Now that's better.. I find a visual
aid very helpful, don't you? ;-)

 

[zoopy]

<http://www.physci.org/test/graph/in...+bzip2-best+&val=148+144+37+37+36+32&width=50>

Andrew, there's something wrong with the images. For instance, I can't see
http://www.physci.org/test/graph/white.png
in my browser.

Maybe your protection against hotlinking?

 

[zoopy]

Andrew, there's something wrong with the images. For instance, I can't see
http://www.physci.org/test/graph/white.png
in my browser.

Maybe your protection against hotlinking?

Please disregard.

I probably is your check for 'referer', but in my browser (Mozilla) I had set an option for 'not
sending referer' [exactly for the same reason for sites that check referer; you probably check 'if
referer notEquals "www.physci.org" then don't serve image' ]

Reset the option, now it's OK.

 

[Andrew Thompson]

Andrew, there's something wrong with the images. For instance, I can't see
http://www.physci.org/test/graph/white.png
in my browser.

....errrm. Can you see it within the graph in the page?

Maybe your protection against hotlinking?

Well.. probably. I noticed my bandwidth had
quadrupled over a three month period while page
hits were actually trailing off. Actually,
the graph says it better..
<http://www.athompson.info/andrew/hotlink.jsp>

I had to take some radical measures that I
am still untwining the curlier file types
from. But I can see no compelling reason
to remove the hotlink protection from the
images if they display correctly in the
pages though.

So I suppose I disagree with your assessment
of 'something wrong', if I understand correctly.

OTOH, it is just a 1x1 white image, you
want me to post it to you? Heck, ..I'll
even throw in the other colors for free! ;-)

 

[Andrew Thompson]

I probably is your check for 'referer', but in my browser
(Mozilla) I had set an option for 'not sending referer'

Aaah.. I had presumed you had made a 'direct
request' for the image, though I could ..not
for the life of me, figure *why*..

This 'not send referrer' is bad news.., yet
another pitfall we web-designers have to
face in trying to deliver content to our
visitors. (sigh)

Reset the option, now it's OK.

Thanks for reporting back. ..

Now I have to figure what to do for others
that might have that setting, as it would affect
a lot of things around the site, and I simply
cannot afford to lift the hotlink protection.

 

[zoopy]

Thanks for reporting back. ..

You're welcome; after all, it was a problem on my side.

Now I have to figure what to do for others
that might have that setting,

It is quite a specific setting, and I don't think a lot of people are aware of it or even have it
enabled. I don't know which browsers other than Mozilla have this option for sending/not sending the
referer. In Mozilla (including Firefox and the ones based on same code, e.g. Netscape, Safari) it's
more or less a hidden option: it's not available in Mozilla's configuration window, but you can
change it by editing the config preferences. A laymen user of Mozilla probably will have no trouble
with your site.

Maybe you could place a footnote on the page (or a bigger message on your homepage) stating that the
browser must send a HTTP referer header or otherwise the page doesn't display properly.

as it would affect
a lot of things around the site,

It sure does: after I changed the option, your site was much more colourful ;-)

and I simply
cannot afford to lift the hotlink protection.

I can understand that.

 

[Andrew Thompson]

A laymen user of Mozilla probably will have no trouble
with your site.
Whew!

Maybe you could place a footnote on the page (or a bigger message on your homepage) stating that the
browser must send a HTTP referer header or otherwise the page doesn't display properly.

(shakes head sadly)
If your browser supports Java these pages offer extra funcionality.

If your broswer was not so broken I could float elements, and
would not have to hide everything invented this millinium from it.

If your monitor were not so damn small and stingy I
could bathe this screen in swathes of glorious color.
.......

At what point does the user shrug and say 'stuff
this site, I'm off somewhere nice.'

It is very difficult to inform a user of their
defective browser/OS/Java usage or lack-thereof..
without offending them. ..Or maybe it's just the
way I say it. ;-)