Java and SHA - with big files...

[=?iso-8859-1?B?QmVub+50?=]

Hi
I want to compare two big files (like films), so I create a fingerprint
and I look...
It works very well with little files, but a java.lang.outOfMemory
appears with big files...
I think the array data in readFile(String filename) function is too
big...so how do I do?

try {
byte[] data = readFile(myfile.getAbsolutePath());
MessageDigest md = MessageDigest.getInstance("SHA");
md.update(data);
byte[] digest = md.digest();
System.out.println("Hexadecimal: " + myfile.getAbsolutePath()+new
BigInteger(digest).toString(16));
}catch (Exception e) {
System.out.println(e);
}

....

private static byte[] readFile(String filename) {
try {
RandomAccessFile raf = new RandomAccessFile(filename, "r");
byte[] data = new byte[(int)raf.length()];
raf.readFully(data);
raf.close();
System.out.println("He");
return data;
}
catch (FileNotFoundException e) {}
catch (IOException e) {}
return null;
}


Thanks

 

[Stefan Schulz]

Hi
I want to compare two big files (like films), so I create a fingerprint
and I look...
It works very well with little files, but a java.lang.outOfMemory
appears with big files...
I think the array data in readFile(String filename) function is too
big...so how do I do?

Take a look at MessageDigest.update() and friends. These methods are
explicitly meant to deal with chunks of data, instead of the whole file.
You can just fill up a fixed-size array until the file has been traversed,
adding one "block" at a time to the digest.

 

[Roedy Green]

Take a look at MessageDigest.update() and friends. These methods are
explicitly meant to deal with chunks of data, instead of the whole file.
You can just fill up a fixed-size array until the file has been traversed,
adding one "block" at a time to the digest.

see http://mindprod.com/jgloss/digest.html

for what you are doing an size/date/adlerian checksum might suffice.
It is much faster to compute.

See http://mindprod.com/jgloss/adler.html