Jakarta Commons FileUpload and only *Partially* uploaded files

[Dundonald]

I'm hoping some in here will have had experience with the Jakarta
commons
file upload utility and inparticular writing an uploaded file to a BLOB
field.

I configure file upload as follows:

DiskFileUpload upload = new DiskFileUpload();
upload.setSizeMax(-1);
upload.setSizeThreshold(4096);
upload.setRepositoryPath("c:/temp");

Any file greater than 4096 bytes will be temporarily stored in to
c:/temp
whilst the FileItem object from an uploaded file exists. This can be
verified by using the isInMemory() method on FileItem. It returns
false if
a temporary file is stored.

I do see files created in c:/temp and the temporary file is the full
and
correct file size. So let's say for example I upload an image at 400k
I do
see a .tmp file in c:/temp at 400k and if I rename the .tmp file to
..jpg
and open it up it is displayed correctly.

However, when I retrieve the file that was written to a Blob field in
MySQL, and this can be any file, it is always a maximum of 65,535 bytes
(or
there abouts). When I open this retrieved file, using the image as an
example, only a portion of the jpg is displayed (65,535 bytes worth).

I was wondering if there was anything significant about this and more
importantly how to overcome this problem.

Any file I upload greater than 65,535 bytes has a problem. Any file
less
than this in size works fine.

I've isolated the problem to the write to database rather than the
retrieve
because I checked the database size before and after a test uploading a
400k file. The database grew by only 65,535 (ish) bytes.

I can post the code I wrote to write a FileItem to a MySQL BLOB field
if it
helps but I first wanted to test the water to see if this is a known
problem. I did google but there was no history of this problem in the
archive. In fact I'll post it anyway at the foot of this post.

Fingers crossed.

*** CODE TO WRITE A FileItem TO BLOB FIELD ****
* Note for background a record has already been created and this code
* uses the inserted record ID to update the Blob field of that record.
* A FileItem object called file is passed in to the method.
***

String sql = "";

try
{
/*
* Updated attached document record with file into
blob
field.
*/
sql = "update "+<table name>+" "+
"set "+<blob field name>+"=? "+
"where "+<record id field
name>+"='"+lastInsertedID+"'";

Connection connection = getDatabaseConnection();

PreparedStatement preparedStatement =
connection.prepareStatement(sql);

/* paramater 1, because there is only 1 ? above. */
preparedStatement.setBinaryStream(1,
file.getInputStream(), (int)file.getSize());

preparedStatement.execute();
preparedStatement.close();
}
catch (SQLException e)
{
DatabaseQueryException dbqe = new
DatabaseQueryException(CLASS_NAME, "insertAttachedDocument", sql + " "
,
e.getMessage());
throw dbqe;
}

 

[Marcin Grunwald]

I'm hoping some in here will have had experience with the Jakarta
commons
file upload utility and inparticular writing an uploaded file to a BLOB
field.

I configure file upload as follows:

DiskFileUpload upload = new DiskFileUpload();
upload.setSizeMax(-1);
upload.setSizeThreshold(4096);
upload.setRepositoryPath("c:/temp");

Any file greater than 4096 bytes will be temporarily stored in to
c:/temp
whilst the FileItem object from an uploaded file exists. This can be
verified by using the isInMemory() method on FileItem. It returns
false if
a temporary file is stored.

I do see files created in c:/temp and the temporary file is the full
and
correct file size. So let's say for example I upload an image at 400k
I do
see a .tmp file in c:/temp at 400k and if I rename the .tmp file to
.jpg
and open it up it is displayed correctly.

However, when I retrieve the file that was written to a Blob field in
MySQL, and this can be any file, it is always a maximum of 65,535 bytes
(or
there abouts). When I open this retrieved file, using the image as an
example, only a portion of the jpg is displayed (65,535 bytes worth).

I was wondering if there was anything significant about this and more
importantly how to overcome this problem.

Any file I upload greater than 65,535 bytes has a problem. Any file
less
than this in size works fine.

I've isolated the problem to the write to database rather than the
retrieve
because I checked the database size before and after a test uploading a
400k file. The database grew by only 65,535 (ish) bytes.

Maximum data size for BLOB and TEXT is (2^16) 65535 bytes.
Use MEDIUMBLOB (2^24 bytes) or LONGBLOB (2^32 bytes).

And also important (from MySQL documentation):
The maximum size of a BLOB or TEXT object is determined by its type, but the
largest value you actually can transmit between the client and server is
determined by the amount of available memory and the size of the
communications buffers. You can change the message buffer size by changing
the value of the max_allowed_packet variable, but you must do so for both
the server and your client program.

 

[Dundonald]

Maximum data size for BLOB and TEXT is (2^16) 65535 bytes.
Use MEDIUMBLOB (2^24 bytes) or LONGBLOB (2^32 bytes).

And also important (from MySQL documentation):
The maximum size of a BLOB or TEXT object is determined by its type, but the
largest value you actually can transmit between the client and server is
determined by the amount of available memory and the size of the
communications buffers. You can change the message buffer size by changing
the value of the max_allowed_packet variable, but you must do so for both
the server and your client program.

What a school boy oversight. Doh! Thanks for the pointer, all working
now.