Copy large files from server to server

[Chris]

What's the best way to copy files from a central server to remote servers?

Our app runs on multiple app servers, so I can control code running on
both ends. I've got no guarantee, however, that the central server will
be able to map or mount remote drives, so doing a simple file copy is
not an option.

My first thought was to package up the file into a POST request and send
it that way. A servlet on the receiving end could handle it correctly.
The trouble is that sometimes app servers have a maximum post size, and
my files could be quite large. Another way would be to send a URL, and
have the remote server pull it from the central server. The trouble with
that approach is error handling; how will I know if there was an error
and the process needs to be restarted?

Is there a guaranteed way to do a file transfer?

 

[Jeroen Wenting]

What's the best way to copy files from a central server to remote servers?

rcp or scp, ftp if you must.

Is there a guaranteed way to do a file transfer?

there are no guarantees in a networked environment. The other machine could
crash halfway through, yours could crash, the network could go down, latency
could cause major packet loss, EMP could cause corruption, someone could cut
a cable or trip over one (had that happen once at a customer, maintenance
tech tripped over the powercord to a mainframe, unplugging the connection
between it and its UPS with predictable results...), construction crews
could cut the cables digging up the street (happens more often than you'd
think), solar flares could take out the comsats you use for your uplink, and
any of a million other causes could cause the transfer to be interrupted.
It's in fact a minor miracle it usually works out just fine.

 

[RedGrittyBrick]

What's the best way to copy files from a central server to remote servers?

In my view, the "best" Protocol for Transferring Files is likely to be
the File Transfer Protocol (FTP).

FTP supports the resumption of interrupted transfers.

There must be many FTP client libraries for Java ... Google finds
http://www.javaworld.com/javaworld/jw-04-2003/jw-0404-ftp.html which is
an elderly review of some of them.

Otherwise I'd consider programs like rsync, but that's off-topic.