Problem with FTPLib and incomplete files on some downloads

[Peter A. Schott]

I download a lot of 4-6 KB files and regularly run into issues with files that
don't get downloaded all the way or otherwise get corrupt.

I do something like:

RemoteList = nlstdir()
for filename in RemoteList:
LocalFile = open(filename, "wb")
LocalFile.write( "get file code here" )
LocalFile.close()
#ftplib call to delete the remote file

I've tried to insert a pause into the code between the close and the remote
delete, but that doesn't seem to help. For some reason it doesn't seem like the
buffer is completely written or read before the remote delete is called. That
leads to a file that's not 100% complete and thus can't be decrypted.

Any ideas on what to look for? I don't have my exact code handy at the moment
or I'd post some actual snippets. It's also not consistent or it would be a
whole lot easier to troubleshoot.

Running Python 2.4.2 on Win32.

Thanks.

-Pete Schott

 

[Martin Franklin]

I download a lot of 4-6 KB files and regularly run into issues with files that
don't get downloaded all the way or otherwise get corrupt.

I do something like:

RemoteList = nlstdir()
for filename in RemoteList:
LocalFile = open(filename, "wb")
LocalFile.write( "get file code here" )
LocalFile.close()
#ftplib call to delete the remote file

I've tried to insert a pause into the code between the close and the remote
delete, but that doesn't seem to help. For some reason it doesn't seem like the
buffer is completely written or read before the remote delete is called. That
leads to a file that's not 100% complete and thus can't be decrypted.

Any ideas on what to look for? I don't have my exact code handy at the moment
or I'd post some actual snippets. It's also not consistent or it would be a
whole lot easier to troubleshoot.

Running Python 2.4.2 on Win32.

Thanks.

-Pete Schott

Pete,


Not sure but here is an example that seems to always work for me:
(tested this morning before pasting here on ~20 files around 7 - 300 k)

import ftplib
import os

os.chdir("D:/TEMP/dump")

ftp = ftplib.FTP("server")
print ftp.login("anonymous", "(e-mail address removed)")

ftp.cwd("pub/Python_Applications")




for rFile in ftp.nlst():
print "Getting : ", rFile
rSize = ftp.size(rFile)
lFile = open(rFile, "wb")
ftp.retrbinary("RETR %s" %rFile, lFile.write)
lSize = lFile.tell()
lFile.close()
if rSize==lSize:
print "Transfer complete"
else:
print "BAD Transfer", rSize, lSize


ftp.close()

HTH
Martin