Creating lots of folders on ftp server

[Raimon Fs]

Hello,


I have to check some folders on a ftp server, if they don't exist,
create it.


The main folder is:

project

the project has some folders, like 1,2,3,4,5,...

and each folder have 12 more folders, one for each month, and this
folders have many folders as days the current month have.


My code works, but I think is pretty slow and always, I'm getting an
EOFError: end of file reached after creating 50 folders, always.

How can I trace to find the error ?

thanks,

r.


date=Date.new(2008,1,1)
res.each{ |tuple|

row_current+=1

print "\n\n Row "+row_current.to_s+"/"+rows_to_fetch.to_s+" => "

print tuple['id'], ' ', tuple['name']

project=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
date.month+"/"+'%02d' % date.day
project_month=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d'
% date.month
project_year=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s
project_id=ftp_acct+"/"+tuple['id']


begin # project_id
ftp.chdir(project_id)
rescue Net::FTPPermError, NameError => boom
ftp.mkdir("/"+project_id)

print "Folder #{project_id} created"
end


begin # project_year
ftp.chdir(project_year)
rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
create it
ftp.mkdir("/"+project_year)
print "Folder #{project_year} created"
end


for i in 1..12

project_month=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
i

begin # project_month
ftp.chdir(project_month)
rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
create it
ftp.mkdir("/"+project_month)
print "Folder #{project_month} created"
end


if i == 12
d=Date.new(date.year+1,1,1)-1
else
d=Date.new(date.year.to_i,i+1,1)-1
end



puts "\n #{d.to_s}"


for days in 1..d.day

project_day=ftp_acct+"/"+tuple['id']+"/"+date.year.to_s+"/"+'%02d' %
i+"/"+'%02d' % days

begin # project_day
ftp.chdir(project_day)
rescue Net::FTPPermError, NameError => boom # it doesn't exist =>
create it
ftp.mkdir("/"+project_day)
print "Folder #{project_day} created"
end

print " \nFolder #{project_day} exists "


end # for days



end # for i in 1..12


} # res.each{ |tuple|

 

[Chris Mowforth]

Paste the error you get. You should see line numbers indicating the
method call(s) that bailed on you.

Incidentally I think you could do this more simply using some recursion
and a couple more ruby idioms.

m0wf

 

[Raimon Fs]

Paste the error you get. You should see line numbers indicating the
method call(s) that bailed on you.

Incidentally I think you could do this more simply using some recursion
and a couple more ruby idioms.

m0wf

The error I get is from the ftp library: reach end of line

As it starts creating folders, is getting slower and slower and finally,
at 50, end of reach line

and yes, I think I can do it more 'ruby like', but I'm coming from a
really bored language ...



thanks,

r.

 

[Hassan Schroeder]

I have to check some folders on a ftp server, if they don't exist,
create it.

This may not fit your use case, but it might be easier and faster to
create the directory structure locally and rsync it to the server.

Just a thought.

 

[Raimon Fs]

Paste the error you get. You should see line numbers indicating the
method call(s) that bailed on you.

EOFError: end of file reached

method readline in ftp.rb at line 211
method getline in ftp.rb at line 211
method getmultiline in ftp.rb at line 221
method getresp in ftp.rb at line 235
method voidresp in ftp.rb at line 251
method voidcmd in ftp.rb at line 274
method synchronize in monitor.rb at line 242
method voidcmd in ftp.rb at line 272
method chdir in ftp.rb at line 674
at top level in createFoldersOnFTP.rb at line 161
method each in createFoldersOnFTP.rb at line 156
at top level in createFoldersOnFTP.rb at line 156
method each in createFoldersOnFTP.rb at line 133
at top level in createFoldersOnFTP.rb at line 133
method each in createFoldersOnFTP.rb at line 96
at top level in createFoldersOnFTP.rb at line 96


maybe could be a memory leak, as it goes slower, slower, slower, and
after 50 folders created, the error appears...

Incidentally I think you could do this more simply using some recursion
and a couple more ruby idioms.

can you give me some hints on this, please ?


thanks,


regards,

r.

 

[Raimon Fs]

I have more information ....

Just create an array and add elements with the folders names you want to
create.


ftp_folder_create.each{ |folder|

print "\n Creating folder ",folder

ftp.mkdir(folder)

end


}

All works great, if they folders doesn't exist and you can create them.


Add error control and check before creating the folder, if it really
exists, and you'll get the error after creating 50 folders.


ftp_folder_create.each{ |folder|

print "\n Creating folder ",folder

begin

ftp.chdir(folder)

rescue Net::FTPPermError, NameError => boom # it doesn't exist => create
it

ftp.mkdir(folder)

end


}


Anyone can confirm this ?

thanks,


r.

 

[Chris Mowforth]

Sounds reasonable.

If you HAVE to use ftp, personally I'd start creating a nested array
representing the folder structure, then start trying to get Net::FTP to
iterate through it.

Alternatively if you could store the absolute paths as array elements, I
think it'd be much more efficient than calling #chdir all the time. Of
course this conflates the folder structure with the ruby array.

And try puts rather than print, might save you some typing.