LWP::Simple getstore with absolute path not working

[emrefan]

I am using the following line of sort in a batch file to fetch things
off the network quickly and mostly I have success but if I need to
indicate an absolute path in the URL, I always get a "not found"
error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
system
where did the test. Help!

perl -MLWP::Simple -e "getstore( '%1', '%2' );"

Oh, the exact URL that I tested it was something like this:
ftp://user[email protected]//dir-under-the-root/some-dir/some-file

I thought the "//" before "dir-under-the-root" should have it working
for me, but no.

 

[Ben Morrow]

I am using the following line of sort in a batch file to fetch things
off the network quickly and mostly I have success but if I need to
indicate an absolute path in the URL, I always get a "not found"
error. I have perl 5.8.x, URI.pm 1.36 and URI::URL.pm 5.03 on the XP
system
where did the test. Help!

perl -MLWP::Simple -e "getstore( '%1', '%2' );"

Oh, the exact URL that I tested it was something like this:

ftp://user[email protected]//dir-under-the-root/some-dir/some-file

I thought the "//" before "dir-under-the-root" should have it working
for me, but no.

FTP urls don't work like that. The standard hack is

ftp://host/%2fdir-under-the-root/some-dir/some-file

where %2f is a url-encoded '/'; but strictly speaking this requires your
FTP server to support a chdir to '/dir-under-the-root' in one go, which
is not required. Most do, however.

Ben

 

[szr]

FTP urls don't work like that. The standard hack is

Actually the url he gave is mostly correct (should be just one "/" after
the host par:

ftp://user[email protected]/dir-under-the-root/some-dir/some-file

This should work just fine.

You may want to test it in a browser and then in your script add checks
to catch cases where adir or file doesn't exist.

ftp://host/%2fdir-under-the-root/some-dir/some-file

where %2f is a url-encoded '/'; but strictly speaking this requires
your FTP server to support a chdir to '/dir-under-the-root' in one
go, which is not required. Most do, however.

Why would it need to do that? Give that url, it should be going to
"/fdir-under-the-root/some-dir/some-file" in one go.

 

[emrefan]

Actually the url he gave is mostly correct (should be just one "/" after
the host par:

ftp://user[email protected]/dir-under-the-root/some-dir/some-file

This should work just fine.

You may want to test it in a browser and then in your script add checks
to catch cases where  adir or file doesn't exist.



Why would it need to do that? Give that url, it should be going to
"/fdir-under-the-root/some-dir/some-file" in one go.

When a user logs in via ftp (including when the user is "anonymous"),
he is placed in wherever is considered his home directory and if this
home directory does not happen to be "/", then the URL with just a
slash after the hostname is not going to work as intended. I tested
that, had problems and so queried. With an "%2f" after the hostname,
it worked. I reckond I'd call that a bug in URI.pm & co. but at least
I now have a workaround.

 

[emrefan]

FTP urls don't work like that. The standard hack is

   ftp://host/%2fdir-under-the-root/some-dir/some-file

where %2f is a url-encoded '/'; but strictly speaking this requires your
FTP server to support a chdir to '/dir-under-the-root' in one go, which
is not required. Most do, however.

Thanks! It worked. I think I bumped into a bug in URI.pm & co, but at
least I now have a workaround.

 

[szr]

When a user logs in via ftp (including when the user is "anonymous"),
he is placed in wherever is considered his home directory

Actually this is server/config (and client/config) dependant. For
example on my personal system, if I login in as a run of the mill user,
it drops me into my home (actually it chroots unless otherwise specified
for that user in the ftpd config.)

and if this home directory does not happen to be "/", then the URL
with just a slash after the hostname is not going to work as intended.

That's because it expects an absolute path. Depending on the ftp
server's configuration, the ftp root that your login "sees" could be in
fact the home dir itself (which is the case when some form of chroot'ing
is used), or the true root of the system, in which case you referernce
your home dir be something like /home/user, depending on the the
platform/OS.

If you want to access directory 'foo' that resides in your home dir, and
your ftp's root / is chroot'ed to your home dir, they this should work:

ftp://server/foo

else, if you are not chrooted (where the ftp's root is the real root of
the system), then

ftp://server/home/johnsmith/foo

is is what you want.

Again, it depends on what / (root) for your ftp login is, which is
dependant on the ftp server software and the platform it runs on.

I tested that, had problems and so queried. With an "%2f" after
the hostname, it worked. I reckond I'd call that a bug in
URI.pm & co. but at least I now have a workaround.

When you login manually via ftp with the same account, and type "cd /"
and they "pwd" or "ls", where do you see your self, in your home
directory, or in the server's real root directory?

Also, do you know what platform your ftp server is running on?

 

[Peter J. Holzer]

It should support a chdir to / (if it has a directory called "/"), so
you should be able to write
ftp://host/%2f/dir-under-the-root/some-dir/some-file
then.

When a user logs in via ftp (including when the user is "anonymous"),
he is placed in wherever is considered his home directory and if this
home directory does not happen to be "/", then the URL with just a
slash after the hostname is not going to work as intended. I tested
that, had problems and so queried. With an "%2f" after the hostname,
it worked. I reckond I'd call that a bug in URI.pm & co. but at least
I now have a workaround.

It's not a bug. That's how FTP URIs are supposed to work. To quote from
RFC 1738:

| The url-path of a FTP URL has the following syntax:
|
| <cwd1>/<cwd2>/.../<cwdN>/<name>;type=<typecode>
|
| Where <cwd1> through <cwdN> and <name> are (possibly encoded) strings
| and <typecode> is one of the characters "a", "i", or "d".
[...]
| The url-path is interpreted as a series of FTP commands as follows:
|
| Each of the <cwd> elements is to be supplied, sequentially, as the
| argument to a CWD (change working directory) command.
|
| If the typecode is "d", perform a NLST (name list) command with
| <name> as the argument, and interpret the results as a file
| directory listing.
|
| Otherwise, perform a TYPE command with <typecode> as the argument,
| and then access the file whose name is <name> (for example, using
| the RETR command.)

So ftp://host/%2fdir-under-the-root/some-dir/some-file means:

Connect to host and log in as anonymous.
cwd /dir-under-the-root
cwd some-dir
retr some-file

Note that /dir-under-the-root is only a directory under the root on unix
and window systems. Other OSs may have other conventions (once upon a
time, FTP servers running VMS or IBM host OSs were common), and the host
OS may not even have the concept of a "root directory".

hp