Parallel::UserAgent with Redirect / How to get the original URL??

[fritz-bayer]

Hi,

I'm using the Parrallel::UserAgent to send several HTTP requests at
once. After that I circle through the result and process the content.

I have redirection turned on, so the agent follows redirects. The
problem I have is retrieving the original urls of the requests.

The method:

my $redirectedUrl = $response->request->url;

will return the redirected url instead of the original one.

How can I retrieve the orginal url of the request, before it was
redirected. Of course I registered the url before, but I do not know,
which url caused the redirect.

Anybody who knows, where the original url can be found? Or is it
discarded?

Fritz

 

[xhoster]

Hi,

I'm using the Parrallel::UserAgent to send several HTTP requests at
once. After that I circle through the result and process the content.

I have redirection turned on, so the agent follows redirects. The
problem I have is retrieving the original urls of the requests.

The method:

my $redirectedUrl = $response->request->url;

will return the redirected url instead of the original one.

It would help if you showed us how you get $response. I can only guess
you got it from the hash returned by wait?

How can I retrieve the orginal url of the request, before it was
redirected. Of course I registered the url before, but I do not know,
which url caused the redirect.

Presumably you know the stringified value of the reference of the request
object which caused the redirect (or where else did response come from?).
So you just need to store the URL of keyed to the same stringified
reference.

Of you could navigate the LWP:arallel::UserAgent::Entry object:

$response->response->previous->request->url();

Xho

 

[John Bokma]

Hi,

I'm using the Parrallel::UserAgent to send several HTTP requests at
once. After that I circle through the result and process the content.

I have redirection turned on, so the agent follows redirects. The
problem I have is retrieving the original urls of the requests.

wait returns a hash reference, the keys are the stringified requests (a
request is a reference) you registered. So when you register a request,
add the stringified request (reference) to a hash table, and use the url
as a value:

my %look_up;
my $request = ....

$pua->register( $request )

$look_up{ $request } = $url;

my $entries = $pua->wait;
for my $key ( keys %$entries ) {

:
print "The original URL is: $look_up{ $key }\n";
:
}

(untested & from top of my head)

Yeah, it's a bit clumsy IMO. No idea if there is an easier way.

 

[fritz-bayer]

wait returns a hash reference, the keys are the stringified requests (a
request is a reference) you registered. So when you register a request,
add the stringified request (reference) to a hash table, and use the url
as a value:

my %look_up;
my $request = ....

$pua->register( $request )

$look_up{ $request } = $url;

my $entries = $pua->wait;
for my $key ( keys %$entries ) {

:
print "The original URL is: $look_up{ $key }\n";
:
}

(untested & from top of my head)

Yeah, it's a bit clumsy IMO. No idea if there is an easier way.

Thanks a lot guys! And yes I'm retrieving them from the hash, which
wait returns...