L
Leif Wessman
Hi!
I want to use LWP:
arallel::UserAgent to handle each request when it
has returned.
I've tried the wait() method. The problem with the wait() method is
that I can parse each response first when all the other requests are
finished.
My second try was to use a callback function. Since I want to handle
the whole response data at the same time, I've set the chunk size to be
very large (100 MB). I was thinking that this would make each chunk to
be the whole page. However, that didn't work. Each cunk becomes about
1460 bytes anyway.
My third try was to use a callback function and put the chunks for a
certain request in a buffer. So that when the whole page is loaded, I
can parse it. However - I can't figure out how to know when the page is
fully loaded. As staded in the docs, the length($content) == 0 doesn't
mean the connection is closed.
Is the some other way to handle each request when it has returned?
Leif
I want to use LWP:
arallel::UserAgent to handle each request when ithas returned.
I've tried the wait() method. The problem with the wait() method is
that I can parse each response first when all the other requests are
finished.
My second try was to use a callback function. Since I want to handle
the whole response data at the same time, I've set the chunk size to be
very large (100 MB). I was thinking that this would make each chunk to
be the whole page. However, that didn't work. Each cunk becomes about
1460 bytes anyway.
My third try was to use a callback function and put the chunks for a
certain request in a buffer. So that when the whole page is loaded, I
can parse it. However - I can't figure out how to know when the page is
fully loaded. As staded in the docs, the length($content) == 0 doesn't
mean the connection is closed.
Is the some other way to handle each request when it has returned?
Leif