R
Rich
I created an asp.net code to scrape data off a website. It calls many
pages on a website.
Coded like this:
for(int i=0;i<2000;i++)
{
HttpWebRequest myHttpWebRequest =
(HttpWebRequest)WebRequest.Create(websiteUrl+pageUrl);
HttpWebResponse myHttpWebResponse =
(HttpWebResponse)myHttpWebRequest.GetResponse();
using (StreamReader sr = new
StreamReader(myHttpWebResponse.GetResponseStream()))
{
strResponse = sr.ReadToEnd();
}
myHttpWebResponse.Close();
}
I encountered a Request timed out error:
System.Web.HttpException: Request timed out.
Exception message: Thread was being aborted.
Method that threw the exception: System.Net.Sockets.SocketError
setsockopt(System.Net.SafeCloseSocket,
System.Net.Sockets.SocketOptionLevel,
System.Net.Sockets.SocketOptionName, Int32 ByRef, Int32)
The timer that used the executionTimeOut value specified in web.config
is not reset after each loop. I cannot set a executionTimeOut value
because I do not have knowledge of how many calls to be made to the
website.
Does anyone know how to solve this?
Possibly force a timer reset or close socket or find the thread that
used the socket and kill it? How to do it?
Thanks in advance,
Rich
pages on a website.
Coded like this:
for(int i=0;i<2000;i++)
{
HttpWebRequest myHttpWebRequest =
(HttpWebRequest)WebRequest.Create(websiteUrl+pageUrl);
HttpWebResponse myHttpWebResponse =
(HttpWebResponse)myHttpWebRequest.GetResponse();
using (StreamReader sr = new
StreamReader(myHttpWebResponse.GetResponseStream()))
{
strResponse = sr.ReadToEnd();
}
myHttpWebResponse.Close();
}
I encountered a Request timed out error:
System.Web.HttpException: Request timed out.
Exception message: Thread was being aborted.
Method that threw the exception: System.Net.Sockets.SocketError
setsockopt(System.Net.SafeCloseSocket,
System.Net.Sockets.SocketOptionLevel,
System.Net.Sockets.SocketOptionName, Int32 ByRef, Int32)
The timer that used the executionTimeOut value specified in web.config
is not reset after each loop. I cannot set a executionTimeOut value
because I do not have knowledge of how many calls to be made to the
website.
Does anyone know how to solve this?
Possibly force a timer reset or close socket or find the thread that
used the socket and kill it? How to do it?
Thanks in advance,
Rich