HttpWebRequest.GetResponse() throwing Timeout

[Jeff B]

I'm trying to create a simple screen scraping application and I kept
getting a System.Net.WebException thrown back with a message of "The
operation has timed-out." At first I thought it was some kind of
connectivity issue on the machine. It didn't make sense though, because I
can open up a browser on the same machine and easily browse
the web.

I'm stumped. I looked over my code for any errors and just couldn't find
what I was doing wrong. So, I went out and
found a code sample from www.gotdotnet.com and tried that on my machine.
When I run the sample code, I get the same error. WebExcpetion is thrown and
the message is "The operation has timed-out."

Any ideas? The code I wrote is below. It's very simple and is just supposed
to go to my local home page (http://localhost/MyWebApplication), scrape the
contents, load it up into a string, and return a page telling me
the length of the string and the contents scraped from my homepage.

Here is my code:


Private Function Page_Load(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles MyBase.Load

Try

If Not Page.IsPostBack Then

Dim strResponse As String
Dim strURL As String = "http://localhost/MyWebApplication/"

Dim objRequest As HttpWebRequest
objRequest = CType(WebRequest.Create(strURL), HttpWebRequest)

'if this line is uncommented then this app will hang
indefinately
'objRequest.Timeout = -1

Dim objResponse As HttpWebResponse
objResponse = CType(objRequest.GetResponse(), HttpWebResponse)

Dim sr As StreamReader
sr = New StreamReader(objResponse.GetResponseStream())
strResponse = sr.ReadToEnd()

sr.Close()
objResponse.Close()
objRequest.Abort()

Response.Write("The scraped contents length is:" &
strResponse.Length.ToString() & "<br><br>")
Response.Write(strResponse)

End If

Catch ex As Exception
'error accessing website
Response.Write(ex.Message)

End Try

End Function

No matter what code I run (this code, or the www.gotdotnet.com sample), the
problem always occurs in
System.Net.Http.WebRequest.GetResponse().

Here is the exact exception message from the event log.

Event Type: Error
Event Source: ExceptionManagerPublishedException
Event Category: None
Event ID: 0
Date: 12/19/2003
Time: 3:02:07 PM
User: N/A
Computer: THEBES
Description:

General Information
*********************************************
Additional Info:
ExceptionManager.MachineName: THEBES
ExceptionManager.TimeStamp: 12/19/2003 3:02:07 PM
ExceptionManager.FullName: Microsoft.ApplicationBlocks.ExceptionManagement,
Version=1.0.1448.24376, Culture=neutral, PublicKeyToken=null
ExceptionManager.AppDomainName:
/LM/W3SVC/1/Root/MyWebApplication-3-127163624072300000
ExceptionManager.ThreadIdentity:
ExceptionManager.WindowsIdentity: THEBES\ASPNET

1) Exception Information
*********************************************
Exception Type: System.Exception
Message: An unhandled exception was passed to the application object.
TargetSite: NULL
HelpLink: NULL
Source: NULL

2) Exception Information
*********************************************
Exception Type: System.Web.HttpUnhandledException
ErrorCode: -2147467259
Message: Exception of type System.Web.HttpUnhandledException was thrown.
TargetSite: Boolean HandleError(System.Exception)
HelpLink: NULL
Source: System.Web

StackTrace Information
*********************************************
at System.Web.UI.Page.HandleError(Exception e)
at System.Web.UI.Page.ProcessRequestMain()
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at
System.Web.CallHandlerExecutionStep.System.Web.HttpApplication+IExecutionSte
p.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean&
completedSynchronously)

3) Exception Information
*********************************************
Exception Type: System.Net.WebException
Status: Timeout
Response: NULL
Message: The operation has timed-out.
TargetSite: System.Net.WebResponse GetResponse()
HelpLink: NULL
Source: System

StackTrace Information
*********************************************
at System.Net.HttpWebRequest.GetResponse()
at ASP.Test_aspx.readHtmlPage(String url) in
C:\Projects\MyWebApplication\Test.aspx:line 13
at ASP.Test_aspx.Page_Load(Object Src, EventArgs E) in
C:\Projects\MyWebApplication\Test.aspx:line 5
at System.Web.UI.Control.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain()

For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

I believe the problem is in the configuration of this machine or some other
install-oriented issue that I have been unable to
solve.

The machine is a Pentium IV 2.0 Mhz w/ 256MB of RAM, and plenty of free HD
space.
I'm running Windows XP Pro with SP1 and IE 6.0 (required by .NET).

Thanks for your help in advance.

Jeff B
Senior System Architect
AAON, Inc.

 

[Rick Strahl [MVP]]

Hi Jeff,

Is it possible you're going through a Proxy Server and not specifying the
proxy? Your IE connection settings aren't automatically pushed into the
HttpWebRequest code.

When you get this timeout does it actually hang for a while before timing
out? THere is a timeout setting on the HttpWebRequest object.

You might also want to see if you can read just a small chunk first. I've
seen some issues with some Apache Servers and HTTP Chunking that can cause
problems. Read the stream in small chunks of a few bytes and see what you
get back if anything...


+++ Rick ---

--

Rick Strahl
West Wind Technologies
http://www.west-wind.com/
http://www.west-wind.com/blog/