HttpWebRequest and Reusing Connections

[Mike]

Hi, I need to modify an existing Windows forms application that (among
many other things) uploads files from the users machine to the web
server which is on the same domain. On the server there is a "Generic
Handler" / .ashx that receive the files. The current implementation
uploads files in one shot as a single byte array, which for very large
files (100's of MBs) is prone to OutOfMemoryExceptions anywhere in the
process, both client and server side. So instead we need to upload the
files in more manageable "chunks".

The code below reads and uploads files 64 KB at a time. It works
except for one big problem. As it's running, it appears from netstat
output that each HttpWebRequest ties up a distinct port on the client
with a status of TIME_WAIT. After about 4,000 or so requests I get
this error on the client:

WebException was unhandled
Unable to connect to the remote server
Only one usage of each socket address (protocol/network address/port)
is normally permitted

So this error message, together with netstat, indicates all the ports
on the client machine are tied up. I would expect that rather than
creating new connections for each request they would be reused based
on the post below, but evidently they aren't:

http://social.msdn.microsoft.com/Forums/en-US/ncl/thread/6d67ff8b-14ff-44b0-9dcd-1c8b9d2ab42b

Any suggestions how I can either release the port immediately after
the request/response completes, or even better reuse the same port for
all of the requests since they are serial/synchronous? To that end,
I've tried using various combinations of KeepAlive,
ConnectionGroupName, UnsafeAuthenticatedConnectionSharing, and others
to force reuse based on other posts I've read, but I wasn't able to
hit on the "magic" combination.

Also, I've read about editing the registry to increase the number of
ports available to the application, but in production I cannot do this
on the user's machines, and besides that doesn't really seem like a
"solution" to me.

Thanks!
MJ

private void TestUpload()
{
  UploadFile(@"C:\Temp\My800MBFile.dat", "http://localhost:1776/
UploadFile.ashx?fileId=" + Guid.NewGuid().ToString());
}

private void UploadFile(string fileName, string url)
{
  using (FileStream fileStream = new FileStream(fileName,
FileMode.Open, FileAccess.Read))
  {
    byte[] buffer = new byte[65536];

    int bytesRead = fileStream.Read(buffer, 0, buffer.Length);
    while (bytesRead > 0)
    {
      HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(url);
      request.Method = "POST";

      using (Stream requestStream = request.GetRequestStream())
      {
        requestStream.Write(buffer, 0, bytesRead);
        requestStream.Close();
      }

      using (HttpWebResponse response = (HttpWebResponse)
request.GetResponse())
      {
        response.Close();
      }

      bytesRead = fileStream.Read(buffer, 0, buffer.Length);
    }

    fileStream.Close();
  }
}

 

[cubaman]

Hi, I need to modify an existing Windows forms application that (among
many other things) uploads files from the users machine to the web
server which is on the same domain. On the server there is a "Generic
Handler" / .ashx that receive the files. The current implementation
uploads files in one shot as a single byte array, which for very large
files (100's of MBs) is prone to OutOfMemoryExceptions anywhere in the
process, both client and server side. So instead we need to upload the
files in more manageable "chunks".

The code below reads and uploads files 64 KB at a time. It works
except for one big problem. As it's running, it appears from netstat
output that each HttpWebRequest ties up a distinct port on the client
with a status of TIME_WAIT. After about 4,000 or so requests I get
this error on the client:

WebException was unhandled
Unable to connect to the remote server
Only one usage of each socket address (protocol/network address/port)
is normally permitted

So this error message, together with netstat, indicates all the ports
on the client machine are tied up. I would expect that rather than
creating new connections for each request they would be reused based
on the post below, but evidently they aren't:

http://social.msdn.microsoft.com/Forums/en-US/ncl/thread/6d67ff8b-14f...

Any suggestions how I can either release the port immediately after
the request/response completes, or even better reuse the same port for
all of the requests since they are serial/synchronous? To that end,
I've tried using various combinations of KeepAlive,
ConnectionGroupName, UnsafeAuthenticatedConnectionSharing, and others
to force reuse based on other posts I've read, but I wasn't able to
hit on the "magic" combination.

Also, I've read about editing the registry to increase the number of
ports available to the application, but in production I cannot do this
on the user's machines, and besides that doesn't really seem like a
"solution" to me.

Thanks!
MJ

private void TestUpload()
{
  UploadFile(@"C:\Temp\My800MBFile.dat", "http://localhost:1776/
UploadFile.ashx?fileId=" + Guid.NewGuid().ToString());

}

private void UploadFile(string fileName, string url)
{
  using (FileStream fileStream = new FileStream(fileName,
FileMode.Open, FileAccess.Read))
  {
    byte[] buffer = new byte[65536];

    int bytesRead = fileStream.Read(buffer, 0, buffer.Length);
    while (bytesRead > 0)
    {
      HttpWebRequest request = (HttpWebRequest)
WebRequest.Create(url);
      request.Method = "POST";

      using (Stream requestStream = request.GetRequestStream())
      {
        requestStream.Write(buffer, 0, bytesRead);
        requestStream.Close();
      }

      using (HttpWebResponse response = (HttpWebResponse)
request.GetResponse())
      {
        response.Close();
      }

      bytesRead = fileStream.Read(buffer, 0, buffer.Length);
    }

    fileStream.Close();
  }

}

First, I would create a WCF Service instead of a ashx to handle the
file upload. Second, i'd try to implement some kind of compression
before uploading the file.
Sorry, cant help with connection re-use.
Best regards.