How to send response headers from proxy to browser?

[hiwa]

This question's got no replies on forum.java.sun.com
through its full four days and three nights life.
The proxy server intercepts every request from the
browser and redirect it to www.yahoo.com. It simply
discards original request. After connecting to yahoo.com,
it sends the received response to the browser,
displaying the first part of the yahoo page. However,
as described in the comment on the code below, it
seems the proxy can't send the response headers to
the browser, it only sends document content. What
should we do to send the response headers from proxy?

This program has no practical usefullness. It's only
an exercise for implementing proxy.

/* SimpleProxy.java
* redirect every request from browser to www.yahoo.com
*/
import java.net.*;
import java.io.*;
import java.util.*;

class SimpleProxy{
ServerSocket pxSrvSocket;
Socket clientSocket;
URL remoteUrl;
URLConnection remoteConnection;
BufferedReader readClient, readRemote;
PrintWriter writeClient;
String clientReq, remoteRes, resphName, resphValue, resphStr, resMsg;
Map<String,List<String>> respHdrs; // response headers
List<String> values; // header values
int len;

public SimpleProxy(){
try{
pxSrvSocket = new ServerSocket(9999);

clientSocket = pxSrvSocket.accept();
System.out.println("browser connected to this proxy");
readClient = new BufferedReader
(new InputStreamReader(clientSocket.getInputStream()));
writeClient
= new PrintWriter(clientSocket.getOutputStream(), true);

// setup connection and request headers
// (from Firefox 1.5.0.1 on Linux)
remoteUrl = new URL("http://www.yahoo.com/");
remoteConnection = remoteUrl.openConnection();
remoteConnection.setAllowUserInteraction(true);
remoteConnection.setDoInput(true);
remoteConnection.setDoOutput(true);
//remoteConnection.setIfModifiedSince(0L);
//remoteConnection.setUseCaches(true);

((HttpURLConnection)remoteConnection).setRequestMethod("GET");

remoteConnection.setRequestProperty("Host", "www.yahoo.com");
remoteConnection.setRequestProperty("User-Agent",
"User-Agent: Mozilla/5.0 (X11; U; Linux i686; ja; rv:1.8.0.1)
Gecko/20060124 Firefox/1.5.0.1");
remoteConnection.setRequestProperty("Accept",
"text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5");
remoteConnection.setRequestProperty("Accept-Language",
"ja,en-us;q=0.7,en;q=0.3");
remoteConnection.setRequestProperty("Accept-Encoding",
"gzip,deflate");
remoteConnection.setRequestProperty("Accept-Charset",
"ISO-8859-1,utf-8;q=0.7,*;q=0.7");
remoteConnection.setRequestProperty("Keep-Alive", "300");
remoteConnection.setRequestProperty("Proxy-Connection",
"keep-alive");

remoteConnection.connect();
System.out.println
("remote connection esatblished : " +
remoteConnection.toString());
readRemote = new BufferedReader
(new InputStreamReader(remoteConnection.getInputStream()));

// dump original request from browser
while ((clientReq = readClient.readLine()).length() > 0){
System.out.println("read from browser : " + clientReq);
}

/*** if we uncoment this part, program doesn't work, only blank screen
***

// get response headers
respHdrs = remoteConnection.getHeaderFields();
Set<String> names = respHdrs.keySet();

// first, write response status line
resphStr = "";
values = respHdrs.get(null);
for (String val : values){
resphStr += val + ","; // e.g. HTTP/1.1 200 OK
}
len = resphStr.length();
if (resphStr.charAt(len - 1) == ','){ // trim last ','
resphStr = resphStr.substring(0, len - 1);
}
System.out.println("resp hdr from remote: " + resphStr);
writeClient.println(resphStr);
// names.remove(null); // UnsupportedOperationException

// next, other response headers
for (String name : names){
if (name != null){
values = respHdrs.get(name);
resphStr = name + ": ";
for (String val : values){
resphStr += val + ",";
}
len = resphStr.length();
if (resphStr.charAt(len - 1) == ','){ // trim last ','
resphStr = resphStr.substring(0, len - 1);
}
System.out.println("resp hdr from remote: " + resphStr);
writeClient.println(resphStr);
}
}
System.out.println();
writeClient.println(); // a blank line

*** end response headers part ***/

// get and write document contents -- this part works normally
// if we comment-out the above part
while((remoteRes = readRemote.readLine()).length() > 0){
writeClient.println(remoteRes);
System.out.println("response from remote : " + remoteRes);
}

writeClient.close();
clientSocket.close();
pxSrvSocket.close();
}catch (IOException e){
e.printStackTrace();
}
}

public static void main(String a[]){
new SimpleProxy();
}
}

 

[Chris Uppal]

This question's got no replies on forum.java.sun.com
through its full four days and three nights life.

Tut! Bunch of wasters...

;-)

The proxy server intercepts every request from the
browser and redirect it to www.yahoo.com. It simply
discards original request. After connecting to yahoo.com,
it sends the received response to the browser,
displaying the first part of the yahoo page. However,
as described in the comment on the code below, it
seems the proxy can't send the response headers to
the browser, it only sends document content. What
should we do to send the response headers from proxy?

Get hold of a copy of Ethereal, and watch what data /really/ goes back and
forth. No arguments, just do it ! You are only wasting your own time if you
try to debug any kind of networking code without a tool like Ethereal.

I suspect that the problem is that you are not correlating what the client sees
with what Yahoo sends back. When you send your request to Yahoo, you have set
a lot of stuff about Keep-Alives, and so so. You then copy /all/ of the client
request onto what you've already set (which is wrong -- e.g. you'll have two
GET lines, although Yahoo'll ignore the second bunch since it'll see them as
the body of the GET request). When it sends back a response, it will be
formatted (keep-alives, compression, etc) in accordance with the settings you
established (ignoring what the client asked for), but you then just copy
Yahoo's answer back to the client -- which will probably get very confused...

I doubt whether the Java HTTP stuff is the right tool to use for this job.
You'd almost certainly be better off working at the level of raw sockets, since
you will need the level of control that gives you. It'll also be easier to see
what's going on, and so probably easier to code too. BTW, you will also need
(if you don't have it already) a reasonable knowledge of the HTTP protocol --
it's not enough just to know how to use HttpURLConnection and its friends.

-- chris

 

[Thomas Weidenfeller]

Get hold of a copy of Ethereal, and watch what data /really/ goes back and
forth. No arguments, just do it ! You are only wasting your own time if you
try to debug any kind of networking code without a tool like Ethereal.

I can only second this! Plus, get a copy of the relevant RFCs to figure
out when which things are done for what purpose. Expect deviations from
the RFCs to a certain extend. Or a Chris has put it:

BTW, you will also need
(if you don't have it already) a reasonable knowledge of the HTTP protocol --
it's not enough just to know how to use HttpURLConnection and its friends.

/Thomas

 

[Thomas Hawtin]

Get hold of a copy of Ethereal, and watch what data /really/ goes back and
forth. No arguments, just do it ! You are only wasting your own time if you
try to debug any kind of networking code without a tool like Ethereal.

Even more usefully, split the program down into manageable chunks. Then
make sure each one works as expected.

Tom Hawtin

 

[Chris Uppal]

Thomas Hawtin wrote:

[me:]

Even more usefully, split the program down into manageable chunks. Then
make sure each one works as expected.

The issues that come up with networking often revolve around not fully
understanding what's actually going on, rather than not being able to
comprehend the code. Make the code as managable as you like, but you'll
/still/ need Ethereal...

-- chris

 

[Roedy Green]

This program has no practical usefullness. It's only
an exercise for implementing proxy.

Find a real proxy and watch what it does with a sniffer. See
http://mindprod.com/jgloss/sniffer.html