Using BufferedReader with a URL

[Eric Capps]

I'm using BufferedReader with a URL to try to read information from that
URL and do something with it. The BufferedReader is basically declared
as follows:

InputStreamReader stream = new InputStreamReader((new
URL(url)).openStream());
BufferedReader fileReader = new BufferedReader(stream);

There are a certain number of lines "length" I'm expecting in the file.
I then read from the BufferedReader as follows:

while(fileReader.ready() && i < length){
String thisLine = fileReader.readLine();
// code i've snipped out involving thisLine
i++;
}

I have simplified this, but you get the idea: I am using readLine() and
the doing something with that line.

The problem: sometimes, this loop seems to terminate earlier than I am
expecting, that is, before "length" number of lines have been read. When
I debug the code, and step through, this NEVER happens, and it only
sometimes happens (and not always on the same invocations) when I
actually run it.

I suspect that what is happening is that somehow the ready() method of
the BufferedReader is returning false prematurely. My questions:

1) Is it possible that the BufferedReader is somehow not able to read
the entire contents of the URL, thus terminating before I expect it to?
2) If so, why is this happening, and how can I work around this?

 

[Gordon Beaton]

I suspect that what is happening is that somehow the ready() method of
the BufferedReader is returning false prematurely. My questions:

1) Is it possible that the BufferedReader is somehow not able to
read the entire contents of the URL, thus terminating before I
expect it to?
2) If so, why is this happening, and how can I work around this?

Your suspicion is right: ready() returns false before you reach the
end of the file, because sometimes you read faster than the data
arrives, so there are times when no data is "ready" to be read. That
also explains why it never happens when you single step through the
code.

The solution is to avoid ready() altogether. Simply use readLine() and
it will return each line as it arrives, blocking when no data is
(temporarily) available.

/gordon

 

[Eric Sosman]

Eric Capps wrote On 08/16/06 14:32,:

I'm using BufferedReader with a URL to try to read information from that
URL and do something with it. The BufferedReader is basically declared
as follows:

InputStreamReader stream = new InputStreamReader((new
URL(url)).openStream());
BufferedReader fileReader = new BufferedReader(stream);

There are a certain number of lines "length" I'm expecting in the file.
I then read from the BufferedReader as follows:

while(fileReader.ready() && i < length){
String thisLine = fileReader.readLine();
// code i've snipped out involving thisLine
i++;
}

I have simplified this, but you get the idea: I am using readLine() and
the doing something with that line.

The problem: sometimes, this loop seems to terminate earlier than I am
expecting, that is, before "length" number of lines have been read. When
I debug the code, and step through, this NEVER happens, and it only
sometimes happens (and not always on the same invocations) when I
actually run it.

I suspect that what is happening is that somehow the ready() method of
the BufferedReader is returning false prematurely.

What do you mean by "prematurely?" The ready() method
returns true if some input data has been received and is
sitting in a buffer somewhere just waiting to be read. Once
you've drained that buffer, ready() will return false until
some more data arrives from the other end of the network.
Once some more data trickles in, ready() will start returning
true again.

(This description is somewhat simplified. The behavior
of ready() depends on the underlying Reader, and Readers of
different kinds of data sources have different ideas of "data
is waiting; come and get it.")

My questions:

1) Is it possible that the BufferedReader is somehow not able to read
the entire contents of the URL, thus terminating before I expect it to?

Yes, of course. The machine at the other end could crash
before sending all the data you expect, or the network could
get confused and route all the data to Timbuktu instead of
to you.

2) If so, why is this happening, and how can I work around this?

The likely cause of your problem is the ready() call.
I'm having a hard time understanding why you wrote it; perhaps
you misunderstood what it tests.