BufferedInputstream problem

[Daniel]

hi!
I have written an application that reads data from the serial-port
with javax.comm. My problem is that it acts a bit strange.
I can issue several commands to the controller card at the other end
of the serial-kable. One command gives me the event-log. This eventlog
is approx. 4500 bytes long. I have the following code to do the
reading

out.write(key.getBytes("US-ASCII"));
out.write(13); // make the card execute the request.
out.flush();
Thread.currentThread().sleep(initalWaitTime); // let the
//card "catch up"
int a=in.available();
do{
a=in.available();
Thread.currentThread().sleep(50);
}while(a!=in.available());

byte[] retval = new byte[in.available()];
in.read(retval,0,retval.length); // get the data

return retval;

key is the command I send. it is given to me as a string. and I write
the bytes to the card. This seems to be working without problems.

initalWaitTime is the time specified by the caller of this method (it
is private so it's only me, and I have full control over this) for
some commands I get very short responses and the need to wait is very
small. Others (like the eventlog) give long responses and with a
longer wait at first, I got better performance.
in is a BufferedInputSteam

The serial port is set to 38400bps so getting the event log should
take approx 1-2 seconds. depending on length.
With the code given here it takes 8.5 seconds!
While this could be explained with things like "the card doesn't send
data at max-speed" I can live with that time. But of course I am
interested to know if there is anything *I* can do to improve this
time.

Second, this code doesn't get all the data. Which confuses me greatly.
The current eventlog is 240 "items" long. I get items 240-10.
If I download the table with settings 117 "items" long. it works
perfectly.
My questions now are
1. is there anything I can do to improve performance even more?
2. Do I do something really stupid?
3. Any ideas why it reads to little data given the code here? (I can
via hyperterminal verify that the data is infact there and is sent to
me)

A really odd thing is that the eventlog seems to be getting fewer and
fewer "items" (or rows) each time I run the application. Each start
gives me one row less! (start of application).
This code has worked before, but I have made a few changes to it now.
(and no I can not go back, the requested changes were really needed)
In short I'm totally lost as to wy this is not working..

 

[Daniel]

A really odd thing is that the eventlog seems to be getting fewer and

fewer "items" (or rows) each time I run the application. Each start
gives me one row less! (start of application).

I noticed on a very close check that I actually get the same ammount
of bytes, but that the graphical display is the culprit. I will look
into that by myself. But the questions of performance still and my
stupidity remains.

 

[Daniel]

I noticed on a very close check that I actually get the same ammount
of bytes, but that the graphical display is the culprit. I will look
into that by myself. But the questions of performance still and my
stupidity remains.

An even closer look..
I always get 4096 bytes. this is 2^12.
I do not see why this is a problem though.. I create the buffered
stream with:
in = new BufferedInputStream(serialPort.getInputStream(),100000);
so that should hold more than 4096 bytes. Should I instead of sleeping
and comparing the ammount of bytes, simply read off a few and put them
in say a LinkedList or Vector or similar structure, and then later
recreate the byte array?
It seems that the buffering somewhere "down" under my control is the
problem here.

 

[Knute Johnson]

An even closer look..
I always get 4096 bytes. this is 2^12.
I do not see why this is a problem though.. I create the buffered
stream with:
in = new BufferedInputStream(serialPort.getInputStream(),100000);
so that should hold more than 4096 bytes. Should I instead of sleeping
and comparing the ammount of bytes, simply read off a few and put them
in say a LinkedList or Vector or similar structure, and then later
recreate the byte array?
It seems that the buffering somewhere "down" under my control is the
problem here.

Daniel:

Lose the available(). Use a BufferedReader and just call readLine() if
your data is a text file. The comm driver buffer is probably adequate
for buffering so I wouldn't adjust the buffer size. Split your reading
and writing into two different threads so you can block on the read.

 

[Daniel]

Daniel:

Lose the available(). Use a BufferedReader and just call readLine() if
your data is a text file. The comm driver buffer is probably adequate
for buffering so I wouldn't adjust the buffer size. Split your reading
and writing into two different threads so you can block on the read.

I tried that, and quickly realized why I had not done so in the first
place. The ready() method seems to always return true, even if there
is no data to be read (must be from the javax.comm package this odd
thing comes) so that means I have no way of telling when I should stop
reading.Since if I block, I can never get out of that, since I can
then not send anything to the card to unblock me. I must say that it
would have been nice to have a
while(in.ready(){
linkedlist.addLast(in.readLine());
sleep(50);
}
but that does not work, as in.ready() always returns true.
Oh well I guess I have to find some workaround..

/daniel

 

[Rogan Dawes]

Daniel:

I tried that, and quickly realized why I had not done so in the first
place. The ready() method seems to always return true, even if there
is no data to be read (must be from the javax.comm package this odd
thing comes) so that means I have no way of telling when I should stop
reading.Since if I block, I can never get out of that, since I can
then not send anything to the card to unblock me. I must say that it
would have been nice to have a
while(in.ready(){
linkedlist.addLast(in.readLine());
sleep(50);
}
but that does not work, as in.ready() always returns true.
Oh well I guess I have to find some workaround..

/daniel

The algorithm that you want to use is:

String line;
while ((line=in.readLine())!= null) {
linkedlist.addLast(line);
}

Note that this will keep reading until the inputstream is closed. If you
want to be able to stop reading when the response to your current
command is finished, you will need to do some protocol handling in the
loop, for example, breaking when you receive a new prompt, or a pair of
blank lines, or something similar. This will depend entirely on the
device that you are talking to, of course.

Rogan

 

[Daniel]

String line;
while ((line=in.readLine())!= null) {
linkedlist.addLast(line);
}

Note that this will keep reading until the inputstream is closed. If you
want to be able to stop reading when the response to your current
command is finished, you will need to do some protocol handling in the
loop, for example, breaking when you receive a new prompt, or a pair of
blank lines, or something similar. This will depend entirely on the
device that you are talking to, of course.

yeah, I now do the following:
int a=in.available();
do{
byte[] temp=new byte[in.available()];
in.read(temp,0,temp.length);
a=in.available();
try{
input.addLast(new String(temp,"US-ASCII"));
}catch(UnsupportedEncodingException e){
input.addLast(new String(temp));
}
Thread.currentThread().sleep(50); // let the card
//"catch up"
}while(a!=in.available());

StringBuffer retval= new StringBuffer();
Iterator retit= input.iterator();
while(retit.hasNext()){
retval.append(input.removeFirst().toString());
}

return retval;

since I must have it as ONE string, not several. This seems to be
working. As I must have protocol handling, I guess I could be
searching for prompt. I have three different prompts to search for.
One of them is just "?" which is issued if you're not logged in. The
problem of course is that I get a "?" if I do something wrong too.
Like for example setting an invalid value, or entering a command in
the wrong mode. So I'm not sure if that protocol handling would
outperform this implementation.
This seems to be fairly okay regarding performance. It also solves the
problem. Or do you see any problem with it?

/daniel

 

[Rogan Dawes]

String line;
while ((line=in.readLine())!= null) {
linkedlist.addLast(line);
}

Note that this will keep reading until the inputstream is closed. If you
want to be able to stop reading when the response to your current
command is finished, you will need to do some protocol handling in the
loop, for example, breaking when you receive a new prompt, or a pair of
blank lines, or something similar. This will depend entirely on the
device that you are talking to, of course.

yeah, I now do the following:
int a=in.available();
do{
byte[] temp=new byte[in.available()];
in.read(temp,0,temp.length);
a=in.available();
try{
input.addLast(new String(temp,"US-ASCII"));
}catch(UnsupportedEncodingException e){
input.addLast(new String(temp));
}
Thread.currentThread().sleep(50); // let the card
//"catch up"
}while(a!=in.available());

StringBuffer retval= new StringBuffer();
Iterator retit= input.iterator();
while(retit.hasNext()){
retval.append(input.removeFirst().toString());
}

return retval;

since I must have it as ONE string, not several. This seems to be
working. As I must have protocol handling, I guess I could be
searching for prompt. I have three different prompts to search for.
One of them is just "?" which is issued if you're not logged in. The
problem of course is that I get a "?" if I do something wrong too.
Like for example setting an invalid value, or entering a command in
the wrong mode. So I'm not sure if that protocol handling would
outperform this implementation.
This seems to be fairly okay regarding performance. It also solves the
problem. Or do you see any problem with it?

/daniel

You may be able to get away with something like:

List lines = new LinkedList(); // or whatever
String line;
while ((line = in.readLine()) != null) {
if (line.equals("?")) break;
lines.add(line);
}
// now do something with your lines

This assumes that your "?" prompt is followed by a carriage return. If
it is not, you may have to take a different approach, for example,
wrapping the actual InputStream from the comm device to give you an
InputStream that terminates whenever it sees the prompt.

Then, each time you issue a command, you get the InputStream containing
the response from that command, read it until the stream is EOF, and
then you can issue your next command.

In effect you are providing a command interface to your device:

InputStream responseStream = cardController.execute("my command");

where cardController keeps track of whether the previous response stream
has been fully consumed, and the state of the card (whether it is ready
to accept a new command, or not)


Say we assume that the controller gives you a "?" as a prompt, with no
carriage return after it. At this point, it is prepared to accept an
instruction, followed by a CRLF, after which it will give you some
output (possibly none?), followed by a CRLF, and a "?", indicating that
the command has completed.

class CardController {

private InputStream in;
private OutputStream out; // these are connected to the COM port

public CardController(InputStream in, OutputStream out) {
this.in = in;
this.out = out;
if (in.available() == 0) {
out.write("\r\n".getBytes()); // nudge the card to get a prompt
}
readUntilPrompt(); // discard what we read, we don't know what it is
}

private InputStream readUntilPrompt() {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
byte[] buff = new byte[1024];
int got;
boolean gotPrompt = false;
while ((got = in.read(buff)) > 0 && ! gotPrompt) {
if (buff[got-3] == '\r' &&
buff[got-2] == '\n' &&
buff[got-1] == '?') {
gotPrompt = true;
got = got - 3; // don't return the prompt
}
buffer.write(buff, 0, got);
}
return new ByteArrayInputStream(buffer.toByteArray());
}

public InputStream executeCommand(String command) {
out.write((command+"\r\n").getBytes());
return readUntilPrompt();
}

}

Then, your calling classes simply do:

InputStream in = cardController.executeCommand("command");

and parse the inputstream into lines.

Alternatively, you may want to handle the conversion between bytes and
characters inside the CardController class, using InputStreamReader and
OutputStreamReader wrappers. This is probably a good idea, but I've
spent too long on this already! ;-)

Please note, I have not compiled or tested any of the above code,
exception handling is left as an exercise for the reader.

Rogan