How to really get the output stream of an unix process?

[Carfield Yim]

I have some code like follow, which get some data from URL pipe to a
unix command, and get the result:

String line = "";
final Runtime rt = Runtime.getRuntime();
final Process p = rt.exec("somecommand");

final BufferedReader urlBR = new BufferedReader(new
InputStreamReader(new URL(url).openStream()));
final Writer os = new BufferedWriter(new
OutputStreamWriter(p.getOutputStream()));
while((line = urlBR.readLine()) != null) os.write(line);
os.close();
urlBR.close();

final BufferedReader din = new BufferedReader(new
InputStreamReader(p.getInputStream()));
while ( (line = din.readLine()) != null ) out.write(line);
din.close();

However it fail with "broken pipe" exception, I believe that as I don't
really pipe the data to the command, so java execute the command as
nothing output to the command.

Thus I try to use
final Process p = rt.exec("|somecommand");

But java complain command not find. How can I tell java that I have
something needed to pipe to the command?
--
\\\|///
\\- - -//
( @ @ )
-----------oOOo-(_)-oOOo------------------------------------------------
Visit my homepage at http://www.carfield.com.hk

Programming discussion groups
Software design: news://news.carfield.com.hk/programming.design
Design Pattern: news://news.carfield.com.hk/programming.design.pattern
java: news://news.carfield.com.hk/programming.java
linux: news://news.carfield.com.hk/programming.linux
------------------------------------------------------------------------

 

[Gordon Beaton]

I have some code like follow, which get some data from URL pipe to a
unix command, and get the result:

String line = "";
final Runtime rt = Runtime.getRuntime();
final Process p = rt.exec("somecommand");

final BufferedReader urlBR = new BufferedReader(new InputStreamReader(new URL(url).openStream()));
final Writer os = new BufferedWriter(new OutputStreamWriter(p.getOutputStream()));
while((line = urlBR.readLine()) != null) os.write(line);
os.close();
urlBR.close();

final BufferedReader din = new BufferedReader(new InputStreamReader(p.getInputStream()));
while ( (line = din.readLine()) != null ) out.write(line);
din.close();

However it fail with "broken pipe" exception, I believe that as I
don't really pipe the data to the command, so java execute the
command as nothing output to the command.

What do you mean you "don't really pipe the data"? You've got calls to
os.write() in the example.

You get broken pipe if you attempt to write to a pipe after the reader
has closed his end of the connection. Did you manage to write any of
the data, or do you get this exception as soon as you try to write
anything?

Do you have the same problem with other commands, or just one
particular "somecommand"?

Thus I try to use
final Process p = rt.exec("|somecommand");

Without another command on the left of the pipe symbol, it would have
been nonsense in a shell that understands redirection. As it is, Java
doesn't even execute the external program in a shell.

But java complain command not find. How can I tell java that I have
something needed to pipe to the command?

Java already assumes that you do and provides p.getOutputStream() for
that purpose. Don't close it if you still need it (you can't get it
back), and make sure that "somecommand" isn't closing its stdin.

/gordon

 

[Steve Horsley]

I have some code like follow, which get some data from URL pipe to a
unix command, and get the result:

String line = "";
final Runtime rt = Runtime.getRuntime();
final Process p = rt.exec("somecommand");

final BufferedReader urlBR = new BufferedReader(new
InputStreamReader(new URL(url).openStream()));
final Writer os = new BufferedWriter(new
OutputStreamWriter(p.getOutputStream()));
while((line = urlBR.readLine()) != null) os.write(line);
os.close();
urlBR.close();

final BufferedReader din = new BufferedReader(new
InputStreamReader(p.getInputStream()));
while ( (line = din.readLine()) != null ) out.write(line);
din.close();

However it fail with "broken pipe" exception, I believe that as I don't
really pipe the data to the command, so java execute the command as
nothing output to the command.

Thus I try to use
final Process p = rt.exec("|somecommand");

But java complain command not find. How can I tell java that I have
something needed to pipe to the command?

A few points I can think of:

BufferedReader.readLine() strips the \r and \n characters from the line.
If you feed the lines direct to the Process, you are giving it a single
long line to work on (and no line ending before you close the pipe). This
may well be why the process produces no discernible output. I suggest that
either re-insert line endings.

Does the URL return binary data? If so, better to deal purely in bytes
than go through a bytes->text->bytes double conversion, with its attendant
characterset conversion foibles. I see no reason at all in the above code
to convert to an intermediate text format.

The code tries to feed the URL contents to the process _completely_ before
attempting any read of the process's output stream. This is unsafe. The
process might block because it's stdout stream buffers are full, in which
cease it will cease reading its input stream, and you are deadlocked. I
suggest that you use multithreading so that you can keep both its input
and output serviced concurrently.

Also, you make no attempt to read the process's error stream. The process
may deadlock with a blocked stderr stream if it has problems, and you will
never know why. I suggest another Thread reading the error stream, even
though you hope it will never have any work to do.

Always close these streams after use, or you may get "too many open file"
errors after a while.

Steve

 

[Carfield Yim]

A few points I can think of:

BufferedReader.readLine() strips the \r and \n characters from the line.
If you feed the lines direct to the Process, you are giving it a single
long line to work on (and no line ending before you close the pipe). This
may well be why the process produces no discernible output. I suggest that
either re-insert line endings.

Does the URL return binary data? If so, better to deal purely in bytes
than go through a bytes->text->bytes double conversion, with its attendant
characterset conversion foibles. I see no reason at all in the above code
to convert to an intermediate text format.

The code tries to feed the URL contents to the process _completely_ before
attempting any read of the process's output stream. This is unsafe. The
process might block because it's stdout stream buffers are full, in which
cease it will cease reading its input stream, and you are deadlocked. I
suggest that you use multithreading so that you can keep both its input
and output serviced concurrently.

Also, you make no attempt to read the process's error stream. The process
may deadlock with a blocked stderr stream if it has problems, and you will
never know why. I suggest another Thread reading the error stream, even
though you hope it will never have any work to do.

Always close these streams after use, or you may get "too many open file"
errors after a while.

Steve

Thx, it solved when output the stderr and stdout in thread