java calling rsync calling ssh: io threads blocked

[pvbemmel-at-xs4all-nl]

Subject:
java calling rsync calling ssh: io threads blocked.

I'm trying to run a java program, that starts rsync, that uses ssh to
send data (--rsh option).

When I run the java program, all my I/O threads are waiting; nothing
happens, and rsync timeouts after 300 seconds.

In Eclipse in Debug mode, I see that all my threads are "Running" .
After I suspend each thread, I can see in what line of my code the
thread is executing:

Thread stdoutRunnable:
line 187: int num = procStdout.read(buf);

Thread stderrRunnable:
line 166: int num = procStderr.read(buf);

Thread stdinRunnable:
line 114: stdinLock.wait();

These are all statements that block.

When I run the same rsync command from a cygwin command shell, the rsync
runs okay.

The rsync.exe and ssh.exe are part of a Cygwin environment installed on
my machine.
(Respectively
rsync version 3.0.7 protocol version 30
and
OpenSSH_5.6p1, OpenSSL 0.9.8o 01 Jun 2010
)

The rsync command is:

C:/cygwin/bin/rsync \
--modify-window=2 \
--verbose \
--recursive \
--no-perms \
--no-group \
--times \
--fuzzy \
--compress \
--log-file="/cygdrive/c/Users/p.vanbemmelen/Documents/rsync/log/+logFile+" \
--rsync-path=rsync \
--stats \
--bwlimit="50" \
--progress \
--rsh="ssh -l p.vanbemmelen -i
/cygdrive/c/Users/p.vanbemmelen/.ssh/sshkey.ssh" \
--backup \
--human-readable \
--partial \
--timeout=300 \
--partial-dir=".rsync/partial/" \
--filter="exclude_p.vanbemmelen/downloads/" \
--chmod=u+rwx \
/cygdrive/c/cygwin/home/p.vanbemmelen \
(e-mail address removed):rbm/manualbackup \


(This command is hardcoded in the Java code; I can change a boolean in
that java code, and recompile, to print the command to stdout; so that
the same command can be issued from the cygwin command shell .)

I use ProcessBuilder from the Java library, and my own ProcessManager
class to listing to stdout, stderr of the rsync process.

I've attached the code.

The main() is in RsyncRunnableRemote.Test .

Note: The goal is to have a java program that can be installed on end
users' computers to give them an easy interface to rsync.
I'm perfectly able to run rsync on my own pc from the cygwin command
shell, but I really need to be able to run rsync from java.

Any help is appreciated,

Paul van Bemmelen.

 

[pvbemmel-at-xs4all-nl]

I just found out that by putting the rsync command in a .bat file, and
running that .bat file from java, does work okay.

But it still would be nice to be able to run the rsync command from java
directly.

Paul van Bemmelen.

 

[Daniel Pitts]

I just found out that by putting the rsync command in a .bat file, and
running that .bat file from java, does work okay.

But it still would be nice to be able to run the rsync command from java
directly.

Paul van Bemmelen.

It is possible that the command you're actually running is not the
command you think you are running. However, since you posted no SSCCE,
we cannot help you diagnose your problem.

<http://virtualinfinity.net/sscce>

 

[pvbemmel-at-xs4all-nl]

Hi Daniel,

Thanks for reacting to my posting.

SSCCE:
Short: maybe not, but handling stdin,stdout of a Process in java
does require a bit of code.
Self Contained: I've included all required java code; the rsync
and ssh executables you can find on the internet.
Compilable: it is
Example: see java code.

Did you see my original posting from 20-9-2010 12:04 ?

Paul van Bemmelen.

 

[Daniel Pitts]

Hi Daniel,

Thanks for reacting to my posting.

SSCCE:
Short: maybe not, but handling stdin,stdout of a Process in java
does require a bit of code.
Self Contained: I've included all required java code; the rsync
and ssh executables you can find on the internet.
Compilable: it is
Example: see java code.

Did you see my original posting from 20-9-2010 12:04 ?

Paul van Bemmelen.

Please do not top post. I've moved your quote to the top for future
readers.

I did see your original post, but I somehow missed the example at the
bottom.

I see at least one race condition. It is possible that your
StdoutRunnable and StderrRunnable will read the data before you're able
to attach the logging listeners.

Another note, StdoutRunnable and StderrRunnable could be made into one
class, where you pass in the InputStream and Listener list in the
constructor.

I also suggest that you put a debugging output in that loop (rather than
relying on the listener), until you find the solution.

 

[pvbemmel-at-xs4all-nl]

Please do not top post. I've moved your quote to the top for future
readers.

I did see your original post, but I somehow missed the example at the
bottom.

I see at least one race condition. It is possible that your
StdoutRunnable and StderrRunnable will read the data before you're able
to attach the logging listeners.

Another note, StdoutRunnable and StderrRunnable could be made into one
class, where you pass in the InputStream and Listener list in the
constructor.

I also suggest that you put a debugging output in that loop (rather than
relying on the listener), until you find the solution.

Hi Daniel,

To solve the race condition, I added a constructor

public ProcessManager(Process proc,
List<OutputStreamListener> stdoutListeners,
List<OutputStreamListener> stderrListeners ) {

that installs the listeners provided as arguments prior to starting the
StdoutRunnable and StderrRunnable threads.

I haven't found the solution to the "io threads blocked" yet.
The workaround using a .BAT file is still necessary.

Greetings,
Paul van Bemmelen

 

[AnAnAsbAnAnAshAkAr]

I've conquered in my past a problem like this when I wanted to call
OpenSSL to generate some certificates on my machine. I finally got a
workaround for this blocking issue: use a shell, and the argument for
the shell is your programm to call. Otherwise I was not able to get
the STDIN and STDOUT to work properly. I think, the OS may call a
shell also if you are running a .bat file.

Greetings,
Mike Grossenbacher

 

[pvbemmel-at-xs4all-nl]

Hi Daniel,

To solve the race condition, I added a constructor

public ProcessManager(Process proc,
List<OutputStreamListener> stdoutListeners,
List<OutputStreamListener> stderrListeners ) {

that installs the listeners provided as arguments prior to starting the
StdoutRunnable and StderrRunnable threads.

I haven't found the solution to the "io threads blocked" yet.
The workaround using a .BAT file is still necessary.

Greetings,
Paul van Bemmelen

I was wrong in assuming there is a difference between running directly
from java versus running via a .bat running from java : if one fails,
then the other fails, if one succeeds then the other succeeds.

I found out that failure or success, is related to the account being
used to run the windows service.

Running under my own user account, p.vanbemmelen (is in administrator
group), or under LocalSystem, the windows service runs okay.
Running under NetworkService, or under LocalService, it fails.

So it's the account, and more specifically, the account's permissions
that determine whether the io blocks.

Greetings,
Paul van Bemmelen.