O
Oleg Konovalov
Hi,
We have a powerful Solaris server which hosts 2 sets of
Apache 2.0.48 -> JBoss 3.07 with Tomcat 4.1.18
using mod_jk2 2.0.2, running a bunch of web applications.
They are configured to run independently (2 server names, 2 virtual servers,
different ports, etc.)
Used to run without a single glitch for about 1/2 year.
We had a power failure a month ago, so the server crashed.
After that we rebooted it, it automatically started Apache,
we manually started JBoss (nobody touched any config files.).
One set was totally restored (no problems whatsoever),
but another one has a problem:
- if you try to run any of the applications, getting error 404
"requested resource not available".
There is nothing showing in error_log, but in access_log:
GET /myapp HTTP/1.1" 404 698
- if I try to go to the main page: http://myserver1.company.com ,
getting index.jsp (Tomcat's main page with the cat);
- but if I try to use port 8080 (http://myserver1.company.com:8080/myapp ),
getting main page and all the apps just fine.
Even killed the second instance of java process for JBoss/Tomcat -
same exact picture on server1
(ghost tomcat? - no tomcat proccesses running)
I interpreted it as a failure of something in the Apache->Tomcat/JBoss
bridge.
So I tried to do the following:
- reboot server - same problem;
- reinstalled Apache, copied all config files and modules (imcl. ) from old
instance of Apache to a new one - same;
- after that restarted JBoss - same.
- reinstalled JBoss, restarted it and Apache - the same.
More Experiments:
1) If I change in workers2.properties URI line
[uri:server1.company.com/*]
from server1.company.com to localhost/* or dnsserver1name.company.com:
and then try to do: http://dnsserver1name.company.com,
it no longer shows the admin page of Tomcat,
but shows page with something like
"Seeing this instead of the website you expected?..." (Apache Test page).
Is that a step in the right or wrong direction?
We had problem on that server resolving localhost (maybe because we have 2
instances of apache & jboss there, 2 servernames, etc.),
so I changed localhost->server1 in all 3 config files:
workers2.properties, and on JBoss - in server.xml and tomcat41-service.xml,
still the same exact behavior.
2) We replaced all "localhost" with the IP address of that virtual server
(it has 2 network cards, 2 IP addresses, etc.) - same result
3) We copied the working version of JBoss and modified its configuration
(port numbers/IPs/servernames) to be as it was on that instance originally -
same result.
It might be worth mentioning that when I restart apache,
see in error_log:
error mod_jk child init 1 -2
error jk2_init() can't find child 5194 in scoreboard
What does that mean ?
Trying to fix it for over a month and still can't find what might be wrong.
Questions:
1) what might have gone wrong in Apache/Tomcat/JBoss configuration or maybe
Unix environment
which prevents from Apache->Tomcat (part of JBoss) redirection (especially
in 2 sets on 1 unix server) ?
Can it be a network card failure after power outage?
2) is it worth trying to move from mod_jk2 (no more support but works on 2
of my servers out of 3)
to mod_jk1.2 or mod_proxy ? Is it worth trying, what's involved? (might
be very difficult, since I can only have
that development/testing server available)
3) On a bit unrelated topic:
It seems that all our other servers works fine, however I have noticed tons
of errors in
Apache error_log, they all follow the following pattern:
[error] channelSocket.receive(): Error receiving message body -1 131
[error] workerEnv.processCallbacks() Error reading reply
[error] ajp13.service() ajpGetReply recoverable error 120000
Can you please tell if they are harmful, the cause and possible fix.
(They seem to be produced at every request).
Any help is VERY appreciated.
Thank you in advance,
Oleg.
We have a powerful Solaris server which hosts 2 sets of
Apache 2.0.48 -> JBoss 3.07 with Tomcat 4.1.18
using mod_jk2 2.0.2, running a bunch of web applications.
They are configured to run independently (2 server names, 2 virtual servers,
different ports, etc.)
Used to run without a single glitch for about 1/2 year.
We had a power failure a month ago, so the server crashed.
After that we rebooted it, it automatically started Apache,
we manually started JBoss (nobody touched any config files.).
One set was totally restored (no problems whatsoever),
but another one has a problem:
- if you try to run any of the applications, getting error 404
"requested resource not available".
There is nothing showing in error_log, but in access_log:
GET /myapp HTTP/1.1" 404 698
- if I try to go to the main page: http://myserver1.company.com ,
getting index.jsp (Tomcat's main page with the cat);
- but if I try to use port 8080 (http://myserver1.company.com:8080/myapp ),
getting main page and all the apps just fine.
Even killed the second instance of java process for JBoss/Tomcat -
same exact picture on server1
(ghost tomcat? - no tomcat proccesses running)
I interpreted it as a failure of something in the Apache->Tomcat/JBoss
bridge.
So I tried to do the following:
- reboot server - same problem;
- reinstalled Apache, copied all config files and modules (imcl. ) from old
instance of Apache to a new one - same;
- after that restarted JBoss - same.
- reinstalled JBoss, restarted it and Apache - the same.
More Experiments:
1) If I change in workers2.properties URI line
[uri:server1.company.com/*]
from server1.company.com to localhost/* or dnsserver1name.company.com:
and then try to do: http://dnsserver1name.company.com,
it no longer shows the admin page of Tomcat,
but shows page with something like
"Seeing this instead of the website you expected?..." (Apache Test page).
Is that a step in the right or wrong direction?
We had problem on that server resolving localhost (maybe because we have 2
instances of apache & jboss there, 2 servernames, etc.),
so I changed localhost->server1 in all 3 config files:
workers2.properties, and on JBoss - in server.xml and tomcat41-service.xml,
still the same exact behavior.
2) We replaced all "localhost" with the IP address of that virtual server
(it has 2 network cards, 2 IP addresses, etc.) - same result
3) We copied the working version of JBoss and modified its configuration
(port numbers/IPs/servernames) to be as it was on that instance originally -
same result.
It might be worth mentioning that when I restart apache,
see in error_log:
error mod_jk child init 1 -2
error jk2_init() can't find child 5194 in scoreboard
What does that mean ?
Trying to fix it for over a month and still can't find what might be wrong.
Questions:
1) what might have gone wrong in Apache/Tomcat/JBoss configuration or maybe
Unix environment
which prevents from Apache->Tomcat (part of JBoss) redirection (especially
in 2 sets on 1 unix server) ?
Can it be a network card failure after power outage?
2) is it worth trying to move from mod_jk2 (no more support but works on 2
of my servers out of 3)
to mod_jk1.2 or mod_proxy ? Is it worth trying, what's involved? (might
be very difficult, since I can only have
that development/testing server available)
3) On a bit unrelated topic:
It seems that all our other servers works fine, however I have noticed tons
of errors in
Apache error_log, they all follow the following pattern:
[error] channelSocket.receive(): Error receiving message body -1 131
[error] workerEnv.processCallbacks() Error reading reply
[error] ajp13.service() ajpGetReply recoverable error 120000
Can you please tell if they are harmful, the cause and possible fix.
(They seem to be produced at every request).
Any help is VERY appreciated.
Thank you in advance,
Oleg.