Linux – How to get Apache2 to wait for network initialization

apache-http-serverdebianlinuxnetworking

Server just upgraded from Squeeze to Wheezy. After reboots, I noted that apache2 wasn't starting, so installed bootlogd to inspect the output of init scripts.

The error output from apache start was,

Tue Feb  3 08:49:55 2015: [....] Starting web server: apache2(99)Cannot assign requested address: make_sock: could not bind to address [0123:4567:890:abc::d]:80
Tue Feb  3 08:49:55 2015: no listening sockets available, shutting down
Tue Feb  3 08:49:55 2015: Unable to open logs
Tue Feb  3 08:49:55 2015: Action 'start' failed.
Tue Feb  3 08:49:55 2015: The Apache error log may have more information.

Timestamps in the boot output show 4s between "Configuring network interfaces" (08:49:51) and starting apache.

I've worked around this by inserting a sleep 5 into /etc/init.d/apache2 after which apache2 starts fine, so I'm reading it as slow network initialization (possibly specific to IPv6 networking). The server is running on VMWare, and I have access only to the VM.

  • What better methods could I use to make Apache wait a few seconds and check its addresses are up and running before start?
  • Or to make the network init script not exit until the addresses are ready?

Best Answer

What better methods could I use to make Apache wait a few seconds and check its addresses are up and running before start?

Well, the method I am outlining obviously does not solve the underlying issue. But I believe using a stand-alone monitoring tool that can handle the “kicking Apache into service” task without having to hack the init.d script is a more stable solution. The key is to use monit which is self-described as:

Monit is a small Open Source utility for managing and monitoring Unix systems. Monit conducts automatic maintenance and repair and can execute meaningful causal actions in error situations.

To install it in Ubuntu, just do this; I am using aptitude and this is on Ubuntu 12.04 FWIW:

sudo aptitude install monit

Once installed I like to setup the default mail server for sending out alerts. This assumes you have postfix or sendmail active on your server. Open up the monit control file with your favorite text editor; I like to use nano:

sudo nano /etc/monit/monitrc

Look for the set mailserver lines and set this line and save the monit control file:

set mailserver localhost

Now check and make sure there is a monit conf.d directory setup; this is where individual monit tasks are setup:

ls -la /etc/monit/conf.d

If somehow that /etc/monit/conf.d was not set up, create it like this:

sudo mkdir /etc/monit/conf.d

Now with that done, let’s create a monit Apache ruleset. First, figure out where the Apache .pid file is set and check that it is set. This is usually where it is set on Ubuntu 12.04 using the default Apache package install:

ls -la /var/run/apache2.pid

If that .pid filepath is correct, let’s create the actual monit Apache ruleset with nano like this:

sudo nano /etc/monit/conf.d/apache2.conf

And place this code in that file and save it:

check process apache with pidfile /var/run/apache2.pid
        start "/etc/init.d/apache2 start"
        stop  "/etc/init.d/apache2 stop"
        if failed host 127.0.0.1 port 80
                with timeout 15 seconds
        then restart
        alert YourEmail@example.com only on { timeout, nonexist }

The logic to the script is fairly simple: The monit Apache ruleset will check the /var/run/apache2.pid file and it knows to use specific init.d directives for start and stop logic. The magic comes from the if/then block which basically monitors Apache port 80 on localhost 127.0.0.1 and will take action to recover Apache if there is a timeout of 15 seconds or more. And the alert line will send e-mail alerts to a specified e-mail address if the conditions of timeout or nonexist are met; that is optional so feel free to comment that out if you don’t have the need to be flooded by e-mail alerts.

Now restart monit:

sudo service monit restart

And you can follow the monit log here to see it do it’s thing and debug it if something doesn’t work as expected:

sudo tail -f -n 200 /var/log/monit.log

So when the dust settles on that server, you will have monit setup to make sure Apache is running. Very useful as a general “keep the Apache web server alive” tool, but in your case monit can take over the task of making sure Apache comes up on reboot/startup so you don’t have to muck around with core Apache init.d script which is really a bit messy and is easily forgotten in the future during an upgrade.

Also note, I can see that you are binding Apache explicitly to IPv6, but this monitoring script assumes that if it can’t reach 127.0.0.1 then it should take action. Maybe you need to change the IPv4 localhost address of 127.0.0.1 to the IPv6 equivalent of address of ::1. Experiment and see what works.

Related Question