securing ssl certificates on web servers

[Amos Shapira]

Hi All,

A queeze for the weekend, if you like:

The standard procedure for production servers hosting secure sites, as
usually proposed by the Apache mod ssl documentation, is to remove the
pass-phrase from the SSL certificate in order to allow apache to
restart without human intervention.

I was trying to think about ways to mitigate this and here is what I
came up with:

1. Keep the certificate encrypted on disk.
2. Configure apache to use SSLPassPhraseDialog
(http://httpd.apache.org/docs/2.2/mod/mod_ssl.html#sslpassphrasedialog)
to execute a program to fetch a passphrase.
3. Write a C program which tries to verify the authenticity of the
call (e.g. try to make sure the system is not compromised, look for
some secret file, check system up time, program parent process tree
etc) then connects to a remote pass-phrase server over SSL to get the
pass-phrase. It can do other tricks like setting up a temporary source
IP address before connecting the pass-phrase server and connect from a
specific source port.
4. The remote pass-phrase server does more checks and hands back the
pass-phrase, logging the transaction.
5. The pass-phrase server keeps the pass-phrases encrypted on its
disk, requiring a human to type a "master password" to unlock them
whenever it starts.
6. The pass-phrase server shall test the front-line servers for
suspicious activity and block them out from further requests (e.g. why
would a front-line server re-request a pass-phrase if it haven't been
rebooted or Apache restarted? Why was Apache restarted? if any
suspicious traffic arrives from the front-line server to the
pass-phrase server, the pass-phrase server will refuse any requests
from that front-line server until securely told otherwise).

The benefit of this excercise are:
1. The pass-phrase server, being on the internal network instead of
the DMZ, is supposedly more secure than tha front-line servers.
2. The pass-phrase server shouldn't have a copy of the certificate,
just the pass-phrase. This way we separate certificate from the
pass-phrase
3. The auto-deploy process deploys protected files, it requires human
intervention to actually gain access to the certificate/pass-phrase.

The down-side is that if the pass-phrase server is not
available/accessible (e.g. if it's in a remote location and we lose
network connectivity to it) then the front-line servers won't be able
to serve the SSL certificates IF apache needs to be restarted. We can
mitigate this by having multiple load-balanced pass-phrase servers,
they'll probably have to synchronize what they think about the
pass-phrase client (and trust each other).

My question - am I re-inventing a well-known wheel? I couldn't find
anything like this when google'ing around.
I know there is a lot of "security by obscurity" here, any PRACTICAL
ideas on how such a system can be improved? (please spare me the "it
can't be totally secure" spiel, I know that).

Thanks,

--Amos