Modern process management for web apps
Part of good systems administration is managing processes. Not that it is a fancy job but rather a job that is necessary to deliver a quality web app. The current state of affairs when it comes to this is unfortunately either outdated or very crude. At the same time, this also makes it easy to improve.
Process management can do much better by doing two things: make your app do less and use of better administration tools. Perhaps that comes off as obvious but I want to share a few thoughts and ideas on how to do better. Applying this to a general purpose application is certainly possible, but this post was written foremost with web apps in mind.
Simplify Expectations
A good way to help you with the task of managing the process(es) of your web app is first and foremost to simplify the expectations of your web app. By that I mean the expectations you have on how your app should behave while running in production. As you’ll see below, it is a lot about doing less when it comes sysadmin side of things of your web app.
Daemonization
Don’t do daemonization from within your app as it will effectively free your sysadmin from any control what so ever of your process and any of it’s child processes, thus making process management near impossible. Running your app as a foreground process is an an enabler for the bullet points below.
PID files
They are a more or less the de facto way of keeping track of running processes on Unix systems. Despite that, they have a tendency of becoming stale or outdated when processes crash unexpectedly and/or fail to clean up after themselves. Avoiding PID files also avoid extra configuration on where to store them. Proper process supervision is a better solution, more about that below.
Forking
The most common use-case of forking web applications is to take advantage of a multi-core system. However when forking, it also becomes the job of your app to keep track of these child processes. That’s something your sysadmin is really better at and probably want to do. When almost every deployment of your web app sits behind a firewall or reverse proxy, getting it to load balance between your processes should pose no problem.
Logging
When logging, send all your log output to stdout. Use whatever logging library you need but don’t bother with a log file or even worse, multiple log files. Overall, log files are a common headache for sysadmins that require special care not to fill up the disk. Simply not doing file based logging free you from doing in-app log rotation and offering configurable log file paths. But for a sysadmin, the greatest benefit is that they can now redirect your logging output wherever he/she pleases: to a log rotation script, logger(1) or /dev/null :).
Adhering to these simple guidelines you will surely make life easier for your sysadmin looking at deploying your web app. And to be fair, that sysadmin is quite often you.
These guidelines are mentioned in the 12 factor app, a document describing characteristics of a modern web app. If these guidelines for processes makes sense to you, explore their site for more inspiration on similar topics.
Tools
Reading that list of guidelines can make it seem that the sysadmin will have one pesky day setting up and running your application. But in reality, this is less the case with good tooling.
To begin with, there’s good old SysV init. Probably the most common (and oldest) process manager you find on Unix systems. But given it’s age, it is also really arcane. A lot of things have happened and improved since then. Managing processes the way your father did is necessarily not the best thing to do. And let’s face it, writing rc.d scripts is not fun, not even the slightest as you need to take care of everything - daemonization, writing PID files and rotating log files and so on. On top of that, you can’t even be notified when a process dies! Truly arcane! Given these hurdles I know that at least I have been guilty of firing up screen/tmux and spawning the processes by hand. A very short-term solution to a nagging problem. Ugh.
Luckily, there are many popular alternatives to init for this task. Upstart, monit, runit to name a few. Supervisor and god are two tools worth an extra mention since they are written in Python and Ruby respectively. Something that web hackers do appreciate as it will make things easier to comprehend and extend. By combining one of these tools with the guidelines above will yield a great combo as they will supplement each other.
Be Modern
So to avoid screen/tmux sessions with running applications and arcane SysV init scripts, do take a look and investigate these tools. Dustin Sallings has a good write-up on modern tools for process management. I suggest you give that a look and then promise yourself to do modern process management.