Multithreaded Socket.IO Server

Dec 16
Turbo charge your NodeJS applications with Multicore

Recently, I’ve been developing a new chat/messaging platform for my site, DarkUmbra.net that will be used throughout the website as well as for upcoming mobile applications. I started running into issues first with the linux kernel killing my chat processes because of open connections but after increasing the ulimit that issue subsided. Because of these crashes, I began looking for a way to run the chat on all 8 cores of the system to optimize performance and add redundancy in the case that one of the worker processes fails.

As you all know, NodeJS only runs on a single thread on one core of the server it’s running on by default. However, it comes installed with a simple utility that allows you to fork extra processes/workers as desired that can all access the same port/server running on Node. This is an incredibly useful feature and I recommend that everyone migrates their applications to multicore as it easily octuples (is that even the right word? Lol) the performance of the application. To illustrate this, I performed some ab benchmarks on my local installation which just served up which worker ID it was connected to. The test was to perform 10,000 requests and the results were really encouraging.

On the single core machine, the test took 4.503s. The multicore server however completed the test in 3.376 seconds, more than an entire second quicker; and that’s just running on my MacBook Pro. Throw in some server-grade hardware and wow does it fly. I’ve posted the Apache Bench results below for those who want to see the full output.

Now multiply that 1.1 second savings over 1M requests and you’ll quickly see the performance increase is exponential! Here’s how to do it:

What you Need

Scaling to Multicore

Let’s start with the basic file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
//-----------------------------------------
// Include our cluster lib packaged with
// Node.JS
//-----------------------------------------
var cluster = require('cluster');
 
//-----------------------------------------
// If this worker is the Master process
// Setup everything we need to scale
//-----------------------------------------
if (cluster.isMaster) {
 
}
 
//-----------------------------------------
// Since our cluster uses the same node script
// we use an if to check if this process
// is a worker process
//-----------------------------------------
if (cluster.isWorker) {
 
}

Configure the Master Process

Now that we have the basic structure that will control both the master process and the worker processes, let’s setup our master process. To make sure socket.io can talk to other socket instances on different workers, we have to configure them to use a Redis Store. Thankfully Socket.io comes packaged with Redis already so we don’t have to install it separately, simply tell socket.io that you want to use a separate Redis server. Let’s add this inside the if( cluster.isMaster )  if block:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
//-----------------------------------------
// Create the server instance but don't let
// http listen yet. The workers will listen.
// This is simply to create the server instance
//-----------------------------------------
var server     = require('http').createServer();
var socketIO   = require('socket.io').listen(server);
var RedisStore = require('socket.io/lib/stores/redis');
var redis      = require('socket.io/node_modules/redis');
 
//-----------------------------------------
// Tell socket.io where to go for the redis
// connection.
//-----------------------------------------
socketIO.set('store', new RedisStore({
    redisPub    : redis.createClient(),
    redisSub    : redis.createClient(),
    redisClient : redis.createClient()
}));

Create Workers

Time to create the additional processes that will the server simultaneously. This can be done manually by calling  cluster.fork() as needed, but I prefer to spawn one process per CPU core:

1
2
3
4
5
6
7
8
9
//-----------------------------------------
// This can be manually done but all this
// does is create a new worker/process for
// each CPU core on your machine
//-----------------------------------------
var numberOfCPUs = require('os').cpus().length;
for (var i = 0; i < numberOfCPUs; i++) {
    cluster.fork();
}

Deal with Worker Crashes

Sometimes an exception will occur that is uncaught by your application causing your server to crash. When this happens, normally the application itself terminates. However, since we are running a multithreaded environment, we can detect a worker process has died and restart one in it’s place:

1
2
3
4
5
6
7
8
9
//-----------------------------------------
// Catch an errors that cause a worker to
// crash, log it to the console, and create
// a new worker process
//-----------------------------------------
cluster.on('exit', function(worker, code, signal) {
    console.error('worker ' + worker.process.pid + ' died');
    cluster.fork();
});

Adding Worker Code

Now that we have our Master process all set up to create worker processes, let’s go ahead and add some worker code to run on the threaded processes. Working now from inside the if( cluster.isWorker )  if block, let’s configure the server and the Redis store like we did before, this time, we’re going to add an extra line to start the server listening on the port:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
//-----------------------------------------
// Create the server instance same as before
// But this time, listen on the port
//-----------------------------------------
var http   = require('http');
var server = http.createServer();
    server.listen(8080);
 
var socketIO   = require('socket.io').listen(server);
var RedisStore = require('socket.io/lib/stores/redis');
var redis      = require('socket.io/node_modules/redis');
 
//-----------------------------------------
// Setup the Redis Store same as before
//-----------------------------------------
socketIO.set('store', new RedisStore({
    redisPub    : redis.createClient(),
    redisSub    : redis.createClient(),
    redisClient : redis.createClient()
}));

Add Socket Bindings

Realistically we’re all good to go now, but our server doesn’t actually do anything yet. Let’s add some simple socket.io bindings:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
//-----------------------------------------
// Perform socket bindings here!
// It's THAT SIMPLE! :)
//-----------------------------------------
socketIO.sockets.on('connection', function(socket) {
 
    console.log('Connected to worker: %s', cluster.worker.id);
 
    //-----------------------------------------
    // Socket Bindings Here...
    //-----------------------------------------
    socket.on("foo", function(){
        socket.emit("bar");
    })
});

That’s It!

If you start the server up, you’ll now see something like this:

Multithreaded Socket.io Server

Multithreaded Socket.io Server

There you go! Your server is now running socket.io in full multicore mode! If you wanted, you could even get fancier and spawn more workers as processor load increases and then kill workers off as the load goes down 😉 But I’ll leave that to you 😛

As always guys if you have any questions or comments, post ’em below and I’ll do my best to answer them 🙂

Apache Benchmark Results

Filed Under: Dev

Sponsored Links