Deploying NodeJS projects with flightplan

Perry Mitchell / 2016-06-26 00:32:10
Deploying NodeJS projects with flightplan

NodeJS makes rapid ap­pli­ca­tion de­vel­op­ment a cinch with its ease of use and ba­sic learn­ing curve, but de­ploy­ing ap­pli­ca­tions built with Node can be chal­leng­ing and fraught with prob­lems. When it comes to de­ploy­ing Node ap­pli­ca­tions to re­mote servers, Flightplan makes the de­ploy­ment con­fig­u­ra­tion and man­age­ment ter­ri­bly sim­ple and easy to main­tain.

Flightplan is an npm mod­ule that uses a flightplan.js file to con­fig­ure de­ploy­ment strate­gies. It uses tar­get spec­i­fi­ca­tions (staging/production etc.) with server de­tails (addresses, cre­den­tials, con­nec­tion method etc.) along­side lo­cal and re­mote tasks.

Here’s a brief ex­am­ple:

let plan = require("flightplan");

// targets
plan.target('production', {
    host: 'myserver.org',
    username: 'deployer',
    agent: process.env.SSH_AUTH_SOCK
});

// run tasks locally
plan.local(function(local) {
    local.log("Building...");
    local.exec("npm run build");

    local.log("Copying files...");
    let filesToCopy = local.exec("git ls-files", {silent: true});
    local.transfer(filesToCopy, "/tmp/deployment/");
});

// run tasks remotely
plan.remote(function(remote) {
    remote.log("Copying files...");
    remote.exec("cp -r /tmp/deployment/ ~/prod/");
});

Because Flightplan uses a JavaScript con­fig­u­ra­tion file, you can eas­ily tap into other npm pack­ages and per­sonal helpers to get the job done. Using the remote func­tion­al­ity you can ex­e­cute com­mands on all end­points as if they were a sin­gle en­tity - every­one gets the some in­struc­tions.

Of course, my ex­am­ple so far does­n’t cover de­ploy­ing ex­e­cutable ap­pli­ca­tions at all, let alone dae­mons - I’ll get to that - But first, let’s start with a real-world ex­am­ple of a de­ploy­able ap­pli­ca­tion.

The ap­pli­ca­tion

Say we have a lit­tle HTTP server that re­sponds with a ran­dom num­ber:

const http = require('http');

const PORT = 8080;

// Handle requests
function handleRequest(request, response){
    response.end("" + Math.ceil(Math.random() * 10));
}

// Server instance
let  server = http.createServer(handleRequest);

server.listen(PORT, function() {
    console.log(`Server listening on port ${PORT}`);
});

When started, this server will con­tinue run­ning un­til halted. When we de­ploy it, we need to up­date the code on the ma­chine and restart the server so that the changes take af­fect. Before we take care of the re­load­ing of our server, let’s get the de­ploy­ment process un­der­way.

Flightplan

Flightplan is a won­der­ful NodeJS li­brary that I’ve used both pro­fes­sion­ally and per­son­ally over the last sev­eral years. It’s helped me and my team move large, com­plex ap­pli­ca­tion servers to their re­mote tar­gets hun­dreds of times with just a sin­gle com­mand and some tens of lines.

Let’s jump straight into the flight­plan file for de­ploy­ing our lit­tle Node app:

let plan = require("flightplan");

// targets
plan.target('production', {
    host: 'test.com',
    username: 'user',
    password: 'pass',
    agent: process.env.SSH_AUTH_SOCK
});

// run tasks locally
plan.local(function(local) {
    local.log("Copying files...");
    let filesToCopy = local.exec(`find . -type f -follow -print | grep -v "node_modules"`, {silent: true});
    local.transfer(filesToCopy, "/tmp/deployment/");
});

// run tasks remotely
plan.remote(function(remote) {
    remote.log("Copying files...");
    remote.exec("cp -r /tmp/deployment/* ~/mysite/");
    remote.log("Booting application...");
    remote.exec("pm2 start ~/mysite/app.js || pm2 restart ~/mysite/app.js");
});

In my file I spec­ify the pro­duc­tion lo­gin cre­den­tials so flight­plan knows how to con­nect to the server (ideally this would be with­out a pass­word and would use keys in­stead). There’s a lo­cal task, which copies the files in the cur­rent di­rec­tory to each re­mote end­point, and a re­mote task, which copies the files to their lo­ca­tion and starts/​restarts the ap­pli­ca­tion us­ing pm2 (another amaz­ing server util­ity).

Run the de­ploy­ment us­ing the com­mand fly production - easy!

pm2 here is at­tempt­ing to start the app (which will suc­ceed upon first de­ploy­ment only), and if it fails, it will try to restart the app. Using a lit­tle bash or’ func­tion­al­ity can help us ex­press this in one line, though it does messy-up the out­put a bit:

flightplan pm2 start restart

Once de­ployed suc­cess­fully, we should be able to key in the ad­dress to see the out­put of our ap­pli­ca­tion:

deployed application output

Most of the frus­tra­tion, at least for me, has been in get­ting the con­nec­tion setup cor­rectly. Once this is taken care of, the rest of the process with flight­plan is pure joy.

Obviously this ex­am­ple is very small-scale, but ex­pand­ing to larger groups of servers is not far re­moved from the code we just looked at. Flightplan sup­ports spec­i­fy­ing groups of servers, and be­cause we have ac­cess to any JavaScript li­brary or npm pack­age in our Flightplan con­fig­u­ra­tion, we can sim­ply add func­tion­al­ity to dy­nam­i­cally gen­er­ate a list of servers to de­ploy to. This can come in very handy when de­ploy­ing to ser­vices such as AWS (but per­haps I’ll cover this in a later post).

Caveats

There’s a few points to men­tion that are ei­ther passed on from dev to dev or learned the hard way. Once such point is the fact that ssh can be a lit­tle harsh and block con­nect­ing when at­tempt­ing to con­nect to a new host. ssh will pro­ceed to wait for user in­put, block­ing our beau­ti­ful au­to­mated de­ploy­ment process.

A lit­tle hack to get around this is to ex­e­cute an ssh-keyscan com­mand on the host so that the con­nec­tion process won’t prompt for in­put:

function sshFix(transport, host) {
    transport.log("Fixing SSH...");
    // Forcibly add the remote key to known_hosts:
    transport.exec(`ssh-keyscan -t rsa,dsa ${host} 2>&1 | sort -u - ~/.ssh/known_hosts > ~/.ssh/tmp_hosts`);
    transport.exec(`mv ~/.ssh/tmp_hosts ~/.ssh/known_hosts`);
}

flightplan.local(function(local) {
    // SSH-fix all endpoints
    flightplan.runtime.hosts.forEach(function(endpoint) {
        sshFix(local, endpoint.host);
    });
});

Branching from this is­sue with ssh is the fact that Flightplan uses rsync un­der­neath the transport method, and this can have com­pli­cated is­sues when us­ing more cus­tom forms of in­ter­ac­tion with ssh (such as our keyscan trick). To get around this, you can use your own trans­port method:

function transfer(transport, hostInfo, localFile, remoteFile) {
    transport.log(`Transferring package to: ${hostInfo.host}`);
    transport.exec(`scp ${localFile} ${hostInfo.username}@${hostInfo.host}:${remoteFile}`);
};

This ob­vi­ously fur­ther com­pli­cates mat­ters, but this is eas­ily added to a boot­strap file if need be. Try Flightplan in vanilla first and move on from there to suit your needs - It’s a solid ap­pli­ca­tion that fan­tas­ti­cally ab­stracts the de­ploy­ment process.