Thursday, September 5, 2013

Setting Up a Node.js Dev Environment with Puppet

On my last post I explained why it's a good idea to set up your dev environment using Vagrant and Puppet. On this post I'm going to provide a specific example: setting up a dev environment for a Node.js/MySQL application.

** All code presented on this post is also available at this repo.
** 2014/03/01: Updated the code to use librarian-puppet to handle puppet modules
** 2014/06/26: Updated the code to use latest Ubuntu and to allow using separate JSON file for configuration

Pre-requisites


Make sure you have Vagrant installed. You can download it from http://downloads.vagrantup.com

Creating the Virtual Machine


Let's start by creating a blank Ubuntu 14.04 virtual machine. Create a new folder for your project and add a file called "Vagrantfile" with the following code:
require 'json'

# ------------------------------ #
#         Config Values
# ------------------------------ #
#
# If you wish to override the default config values, create a JSON
# file called Vagrantfile.json on the same folder as this file
#

configValues = {
  # box to build from
  "box" => "Official Ubuntu 14.04 daily Cloud Image amd64 " +
           "(Development release, No Guest Additions)",

  # the url from where the 'config.vm.box' box will be fetched if it
  # doesn't already exist on the user's system
  "box_url" => "https://cloud-images.ubuntu.com/vagrant/trusty/"    +
               "current/trusty-server-cloudimg-amd64-vagrant-disk1" +
               ".box",

  # private IP address for the VM
  "ip" => '192.168.60.2',

  # hostname for the VM
  "hostname" => "dev.nodejs"
}

if File.exist?('./Vagrantfile.json')
  begin
    configValues.merge!(JSON.parse(File.read('./Vagrantfile.json')))
  rescue JSON::ParserError => e
    puts "Error Parsing Vagrantfile.json", e.message
    exit 1
  end
end

# ------------------------------ #
#        Start Vagrant
# ------------------------------ #

Vagrant.configure("2") do |config|
  config.vm.box = configValues["box"]
  config.vm.box_url = configValues["box_url"]
  config.vm.network "private_network", ip: configValues['ip']
  config.vm.hostname = configValues['hostname']
end

As you can see, we first define some configuration values.
box
what type of image we want to use for our virtual machine (ubuntu 14.04 64bits on this case).

box_url
where to download the image from (in case the user doesn't have it already).

ip
which IP address it should assign to the virtual machine.

hostname
what hostname to use for the virtual machine.
Now you can cd into your directory and start the virtual machine with this command:
vagrant up
The first time you run it, vagrant is going to download the entire ubuntu 14.04 image, so it will take a while. After the image is downloaded, vagrant will boot up a new virtual machine with the settings we specified on the Vagrantfile.

You can ssh into your machine by running
vagrant ssh

Installing Puppet


Some vagrant boxes come with Puppet preinstalled, but most of them are outdated and lack important features. That's why I prefer using a blank Ubuntu box (like the one we're using on this project) and install Puppet myself.

Create a "shell" directory containing two files: "install-puppet.sh" and "install-librarian-puppet.sh". Your directory structure should look like this:
shell/
    install-puppet.sh
    install-librarian-puppet.sh
Vagrantfile
Copy the contents of this file into install-puppet.sh and this other file into install-librarian-puppet.sh.

These two shell scripts will install puppet and librarian-puppet, respectively. Librarian-puppet is a gem that makes it a lot easier to manage Puppet modules.

Now add the following code to your Vagrantfile:
  config.vm.provider :virtualbox do |vb|
    # This allows symlinks to be created within the /vagrant dir
    vb.customize [
      "setextradata", :id,
      "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root", "1"
    ]
  end

  # install puppet and librarian-puppet
  config.vm.provision :shell, :path => "shell/install-puppet.sh"
  config.vm.provision :shell, :path => "shell/install-librarian-puppet.sh"
And run "vagrant provision" to get puppet installed.

Adding Node.js and MySQL


Now that we have a basic Ubuntu vm up and running, let's add some puppet configurations to install Node.js and MySQL.

Create a "puppet" folder containing a file called "Puppetfile" and two subfolders: "modules" and "manifests". Your directory structure should look like this:
puppet/
    modules/
    manifests/
    Puppetfile
shell/
    install-puppet.sh
    install-puppet-librarian.sh
Vagrantfile
The Puppetfile is used to declare all puppet modules we want to use. Since we want to use the stdlib, apt, and mysql modules, add the following content to your Puppetfile:
forge "http://forge.puppetlabs.com"

mod "puppetlabs/stdlib", "3.2.1"
mod "puppetlabs/apt", "1.5.0"
mod "puppetlabs/mysql", "2.2.3"
Create a "default.pp" file inside the manifests folder, with the following content:
Exec {
  path => ['/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/local/bin']
}

# --- Preinstall Stage ---#

stage { 'preinstall':
  before => Stage['main']
}

# Define the install_packages class
class install_packages {
  package { ['curl', 'build-essential', 'libfontconfig1', 'python',
             'nodejs', 'npm', 'g++', 'make', 'wget', 'tar', 'mc', 'htop']:
    ensure => present
  }
}

# Declare (invoke) install_packages
class { 'install_packages':
  stage => preinstall
}

# Setup your locale to avoid warnings
file { '/etc/default/locale':
  content => "LANG=\"en_US.UTF-8\"\nLC_ALL=\"en_US.UTF-8\"\n"
}

# --- NodeJS --- #

# Because of a package name collision, 'node' is called 'nodejs' in Ubuntu.
# Here we're adding a symlink so 'node' points to 'nodejs'
file { '/usr/bin/node':
  ensure => 'link',
  target => "/usr/bin/nodejs",
}

# --- MySQL --- #

class { '::mysql::server':
  root_password => 'foo'
}
Let's go over this file one block at a time.
Exec {
  path => ['/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/local/bin']
}
We start by adding several folders to our $PATH, so Puppet knows where to find system commands like wget, curl, etc.
stage { 'preinstall':
  before => Stage['main']
}
We then define a new stage called 'preinstall'. Inside Puppet, stages are containers that allow you to execute blocks of code in a certain order. By default everything is executed inside the 'main' stage, but you're free to define additional stages if you want to.

On our example, we're telling Vagrant that everything inside the 'preinstall' stage should be executed before the 'main' stage starts.
class install_packages {
  package { ['curl', 'build-essential', 'libfontconfig1', 'python',
             'nodejs', 'npm', 'g++', 'make', 'wget', 'tar', 'mc', 'htop']:
    ensure => present
  }
}
On the next block we're defining a new class called 'install_packages'. Like its name suggests, this class will let you install a bunch of useful packages on your development machine. We make use of the package directive to let apt-get install these packages for us.

class { 'install_packages':
  stage => preinstall
}
Even though the syntax looks very similar to the class declaration, on this block we're actually invoking the 'install_packages' class. Notice we add a stage parameter to make sure the class is executed inside the preinstall stage.

file { '/usr/bin/node':
  ensure => 'link',
  target => "/usr/bin/nodejs",
}
Here we're creating a symlink from /usr/bin/node to /usr/bin/nodejs. This is required because Ubuntu has a different package called 'node', so apt installs node as 'nodejs'.

class { '::mysql::server':
  root_password => 'foo'
}
The '::mysql::server' class is defined on the puppetlabs/mysql module, and it will install a MySQL server for you. We're invoking it with a root_password param, telling it to set the MySQL root password to 'foo'.

Now we need to tell Vagrant where our Puppet files are. Just add the following to your Vagrantfile:
# provision with Puppet stand alone
  config.vm.provision :puppet do |puppet|
    puppet.manifests_path = "puppet/manifests"
    puppet.manifest_file = "default.pp"
  end
And run the following command to apply changes:
vagrant provision
After the command finishes running, your virtual machine will have Node.js and MySQL installed. You can test it by running:
vagrant ssh
node --version
mysql --version
You can also login to the root mysql account by running:
mysql -u root -pfoo

Our App


Our dev environment is ready now, so let's take it for a spin by writing a very simple Express.js app.

Add a "package.json" file with the following content:
{
  "name": "hello_world",
  "description": "hello world test app",
  "version": "0.0.1",
  "private": true,
  "dependencies": {
    "express": "3.3.8",
    "mysql": "2.0.0-alpha9"
  },
  "scripts": {
    "start": "node ./app.js"
  }
}

And an "app.js" file with the following content:
var express = require('express');
var mysql = require('mysql');
var app = express();

// This is for demonstration purposes only.
// You should NEVER store your db credentials on an
// unprotected, plain-text file
var connection = mysql.createConnection({
  host     : 'localhost',
  user     : 'root',
  password : 'foo',
});

app.get('/', function(req, res){
  res.writeHead(200, {
    'Transfer-Encoding': 'chunked',
    'Content-Type': 'text/plain; charset="utf-8'
  });

  res.write("Hello World!");
  res.write("\n\nNow connecting to mysql...");

  connection.connect(function(err) {
    if(err) {
      res.write("\n\nConnection failed");
    }
    else {
      res.write("\n\nConnection successful!");
    }
    res.end();
  });
});

app.listen(3000);
console.log('Listening on port 3000');

Your directory structure should look like this:
app.js
package.json
puppet/
    modules/
    manifests/
        default.pp
    Puppetfile
shell/
    install-puppet.sh
    install-puppet-librarian.sh
Vagrantfile

Now we can start our app by running:
vagrant ssh
cd /vagrant
npm install
npm start

Your app will be running on http://192.168.60.2:3000/

** For a more robust puppetfile containing PostgreSQL and node moudule installations, see this gist
Post a Comment