Introduction to Node on Pivotal Web Services

Node.js is one of the most popular runtime environments on Pivotal Web Services (PWS). A solid introduction to node.js and the popular Express framework is from the Mozilla foundation. In this multi-part tutorial, you learn about how to install and configure node locally, how to create a simple database driven application using MongoDB and how to deploy it to production. What it does not tell you is how to deploy that same tutorial to PWS. (It uses some other popular non-Open Source PaaS)

This post covers how to modify the LocalLibrary application from the tutorial for deployment on PWS and Cloud Foundry. In doing so, it covers the basics of deploying any node.js application to PWS with the following steps.

• Configuring the package.json file to run with the engines available on PWS
• adding and installing the ‘cfenv’ node module to make working with services easier
• using the cfenv module to connect to a MongoDB instance from mLab that was created and bound using the PWS marketplace
• use the cf CLI tool to create a new mongoDB service instance and bind it to the local library application
• how to set environment variables for Node using the cf CLI
• how to look at logs, again using the cf CLI tool

So let’s get started. You have two options, you can go through the tutorial from the Deploying LocalLibrary on PWS

This section provides a practical demonstration of how to deploy the LocalLibrary sample application on the Pivotal Web Services PaaS cloud. Before we get started, let’s cover some of the reasons why PWS is ideal for Node.js apps.

Why PWS?

Pivotal Web Services is a public instance of the Open Source Cloud Foundry Platform. It is a polyglot platform supporting many different languages including Node.js, Java, PHP, Python, Staticfiles and Ruby. It has a introductory free trial and is incredibly efficient for running Node applications! As Node and Express are open source projects, there is consistency with working with an open deployment platform such as Cloud Foundry. You can get under the hood and see how an application is hosted. Also, as an open platform, there is freedom from vendor lock in as you can replicate your workflow to many providers.

There are multiple reasons to use PWS!

• PWS has a flexible pricing that is extremely well tuned for small runtimes such as node. It’s possible to run a redundant pair of your app for less than $5 per month. That includes failover.
• As a Applications Platform or PaaS, PWS takes care of a lot of the web infrastructure for us. This makes it much easier to get started, because you don’t worry about servers, load balancers, routing, restarting your website on a crash, or any of the other web infrastructure that PWS provides for us under the hood.
• Because PWS is using Cloud Foundry. You can easily deploy your application to other Cloud Foundry providers such as IBM BlueMix, Swisscom or AnyNines.
• While it does have some limitations, these will not affect this particular application. For example:
  • PWS and Cloud Foundry provides only short-lived storage so user-uploaded files cannot safely be stored on PWS itself. You can use a third party blog store to do that.
  • The free trial is good for a year and only up to $87 of app usage. For a typical Node app that means you can run an app for the entire year. Upgrade your account to a paid account and use the code ‘EXPRESS25’ for an additional $25 in app usage credit.

Modifying the LocalLibrary for PWS

Deployment of a Node application on Cloud Foundry takes the following steps. Using the downloaded ‘cf’ CLI tool on your environment, your source code and supporting metadata files are uploaded to Cloud Foundry which will assemble and package the components of your application. Note that your files need to be located on your system to deploy or as a zip archive somewhere accessible on the internet. We’ll assume the former.

This means, no assumptions about which source control system is used. As long as you have a complete source tree in your local file system you can deploy the app.

There are some things you have to make available to ensure the correctly assembly of your Node application. First Cloud Foundry cf CLI will look for the presence of the ‘package.json’ file to determine the dependencies and download the necessary components. The rules of this assembly are defined in Cloud Foundry’s nodejs buildpack. An optional cloud foundry manifest file can specify information about your application such as name, size and start command if non-standard.

In addition to deploying the application, the cf CLI tool can also configure services, set environment variables and view logs.

That’s all the overview you need in order to get started (see Getting Started on Pivotal Web Services for a more comprehensive guide).

Let’s start making the changes so you’ll need to deploy the LocalLibrary application to  PWS.

Set node version

The package.json contains everything needed to work out your application dependencies and what file should be launched to start your site. Cloud Foundry and PWS detects the presence of this file, and will use it to provision your app environment.

The only useful information missing in our current package.json is the version of node. We can find the version of node we’re using for development by entering the command:

>node --version
v6.10.3

Open package.json with a text editor, and add this information as an engines > node section as shown (using the version number for your system). As of this writing PWS support 6.10.3

{
  "name": "express-locallibrary-tutorial",
  "version": "0.0.0",
  "engines": {
    "node": "6.10.3"
  },
  "private": true,
  ...

Database configuration

So far in this tutorial we’ve used a single database that is hard coded into app.js file. Normally we’d like to be able to have a different database for production and development, so next we’ll modify the LocalLibrary website to get the database URI from the OS environment, and otherwise use our development database that we added manually earlier.

Cloud Foundry has a very flexible services model that enables multiple services of the same type to exist in the environment. It stores all services related configurations in a single parseable JSON object called VCAP_SERVICES. A typical VCAP_SERVICES variable looks like :

{
 "VCAP_SERVICES": {
  "mlab": [
   {
    "credentials": {
     "uri": "mongodb://CloudFoundry_test_dev:[email protected]:57971/CloudFoundry_dbname"
    },
    "label": "mlab",
    "name": "node-express-tutorial-mongodb",
    "plan": "sandbox",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     "Cloud Databases",
     "Developer Tools",
     "Web-based",
     "Data Store",
    ],
    "volume_mounts": []
   }
  ]
 }
}

Writing the code to extract and parse this environment variable is not hard, but it doesn’t add a lot of value when others have written libraries to do this. In this case, there is a node.js package called cfenv. To install the package, go to your terminal and make sure you are in the directory where the package.json file for LocalLibrary is.

From the command line, type

npm install cfenv

and this will download the cfenv module and its dependencies, plus modify the package.json file.

Open app.js and find the block with all the ‘requires’ that load the modules into your application. In this example look for the line that looks something like this:

var expressValidator = require('express-validator');

If you cannot find that exact line, look for the blocks of ‘requires’ and look for the last one. Then add the following text after that line

var cfenv = require('cfenv');

Now that you have loaded the module, this next line will instantiate an object that will contain the app and services information required for . Add the following after the line that contains app.use(helmet()); and add

// Set up CF environment variables

var appEnv = cfenv.getAppEnv();

When this line executes, all the Cloud Foundry application environment information will become available to the application in the object appEnv.

Now it is time to update the application to use the database configured by the platform. Find the line that sets the mongoDB connection variable. It will look something like this:

var mongoDB = process.env.MONGODB_URI || dev_db_url;

You will now modify the line with the following code  appEnv.getServiceURL('node-express-tutorial-mongodb') to get the connection string from an environment variable that is being managed by the cfenv  module. If no service has been created and bound it will use your own database URL you created as part of the tutorial instead of the one from the environment.

So replace the line above with the below.

var mongoDB = appEnv.getServiceURL('node-express-tutorial-mongodb') || dev_db_url;

Now run the site locally (see Testing the routes for the relevant commands) and check that the site still behaves as you expect. At this point you app has been made ready to be used with Cloud Foundry and Pivotal Web Services.

Get a Pivotal Web Services account

To start using Pivotal Web Services you will first need to create an account (skip ahead to Create and upload the website if you’ve already got an account and have already installed the PWS cf CLI client).

• Go to https://run.pivotal.io and click the SIGN UP FOR FREE button.
• Enter your details and then press CREATE FREE ACCOUNT. You’ll be asked to check your email for a sign-up email.
• Click the account activation link in the signup email. You’ll be taken back to your account on the web browser and you will complete the registration.
• You will set your password and go through the rest of the new user sign up and how to claim your free trial account. Note you need a mobile phone to confirm your account. You will receive an “org” account funded with $87 of application usage credit. Note your email account can be in multiple orgs on pws. This is similar to your user account on services like GitHub.
• Go to https://console.run.pivotal.io and login in. You’ll then be logged in and taken to the PWS dashboard: https://console.run.pivotal.io.

Install the cf CLI client

The cf CLI client is a software tool to manage and deploy your application. Download and install the PWS cf CLI client by following the instructions on Pivotal Web Services here. Be sure to download the correct version for your computer.

After the client is installed you will be able run commands. For example to get help on the client:

cf help

We’ll now go through the steps to login to PWS using the CLI and deploy or in Cloud Foundry parlance “push” your app.

Create and upload the website

To create the app we navigate to the directory where our modified files are. This is the same directory where the LocalLibrary package.json file is located.

First, let’s tell the cf CLI which Cloud Foundry instance you want to use. We need to do this, since the cf CLI tool can be used with any standard Cloud Foundry system, so this command indicates which specific Cloud Foundry you are using.

cf api api.run.pivotal.io

cf login
Email: enter your email
Password: enter your password

Note: you may be prompted to choose an org and space if you already have an existing account. If so, just choose the location from where you want to deploy your app.

We can now push our app to PWS. In the below example. replace ‘some-unique-name’ with something you can remember that is likely to be unique. If it isn’t unique, the system will let us know. The reason this name has to be unique to the PWS system is it is the address we will use to to access your LocalLibrary application. I used mozilla-express-tutorial-xyzzy. You should use something else.

cf push some-unique-name -m 256MB

Note the -m flag we added is not required. We just included it so that we only use 256MB of memory to run your app. Node apps typically can run in 128 MB, but we are being safe. If we didn’t specify the memory, the CLI would use the default 1 GB of memory, but we want to make sure your trial lasts a long time.

You should now see a bunch of text on the screen. It will indicate that the CLI is uploading all your files, that it’s using the node buildpack and it will start the app. If we’re lucky, the app is now “running” on the site at the url https://some-unique-name.cfapps.io. Open your browser and run the new website by going to that URL.

Setting configuration variables

You will recall from a preceding section that we need to set NODE_ENV to ‘production’ in order to improve our performance and generate less-verbose error messages. We do this by entering the following command:

>cf set-env some-unique-name NODE_ENV production

We should also use a separate database for production. Cloud Foundry can take advantage of a marketplace to create a new service and automatically bind it to your app. Bind means place the service database credentials in the environment variable space of the container running your application for you to access.

>cf create-service mlab sandbox node-express-tutorial-mongodb
>cf bind-service some-unique-name node-express-tutorial-mongodb

You can inspect your configuration variables at any time using the cf env some-unique-name command — try this now:

>cf env some-unique-name

In order for your applications to use the new credentials you will have to restage your application to restart and take in the environment variables. This can be done using the ‘cf restage some-unique-name’.

>cf restage some-unique-name

If you check the home page now it should show zero values for your object counts, as the changes above mean that we’re now using a new (empty) database.

Debugging

The PWS cf client provides a few tools for debugging:

cf logs some-unique-name --recent  # Show current logs
cf logs some-unique-name # Show current logs and keep updating with any new results

You have now deployed the LocalLibrary app to PWS. Try to deploy your own Node.js app to PWS and tell us what you think.