Node.js Code Coverage with Istanbul and Mocha

Code coverage is a measure of how much of your code has been  tested. Code coverage tools run a set of metrics in order to determine if your code has been completely tested, reducing the chance of  unwanted bugs.

You have to take into account that even if your code has 100% code coverage, that doesn’t guarantee all your tests are correct, there are some logical bugs you might miss, but as with anything, practice will help you with that.

Istanbul

Yet another JS code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests.

Istanbul Github

Istanbul runs some  of the common coverage metrics/criteria to check whether your code meets the code coverage expectations. You can learn more about basic metrics here http://en.wikipedia.org/wiki/Code_coverage

Prerrequsites

Node.js >= 0.12, npm

You can also read my previous post Setting Up your Node.js Unit Tests with Chai And Mocha.

Steps

• Create a new directory. In your terminal run:

mkdir mochaistanbul && cd mochaistanbul.

• Run npm init.

npm init

• Install Mocha, Chai and Istanbul:

npm install mocha chai istanbul --save-dev

• Create  a module «index.js» in the root directory (mochaistanbul).

module.exports.getCoverageHello = function(to) {

return 'Hello, this is your first coverage test ' + to;
};

• Create a test directory

mkdir test

• Create a simple test file index.js under the test directory.

//test/index.js

/require chai and our simple 'index.js' module
require('chai').should();
//If you require an specific module, it will be included in the coverage test
var testModule = require('../index.js');

//simple test suite

describe('Istanbul Code Coverage', function(){

});

• In your package.json file add (or replace) the following script.

...
"scripts": {
"test": "node ./node_modules/.bin/istanbul cover ./node_modules/.bin/_mocha -- --check-leaks"
},
...

• Run ‘npm test’ in order to execute your test script.

npm test
=============================== Coverage summary ===============================
Statements   : 50% ( 1/2 )
Branches     : 100% ( 0/0 )
Functions    : 0% ( 0/1 )
Lines        : 50% ( 1/2 )
================================================================================

When you ran that command a new «coverage» folder was created, which contains a report of your code coverage. Check this file: mochaistanbul/coverage/lcov-report/mochaistanbul/index.js.html

Based on the report you can tell that one function is not being covered by the test suite. Let’s write a test in order to fix that
.

• Write a test for our function.

it('should return a hello message for an specific name', function() {
var name = 'otto';
var result = testModule.getCoverageMessage(name);
result.should.contain(name);
});

• Run ‘npm test’ again

npm test
=============================================================================
Writing coverage object [/home/otto/workspace/tutorial/mochaistanbul/coverage/coverage.json]
Writing coverage reports at [/home/otto/workspace/tutorial/mochaistanbul/coverage]
=============================================================================

=============================== Coverage summary ===============================
Statements   : 100% ( 2/2 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 1/1 )
Lines        : 100% ( 2/2 )

The output reflect the new test we’ve implemented, so the code coverage is in 100%.

Conclusion

This was a simple example on how to get started with Code Coverage in Node.js. With Istanbul you can validate that your tests are covering your code based on specific metrics, which helps you  reduce the chance of defects. You should be aware that code coverage won’t guarantee your code is free of defects, but it’s a great tool for helping you with that.

Code available on github https://github.com/ottogiron/mochaistanbul

Setting Up your Node.js Unit Tests with Chai And Mocha

In this post I’m going to list the steps for setting up a Node.js project with unit testing.

The Tools

Mocha

Mocha is a JavaScript test framework. You can use Mocha in both server (Node.js) and the browser. Given the asynchronous nature of JavaScript programming model in both Node.js and browser, Mocha provides a neat syntax for performing asynchronous test.

Chai

Chai is a BDD/TDD assertion library.

Chai comes in two flavors, BDD and the clasical TDD assert style. In this how to, I’m going to use the ‘Should’ (BDD) interface, since it’s the one I enjoy the most.

Prerrequisites

Node.js >=0.12 and npm

Setting up your project

• Create a new directory. In your terminal run:

mochachaitest && cd mochachaitest

• Initialize a new Node project. Run ‘npm init’, and follow the steps.

npm init

name: (mochachaitest)
version: (1.0.0)
description:
entry point: (index.js)
test command:
git repository:
keywords:
author: Otto Giron
license: (ISC) MIT

• Install Mocha and Chai. In your terminal run:

npm install mocha chai --save-dev

Note that I’ve added the ‘–save-dev’ flag at the end, since these dependencies will be useful only in your development environment.

Creating a simple test file

By default, Mocha runs *.js test files inside a test folder under your root directory (mochachaitest).

• Create a test folder

mkdir test

• Create a file ‘first-test.js’ under test folder with the following content

// require chai and initialize 'shoud()'
var should = require('chai').should();

//Describe the purpose of your unit tests suite

describe('Showcase of mocha and chai', function() {

it('should show an example of synchronous unit test', function() {

//simple test

var sum = 1 + 1;
//after initializing chai.should() every variable in
//this scope has the special property 'should'
sum.should.be.equal(2);
});

//For an asynchronous test we can add the especial parameter 'done' (you can use any name)
// which is a callback that informs mocha when the test has finished
it('should show an example of an asynchronous unit test', function(done) {

//some async call
setTimeout(function() {

//simple test

var sum = 1 + 1;
sum.should.be.equal(2);
done(); //inform mocha that this test is done
}, 1000);

});

});

Running your tests

• In your package.json edit scripts.test property and set the following value

//package.json file
...
"scripts": {
"test": "node ./node_modules/.bin/mocha"
}

• In your terminal run ‘npm test’

npm test

> mochachaitest@1.0.0 test /home/otto/tutorial/mochachaitest
> node ./node_modules/.bin/mocha
Showcase of mocha and chai
✓ should show an example of synchronous unit test
✓ should show an example of an asynchronous unit test (1005ms)
2 passing (1s)

This will run the script you’ve just added to the package.json file.

Conclusion

There are several options for implementing your tests in Node.js. Check HapiJS Lab it is another test framework I liked. You can choose the option that best suit to your needs, style, and existing tooling. When reviewing your options you might take into account features such asynchronous support, compatible assertion libraries, options for tests code coverage and code style (eslint, jslint etc).

Code for this example is available on github https://github.com/ottogiron/mochachaitest

Waterline: Node.js adapter-based ORM

Waterline is an ORM/ODM (Object Document Mapper), part of the Sails Framework.

Waterline is agnostic of the data store so it allows you to map different types of databases to a common model structure, focusing on business logic for CURD operations, querying records and models relations mapping.

Pros

Standardized Data Access Interface

In a project is always necessary to have a set of standards or architectural guidelines that the developers can follow, so they can focus on business logic. As developers thinking one from many ways of doing a something adds mental load and it makes us waste time. Also depending in our experience, we may end up writing a lot of boilerplate code that might not be even modular and reusable.

Waterline provides a common language the developers can agree on, it also reduces the boilerplate code implementing common database operations.

Flexibility

Imagine that you are working in a project using let’s say Mongo. Then you realize that the company has no previous experience with Mongo, they have invested time and resources in MySQL and they want to stick to that. You have already invested a couple of weeks developing a prototype using Waterline. No problem! since you’re using Waterline you just need to change a couple of configuration lines.

Adapters

An adapter is an interface that maps methods like find() and create() to lower level syntax like «SELECT * FROM» and «INSERT INTO». Sails core team maintains a set of adapters for the most popular databases. A great thing is that you can also contribute with your own adapters, or the existing ones.

Cons

Maturity Or Availability Of Adapters

For some databases adapters might be at early stages, or nonexistent. The good thing is that you can implement your own adapter or contribute existing ones. Right now I’m working on an adapter for Elasticsearch https://github.com/ottogiron/sails-elasticsearch

Example

var waterline = require('waterline');

// Instantiate a new instance of the ORM
var orm = new Waterline();

var config = {
adapters: {
mysql: require('sails-mysql')
},

connections: {
myLocalMySql: {
adapter: 'mysql'
host: 'localhost',
database: 'test'
}
}
};

var User = Waterline.Collection.extend({
identity: 'user',
connection: 'mysql',
attributes: {
first_name: 'string',
last_name: 'string'
}

})

//Load the models
orm.looadCollection(User);

// Start Waterline passing adapters in
orm.initialize(config, function(err, models) {
models.user
.find({first_name: 'something'})
.execute(function(err, result){
// do something
});
})

Find additional examples on how to map models and using them here: http://sailsjs.org/#!/documentation/concepts/ORM

Conclusion

With Waterline you can create an abstraction of different databases, reducing the boilerplate code for common operations, and giving the developers a common data access language they can work with, letting them focus in the business logic.

Weasel: Node.js- Waterline Simple Mocking Library

Weasel is a small nodejs module I wrote for mocking Waterline models. His name is inspired in the famous and beloved weasel woodpecker rider.

About Waterline

Waterline is a brand new kind of storage and retrieval engine.

It provides a uniform API for accessing stuff from different kinds of databases, protocols, and 3rd party APIs. That means you write the same code to get and store things >like users, whether they live in Redis, mySQL, LDAP, MongoDB, or Postgres.
(Waterline Github repository)
https://github.com/balderdashy/waterline

Waterline is an ORM, good examples of known ORM’s are mongoose for nodejs and Hibernate (for Java).

A good thing about Waterline is that you can implement custom «Adapters». Adapters provide an interface for connecting to different data stores. You can implement your own adapter for your favorite database if it doesn’t exists or if an existing adapter doesn’t fit to your needs. You can also contribute to your favorite existing database adapter 🙂 . At the time of this post there are adapters for PostgreSQl, MySQL, MongoDB, Memory, Disks, Twitter, Neo4j among others.

Weasel Quick Example

var weasel = require('weasel');
// Create a model mock instance
var ModelMock = weasel.mock();
// Set results for find method
ModelMock.setResults('find', [
  [{name: 'pecker'}]
]);

ModelMock.find({})
         .where({})
         .populate('anything')
         .exec(function(err, results){
           console.log(results); // [{name: 'pecker'}]
         });

Find out more in the github repository https://github.com/ottogiron/weasel

Web Components

Web components is a set of standards which brings technologies we were used to, but normally provided by frameworks or libraries such angular, ember, knockout (among others). Web Components standard natively supports in the web browser custom HTML elements, HTML imports and templates, also «shadow DOM» which extends the current DOM specification providing boundaries and better encapsulation between DOM trees.

Custom Elements

It let’s the author to define custom DOM Elements. Imagine that you have an application for managing a todo list, with custom elements we might come with something like this:

<todo-list>
    <todo-item value="Learn Haskell></todo-item>
    <todo-item value="Write a blog post></todo-item>
</todo-list>

Benefits

The end of the div soup ?

One of the problems of the way we develop web interfaces using the current HTML standard is that HTML tags don’t cover all the visual and functional requirements we need, so the approach we’ve been using in order to create custom visual components and behavior is a mix of what we already have (divs, spans, inputs etc) and some JavaScript and CSS.

What you see above it’s what some call «the DiV soup» a bunch of nested DIVS which don’t say much about the real purpose of the HTML we are using, this might lead to problems in code maintenance for developers and web accessibility, it also evidences a lack of semantics.

Custom elements will let you define an HTML structure which will represent and transmit what you are actually building.

HTML Imports

HTML imports allows you to reuse your web components accross different pages.

Example

<head>
  <link rel="import" href="/path/to/imports/myCustomElement.html">
</head>
<body>
  <myCustomElement></myCustomElement>
</body>

Templates

We are familiar with existing templating libraries such handlebars, jade, mustache etc. Template engines are important for keeping our view rendering semantic separated from our logic. The use of template engines is already accepted as a good practice when developing web interfaces, so it’s seems natural to have a built in (native) template engine in our browser.

Example

First we need to define a template element:

<template id="template">
  <style>
    ...
  </style>
  <div>
   <h1>Hello Templates</h1>
  </div>
</template>

Then we can actually use template html and append it to the dom.

<script>
  var template = document.querySelector('#template');
  var clone = document.importNode(template.content, true);
  var host = document.querySelector('body');
  host.appendChild(clone);
</script>

Templates don’t support data binding which is a concept normally tied to libraries and frameworks. There are frameworks supporting this kind of abstractions and extra functionality, check Polymer.

Shadow DOM

Shadow DOM provides a way of scoping HTML and CSS. One common issue Shadow DOM address is when we have CSS rules with common names, now we have a way for encapsulate styles of different components so they don’t mess with each other.

Conclusion

Web Components is a new exciting standard supported in modern web browsers bringing a solution to problems we have tried to solve in different ways. The standard provides a good foundation that eventually might be used by the different frameworks we already love and use, and by the frameworks to come.

Apache Thrift Java Server and NodeJS Client

As an exercise for learning Apache Thrift I’m writing a simple API for Apache Jackrabit 3 using this software framework developed by Facebook.  Apache Thrift is useful for writing cross-language services for many languages such Java, Python, PHP, JavaScript and others.

Apache Jackrabit 3 is an implementation of Java Content Repository  and for now it can only be used through It’s Java API (it also has a REST interface I haven’t used),  so the idea is to reproduce more or less the Jackrabbit 3 API more specifically the «Oak API» with Apache Thrift so it can be used on NodeJS.

Thrift Java Server

https://github.com/ottogiron/jcr-oak-rpc/blob/development/jcr-oak-rpc-services/src/test/java/org/jumlabs/jcr/oak/rpc/api/JcrOakThritServerTests.java

For the java server I have used TNonblockingServerSocket transport  and binary protocol, basically we have to register the available service processors, for this I have used TMultiplexedProcessor.

 @Test
 public void testServerServe() throws TTransportException {
    //simple(sessionProcesor);
     TMultiplexedProcessor processor = new TMultiplexedProcessor();
     processor.registerProcessor("TRootService", rootProcessor);
     processor.registerProcessor("TTreeService", treeProcessor);
      nonBlocking(processor);
}

public static void nonBlocking(TProcessor processor) throws TTransportException {
        TNonblockingServerSocket serverTransport = new TNonblockingServerSocket(9090);
        THsHaServer server = new THsHaServer(
                new THsHaServer.Args(serverTransport)
                .processor(processor)
                .protocolFactory(new TBinaryProtocol.Factory(false, false)));
        System.out.println("Starting server on port 9090 ...");
        server.serve();
    }

Thrift JavaScript Client

https://github.com/ottogiron/jcr-oak-rpc/blob/development/nodejs/jcr-oak-api/test/jcr-oak-rpc-api-nodejs_test.js

After running our server, we can consume the registered services.

 testJcrOakClient: function(test) {

        var connection = thrift.createConnection('localhost', 9090, {transport: thrift.TFramedTransport}),
        multiplexer = new  thrift.Multiplexer(),
        root =  multiplexer.createClient('TRootService',TRoot,connection);        

        connection.on('error', function(err) {
            console.error(err);
            test.done();
        });
        var path = "/";
        root.getTree(path, function(error, tree) {
            if (error) {
                console.log(error);
            } else {
                test.ok(tree, 'Session has not returned a result');
                test.equal(tree.path, path, 'The path of the tree should be equal the required path /');
                console.log(tree);
                connection.end();
            }

            test.done();
        });

    }

Please be aware that I’m using the master branch of Apache Thrift, «Multiplexer processor» is not available in 0.9.x branch for NodeJS. The multiplexer allows to register and consume more than one service on a server, otherwise you’ll need a server for each service.

All the code is available on GitHub

https://github.com/ottogiron/jcr-oak-rpc