Thinking Asynchronously in CoffeeScript/JavaScript: Loops and Callbacks

Awhile back, I wrote about my new experience in learning Node.js: A Node.js Experiment: Thinking Asynchronously, Using Recursion to Calculate the Total File Size in a Directory.

Consider this snippet of code:

var names = ['JP', 'Chris', 'Leslie'];
for (var i = 0; i < names.length; ++i){
  var name = names[i];
  setTimeout(function(){
    alert(name);              
  },10);
}​

Equivalent CoffeeScript:

names = ['JP', 'Chris', 'Leslie']
for name in names
  setTimeout(->
    alert(name)
  ,10)

Click here to run it.

If you guessed that the loop would alert “Leslie” three times, then you’d be correct.

The problem is, that before the callback executes, the loop has completed. Thus callback always has the last value.

How do you solve this problem? You wrap the callback in a closure that executes immediately.

JavaScript:

var names = ['JP', 'Chris', 'Leslie'];
for (var i = 0; i < names.length; ++i){
  var name = names[i];
  (function(name){
    setTimeout(function(){
      alert(name);              
    },10);
  })(name);
}​

CoffeeScript:

names = ['JP', 'Chris', 'Leslie']
for name in names
  do (name) ->
    setTimeout(->
      alert(name)
    ,10)

Click here to run it.

These solutions execute the block of code in a parallel manner. Using the alert’s are not a good indication in showing this behavior. However, if you were opening files, all of them would be opened approximately (not exactly) at the same time.

What if you wanted to perform the action in the callback in a serial manner?

Using the previous simple example, it’d look like this:

JavaScript:

var names = ['JP', 'Chris', 'Leslie'];
loop = function(i){
    setTimeout(function(){
      alert(names[i]);
      if (i < names.length - 1)
        loop(i + 1);       
    },10);
}
loop(0);

CoffeeScript:

names = ['JP', 'Chris', 'Leslie'];
doloop = (i) ->
  setTimeout(->
    alert(names[i])
    if i < names.length - 1
      doloop(i + 1)       
  ,10);
doloop(0)

Run it.

If you were doing file processing in the loop, it would be executed sequentially.

Hopefully this helps you to better understand asynchronous design of algorithms in JavaScript.

Update:
I forgot about the forEach function that exists in Node.js and most modern browsers. This function pretty much solves the problem.

Here’s the JavaScript code:

var names = ['JP', 'Chris', 'Leslie'];
names.forEach(function(name){
  setTimeout(function(){
    alert(name);              
  },10);
}​);

Much cleaner. Thanks to smog_alado [Reddit] for the reminder.

Checkout Gitpilot, a different kind of Git GUI.

Follow me on Twitter: @jprichardson

-JP

Why Do All the Great Node.js Developers Hate CoffeeScript?

Why do all the great Node.js developers hate CoffeeScript?

Take a look at the following Github repositories of the well-known Node.js developers:

Did you look at them? Not one of them has a project (that isn’t forked) that is written in CoffeeScript. So does the absence of CoffeeScript on Github imply these developers hate it? Absolutely not. Listen to episode 18 or 19 of Nodeup (don’t remember which one) but there are a couple of instances where they (expert Node.js devs) joke and laugh about writing in CoffeeScript. If this offensive? Of course not. But the attitude is curious to me.

One of the aforementioned developers said the following about a technology:

What if we could omit braces? How about semi-colons?

Sounds like the developer is talking about CoffeeScript, doesn’t it? No, it was TJ Holowaychuk describing Stylus, his CSS replacement language. Look at Stylus, look how CoffeeScript-esque it is. This is the the same TJ that doesn’t like CoffeeScript. This is meant to be partially tongue & cheek, but it does lend credance to my point.

Can you guess what the second most depended-upon package is on NPM? If you guessed CoffeeScript, you’d be right!

So if it’s the second most depended-upon package, it must be in use by us mere-mortal developers. Having defected from Rails, I love CoffeeScript. But, I ask again, why do the greats have a haughty attitude towards CoffeeScript? This isn’t meant to be a crusade trying to get people to convert to the holier-than-though CoffeeScript, but a genuine lack of understanding of why the disdain exists. Especially given the acceptance towards Haml, SASS, SCSS, Jade, etc. I mean, when it comes down to it, write in whatever makes you happy, but I feel like I’m missing something. If you’re part of the Node.js community, you’ll know what I’m talking about.

Looking over the CoffeeScript page, I think that you can safely conclude that in general, you’ll write less lines of code using CoffeeScript. Code is our enemy so that’s a good thing.

What do you think about CoffeeScript? Why do you think these developers don’t like CoffeeScript?

More fun CoffeeScript hatred:

If you use Git with others, you should checkout Gitpilot to make collaboration with Git simple. We would love your advice.

If you made it this far, follow me on Twitter: @jprichardson

-JP

Quick and Dirty Screen Scraping with Node.js using Request and Cheerio

I wrote my own screen scraping module built on PhantomJS, but unfortunately it’s too slow for most screen scraping tasks that don’t require browser-side JavaScript. One easy way to scrape pages with Node.js is to use Request and Cheerio.

Here is an example of scraping Bing to get all of the search results:

var request = require('request');
var cheerio = require('cheerio');

var searchTerm = 'screen+scraping';
var url = 'http://www.bing.com/search?q=' + searchTerm;

request(url, function(err, resp, body){
  $ = cheerio.load(body);
  links = $('.sb_tlst h3 a'); //use your CSS selector here
  $(links).each(function(i, link){
    console.log($(link).text() + ':\n  ' + $(link).attr('href'));
  });
});

Cheerio acts a jQuery replacement for a lot of jQuery tasks. It doesn’t replicate jQuery in every way, and most importantly it’s not meant for the browser but for the server. But it beats the pants off of the jsdom/jQuery combo for screen scraping.

Do you use Git? If so, checkout Gitpilot to make collaborating on software development easy.

You should follow me on Twitter: @jprichardson.

-JP

Comparing Two Javascript Objects

Recently, I was faced with a problem where I needed to compare two Javascript objects. My initial strategy was to convert them to JSON and compare the JSON strings.

Sort of like this:

var a = JSON.stringify(person1);//'{"firstName":"JP","lastName":"Richardson"}'
var b = JSON.stringify(person2);//'{"firstName":"JP","lastName":"Richardson"}'

assert(a === b);

Simple enough, right?

Not so fast. I encountered a case like this:

var a = JSON.stringify(person1);//'{"firstName":"JP","lastName":"Richardson"}'
var b = JSON.stringify(person2);//'{"lastName":"Richardson","firstName":"JP"}'

assert(a === b);

The data is the same, but the string is different. Fortunately, Stackoverflow had a nice Javascript object comparison algorithm to dump into my app.

Object.prototype.equals = function(x)
{
  var p;
  for(p in this) {
      if(typeof(x[p])=='undefined') {return false;}
  }

  for(p in this) {
      if (this[p]) {
          switch(typeof(this[p])) {
              case 'object':
                  if (!this[p].equals(x[p])) { return false; } break;
              case 'function':
                  if (typeof(x[p])=='undefined' ||
                      (p != 'equals' && this[p].toString() != x[p].toString()))
                      return false;
                  break;
              default:
                  if (this[p] != x[p]) { return false; }
          }
      } else {
          if (x[p])
              return false;
      }
  }

  for(p in x) {
      if(typeof(this[p])=='undefined') {return false;}
  }

  return true;
}

Test passed. I eventually hit a situation where I had some code with an Object that had a Person prototype and some data that came from JSON. Kinda like this:

var person1 = new Person('JP', 'Richardson');
var person2 = JSON.parse('{"firstName":"JP","lastName":"Richardson"}');

//deepEquals is code snippet above ^
person1.deepEquals(person2); // <--- THIS FAILS

I only cared about comparing the data. The methods associated with the object (Prototype) didn’t matter. Let’s modify the above algorithm. I use CoffeeScript. Here’s the modification:

Object::jsonEquals = (x) ->
  #we do this because two objects may have the same data fields and data but different prototypes
  x1 = JSON.parse(JSON.stringify(this))
  x2 = JSON.parse(JSON.stringify(x))

  p = null
  for p of x1
    return false if typeof (x2[p]) is 'undefined'
  for p of x1
    if x1[p]
      switch typeof (x1[p])
        when 'object'
          return false unless x1[p].jsonEquals(x2[p])
        when 'function'
          return false if typeof (x2[p]) is 'undefined' or (p isnt 'equals' and x1[p].toString() isnt x2[p].toString())
        else
          return false  unless x1[p] is x2[p]
    else
      return false if x2[p]
  for p of x2
    return false if typeof (x1[p]) is 'undefined'
  true

This causes the situation like I described above to pass. Essentially convert to JSON to remove the prototype. I suppose you could make this more efficient my just manually setting the prototype to Object before doing the comparison, but oh well this works for the time being.

Do you use Git? If so, checkout Gitpilot to make project management and collaborating on projects seamless.

Follow me on Twitter: @jprichardson and read my blog on entrepreneurship: Techneur.

-JP Richardson

Node.js Exec Like Ruby Exec and Writing a Node.js Native Add On Module

Recently, I was faced with a problem that required my Node.js programs process to execute another process and have the procoess that’s passed to the exec function completely replace the Node.js process. In short, I wanted an ‘exec’ function like Ruby’s ‘exec’ function. Unfortunately, out of the box, Node.js doesn’t support this functionality. I asked on Stackoverflow.com, and someone had a response that I should use the POSIX exec functions to solve my problem and to consider writing a native Node.js extension.


npm install kexec

You can then use it like:

var kexec = require('kexec');
kexec('top'); //you can pass any process that you want here

Here is the C++ source for Node Kexec:


#include <v8.h>
#include <node.h>
#include <cstdio>

//#ifdef __POSIX__
#include <unistd.h>
/*#else
#include <process.h>
#endif*/

using namespace node;
using namespace v8;

static Handle<Value> kexec(const Arguments& args) {
    String::Utf8Value v8str(args[0]);
    char* argv2[] = {"", "-c", *v8str, NULL};

    execvp("/bin/sh", argv2);      
    return Undefined();
}

extern "C" {
    static void init (Handle<Object> target) {
        NODE_SET_METHOD(target, "kexec", kexec);
    }

    NODE_MODULE(kexec, init);
}

As you can see, writing a C++ add on in Node.js isn’t too difficult. You can use it in your Node.js Javascript like so:

var kexec;

try {
  kexec = require("./build/default/kexec.node"); //Node.js v0.4
} catch(e) {
  kexec = require("./build/Release/kexec.node"); //Node.js v0.6
}

module.exports = kexec.kexec; //function of kexec module is named kexec

Don’t forget your wscript file, which ironically is Python code:

def set_options(opt):
  opt.tool_options("compiler_cxx")

def configure(conf):
  conf.check_tool("compiler_cxx")
  conf.check_tool("node_addon")

def build(bld):
  obj = bld.new_task_gen("cxx", "shlib", "node_addon") 
  obj.cxxflags = ["-g", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE","-Wall"]
  obj.target = "kexec"
  obj.source = "src/node_kexec.cpp"

In your package.json, include this bit:

"scripts": { "install": "node-waf configure build" }

Github Sourcecode: Node.js kernel exec

I’ve also included other resources for writing a Node.js Native Add On Module:

  1. Google V8 Engine Getting Started
  2. Google V8 Embedder’s Guide
  3. How to Roll Your Own Javascript API with V8
  4. How to Write Your Own Native Node.js Extension
  5. Writing Node.js Native Extensions
  6. Node.js Native Extension with Hammer and a Prayer
  7. Mastering Node; Add Ons
  8. Node.js Documentation: Add Ons
  9. Postgres Node.js Module
  10. V8 Sample: shell.cc
  11. V8 Objects
  12. There’s C in My JavaScript
  13. Converting V8 Arguments to C++ Types

Do you use Git? If so, checkout Gitpilot to make using Git easy.

Follow me on Twitter: @jprichardson and read my blog on entrepreneurship: Techneur.

-JP Richardson

A for-loop conversion from JavaScript to CoffeeScript

I was stumbling along StackOverflow when I ran into this question:

How can I convert a Javascript for-loop to Coffee? With this example:

for (i = 0; i < 10; i++) {
    doStuff();
}

The answer: (I’ve since edited and updated the Stackoverflow answer)

doStuff() for i in [0 .. 9]

http://jashkenas.github.com/coffee-script/#loops

This answer works for this contrived case.

What happens when you have a case like this:

for (var x = 0; x < myArray.length; ++x)
  alert(x);

What’s the answer smarty pants?
If you answered:

alert(i) for i in [0 .. myArray.length-1]

You get a pass if myArray has anything in it. What will happen if myArray is an empty (not null) array?
Go ahead and execute it. Go here to the CoffeeScript page, and paste this code in the “Try CoffeeScript”, I’ll wait.

myArray = []
alert(i) for i in [0 .. myArray.length-1]

What’s that? You say it returned “0″ and “-1″???

The secret is in the ".." which implies "=>" or "<=" 
and the "..." implies "&lt" or "&gt".

So this is probably what you expect:

myArray = []
alert(i) for i in [0 ... myArray.length]

It’s too bad the loops section at this time doesn’t mention this.

CoffeeScript is a great tool. Here is another tool that allows you to convert JavaScript to CoffeeScript. Try our examples here and see what the tool generates. Hint: it isn’t always optimized.

Do you use Git? If so, checkout Gitpilot to make using Git thoughtless.

Follow me on Twitter: @jprichardson and read my blog on entrepreneurship: Techneur.

-JP Richardson

Follow

Get every new post delivered to your Inbox.