A Meaningful Client Side Alternative To node require()

TL;DR

Here you have the solution to all your problems ... no? So take a break, and read through :)

Current Status

Using a generic "module loader" for JavaScript files is becoming a common technique, and full of wins, not only on the node.js server side. There are different client side alternatives, not entirely based over the module concept, and apparently only one concrete proposal, called AMD, able to work in both server and client following a build/parsing procedure.
However, as somebody pointed out already ...

AMD is Not the Answer

This post does not even tell everything bad about AMD problems, and I am getting there, but it's surely a good starting point.
Tobie article is another resource that inspired somehow what I am going to propose here, so before discriminating this or that technique, I hope you'll find time to understand the whole problem ... found it? Let's go on then :)

The Beauty Of node.js require Function

I don't think I should even spend many words here ...

// module: compress

// in the global scope, it's just a module
// so no pollution to the global/window object
// no need to put everything in yet another closure
var fs = require("fs");

// just as example
this.compress = function (file, fn) {
  fs
    .createReadStream(file)
    .pipe(
      // check this out!
      require("zlib").createGzip()
    )
    .pipe(
      fs.createWriteStream(file + ".gz")
    ).on("close", function () {
      var content = fs.readFileSync(file + ".gz");
      fs.unlinkSync(file + ".gz");
      fn(content);
    })
  ;
};

So basically, with a synchronous require(module) call we can decide whenever we want where that module should be included in our logic. This means we don't need to think in advance to all needed dependencies for our current module, as well as this doesn't require to split in many nested functions our asynchronous logic.
Look at above example ... in node world the require("fs") is performed basically in every single module ... the FileSystem is that important, hell yeah!
We dont' care much about requiring it at the very beginning of above stupid module 'cause surely that operation will cost nothing! Each module is cached indeed so there's actually zero impact if we use a require hundreds of time inline ... really, it's that fast, but the very first one might cost a bit.
This is why require("zlib") is loaded only once, and only when it's needed ... so that memory, disk, cpu, and everything else, can live in peace before the very first call of that exported method while after that, that call will cost again nothing.

Is a mandatory nested function call through AMD logic as fast as above example? I tell you, NO, but this is not the real problem, is it?

AMD Does Not Truly Scale

... as simple as that. If you need a build process behind something that could fail in any case, I would say you are doing it wrong. I am talking about current era, where internet connection could be there or not ... I am talking about mobile and I give you a very basic example.
I have created my freaking cool website or webapp that loads asynchronously everything and I am on the road with my supa-dupa smart phone.
When "suddenly, le wild spot without internet coverage appears"!
Now I press that lovely button in the lovely application that loads asynchronously the lovely next part of my action and guess what happens ... are you there? Nothing!
That instant without my mobile network will screw up my flow, will break my action, will let me wait "forever" and in most common cases, it will block me to use properly the app.
Of course, if that action required server side interaction, that action won't work in any case ... but how come I don't even see a notification? How come pieces of my application are not showing up telling me that there is actually a problem, rather than letting me wait without any notification, 'couse probably even notification logic was created as AMD module?

And what if connection is simply bad? The whole application or website is responding decently but that part, oh gosh how much I was planning to use that app part, is taking ages to load! How would you feel in front of such program?

AMD Resolves Lazy/Circular Load Synchronously

That's correct ... gotcha! Have you ever read what you should do in order to resolve modules that load asynchronously other modules inside themselves?

//Inside b.js:
define(["require", "a"],
    function(require, a) {
        //"a" in this case will be null if a also asked for b,
        //a circular dependency.
        return function(title) {
            // me: Excuse me ... WUT?
            return require("a").doSomething();
        }
    }
);

So here the thing, if "by accident" you have a circular dependency you should use require() as node.js does ... oh well ...

On Circular Dependencies / Cycles

This topic is one of the biggest paradox about programming ... we try to decouple everything, specially when it comes to write modules, so that not a single part of the app should be aware of the surrounding environment ... isn't it? Then someone said that circular dependencies are bad ... but how come?
Here an example, a truly stupid one:

Hi, I am a human, the human module, and I am completely sufficient, but I need to go in a place that would take too much by my own ... so I need the module car.

Hi, I am a car, the module car, and I am fabulous by my own, but I need the module human to be able to go somewhere.

A partner, better than a car, could also explain the fact we actually think in circular references all the time ... am I wrong?
The AMD take in this case is that "we should change the require logic when this happens and we should be aware in advance in order to solve this" ... yeah, nice, so now in AMD we have two ways to require and return what we export ... and once again, in my humble opinion, this does not scale ... at all!

Double Effort, Double Code!

With AMD we don't only need to change our AMD code/style/logic when things are not known in advance, as showed before ... we also need to write code for modules that is not compatible with node.js, resulting in such piece of redundant code that I believe nobody truly want to write more than once in a programmer life.
Take an excellent library as lodash is, and check what it has to do in order to be compatible with AMD too ...

// expose Lo-Dash
// some AMD build optimizers, like r.js, check for specific condition patterns like the following:
if (typeof define == 'function' && typeof define.amd == 'object' && define.amd) {
  // Expose Lo-Dash to the global object even when an AMD loader is present in
  // case Lo-Dash was injected by a third-party script and not intended to be
  // loaded as a module. The global assignment can be reverted in the Lo-Dash
  // module via its `noConflict()` method.
  window._ = lodash;

  // define as an anonymous module so, through path mapping, it can be
  // referenced as the "underscore" module
  define(function() {
    return lodash;
  });
}
// check for `exports` after `define` in case a build optimizer adds an `exports` object
else if (freeExports) {
  // in Node.js or RingoJS v0.8.0+
  if (typeof module == 'object' && module && module.exports == freeExports) {
    (module.exports = lodash)._ = lodash;
  }
  // in Narwhal or RingoJS v0.7.0-
  else {
    freeExports._ = lodash;
  }
}
else {
  // in a browser or Rhino
  window._ = lodash;
}

Now ... ask yourself, is this what you want to write for every single module you gonna create that might work in both server and client side?

Combined AMD Modules

Here another possibility, super cool ... we can combine all modules together into a single file: YEAH! But if this what you do for your project, don't you wonder what is the point then to use all those "asynchronous ready" callbacks if these will be executed in a synchronous way in production? Was that different syntax truly needed? And what about JS engines parsing time? Is processing the whole project at once in a single file a good thing for both Desktop and Mobile?
Why are you developing asynchronously with all those nested callbacks if you provide a synchronous build? Is the code size affected? Does all this make any sense?

AMD, The Good Part... Maybe

OK, there must be some part of this logic that conquered many developers out there .. and I must admit AMD "solved with nonchalance" the fact JavaScript, in the client side, has always had problems with the global scope pollution.
The fact AMD forces us to write the module inside a function that receives already arguments with other needed modules, is a win ... but wait a second, wasn't that boring before that nobody until now wrote a single bloody closure to avoid global scope pollution?
I think AMD is a side effect, with all possible noble and good purpose, of a general misunderstanding of how is JS code sharing across libraries.
Let's remember we never even thought about modules until we started clashing with all possible each other polluting namespaces, global variables, Object.prototype, and any sort of crap, thinking we are the only script ever running in a web page ... isn't it?
So kudos for AMD, at least there is a function, but where the hack is the "use strict" directive suggested for every single bloody AMD module in any example you can find in the documentation? where is the global pollution problem solver, if developers are not educated or warned about the problem itself?

node.js require ain't gold neither

When network, roundtrips and latency come into the game, the node.js require() solution does not fit, scale, work, neither.
If you understand how Loading from node_modules Folders logic works, and you have an extra look into All Together diagram, you will realize that all those checks, performed through an HTTP connection, won't ever make sense on the client side.
Are we stuck then? Or there's some tiny little brick in the middle that is not used, common, public, or discussed yet?

A node require() Like, For Client Side

Eventually, here where I was heading since the beginning: my require_client proposal ... gosh it took long!
Rewind ... How about having the best from all techniques described "here and there" in order to:

• avoid big files parsing, everything is parsed once and on demand
• provide an easy to use builder for production ready single file code
• use one single syntax/model/style to define modules for both node or client side
• solve cycles exactly as node does
• forget 10 lines of checks to export a bloody module
• organize namespaces in folders
• obtain excellent performance
• make a single file distributable

... and finally, compare results against all other techniques?
Here, the AMD loader, versus the inline and DOM script injection loader, versus the dev version of my proposal, and finally the production/compiled version of my proposal ... how about that? You can use any console, profiler, dev tool/thing you want, to compare results, it's about a 150Kb total amount of scripts with most of them loaded ASAP, and one loaded on "load" event.
You can measure jquery, underscore, backbonejs, and the ironically added as last script head.js script there within their loading/parsing/ready time.

Reading Results

If you think nowadays the DOMContentLoaded event is all you need to startup faster your web page/app, you are probably wrong. DOMContentLoaded event means almost nothing for real UX, because a user that has a DOM ready but can't use, because modules and logic are not loaded yet, or see, because CSS and images have not been resolved yet, the page/app, is simply a "user waiting" rather than interacting and nothing else.
Accordingly, if you consider the code flow and the time everything is ready to be used, the compiled require() method is the best option you have: it's freaking fast!

"use strict" included

The best part I could think about, is the "use strict"; directive by default automatically prepended to any module that is going to be parsed.
This is a huge advantage for client side code because while we are able to create as many var as we want in the module scope, the engine parser and compiler will instantly raise an error the line and column we forgot a variable declaration. All other safer things are in place and working but .. you know, maybe you don't want this?
That's why the require_client compiler makes the strict configuration property easy to spot, configure, and change ... as long as you know why you are doing that, everything is fine.

How Does It Work

The compiler includes a 360 bytes once minzipped function that when is not optimized simply works through XHR.
This function could be the very only global function you need, since every module is evaluated sandboxed and with all node.js module behaviors.
You can export a function itself, you can export a module, you can require inside a module, you can do 90% of what you could do in a node.js environment.
You don't need to take care of global variables definition, those won't affect other modules.
What you should do, is to remember that this is the client so the path you choose in the development version, is the root, as any node_modules folder would be.
If you clone the repository, you can test via copy and paste the resulting build/require.js in whatever browser console traps such require_client ~/folder_you_put_staff/require_client/js or require_client ~/folder_you_put_staff/require_client/cycles.
require("a") in the first case and require("main") in the second.
In order to obtain a similar portable function you should create a folder with all scripts and point to that folder via require_client so that a script with all inclusions will be created.

A Basic Example

So here what's the require_client script is able to produce.
Let's imagine this is our project folder structure:

project/
        css/
        js/
        require.dev.js
        index.html

The index.html file can simply have a single script in its header that includes require.dev.js and the bootstrap module through require("main");, as example.
So, let's imagine we have module a, b, and main inside the js folder, OK?
require_client project/js project/require.js
This call will produce the require.js file such as:

/*! (C) Andrea Giammarchi */
var require=function(c,d,e){function l(n,m){return m?n:g[n]}function b(o){var m=a[o]={},n={id:o,parent:c,filename:l(o,h),web:h};n[k]=m;d("global","module",k,(e.strict?"'use strict';":"")+l(o)).call(m,c,n,m);j.call(m=n[k],i)||(m[i]=h);return m}function f(m){return j.call(a,m)?a[m]:a[m]=b(m)}var k="exports",i="loaded",h=!0,a={},j=a.hasOwnProperty,g={
 "a": "console.log(\"a starting\");exports.done=false;var b=require(\"b\");console.log(\"in a, b.done = \"+b.done);exports.done=true;console.log(\"a done\");",
 "main": "console.log(\"main starting\");var a=require(\"a\");var b=require(\"b\");console.log(\"in main, a.done=\"+a.done+\", b.done=\"+b.done);",
 "b": "console.log(\"b starting\");exports.done=false;var a=require(\"a\");console.log(\"in b, a.done = \"+a.done);exports.done=true;console.log(\"b done\");"
};f.config=e;f.main=c;return f}(this,Function,{
strict:true
});

Now the index.html could simply include reuiqre.js rather than the dev version ;)
Above program is the same showed in node.js API documentation about cycles. If you copy and paste this code in any console and you write after require("main"); you'll see the expected result.
As summary, require_client is able to minify and place inline all your scripts, creating modules names based on files and folders hierarchy.
All modules will be evaluated with a global object, already available, as well as module, exports, and the latter used as the module context.
The simple object based cache system will ensure these modules are evaluated once and never again.

What's YAGNI

Few things, that could change, are not in on purpose. As example, the module.parent is always the global object, since in fact, it's in the global scope, through Function compilation, that the module will be parsed the very first time. Not sure we need a complicated mechanism to chain calls and also this mechanism is error prone.
If you have 2 scripts requiring the same module, first come, first serve. The second one should not affect runtime the already parsed module changing suddenly the module.parent property ... you know what I mean?
The path resolution is a no-go, rather than trying to fix all possible messes with paths and OS, put your files in a single JS folder and consider that one your virtual node_modules one for clients.
If you have folder links inside the JS folder it's OK, but if you have recursive links you are screwed. Please keep it simple while I think how to avoid such problem within the require_client logic, thanks.

What Is Not There Yet

If your project is more than a megabyte minzipped, you might want to be able to split in different chunks the code so that the second, last, injected, require, won't disturb the first one. This is not in place yet since this free time project was born for a small/medium web app I am working on that will be out pretty soon ... an app that surely does not need such amount of code as many other web app should NOT ... but you know, I've been talking about scaling so much that a note about the fact this solution won't scale so nicely with massive projects is a must say.

UpdateJust landed a version that does not cause conflicts with require itself. The first defined require will be the one used everywhere so it's now possible to name a project and include it in the main page so ... parallel projects are now available ;)

If you would like to reuse node modules that work in the client side too, you needto copy them inside the path folder.
The configuration object is the one you can find at the end of the require.js file ... there are two defaults there, but you can always change them via require.config.path = "different"; as well as you can drop the require.config.strict = false; directive so that modules will be evaluated without "use strict"; directive.
Anything else? Dunno ... you might come up with some hint, question, suggestion. And thanks for reading :), I know it was a long one!

Last Thoughts

If AMD and RequireJS comes with a compiler able to make everything already available somehow, think how much pointless become the optimization once you can have already available all dependencies without needing to write JS code in a different way, regardless it's for node.js or the client web normal code.
There are NOT really many excuse to keep polluting the global scope with variables, we have so may alternatives today that keep doing it would result as evil as any worst technique you can embrace in JS world.

Y U NO use libraries and add stuff

This is an early introduction to a project I have been thinking about for a while.
The project is already usable in github but the documentation is lacking all over the place so please be patient and I'll add everything necessary to understand and use yuno.

Zero Stress Namespace And Dependencies Resolver

Let's face the reality: today there is still no standard way to include dependencies in a script.
If we are using a generic JS loader, the aim is to simply download files and eventually wait for one or more dependency in order to be able to use everything we need.
The require logic introduced via node.js does not scale in the browser due synchronous nature of the method itself plus the sandbox not that easy to emulate in a browser environment.
The AMD concept is kinda OKish but once we load after dependencies, there is no way to implement a new one within the callback unless we are not exporting.
I find AMD approach surely the most convenient but still not the best one:

1. we cannot implement a provide like procedure, whenever this could be handy or not
2. it's not clear within the module code itself, what we are exporting exactly

Specially the last point means that AMD does not scale properly with already combined code because AMD relies in the module/folder structure itself ... so, cannot we do anything better than what we have so far?

The yuno Concept

Directly out of a well known meme, yuno logic is quite straightforward:

• automagically resolved path, you point once to yuno.js file in your page and you are ready to go
• compatible with already combined files (smart builder coming soon)
• yuno.use() semantic method to define dependencies, if necessary
• yuno.use().and() resolved callback to receive modules AMD style once everything has been loaded
• yuno.add() standard ES5 way to define new namespaces, objects, properties, or constructors ( so no extra note in the documentation is needed )
• cross referenced dependencies automagically resolved: if two different scripts needs same library, this will be loaded once for both
• external url compatible, because you may want to include a file from some known CDN rather than put all scripts in your own host ( speed up common libraries download across different libraries that depend on same core, e.g/ jQuery )
• modules, namespaces, or global objects, cannot be reassigned twice, which means if we are adding twice same thing we are doing it wrong, but if we are not aware of other script that added same thing before we have a notification
• something else I may decide to add after this post

Here some example:

// define a jQuery plugin
yuno.use(
  "jQuery",
  "extraStuff"
).and(function (jQuery, extraStuff) {
  yuno.add(jQuery.fn, "myPlugin", {value:function () {
    // your amazing code here
  }});
  // we may opt for just this line
  jQuery.fn.myPlugin = function () {};
  // in order to export our plugin
  // however, the purpose of yuno is to have
  // a common recognizable way to understand
  // what the module is about
  // plus the "add" method is safer
});

Let's imagine that extraStuff contains similar code:

// define extraStuff
yuno.use(
  "jQuery"
).and(function (jQuery) {
  yuno.add(jQuery.fn, "extraStuff", {value:function () {
    // your amazing code here
  }});
});

Both plugin and extraStuff needs jQuery to be executed ... will jQuery be loaded twice? Nope, it's simply part of a queue of modules that needs to be resolved.
As soon as it's loaded/added once, every module that depends on jQuery will be notified so that if the list of dependencies is fully loaded, the callback passed to and will be executed.

Y U NO Add

Modules are only one part of the proposal since we may define a script where no external dependency is needed.

// note: no external dependency, just add
yuno.add(this, "MyFreakingCoolConstructor", {value:
  function MyFreakingCoolConstructor() {
    // freaking cool stuff here
  }
});
// this points to the global object so that ...
MyFreakingCoolConstructor.prototype.doStuff = function () {
  // freaking cool method
};

The yuno.add method reflects Object.defineProperty which means for ES5 compatible browsers getters, setters, and values, are all accepted and flagged as not enumerable, not writable, and not configurable by default.
Of course we can re-define this behavior but most likely this is what we need/want as default in any case ... isn't it?
For those browsers not there yet, the Object.defineProperty method is partially shimmed where __defineGetter/Setter__ or simply the value property will be used instead.
Bear in mind this shim may change accordingly with real needs but so far all mobile browsers should work as expected plus all Desktop browsers except IE less than 9 ... not so common targets for modern web sites.
Last, but not least, yuno logic does not necessarily need the add call so feel free to simply define your global object or your namespace the way you want.
However, as I have said before, the add method is a simple call able to make things more robust, to speedup and ensure notifications, and to use a standard, recognizable pattern, to define our own objects/functions being sure nobody did before thanks to defaults descriptor behavior which is not writable and not configurable indeed.

To DOs

This is just an initial idea of what the yuno object is able to do but few things are in my mind. On top of the list we have the possibility to shortener CDN calls via prefixes such "cdn:jQuery", as example, in order to use most common CDNs to load widely shared libraries.
Last, but not least, the reason I am writing this is because I am personally not that happy with any solution we have out there so if you are willing to contribute, please just leave a comment, thanks.

[Q&D] Fix Decimal Module Operator

This is a Q&D solution for some JavaScript precision problem when we use the module operator with floating point numbers.

The Problem

If we are dealing with the module operator we could have this case:

alert(
    8.25 % .05
);
// 0.049999999999999545 

We expected 0 but we have a floating point inconsistency instead. This is a well known problem, and not only with JavaScript, truly annoying when we have to show run-time computed money/trends operations.

The Solution

This comes from my reply in a JS forum, knowing that the module will be a simply floating point, let's say .1 to .001, if we remove floating point from the right side of the operation everything works just fine:

function mod(num, mod){
    // Another WebReflection silly idea
    var pow = Math.pow(10, (("" + mod).split(".")[1] || "").length);
    return ((num * pow) % (mod * pow)) / pow;
};

alert(
    mod(8.25, .05)
);  // 0


alert(
    mod(8.25, .1)
);  // 0.5

That's it, have a nice Sunday