I live in the Hayes Valley neighborhood of San Francisco. While it does seem neat, clean, and nice, if you pull things back and look beyond the gentrification, the boutiques, the parks, the disappeared freeway, and the earthquake – back into the 80’s – you would find the identity and experience of this neighborhood completely foreign. Back then the neighborhood (but, honestly, most parts of most cities) was pretty edgy. Indeed, in 1979, Chris Pirsig (from ZAMM) was murdered a few blocks away from my apartment.

However, despite all of the drama of the moment, in history, most events are forgotten and stories remain untold. I’d reserve judgment on whether or not this is a good thing, but, regardless, this does make the discovery of those stories, regardless of provenance, something to behold.

Anyways, not to make to too much of it, a while ago I dug up a short story about growing up in this neighborhood – way before I ever stepped foot in it – and I think it’s pretty interesting:

Chronicles Of A True Hustler: Pt. 1, Pt. 2, Pt. 3, Pt. 4, Pt. 5, Pt. 6

In my case, I needed to present some information on the toolbar, but needed it to be non-interactive, have a special layout, and use some existing vector images. To that end, here are some category methods that made it straight forward: https://gist.github.com/2465637.

Given that, etching can be had with a few lines. It’s not efficient, but it is easy.

[[[[UIImage ct_imageWithPDFNamed:@"time" size:CGSizeMake(26, 26)]
  ct_imageWithOverlayColor:[UIColor colorWithRed:112/255. green:119/255. blue:123/255. alpha:1]]
  ct_imageWithInnerShadowOffset:CGSizeMake(0, 1) color:[UIColor colorWithWhite:0 alpha:.6]]
  ct_imageWithShadowOffset:CGSizeMake(0, 1) blur:0 color:[UIColor colorWithWhite:1 alpha:.6]]

It is my weak protest against the garish implementations that litter the App Store. It’s not something I’d plan to profit from (game development has always seemed to be an inevitably feral existence), but an argument for quality in the App Store, where barring a few exceptions it can seem like there is so little.

…It was also fun to do.

† How many times have I heard that said before?

This isn’t an unrecognized problem and it has been solved elsewhere, but the accepted approach seems require JavaScript and class names, which strikes me as unnecessary – CSS declarations alone should suffice.

To that end, the approach I use is this:

@media only screen and -webkit-min-device-pixel-ratio:2 {
  img[src="images/icon.png"] {
    content: url("images/icon@2x.png");
  }
  img[src="images/store_small.png"] {
    content: url("images/store_small@2x.png");
  }
  img[src="images/screen_fail.png"] {
    content: url("images/screen_fail@2x.png");
  }
  img[src="images/screen_play.png"] {
    content: url("images/screen_play@2x.png");
  }
}

*I swear I read about content negotiation for this at some point, but all I can find is an old, unresolved discussion at webkit.org.

Principles

This work is related to that done in RequireJS and CommonJS, but hardly bound by them. Instead, the results are a product of the following principles:

• Improving the loading characteristics of a JavaScript project should be approached as an incremental optimization problem.
• Simplicity is best. Only implement the necessary features.
• Caching should be exploited. Expensive one-time operations are acceptable provided their responses are reusable.
• Modules are well defined, widely used, and well founded. The five different asynchronous loading specifications, not so much.

This leads to what I’m given to think is a much simpler version that works than existing implementations. The other nice thing is that the packaging tool takes an original approach to solving the dependency issue.

Observations

Optimization

Naïve implementations are good. Those implementations may be slow, but they are also cheap and set the stage for proper optimization. Chances are, that many possible optimizations are rendered unnecessary by the right tools.

Dependencies

Most current implementations use one of five ways to wrap a module’s code with a description of the dependencies that that code requires, and which a library will fetch asynchronously, finally evaluating the modules code once all are loaded. The thought being that, once that module is received, first all of its dependencies need loading (asynchronously so they are non-blocking, natch).

IMHO, that obscures the more obvious and important observation, that having dependencies that aren’t loaded by the time the current module is loaded, asynchronous or not, is never good. If this is to be treated as an optimization problem, then the issue is one of packaging. If the packaging works well, the question of synchronous/asynchronous loading is moot.

Packaging

Existing packagers all perform some sort of parsing on source files, usually a regular expression, maybe a full preprocessor language. Both approaches have the downsides of requiring boilerplate code or being unreliable. There is also the additional complication of describing lazy dependencies so that they do not get confused with loading dependencies.

The good news is that, given the availability of non-browser interpreters, there is a third way, where the code itself can be evaluated offline and dependencies extracted during run-time. Not only would this extract only those dependencies needed exactly at load time and require no boilerplate, using the same kernel in both environments, it would keep both the client and the packager’s results consistent.

Versioning

The current practice for deploying JavaScript is to set a query parameter like bust=v_n+1 on the URL of the script’s location to, in effect, invalidate the cache. This happens to work in the monolithic file case, however, lazy loading code makes versioning a problem that cannot be ignored. While new clients will use v_n+1, clients using v_n code must continue to receive v_n code. For this reason, versioning should be reflected in the base URI.

http://assets1.example.com/js/src/n/
http://assets1.example.com/js/src/n+1/
http://assets1.example.com/js/src/n+2/

Caching

It’s a common observation that different areas of a project change at different rates. This is certainly the case in web applications where library code will change much slower than application code. It follows then that updates to application code should have no effect on still cacheable library code.

Convention already specifies this using a leading slash for '/application/code' and none for 'library/code'. This is simple to exploit by allowing different URIs for the two classes of code.

http://assets1.example.com/js/lib/0.1.2/
http://assets1.example.com/js/src/0.3.0/

Implementation

The tool that implements this loader provides two things things, a kernel and a module compiler. For the moment it is on an experimental branch of the Modulizer project, though I’m beginning to like “Yajsml” more and more.

The kernel is a terse bit of JS that provides the module loading and fetching functionality. It has no references to the global environment and, by default, exports to the require symbol. It’s the part that enables a simple page like this:

<script type="text/javascript" src="/kernel.js"></script>
<script type="text/javascript">
  require.setRootURI('/js/src/');
  require.setLibraryURI('/js/lib/');
  require.setGlobalKeyPath('require');
  app = new (require('/app').Application)({
    "userId": 1234
  , "baseURI": "http://example.com/"
  });
</script>

The module compiler takes a number of paths and compiles them into a require.define() call. Using the command line tool:

../modulize --output code.js --root-path ./src --library-path ./lib --import-dependencies -- ./src/app.js

Produces the following package:
require.define({
  "/app": null
, "/app.js": function (require, exports, module) {
    var models = require('/models');
    var util = require('util');
  }
, "/models": null
, "/models.js": null
, "/models/group": null
, "/models/group.js": function (require, exports, module) {
    exports.Group = function () {
      /.../
    };
  }
, "/models/index": null
, "/models/index.js": function (require, exports, module) {
    exports.User = require('./user').User;
    exports.Group = require('./group').Group;
  }
, "/models/user": null
, "/models/user.js": function (require, exports, module) {
    exports.User = function () {
      /.../
    };
  }
, "util": null
, "util.js": null
, "util/index": null
, "util/index.js": function (require, exports, module) {
    exports.escapeHTML = function () {};
    exports.escapeHTMLAttribute = function () {};
    exports.importantURL = 'http://example.com/';
  }
});

Future

This the first iteration in a longer project with several more big ideas to adopt, but, IMHO, aside from one or two missing features, this is a pretty comprehensive solution for the problem of distributing code from the client’s perspective. The remaining improvements revolve around improving the optimization of packaging and using the cache more effectively.

Regarding effective use of the cache, having module requests get redirected to designated/canonical packages has lots of potential to increase cache hits when loading order varies – such as across pages. As far as finding an optimal packaging goes, it’s the kind of problem that sounds like the perfect job for some sort of nondeterministic heuristic-ish algorithm.

And finally, while the two buckets, libraryURI and rootURI, are probably sufficient for most projects, the thought of allowing for multiple library paths is appealing. Searching would of course be made more expensive for some modules, but I suspect that ordering the search paths by increasing frequency of updates may allow caching to compensate for this.

Updates

It’s since become clear that parts of this discussion are reasonably independent of each other, so they’ve been broken into their own projects:

• require-kernel: A minimalist implementation of require that supports asynchronous retrieval.
• yajsml: An asset server that performs packaging and clever things like redirecting to a canonical resource.
• modulizer: A tool that finds dependencies at runtime.

var global = (typeof window != 'undefined' ? window : global)

Below is a snippet that will return the global object independent of scope and interpreter.

var global = (function () {return this})();

Note: except in the rarest of cases, direct address of the global object is illegitimate regardless of approach, using this more robust snippet is no excuse.

• Birds, reptiles and amphibians only have one orifice for their intestinal and urinary tract. This orifice is called the Cloaca, from the Latin for sewer. If that’s not cool enough get this: some turtles have gills in their cloacal cavity.

• Belt Drives are cool! Because of the repetitive nature of their use, wear is a major concern. Thankfully there are clever ways to reduce the wear. 1) flat belts can be made into a Möbius strip. 2) timing belts can made so that the number of teeth on the belt and all of the sprokets are coprime.

• Gastropoda have diversified so much over geological time that evolutionarily things have gotten very confused. There are land species with gills as well as marine species with lungs. Slugs (Gastropoda without much of a shell) are also interesting because most go through torsion where the internals and the shell of the gastropod rotate. This can be problematic since it puts the anus right by the mouth – but evolutionarily speaking it must of been helpful for something.

• The parasitic flatworm Leucochloridium paradoxum is an amazing example of aggressive mimicry. The flatworm’s sporocyst grows within a snail, invading the tentacles of the snail, with a preference to the left. The now swollen and colored tentacle mimics the appearance of a caterpillar – which a bird host eats. A snail then consumes the birds feces completing the cycle. Some interesting bits about the infected tentacle’s effect on the snail: 1) it cannot retract into its shell, leaving the flatworm always exposed. 2) it appears to reduce light sensitivity in the snail which keeps it out in the open in view of potential hosts.

• The Tempest Prognosticator was a curious invention from the 1850’s. It was a barometer made of leeches. The inventor, George Merryweather, found that leeches would respond to changes in the atmosphere by climbing to higher ground. It consisted of a number of leaches each in their own glass jar, at the top of which was a piece of whale bone tied to a string. [diagram] When the leach would climb the jar it would dislodge the whale bone which would pull a mallet which would strike a bell – you would guess the likelihood of a storm by the number of bell rings. He had lobbied for it to be distributed throughout the coast of England, but the British Crown decided instead on the (still not entirely understood) storm glass.

• Neurocysticercosis is a condition caused by the infestation of the brain by the tapeworm Taenia solium. If you aren’t afraid of undercooked meat yet, I think you might now – the usual vector for infection is through larvae in pork meat (though autoinfection through vomit or feces is also possible). Treatment of Neurocysticercosis can be very problematic. The problem is that living cysticerci are typically asymptomatic, the complications arise when the organism dies, triggering the host’s immune response and leading to the inflammation, scarring, etc. that can lead to seizures, paralysis, and death.

• While all flies will occasionally mistake fetid wounds for dead flesh, Cochliomyia hominivorax, a spechies of screwworm fly, is notable for targeting fresh wounds and even exposed skin (e.g. the navel) for laying their larvae, causing Myiasis. The larvae are also very aggressive, if disturbed they will burrow further into the host, hence their name. They are known to infect humans and are also a major concern for merino sheep, where the condition is called flystrike. However, before you get too worried, know this – that the fly caused so much destruction to American cattle herds that the sterile insect technique was developed as a means of controlling the screw worm. This led to its eradication in the United States in 1982.

It started with my work on LaserLine. I had written a parser for ILDA files, but needed a way to test its output. Naturally, I couldn’t accept Aqua’s presence in an app associated with an app for Lasers – which are all about bright lights and dark rooms. Granted, it was only two controls and some custom images, but by the time I was done there was no doubt in my or anybody else’s mind that the final app would be “dark”.

Knowing that a complete app would need a more complete library of widgets and subclassing AppKit seemed like a good way to get intimately acquainted with its internals, I didn’t stop there. A full accounting of my adventures in AppKit really deserves its own (lengthy) post that I’ll for later, I will just say this — kids, don’t try this at home. Even I, in my pixel perfecting, eye candy licking, gradient loving glory, will admit that the whole endeavor was likely not worth the time and frustration involved. But the work was done, so I might as well publish it.

Disclaimer: DarkKit is full of sketchy code. It began as a hack – and it still is a hack. It does a whole lot of things AppKit doesn’t want you do to; a whole lot of things you shouldn’t do. It uses private, undocumented methods and even shadier things (IIRC there was toying with a super’s instance variables somewhere). Keep this in mind if you think you might need help from AppKit folks – I doubt abusing their framework will endear them to you. Use at your own risk.

What does it cover? not everything, but hopefully enough to be kinda useful. Most of the controls have their dark alternatives and DKButton covers some of the button variants present in AppKit (the normal shiny capsule button, the square beveled button, and square gradient button). Where things get iffy are Views… NSScrollViews were not designed with alternative looks in mind (not that that’s a bad thing) and that causes problems when it comes to TableViews, etc.

Using DarkKit

If you’ve ever gone about working out in the dark, then you probably have a good appreciation for one of the reasons I decided DarkKit would be a nice thing to have. When everything around you is pitch black, the fluorescent glow of the computer’s screen stresses your eyes terribly, so much so that when I’d be typesetting text or some other project that extended way into the night I would regularly enable high-contrast mode (⌃⌥⌘8) to give my eyes a chance to relax. So, if you happen to be designing an application that is going to be used in low-light settings (a laser show for example) a good low-light interface approaches the point of being a requirement.

IMHO, a really dark interface also does an excellent job of placing content front-and-center. The huge contrast between your app’s monochrome controls and your app’s colorful/interesting content tells your eyes exactly what to look at and drastically reduces visual clutter.

So yea, if you happen to be doing a pro-thing (and you accept the disclaimer from earlier) DarkKit might be for you. And, if you hadn’t noticed yet, jet black interfaces seem to have a way of kicking an app’s sex appeal up a notch. So, if on the other hand you find yourself looking for cheap UI pixie dust DarkKit might also be helpful – not that I endorse pixie dust… just sayin’.

Seeing that doing something as damaging to you application’s stability, maintainability, and general code-quality as adding an illegitimate framework like DarkKit should only be done with the greatest care and after considered thought, I built a IBPlugin* so you can build all your dark interfaces with minimal effort right from interface builder. Enjoy!

*There was also a palette, back when that was cool…

» Continue on to svn.oofn.net

I’m my past post I forgot to mention just how significant distraction can be to keeping projects from getting finished. Well, a bit ago I got Atmel’s AT90USBKey and last week, while up in a cabin in Wisconsin’s northwoods, I indulged my sentimental side and started reading Thoreau’s Walden. I’ll plan on getting back on the wagon, just not right now…