It’s just too many clicks to leave the ol’ feed reader to see an article and save it to del.icio.us. I’m not into that kind of commitment. I am, however, into Greasemonkey so I headed to userscripts.org and discovered two fairly poor attempts to do what I wanted. The first one just took me to the del.icio.us/post page which is a different kind of bad solution. The second one displayed a nice box in the page for editing the info before saving away to del.icio.us but it appeared halfway off of my screen and I didn’t have the patience to figure out why. Here are those two:

• http://userscripts.org/scripts/show/11233
• http://userscripts.org/scripts/show/7957

Mine has many virtues. First off it actually works and on any browser window size larger than 422 pixels. Secondly it is visually a bit more disruptive than the small Google-styled “window” that one displays but not so obnoxious as to take you off the page entirely. And finally, it automatically pulls the title, link and selected text from the article. Fully featured! As a bonus you get the timeless, classic and beautiful feel of my favorite color, #eee.

Because of the way del.icio.us’ API works, it’ll ask you for your username and password the first time through but after that should behave nicely.

http://svn.rcrowley.org/svn/greasemonkey/readelicious.user.js or http://userscripts.org/scripts/show/22494

Why can’t XULRunner do this already?

Without really saying what “this” is, Mozilla wants XULRunner to be the standard play for cross-platform GUI applications and yet it has a bit of trouble interacting with 3rd party libraries. Most mature libraries (say, GraphicsMagick) still use char * or std::string to represent file paths. To interact with them, then, you have to be able to get to a char * representation of files you’re trying to open.

Windows and Mac character encodings 101

As is becoming tradition, Windows makes my life difficult by using UTF-16 as its Unicode of choice. UTF-16 uses 2 bytes (usually) to represent each character, meaning it isn’t binary compatible with ASCII, which uses 1 byte per character. These 2-byte characters are stored in wchar_ts which are twice as wide as chars. It isn’t possible to pass a wchar_t * through normal C libraries that expect strings as char *.

Mac OS X uses UTF-8 internally and is my hero. UTF-8 is used internally to represent strings and has the advantage of using single bytes (that’s char for C programmers playing along) as its base unit. While a single character in UTF-8 can span up to 8 bytes, the in-memory representation of these characters will pass muster with a C compiler. When the UTF-8 bytes are passed through some C library they can arrive intact to the fopen system call, which understands the UTF-8 bytes and acts like you’d expect.

There’s an easy way?

There could be. Using the @mozilla.org/file/local;1 it might be possible to get the path as an nsACString, Mozilla’s portable representation of single-byte ASCII strings. But, the reference warns that the native path available from nsILocalFile is not for passing to C libraries and isn’t guaranteed to be correct. So it’s up to me.

No pain, no gain

Bear with me, this gets ugly. The solution that lets me take paths from JavaScript-land to C-land, open Unicode paths with a normal char * and do so on Windows and Mac OS X is stupefying (and stupid). The iinitial path is stored as a JavaScript string, so the boundary between JavaScript and C will be an nsAString, which represents strings as UTF-16.

The string takes quite a journey here, so let’s cover the easy side first. Macs understand UTF-8, and I’m making a transformation to UTF-8 in JavaScript. This means that the only transform needed in C is a type change, not an encoding change. The UTF-8 string enters C-land as wide characters and is simply cast character-by-character down to an array of bytes.

Windows is quite a bit more work, since a path in Windows that needs Unicode characters can’t easily be represented as a char *. Enter the Windows API function GetShortPathName. This can convert any path to an old-school Windows path that fits in ASCII and uses that familiar “8.3″ naming convention. I got the Mac half working first, so the Windows implementation is actually more complicated than it needs to be — more on that later. Without further ado:

string * conv_path(const nsAString & fake) {

	// Fun with Windows paths
#ifdef XP_WIN

	// UTF-16 but really UTF-8 nsAString to really UTF-8 nsCString
	nsCString utf8 = NS_LossyConvertUTF16toASCII(fake);

	// UTF-8 nsCString to UTF-16 nsEmbedString
	nsEmbedString & utf16 = NS_ConvertUTF8toUTF16(utf8);

	// UTF-16 nsEmbedString to wchar_t[]
	wchar_t * w_arr = new wchar_t[utf16.Length() + 1];
	if (0 == w_arr) return 0;
	wchar_t * w_arr_p = w_arr;
	PRUnichar * w_start = (PRUnichar *)utf16.BeginReading();
	const PRUnichar * w_end = (PRUnichar *)utf16.EndReading();
	while (w_start != w_end) {
		*w_arr_p++ = (wchar_t)*w_start++;
	}
	*w_arr_p = 0;

	// GetShortPathName to get guaranteed ASCII
	wchar_t s_arr[4096];
	if (0 == GetShortPathNameW(w_arr, s_arr, 4096)) {
		delete [] w_arr;
		return 0;
	}
	delete [] w_arr;

	// wchar_t[] to ASCII nsEmbedString
	nsEmbedString ascii;
	wchar_t * s_arr_p = s_arr;
	while (*s_arr_p) {
		ascii.Append((char)*s_arr_p++);
	}

	// Macs don’t need any help since they understand UTF-8
#else
	nsEmbedString ascii;
	ascii.Assign(fake);
#endif

	// Convert the nsEmbedString into a std::string
	char * c_arr = c_arr = new char[ascii.Length() + 1];
	if (0 == c_arr) return 0;
	char * c_arr_p = c_arr;
	PRUnichar * c_start = (PRUnichar *)ascii.BeginReading();
	const PRUnichar * c_end = (PRUnichar *)ascii.EndReading();
	while (c_start != c_end) {
		*c_arr_p++ = (char)*c_start++;
	}
	*c_arr_p = 0;
	string * str = new string(c_arr);
	delete [] c_arr;
	return str;

}

This works simply because, on either platform, the result is a char * that actually uniquely represents the desired file.

My code sucks, let me count the ways

There are of course things that could be better. First of all, after reading the code again (I wrote it Thursday and haven’t looked back), I see that I should refactor a bit. By leaving the string in UTF-16 representation for passing to C-land, I can add a transform from UTF-16 to UTF-8 to the Mac version and greatly simplify the Windows version. I’ll put that on my to-do list. The other place I hope to improve is in the event that GetShortPathName fails. In this case, copying the file under a new ASCII filename to a temporary directory (found using GetTempPath) would still let the file be accessed with a char * path.

Why did I have to do this?

It would be sweeeeeet if Mozilla integrated this code (after a healthy dose of optimization) into the toolkit as something like NS_ConvertToNativePath(nsAString) or some such function. Having this will make integrating with third-party libraries much easier.

Rather than including an outside JavaScript library just to do MD5 sums, how about using Mozilla’s nsICryptoHash interface? For the Flickr API, this took slightly more effort than expected because it only returns data in base64-encoded ASCII or binary. Fortunately for me, the binary format was packed as a string, so making this usable was a matter of a little bin2hex function. Behold:

var _md5 = null;
try {
	_md5 = Cc['@mozilla.org/security/hash;1'].createInstance(Ci.nsICryptoHash);
} catch (err) {
	Components.utils.reportError(err);
}
var md5 = function(str) {
	if (null == _md5) {
		return ”;
	}

	// Build array of character codes to MD5
	var arr = [];
	var ii = str.length;
	for (var i = 0; i < ii; ++i) {
		arr.push(str.charCodeAt(i));
	}
	_md5.init(Ci.nsICryptoHash.MD5);
	_md5.update(arr, arr.length);
	var hash = _md5.finish(false);

	// Unpack the binary data bin2hex style
	var ascii = [];
	ii = hash.length;
	for (var i = 0; i < ii; ++i) {
		var c = hash.charCodeAt(i);
		var ones = c % 16;
		var tens = c >> 4;
		ascii.push(String.fromCharCode(tens + (tens > 9 ? 87 : 48)) +
			String.fromCharCode(ones + (ones > 9 ? 87 : 48)));
	}

	return ascii.join(”);
};

Today Om is reporting that Meebo is launching its own platform for third-party Meebo widgets. I could care less about the widgets, but could this be a REST API for sending IMs? No (dammit).

The Meebo API is actually an ingenious bit of Flash with a JavaScript wrapper that talks to Meebo. Once your hosted app gets some params from Meebo, you can do exciting things like listen for new messages. Through the same JavaScript-wrapped Flash object, you can also send messages. Praise be! For more, check out the Meebo JavaScript docs, which are minimal to say the least. Oh well, there’s sample code.

A couple weeks back, Instantbird did a lap around the Internet. I must call failure here, because when I tried to sign into Yahoo! Messenger it crashed immediately, spectacularly and consistently (tried four times). I know what you’re thinking: but Yahoo! Messenger is a closed system, of course there will be problems. Yeah, but Adium is built on the same library and pretty much always signs in happily (but never, ever shows me when Cal is online).

Seeing as Instantbird is built on XULRunner (my new life partner), a nice JavaScript way to send IMs would be just sweet. From my limited research, it seems embedding Flash in XULRunner is possible, as is opening a browser window within Instantbird to snag the authentication params from Meebo. Then it’s smooth sailing. Maybe.

The glaring hole in all of this is that while you can sendMsg to your heart’s content, thus far you have no one to send to.  This isn’t a full-blown API, folks.  It’s a widget platform.  There’s no way to get at someone’s buddy list.  Back to libpurple, I suppose.  But if Meebo made the rest of their service available as a REST API (to complement the far superior Flash method of sending/receiving messages), I’d be sold.

Despite my blog being stale like the bread in my cabinet, I’ve still been elbow deep in XULRunner all day every day. Now 3 1/2 months in, the ugly stuff has taken shape and it’s time to lock down the user interface. I still haven’t found an elegant way to keep clicks on the scrollbar from firing my sequence of mousedown, mousemove and mouseup events, but gimme a second chance.

As expected, I’m learning a lot about releasing software in the real world, where it has to stand up to real revisions, new features and public scrutiny. I like to think I’ve always built software that’s up to the task but a UI worthy of the Flickr name is a lot of work. And… Surprise! My engineer’s touch is not good enough. Gino and George, with a healthy bit of Stewart, have made it look awesome. And my role in the design-y stuff? Implementing some of the ideas so we can play with them before springing them on unsuspecting users.

But stay tuned. It turns out that work not only fluctuates between C++ and JavaScript but between engineering (then) and design (now). After round one is out in the wild it will be time for more hacking in the great undocumented. Lather, rinse, repeat.

The XPCOM tips will be back soon!

JSONRequest was born from Crockford’s head more than a year ago, but only this month has it really gotten interesting. Collin Jackson has released a Firefox extension implementing the JSONRequest spec. I was naturally compelled to play with it.

JSONRequest is a proposed new standard that could replace XMLHttpRequest and shed the crippling “same domain policy” that makes it a pain to consume outside APIs in Javascript. It’s secure enough for cross-site-scripting for a couple of reasons. I’m lazy, so here are Crockford’s words:

1. JSONRequest does not send or receive cookies or passwords in HTTP headers. This avoids false authorization situations. Knowing the name of a site does not grant the ability to use its browser credentials.
2. JSONRequest works only with JSON text. The JSONRequest cannot be used to access legacy data or documents or scripts. This avoids attacks on internal websites which assume that access is sufficient authorization. A request will fail if the response is not perfectly UTF-8 encoded. Suboptimal aliases and surrogates will fail. A request will fail if the response is not strictly in JSON format. A request will fail if the server does not respond to POST with a JSON payload.
3. Reponses will be rejected unless they contain a JSONRequest content type. This makes it impossible to use JSONRequest to obtain data from insecure legacy servers.
4. JSONRequest reveals very little error information. In some cases, the goal of a miscreant is to access the information that can be obtained from an error message. JSONRequest does not return this information to the requesting script. It may provide the information to the user through a log or other mechanism, but not in a form that the script can ordinarily access.
5. JSONRequest accumulates random delays before acting on new requests when previous requests have failed. This is to frustrate timing analysis attacks and denial of service attacks.

From this list, #1 and #3 make the most difference. Without sending cookies, it will be very difficult to send requests impersonating a logged-in user. By requiring the application/jsonrequest content type on the response, the standard requires that the API provider take action to allow JSONRequests. This leaves no excuse for lazy API providers who don’t make their API secure.

As for other security holes, I’ve been poking around a bit against the Flickr API. I had to steal an API key, the shared secret for that key and a user’s token that came from that API key before I could do anything nasty. Now, since getting these three pieces of information gives you free reign even without JSONRequest, I don’t see much of a threat here.

As a bit of an aside, OAuth seems to work very much like the Flickr API and so JSONRequest will be equally innocuous for OAuth users.

OK, finally the amendment part

Requiring that all POST requests be sent as well-formed JSON data severely limits JSONRequest’s ability to take web apps to the next level. The inability to send traditional POST payloads isn’t such a big deal, but the inability to send multipart POSTs for file uploads is tragic.

Of course, it isn’t as easy as just lifting the JSON-only restriction on POST payloads. As Mike pointed out to me last night, old sites using GET variables to handle authentication (PHPSESSID anyone?) are pretty much sitting ducks. But there’s always another way.

Allowing traditional or multipart POSTs without some prior setup can be hazardous. Although these will always return an error to the caller, on the server side they may be successful which could really muck up your databases. The solution is simply to require a successful JSON POST immediately prior to accepting the multipart request as a way for the server to say “yes, I want that multipart POST.” In pseudocode, something like this:

JSONRequest.post_multipart = function(url, send, done, timeout) {
  JSONRequest.post(url, {'multipart': 1}, function(requestNumber, value, exception) {
    if (value) {
      // Here we would take params passed to post_multipart and actually
      // send the multipart request.  The result of that would be passed
      // to the function the called passed in the done param.  The
      // intermediate JSON POST is essentially invisible.
    } else {
      console.log(exception.message);
    }
  }
};

With this we will finally have hack-free file uploads in addition to safe cross site scripting. Even Microsoft can’t say no to implementing that.

For reference, here’s what sent me over the edge: Ext 2.0: Scrolling Tabs, Anchor Layout, the Web Desktop, and more (Ajaxian)

I might break up with the Internet if I keep seeing so much misguided effort spent on making a website behave like a whole operating system. Because I think I’m dealing with some rather thick skulls, I’ll put this simply: stop. Now.

An arguably great feature of web apps is that they’re more two-dimensional than three-dimensional. It promotes focus. It discourages repeated seizure-inducing Exposé usage. While I’m on the topic, I can’t wait for someone to do Exposé in Javascript. That’s exactly what we need. (It’ll give me something to write about, at least.)

I’d like to propose a couple of rules that will kill stupid stuff like web desktops, all the while promoting smart use of Javascript wizardry.

• If any two things can partially overlap, you’re fired.
• If any drag & drop events don’t result in database writes, get out.
• If your website looks like Microsoft Windows, you’ve stepped over the line.

jQuery provides a ton of ways to select sets of elements but pretty much no way to pare down a selection in an arbitrary way. Fortunately for me it’s easy to extend jQuery, so I wrote a plugin that allows you to apply a function to your set of elements to pare the set down before applying an action. Source first, talk later:

jQuery.fn.if = function(arg) {
    var elems = [];
    var all = this.get();
    var ii = all.length;
    for (var i = 0; i < ii; ++i) {
        if (arg.apply(all[i])) {
            elems.push(all[i]);
        }
    }
    return jQuery(elems);
};

jquery.if.js (version 0.1)

The plugin takes as its argument a function that must return a boolean value. This function will be applied to every argument in the current set. Those that the function finds true will be returned in the final subset.

The use-case that inspired this quickie plugin is this: I wanted an action to happen to form fields that were empty, which previously required a bit of work. If I had more work to do with each match or a more complex matching requirement, this could get ugly.

$('input.promptable').focus(function() {
    var to = $(this);
    if ('' == to.val()) {
        to.val(this.id);
        to.addClass('prompt');
    }
});

With this plugin, the chaining action jQuery is known for is not interrupted. I can select all input elements with the promptable class and pare down that selection to only those whose value is empty. On the set returned there I can set values and add classes without breaking the chain.

$('input.promptable').focus(function() {
    $(this).if(function() {
        return '' == $(this).val();
    }).val(this.id).addClass('prompt');
});

jquery.if.js (version 0.1)

XULRunner provides a nice way to localize apps through the use of custom document-type declarations (*.dtd files) and *.properties files used from within your Javascripts (more info on ). The markup below brings one of these “string bundles” into the app:

<stringbundleset>
	<stringbundle id="locale" src="chrome://app/locale/main.properties" />
</stringbundleset>

Then just one line of Javascript brings it from the XUL world into the Javascript world. The string bundle element itself contains methods getString and getFormattedString that will bring localized content into your Javascript.

var locale = document.getElementById('locale');
locale.getString('foo');
locale.getFormattedString('bar', [asdf, qwerty]);

But getFormattedString doesn’t work right in XULRunner 1.9a7. It substitutes only the first character of the string (in my experience). Here is a transparent Javascript-only fix:

var locale = document.getElementById('locale');

// Now hack locale.getFormattedString to work like it should
locale.getFormattedString = function(id, args) {
	var str = locale.getString(id);
	var ii = args.length;
	for (var i = 0; i < ii; ++i) {
		var regex = new RegExp('%' + (i + 1) + '\$[ds]‘);
		str = str.replace(regex, args[i]);
	}
	return str;
};

This properly takes care of the replacement of each element in args. In the example above where the variables asdf and qwerty were passed, the contents of asdf would replace %1$s and the contents of qwerty would replace %2$s. Note the numbering of the placeholders differing from normal array indexing. Also note that for semantic reasons, a d can be used in the placeholder to denote a number, but no special case is necessary in Javascript. Happy localizing!

Things change quickly in Internet-land, and between versions 1.8 and 1.9, the Gecko SDK received a major upgrade to its thread library. What once took multiple XPCOM objects, more than few hacks and a prayer for good measure can now be done in a single Javascript file with minimal interference from the world of C++ beneath.

Here is a sample of how one might spin an event off onto a background thread acting as a job queue. The execution is fairly straightforward: a background thread is created; the UI dispatches events onto this background thread; the job on the thread dispatches an event back to the UI thread when it is finished.

Before we start, let’s define the objects used to cross the thread boundary. They must be XPCOM objects (but can be implemented in pure Javascript and don’t even need to live in the components/ directory) and must implement the nsIRunnable interface. This interface requires that QueryInterface respond positively to nsIRunnable and that a run() function be defined.

var Foo = function(id) {
	this.id = id;
};
Foo.prototype = {
	run: function() {
		try {

			// Do some work, for now just print something
			Components.utils.reportError('This is where the background work happens.');

			// Callback to the main thread
			main.dispatch(new FooCallback(this.id, 'SUCCESS'),
				background.DISPATCH_NORMAL);

		} catch (err) {
			Components.utils.reportError(err);
		}
	},
	QueryInterface: function(iid) {
		if (iid.equals(Ci.nsIRunnable) || iid.equals(Ci.nsISupports)) {
			return this;
		}
		throw Components.results.NS_ERROR_NO_INTERFACE;
	}
};
var FooCallback = function(id, result) {
	this.id = id;
	this.result = result;
};
FooCallback.prototype = {
	run: function() {
		try {

			// This is where we can work with the main thread after the job
			alert('id: ' + id + ', result: ' + result);

		} catch (err) {
			Components.utils.reportError(err);
		}
	},
	QueryInterface: function(iid) {
		if (iid.equals(Ci.nsIRunnable) || iid.equals(Ci.nsISupports)) {
			return this;
		}
		throw Components.results.NS_ERROR_NO_INTERFACE;
	}
};

Here is the code we need to actually use these objects to spawn a background thread and dispatch a job to it. The callback will end up on the main thread’s event queue.

background = Cc['@mozilla.org/thread-manager;1'].getService().newThread(0);
main = Cc['@mozilla.org/thread-manager;1'].getService().mainThread;
background.dispatch(new Foo(id), background.DISPATCH_NORMAL);

The first two lines here should be executed on application startup and will leave thread references in main and background. The third line actually puts a job on the background queue. This can be called anywhere and will likely be used inside a loop to push a lot of compute-intensive work onto the background thread, allowing the UI thread to remain responsive.

A note about the id passed to the thread objects: this is not strictly necessary, but I find it very convenient to have the option of tracking individual jobs through the event queue. The id is especially useful when you’re using the callback to update a certain element of the UI with the result of the job. Then in the callback it’s as simple as document.getElementById(this.id).

Apologies for the bits-and-pieces examples. I’m ripping this out of unreleased code so I don’t have time to create a demo app for each one. I recommend starting with Mark Finkle’s hello world.