Eli Grey

CPU core estimation with JavaScript

(Update) Standardization

navigator.cores has been standardized as navigator.hardwareConcurrency and is now supported natively in Chrome, Safari, and Opera. Our polyfill has renamed the APIs accordingly. Since the initial blog post, Core Estimator has been updated to estimate much faster and now has instant estimation in Chrome through PNaCl.

navigator.cores

So you just built some cool scalable multithreaded feature into your webapp with web workers. Maybe it’s machine learning-based webcam object recognition—or a compression algorithm like LZMA2 that runs faster with the more cores that you have. Now, all you have to do is simply set the number of worker threads to use the user’s CPU as efficiently as possible…

You might be thinking “Easy, there’s probably a navigator.cores API that will tell me how many cores the user’s CPU has.” That was our thought while porting xz to JavaScript (which will be released in the future as xz.js), and we were amazed there was no such API or any equivalent whatsoever in any browser! With all the new features of HTML5 which give more control over native resources, there must be a way to find out how many cores a user has—we are already running code on them.

I immediately envisioned a timing attack that could attempt to estimate a user’s CPU cores to provide the optimal number of workers to spawn in parallel. It would scale from one to thousands of cores. With the help of Devin Samarin, Jon-Carlos Rivera, and Devyn Cairns, we created the open source library, Core Estimator. It implements a navigator.cores value that will only be computed once it is accessed. Hopefully in the future, this will be added to the HTML5 specification.

Live demo

Try out Core Estimator with the live demo on our website.

screenshot of the demo being run on an i7 3930k

How the timing attack works and scales

The estimator works by performing a statistical test on running different numbers of simultaneous web workers. It measures the time it takes to run a single worker and compares this to the time it takes to run different numbers of workers simultaneously. As soon as this measurement starts to increase excessively, it has found the maximum number of web workers which can be run in parallel without degrading performance.

In the early stages of testing whether this would work, we did a few experiments on various desktops to visualize the data being produced. The graphs being produced clearly showed that it was feasible on the average machine. Pictured are the results of running an early version of Core Estimator on Google Chrome 26 on an Intel Core i5-3570K 3.4GHz Quad-Core Processor with 1,000 time samples taken for each core test. We used 1,000 samples just to really be able to see the spread of data but it took over 15 minutes to collect this data. For Core Estimator, 5 samples seem to be sufficient.

The astute observer will note that it doesn’t test each number of simultaneous workers by simply counting up. Instead, Core Estimator performs a binary search. This way the running time is logarithmic in the number of cores—O(log n) instead of O(n). At most, 2 * floor(log2(n)) + 1 tests will be done to find the number of cores.

Benefits

Previously, you had to either manually code in an amount of threads or ask the user how many cores they have, which can be pretty difficult for less tech savvy users. This can even be a problem with tech savvy users—few people know how many cores their phone has. Core Estimator helps you simplify your APIs so thread count parameters can be optional. The xz.js API will be as simple as xz.compress(Blob data, callback(Blob compressed), optional int preset=6, optional int threads=navigator.cores), making it this easy to implement a “save .xz” button for your webapp (in conjunction with FileSaver.js):

save_button.addEventListener("click", function() {
    xz.compress(serializeDB(), function(compressed) {
        saveAs(compressed, "db.xz");
    });
});

Supported browsers and platforms

Early Core Estimator has been tested to support all current release versions of IE, Firefox, Chrome, and Safari on ARM and x86 (as of May 2013). The accuracy of Core Estimator on systems with Intel hyper-threading and Turbo Boost technology is somewhat lesser as the time to complete a workload is less predictable. In any case it will try to tend towards estimating a larger number of cores than actually available to provide a somewhat reasonable number.

Saving generated files on the client-side

Have you ever wanted to add a Save as… button to a webapp? Whether you’re making an advanced WebGL-powered CAD webapp and want to save 3D object files or you just want to save plain text files in a simple Markdown text editor, saving files in the browser has always been a tricky business. Usually when you want to save a file generated with JavaScript, you have to send the data to your server and then return the data right back with a Content-disposition: attachment header. This is less than ideal for webapps that need to work offline. The W3C File API includes a FileSaver interface, which makes saving generated data as easy as saveAs(data, filename), though unfortunately it will eventually be removed from the spec. I have written a JavaScript library called FileSaver.js, which implements FileSaver in all modern browsers. Now that it’s possible to generate any type of file you want right in the browser, document editors can have an instant save button that doesn’t rely on an online connection. When paired with the standard HTML5 canvas.toBlob() method, FileSaver.js lets you save canvases instantly and give them filenames, which is very useful for HTML5 image editing webapps. For browsers that don’t yet support canvas.toBlob(), Devin Samarin and I wrote canvas-toBlob.js. Saving a canvas is as simple as running the following code:

canvas.toBlob(function(blob) {
    saveAs(blob, filename);
});

I have created a demo of FileSaver.js in action that demonstrates saving a canvas doodle, plain text, and rich text. Please note that saving with custom filenames is only supported in browsers that either natively support FileSaver or browsers like Google Chrome 14 dev and Google Chrome Canary, that support <a>.download or web filesystems via LocalFileSystem.

How to construct files for saving

First off, you want to instantiate a Blob. The Blob API isn’t supported in all current browsers, so I made Blob.js which implements it. The following example illustrates how to save an XHTML document with saveAs().

saveAs(
      new Blob(
          [(new XMLSerializer).serializeToString(document)]
        , {type: "application/xhtml+xml;charset=" + document.characterSet}
    )
    , "document.xhtml"
);

Not saving textual data? You can save multiple binary Blobs and ArrayBuffers to a Blob as well! The following is an example of setting generating some binary data and saving it.

var
      buffer = new ArrayBuffer(8) // allocates 8 bytes
    , data = new DataView(buffer)
;
// You can write uint8/16/32s and float32/64s to dataviews
data.setUint8 (0, 0x01);
data.setUint16(1, 0x2345);
data.setUint32(3, 0x6789ABCD);
data.setUint8 (7, 0xEF);
 
saveAs(new Blob([buffer], {type: "example/binary"}), "data.dat");
// The contents of data.dat are <01 23 45 67 89 AB CD EF>

If you’re generating large files, you can implement an abort button that aborts the FileSaver.

var filesaver = saveAs(blob, "video.webm");
abort_button.addEventListener("click", function() {
    filesaver.abort();
}, false);

Title image files in Opera

I recently discovered a method to title image files in Opera. I was experimenting with CSS generated content in regards to the <title> element in various browsers, and discovered that as long as the <head> and <title> elements are not display: none, generated content applied before and after the <title> element is added to the page title itself in Opera. It was obvious to me that I should combine this with HTTP Link: headers containing stylesheets, as to make it possible to modify the title of usually non-titleable media, such as images, plain text, audio, and video.

In this demo, the following CSS rules are applied in Opera.

head, title {
	display: block;
	width: 0;
	height: 0;
	visibility: hidden;
}
 
title::before {
	content: "Just an image — ";
}
Tagged: , ,

Voice Search Google Chrome extension

Voice Search screenshot Voice Search is an open source Google Chrome extension I made that allows you to search by speaking. For example, just click on microphone and say kittens to search for kittens. If you specifically want pictures of kittens, say google images kittens. Want to learn more about World War II? Say wikipedia world war two. The source code for Voice Search is on GitHub.

Voice Search comes pre-loaded with many popular search engines by default, and you can add your own user-defined search engines. It also integrates a speech input button for all websites using HTML5 search boxes, all of the default search engines’ websites, Facebook, Twitter, reddit, and GitHub.

In later versions, I plan to introduce ability to import/export settings, create aliases (e.g. map to Google Maps and calculate to Wolfram|Alpha), OpenSearch description detection, and scripted terms that do more than open URLs.

Better font smoothing in Google Chrome on Windows

Screenshots: before and after.

Firefox 4 and Internet Explorer 9 already support improved font smoothing offered by the DirectWrite API in Windows 7 and Vista. Google Chrome (WebKit) has yet to support DirectWrite, which may be the deal breaker for you when choosing to use either Google Chrome or Firefox if you are primarily a Windows user.

I recently discovered while messing with the CSS3 text-shadow property in Google Chrome that it somehow improves font smoothing in Google Chrome (but surprisingly not the WebKit-based Safari too). To use the better font smoothing on your website, just use text-shadow: rgba(0, 0, 0, .01) 0 0 1px in your CSS on whatever you want to have better font smoothing. I have also created a Google Chrome extension called “Enhanced Windows Font Smoothing“, which applies this CSS hack to every website and to all text. Please note that smaller text may look a little unsightly, though it will still be completely readable. For a good example website try the extension on, see how the text logo on the Ubuntu Font Family website looks before and after installation of the Google Chrome extension. Please note that this CSS hack may cause adverse side-effects on Mac OS X, so I suggest that you try to target Windows UAs only.

Tagged: ,

Encrypted DuckDuckGo with Google Suggest search plugin

I use a modified version I made of Nathan Friedly’s DuckDuckGo + Google Suggest search plugin as my default search provider, and I would like to share it with everyone else. It uses the encrypted versions of DuckDuckGo and Google Suggest so no one can eavesdrop on your searches (except DuckDuckGo and Google, of course).

Installation

In most browsers, the following link should work for you: Install encrypted DuckDuckGo with Google Suggest search plugin. If you are having any trouble installing the search plugin, refer to the detailed instructions by Nathan Friedly for the original search plugin. For manual installation in Opera, replace every instance of http:// with https:// and replace every instance of suggestqueries.google.com with encrypted.google.com.

Encrypted Google Search and Suggest search plugin

If you’re looking for an encrypted search plugin but want to keep using Google, I made that too. Install encrypted Google Search and Suggest search plugin.

Android advanced settings webapp

QR Code for Android Advanced Settings webapp In Android, there’s an advanced settings application called “Testing” which can be launched by entering *#*#4636#*#* (*#*#INFO#*#*) into the phone application’s dialer. I then attempted to bookmark a tel: URI of the number but found out that the browser application does not allow tel: URIs as bookmarks, claiming they are invalid. I have made a simple Android webapp that redirects all subsequent visits after the first visit to this application, and gave it a fancy launcher icon from the Tango Icon Library. The QR Code in this post points to the webapp.

Tagged: ,

Passive localization in JavaScript

Today, I created a passive localization JavaScript library named l10n.js. l10n.js is a JavaScript library that enables passive localization through native JavaScript methods, gracefully degrading if the library is not present. You can make Ajax applications, JavaScript libraries, etc. that can be localized but not require l10n.js to function. There is already a placeholder method for all API calls as specified in the ECMAScript specification and is present in all JavaScript engines, so when l10n.js isn’t present, your application works fine.

Demo

You can try out the online demo to see l10n.js in action. Currently, only the languages mentioned in the readme are supported, but more will eventually be added.

Usage

API

API documentation can be found in the readme. All API calls gracefully degrade, so calling them even without l10n.js loaded causes no problems.

Localizing strings

Calling toLocaleString() on every localizable string can create a lot of extra typing and bloat for sending your JavaScript down the wire. I recommend using the following helper function to localize strings. The reason I don’t define this in l10n.js is to not introduce any new globals, which keeps l10n.js a one of the JavaScript libraries least-prone to conflicts with other libraries.

var l = function (string) {
    return string.toLocaleString();
};

With this helper function, you can start writing l("Your localizable string") instead of "Your localizable string".toLocaleString(). I chosel instead of _ (an underscore), because it’s easier to spot so you can quickly skim your code to see which strings are localizable.

Variable replacement

If you don’t mind requiring l10n.js for your JavaScript application or library to function, I suggest using short variable strings instead of default strings. It saves bandwidth by decreasing the size of localization files, and it enables you to write nice, short code as such in the following.

  • document.title = l("%title.search")
    • Example results: "Seach - Acme, Inc."
  • confirm(l("%confirm.deleteAccount"))
    • Example results: "Are you sure you want to delete your account?"
  • link.href = "http://www.google" + l("%locale.tld")
    • Example results: "http://www.google.co.uk"

Often, string concatenation is used instead of replacement in JavaScript. With l10n.js, to make localization easier, you may have to use replacements instead. You might want to use a JavaScript library that implements something similar to C++’s sprintf(). A nice JavaScript implementation I’d recommend is php.js’s sprintf().

When localizations are downloaded

If you are using single localization URLs (<link rel="localization" hreflang="..." href="..." type="application/vnd.oftn.l10n+json"/>), they will only be downloaded when needed. If you are using multiple localizations in one (<link rel="localizations" href="..." type="application/vnd.oftn.l10n+json"/>), then the file will be downloaded right away, but externally linked localizations in the localization file will not be. If you provide an interface for your users to change locales, any non-loaded localization files will be loaded when necessary.

Including localizations with link elements

Multiple localizations can be included with one localization JSON file, with all of the top properties being language codes. Instead of putting all of the localized strings directly in the file, you may want to assign a specifc localization JSON URL to each locale, as to save bandwidth by only downloading locales the user needs.
The following is an example localization file for <link rel="localizations" href="path/to/localizations.json" type="application/vnd.oftn.l10n+json"/>.

{
  "en-US": {
      "What is your favourite colour?": "What is your favorite color?"
  },
  "fr": "path/to/french-localization.json"
}

Using localization files is the same as calling String.toLocaleString() with the JSON localizations object as the first parameter.

You can also include single localizations by specifying the standard HTML5 hreflang link element attribute and using a rel of localization instead of localizations with an ‘s’, as shown in the following.

<link rel="localization" hreflang="en-US" href="american-english.json" type="application/vnd.oftn.l10n+json"/>

The JSON file for the localization might look like the following.

{
    "What is your favourite colour?": "What is your favorite color?"
}

tinylog

tinylog is a minimalistic logging platform JavaScript library I created which is primarily intended for online IDEs and implementing console.log() for browsers without native consoles. There is also a lite version intended for embedding in other JavaScript libraries. One such library that embeds tinylog lite is Processing.js, which uses it to implement Processing’s println().

There are also online demos of using tinylog that you can try out. The tinylog saved log viewer demo only works in browsers that support the W3C File API which is only Firefox as of now.

Pausing JavaScript with async.js

async.js is a library that aims to make it so you don’t have to mess with callbacks when making applications in JavaScript 1.7 or higher by using the yield statement to pause function execution.

Examples

Please note that user interaction with the page is not blocked during the course of any of these examples.

node.next(eventType) method

The node.next(eventType) method would pause a function until the specified event is fired on the node that next was called on and would return the captured event object.

var listenForNextEventDispatch = function ([node, eventType], callback) {
    var listener = function (event) {
        node.removeEventListener(eventType, listener, false);
        callback(event);
    };
    node.addEventListener(eventType, listener, false);
};
Node.prototype.next = function (eventType) {
    return [listenForNextEventDispatch, [this, eventType]];
};

You could now then the following in an asynced function to handle the next click event on the document.

var clickEvent = yield document.next("click");
// handle click event here

Asking the user for their impressions of async.js

The following code does not use any obtrusive and annoying functions like prompt or alert yet still can utilize execution-blocking features.

yield to.request("feedback", "POST", (
    yield to.prompt("What are your impressions of async.js?")
));
yield to.inform("Thanks for your feedback!");
// do more stuff here

As opposed to the following, which is functionally equivalent to the previous code but doesn’t use async.js’s blocking features.

async.prompt(
    ["What are your impressions of async.js?"],
    function (response) {
        async.request(
            ["feedback", "POST", response],
            function () {
                async.inform(
                    ["Thanks for your feedback!"],
                    function () {
                        // do more stuff here
                    }
                );
            }
        );
    }
);

That’s a lot of callbacks, all of which are implied when you use async.js.

Creating an async.js module for thatFunctionThatUsesCallbacks

async.yourMethodName = function ([aParameterThatFunctionUses], callback) {
    thatFunctionThatUsesCallbacks(aParameterThatFunctionUses, callback);
};

You could then use yield to.yourMethodName(aParameterThatFunctionUses) and immediately start writing code that depends onthatFunctionThatUsesCallbacks function after the statement.