Firefox Extension, Mozilla Labs

Ubiquity is a natural language command-line interface to the web, developed by Mozilla Labs. It allows the user to input a command which they would like the browser to execute, and Ubiquity returns suggested actions based on its interpretation of that command. Common commands include interfacing with web services, such as Twitter or Gmail, or performing common tasks of the browser, such as opening a tab or doing a search. A user can update their twitter, send an email, check the weather, and locate a restaurant on google maps, all through the Ubiquity command-line. Developers can also write their own actions for Ubiquity, which users can then install and use.

A crucial component of the Ubiquity back-end is its nountype detection system. When the user inputs a noun into the Ubiquity command-line without entering an accompanying verb, Ubiquity returns suggested actions based on how it classifies that noun. For example, if the user enters the name of a well-known restaurant, Ubiquity will identify that noun as being of type "restaurant" and return actions which can be performed on a "restaurant", including looking up the restaurant on Yelp or getting the address to the restaurant on Google Maps. To identify that the input was of nountype "restaurant", Ubiquity has to perform asynchronous calls in the background which check online data sources like Yelp to see if the input matches any known restaurants. One of my primary contributions to the project was writing the Javascript code that handles this asynchronous information retrieval and storage.

Firefox Extension, MIT User Interface Design Group

Chickenfoot is a Firefox extension which embeds a programming environment in the browser. This programming environment is displayed in a sidebar, and allows users to write scripts which manipulate web pages and automate web browsing. The scripts can be written using a combination of Javascript, XUL, and a high-level library designed specifically for the Chickenfoot project. This library provides simple and powerful methods of writing compact scripts which interact with websites. As an example, here is a script that will navigate to google, type "Brandon Pung" into the search bar, and click the Search button.

I contributed to the project through the summer and fall of 2007, improving its development environment and expanding the Chickenfoot function library.

Firefox Extension, MIT User Interface Design Group

Kangaroo is a Firefox extension which makes extensive use of the Chickenfoot function library. It was built to demonstrate the power of the Chickenfoot programming environment for rapid development of sophisticated Firefox extensions. Kangaroo displays a grid of headshot photos within the compose window of an email client corresponding to the people to which the user is sending an email. It does this by parsing the To, Cc, and Bcc fields of the compose window and looking up those names or email addresses on Facebook and Google Images. It then performs facial recognition on the images it finds to pick the one that most closely resembles a headshot, and embeds that image in the Compose window of the email client. It makes appropriate use of caching, and also allows users to manually select an image that they want associated with a particular name or address. I was one of the primary developers of this extension, which was build and released in late 2007.