Panorama went through a pretty rapid development cycle, making it into Firefox 4 which was released today (yay!). Moreover, for a while we were developing outside of mozilla-central, without the regular “patches require tests” requirement. This makes checking its test coverage particularly important.

Check out the final result, the Panorama test coverage report. The good news: our code coverage is 86%! (Some notes on what improvements can be made are in the bug.)

PhiliKON had previously worked on hooking into the JS Debugger service’s interruptHook to test xpcshell tests. I modified this code to run instead in the Mochitest browser chrome tests. This code can be found on the bug.

With this patch applied, I invoked the test suite with the following code: TEST_PATH=browser/base/content/tests/tabview COVERAGE_FILTER="*tabview*" COVERAGE=true make -C obj-ff-dbg mochitest-browser-chrome . That’s a regular mochitest-browser-chrome invocation with the COVERAGE=true flag which turns on code coverage checking, and COVERAGE_FILTER=*tabview* which filters out results from files which don’t have “tabview” in their paths. This creates a file called coverage.json in the working directory of the test suite, meaning, for me, obj-ff-dbg/_tests/testing/mochitest/.

This JSON file is a multidimensional array, with file paths and then line numbers as keys. The file paths here, as best as possible, have been converted into local filesystem paths. PhiliKON built a script which produces beautiful reports based on this output.

A word of warning: running with this JSD interruptHook is ridiculously slow. A number of tests for Panorama are timing-dependent (drag-drop tests, for example), making some of them fail, but that’s okay… as long as it completed not via a timeout, it actually did run through all the code. In order to get this to run through everything with some degree of control, I split up the mochitest tabview suite in to a few chunks. I then took the multiple resulting coverage.json files and passed them into another script, in tools/coverage/aggregate.py, which takes multiple JSON results like this and puts them together into a single JSON file. I then passed this aggregate JSON file to PhiliKON’s wonderful report script and—voila—the Panorama test coverage report! Easy as pie.

Related posts:

1. Performance vs Responsiveness —or— How I Made the Parser Twice As Fast in One Day
2. Beginning development with Jetpack SDK 0.2
3. Keep up with Yet Another Related Posts Plugin with RSS!

Related posts brought to you by Yet Another Related Posts Plugin.

JSConf.us:

I’ll be putting my Mozilla Jetpack Ambassador hat on to represent Mozilla Labs’ Jetpack project at the premier Javascript conference in North America, JSConf.us, which this year will be April 17-18 in Washington D.C. and has a pirate theme.¹ I’ll be giving a short talk in the main session and will also lead a hands-on Jetpack workshop in the hacker lounge. I’ve heard that JSConf is a lot of fun and I’m really looking forward to it!

WordCamp San Francisco:

I’m honored to have been invited to give a talk at WordCamp San Francisco 2010. WordCamps are community-organized events for the WordPress community, and the San Francisco WordCamp is the original and biggest. WordCamp SF will be at the Mission Bay Conference Center on May 1st. Tickets available.

My talk is tentatively titled “Abstract Your Code.”² WordPress is a great platform to build amazing content-rich applications on, and many of us have written new functionality in the form of plugins. I hope to encourage developers to make their code more portable and reusable after the project is done—or, ideally, to even start with abstraction in mind—to add to the “life” of the code and to consider then open-sourcing that functionality.

Hope to see you there!

International Association of Chinese Linguistics (IACL) 18:

Finally, I’m thrilled to say that I got a paper accepted to the annual meeting of the International Association of Chinese Linguistics which this year is at Harvard on May 20-22. IACL is the big conference for Chinese linguistics, with about 180 papers presenting. I’ll be presenting Two Onlys in Mandarin Chinese, my recent work on the formal syntax/semantics of two only words in Chinese: zhǐ (只) and éryǐ (而已). I’ve put up a handout of some of this material in work-in-progress form which I recently presented at SNEWS.

Related posts:

1. WordCamp Boston 2010
2. Jetpacking in Boston
3. Mashing up the browser in Maine

Related posts brought to you by Yet Another Related Posts Plugin.

Shadowbox is organized around different “players”—one for each kind of media that will be displayed. The library by default comes with players for Flash, HTML fragments, iframes, QuickTime, and Windows Media. Some of these players, like those for images and video, automatically recognize the media size and adjust the lightbox accordingly, while others such as the iframe player can use a set size or can fill the screen. For the Edgerton site, though, we had a need for displaying an iframe but in the dimensions of a set image, so that we could display the image with an overlay. Here are some notes on how to implement a custom player for Shadowbox.

The first step to getting Shadowbox to recognize a custom player is to modify the players loaded by shadowbox.js. I called my player “hybrid,” as it’s like a cross between the iframe and img players.

Shadowbox.options.players=["html","iframe","img","hybrid"];

Now Shadowbox will try to look for the file players/shadowbox-hybrid.js. That’s where we will create our custom player. Start by duplicating the iframe player file. Next, we can make one more change to shadowbox.js. In the getPlayer function, add a line that will return “hybrid” when it recognizes a lightbox which should use the hybrid player. I added the following line, which uses the hybrid player when the text “galleries” is in the a tag’s href attribute.

if (/galleries/.test(content)) return "hybrid";

Finally, you must actually write the new player. There’s a readme file which describes the interface:

All players should implement the same interface. This makes it possible for the Shadowbox class to know what methods to call and properties to check on player objects.

The interface is described here, with some simple explanations of how each method and/or property is to be used.

height

(Number) The height of the object (in pixels)

width

(Number) The width of the object (in pixels)

ready

(optional, Boolean) True if the content is ready to be loaded, false otherwise. Useful when the script should wait until the content loads before proceeding (see below)

resizable

(optional, Boolean) True if the content can be dynamically resized by the script (e.g. images, but not most movie formats)

append()

Appends this object to the DOM

remove()

Removes this object from the DOM

onLoad()

(optional) Called after the content is loaded and the loading layer is hidden

onWindowResize()

(optional) Called when the window is resized

I found these directions quite easy to follow. For our purposes, there was also a note on how to set an object’s height and width based on an image.

Hopefully these notes are helpful for others who hope to take full advantage of this very powerful JavaScript library.

Related posts:

1. I’m Seriously Dreaming of a White Christmas
2. The Gift That Keeps On Giving
3. Atayal cultural festival

Related posts brought to you by Yet Another Related Posts Plugin.

Being a longer time slot than I previously have used to talk about Ubiquity, I decided to dedicate a good portion of the talk to Jetpack. Being outside of Mozilla for the past few months, this gave me an opportunity to get reacquainted with the Jetpack APIs. I myself was impressed by how easy it was to develop a quick Jetpack. I ended up preparing two to live-code during the talk: one called Helvetica which, with one click, replaces all fonts on the current page with Helvetica; and You Are Here which uses an open API from IPinfoDB to display the physical location of the domain you are currently visiting in the status bar. Both are now on the Jetpack Gallery.

Unfortunately there was a bit of a snowstorm leading up to the event, but there was still a nice turnout and I got to meet some fantastic people there. Ken Shoemake of slerp and quaternion fame came up to me after my talk and said “the Ubiquity parser reminded me of the dancing bear… it’s less surprising that it works well as that it works at all.” I also enjoyed the other great presentations in the technology track, covering the virtues of REST and basic iPhone development.

Mashup the Browser with Ubiquity and Jetpack

Related posts:

1. Extending WordPress talk at the Boston WordPress Meetup
2. Talking Ubiquity in Japan: 拡張機能勉強会にて発表
3. Ubiquity presentation at Tokyo 2.0

Related posts brought to you by Yet Another Related Posts Plugin.

In order to facilitate this discussion, we created the Persistence tool. With the Persistence tool you can quickly try out new design and interaction ideas, mocking things up with some simple jQuery-powered JavaScript and CSS and see your changes live. The Persistence tool is bundled with our latest Ubiquity beta (install link).

The Ubiquity Persistence Project: exploring a persistent Ubiquity in the toolbar from mitcho on Vimeo.

I just put together a screencast introducing the initiative, demoing the Persistence tool, as well as talking about this project’s relation to the ongoing work on Taskfox. We’ll look forward to your comments and designs!

Related posts:

1. User-Aided Disambiguation: a demo
2. Count command for Ubiquity
3. Exploring Command Chaining in Ubiquity: Part 1

Related posts brought to you by Yet Another Related Posts Plugin.

Today I hope to cover some of the technical considerations required in implementing command chaining in Ubiquity, and I will follow up soon with a blog post on the linguistic considerations required as well.

Technical considerations: hooking the pipes together

I’d first like to lay out some technical challenges and questions. These can be broken into two different categories: (1) how the parse and display of suggestions is affected and (2) how the execution is affected.

Matching inputs and outputs

We’ll first consider how command chaining may affect the parsing. Ubiquity commands each specify the types of argument inputs that it expects using different noun types, such as noun_arb_text which accepts anything, noun_type_number which accepts numbers, or noun_type_language, which takes the name of a language. For example, the translate verb takes maximally three arguments: a noun_arb_text object, a noun_type_language goal (the language to translate into), and a noun_type_language source (the source language). In implementing command chaining, it will be necessary to identify the appropriate noun types for the output of a command.

The first question we must address here is “what is the chaining output of a command”? Is it the preview text? Some text output from the execution?

Big fish eat da lil fish by joemud, CC-SA-NC

To put this question into perspective, we note that Ubiquity commands can be broadly classified into two types: lookup and action execution. Here’s a classification which I believe to be exhaustive:²

In light of this classification I believe we can say that lookup commands are much more likely to be the first verb in a command chain—conversely, chains such as email hello to Blair and then do ... or twitter hello and then ... are quite unlikely. What is much more likely is for the first verb to be a lookup function.

Thus in the same way that not all commands have a useful execution³ perhaps only lookup commands will have a chainable output: the results of the lookup. Even with this restriction, we will most likely need to implement a new “chainable output” method or getter in these commands. This means that commands will need to opt-in to become chainable, but I believe this is a necessary evil.

The second question we must address is “when do we establish the noun type of a command’s chainable output?” One unsung but crucial feature of the way Ubiquity works now is that suggestions’ previews are not computed until that suggestion is selected (except for the first suggestion, which in most skins gets previewed immediately). Should we wait for all of the first verbs’ chainable output to be computed and then run them through the noun type detection system? Or should verbs with chainable output also a priori specify what noun types their output will be?

Both of these approaches have their problems. If we compute the chainable output of the first verb, run a noun type detection on it and then suggest the full combination if it matches what the second verb was expecting, this will have clear performance implications, not to mention that it could greatly complicate our parsing algorithm. While the latter approach doesn’t have these performance implications, it does mean that it will have to list (by name or reference) the noun types that will match its output, meaning that if a command author is unaware of someone else’s noun type, that chain will be impossible, even if the chainable output itself does indeed match that noun type. The a posteriori approach would never have this issue. What other benefits or problems do you forsee with either of these approaches? Is there another approach which avoids these pitfalls?

(A)synchronous composability

Once we have the noun types, parsing, and suggestions down, all that remains is to compute the previews and implement the composite execution. Since the Ubiquity command manager already wraps the preview and execute functions in a wrapper to facilitate localization, among other uses, it would be easy to make the command manager compose asynchronous processes pseudo-synchronously. No major changes should be necessary to do the previews and executions, though, again, there will be a performance cost.

Conclusion

There are a number of technical questions which must be answered, mostly in the parsing/suggesting stage. The key questions to answer are:

1. What is the chaining output of a command?
2. When do we establish the noun type of a command’s chainable output?

I’ll make another post soon on the linguistic considerations necessary in making command chaining happen in a natural fashion.

Related posts:

1. Command Chaining with Oni?
2. Ubiquity Commands by The Numbers
3. Ubiquity Parser: The Next Generation Demo

Related posts brought to you by Yet Another Related Posts Plugin.

The short story:

Inspired by some comments by Blair, yesterday I was able to make significant (roughly 100%) performance gains in Parser 2, resulting in 40-60% faster parses, depending on the query. This change has been committed and will be released as part of our forthcoming minor update, Ubiquity 0.5.4. Yay!

The long story: asynchronous parsing

Given that parsing in Ubiquity, combined with the post-parse of displaying suggestions, takes a good few dozen milliseconds, it is important to make sure it doesn’t block the main execution thread in order for the UI to stay responsive throughout. In other words, we needed to make it asynchronous.¹

When we began work on Parser 2 a few months ago, Blair stepped up to the plate to make it run asynchronously. For various reasons, the parser doesn’t run in a Worker thread for truer threading. Instead, what we did was to put the parser’s steps into a generator called _yieldingParse. The keyword yield is scattered in points throughout this generator.

function _yieldingParse(...) {
  // step 1
  ...
  yield true;
 
  // step 2
  ...
  {
    ...
    yield true;
  }
  ...
 
}

We then iterate over a _yieldingParse object in a function called doAsyncParse. Each time we go invoke doAsyncParse, it invokes next which advances from the last yield point of the parse to the next one. doAsyncParse checks after each step whether we should keepworking or not and then asynchronously advances to the next step by calling itself with a setTimeout. (Note the code below is a simplification.)

var parseGenerator = _yieldingParse();
 
function doAsyncParse() {
  var ok = parseGenerator.next();
  if (ok && keepworking)
    Utils.setTimeout(doAsyncParse,0);
}
 
// get this party started
Utils.setTimeout(doAsyncParse,0);

The more often we yield, the more responsive the UI would be. However, there is a certain overhead to yielding each time due to the setTimeouts we call. This point hit home when Blair told me the other day that the parser was much faster without any of the setTimeouts. Indeed, in my own testing running queries completely synchronously (replacing out all the setTimeouts), parses would run in roughly half the original time. However, by virtue of being completely synchronous, the parser would then completely lock up the UI.

I thus set out to strike a balance between performance and responsiveness by taking out and moving some of the yields in our _yieldingParse (#856).

Tests, tests, tests

Keeping this in mind, I ran a number of tests as I proceeded with my “refactoring.”

Here are some final parse time results:²

Four different queries (“hello to span”, “goo hello”, “22.7”, “tw as test” with a selection context of “hello world”) were run using each algorithm. The blue bar is the performance of the algorithm prior to adjustment of setTimeouts—that of Ubiquity 0.5.3. The gold bar is the time from a completely synchronous parse where all the setTimeouts were replaced. This algorithm completely locks up the UI, but is clearly the fastest, and should be seen as a baseline for all other yielding optimizations. The green bar is our new algorithm. As you can see, the parser is now roughly twice as fast.

Moreover, the average time difference between yields went from 0.7ms to 3.9ms which still should be no problem in terms of responsiveness.

Cancellability

This doAsyncParse is also the key for cancellability of the query. When a user changes the input or closes Ubiquity while a query is in progress, we want to cancel that query as soon as possible so the user and UI can advance. keepworking is set to false when the query is cancelled, so making sure that we yield early enough and often enough in the parse are important for issues like keystrokes being lost when typing too fast.

While the parser was indeed yielding often enough (in fact, more often than necessary) before, I noticed that our first yield was often 15-20 ms into the parse. This was because step 1 of our parse derivation was happening outside of the doAsyncParse loop. By moving this into that loop, I was able to bring this initial synchronous time down to around 10 ms. Of course, setting up the parse generator itself takes a little overhead, so this can never go down to 0, but perhaps this will improve the keystroke issue as well. I’d love to get anecdotal feedback on whether this update improves the disappearing keystrokes issue from 0.5.4 users.

Related posts:

1. Ubiquity Parser: The Next Generation Demo
2. This week on Ubiquity Parser: The Next Generation
3. Changes to Ubiquity Parser 2 and the Playpen

Related posts brought to you by Yet Another Related Posts Plugin.

This video walks through the process of converting your Ubiquity commands to Ubiquity 0.5 with Parser 2. For more information, please consult the command conversion tutorial.

Related posts:

1. A Demonstration of Ubiquity Parser 2
2. Changes to Ubiquity Parser 2 and the Playpen
3. Foxkeh demos Ubiquity Parser: The Next Generation

Related posts brought to you by Yet Another Related Posts Plugin.

Command localization in Ubiquity 0.5

A few important design decisions have already been made in implementing command localization in Ubiquity 0.5. The first was the choice of the gettext po (portable object) file format. The po format is a de facto industry standard with many tools built for the format and this design choice hopefully lower the bar for prospective localizers.

Second, in order to simplify the matching of localizations and commands, we require that each command feed have one localization po file, rather than splitting the localizations of different commands across multiple po files.

Finding the appropriate localization po file for a JavaScript command feed is very simple with bundled commands as the files are managed in our main repository so we know exactly where to find them. We just take the command feed’s filename sans extension—the feed key—and look for localizations/lang/feed key.po, where lang is the active language code. This gives us a simple one to one relationship between the JavaScript command feed source and the appropriate localization.

Background

The ability of users to easily write their own Ubiquity commands has always been a huge strength of the Ubiquity platform. Users can also “subscribe” to commands written by other users on other servers. In this case, local copies of those command sources are made.

The herd was developed as a dynamic aggregator of community Ubiquity commands. The herd keeps its own copy of each command source. The herd groups mirrors of command feeds on multiple servers together as well, giving each unique command feed a unique ID.

Localizing distributed resources

This distributed nature of the command feeds is exactly what complicates the localization of community commands. The original sources themselves are distributed, and we also want a community of localizers to be able to localize the commands—i.e., for the localizations to be (in some sense) distributed. Here I will present three options which I believe could be the basis for a winning solution, in order of more distributed to less distributed.

All three of these options have the property that localizations do not need to be “approved” or “managed” by the command author. For example, a possible standard where URL’s of localizations must be in the command feed is not considered. I believe this is a crucial property of any approach we decide on.¹

Option 1: a completely distributed option

In this option, localizers simply put the po files on their own servers (or some code snippet site such as github) and the user must then subscribe to those localizations much as they subscribe to commands now.

Option 2: registration and discovery through the herd

In this option, localizers put po files up on their own servers and then register that po file’s URL with the herd. The herd keeps track of each command feed’s localizations in different languages.

Option 3: localizations on the herd

In this option, po localizations are uploaded right onto the herd. The herd is the centralized repository of all localizations for each command feed.

Users first

In coming up with a criteria for judging different models of community command localization, I think it is helpful to think of the end-user experience. Right now to subscribe to a new command a user must find the command (perhaps via the herd) and click on the subscribe button, then in most cases confirm that they are aware of the possible dangers and confirm subscription.² What work is required for a user to subscribe to a new command and get it in their language?

Under option one, the user would somehow have to find the localization scattered someplace on the internet of their own accord, and then install that localization by themself. In my mind, this is clearly a no go. With options two or three, however, when a user subscribes to a command feed Ubiquity can check with the herd to see whether there are any localizations available. The localizations could be offered to the user or the localization for the currently active language could be automatically installed. There are, on the other hand, disadvantages to requiring a centralized authority, exemplified by the fact that the current iteration of the herd itself has often been down.

Summary and a call for comments

As laid out in this visual guide, I personally have a couple main criteria which I believe we should follow:

1. localization independence: Command authors should not have to manage their commands’ localizations. (See footnote 1.)
2. friendly discovery and subscription: Users should not have to go out and find localizations by themselves. Localizations should be offered to the user.

With these criteria, I’m pretty sure the only logical conclusion is that we need some level of centralization, pointing to options two and three. If you think of another option which satisfies these criteria, or disagree with the criteria above, I would love to know. In addition, have you encountered similar problems of localizing distributed resources elsewhere? What worked there?

If this problem has been solved before, there is no need to reinvent the wheel. As far as I can tell, however, this could be a particularly hairy problem.

Related posts:

1. Ubiquity Localization Update
2. Localizing Commands for Ubiquity 0.5
3. Ubiquity Localization: What’s New, What’s Next

Related posts brought to you by Yet Another Related Posts Plugin.

If you are reading this through a feed reader or planet, click on the permalink to view the banner embedded below:

Place one of these personalized canvas-powered virtual tree banners on your site to spread the word on Firefox 3.5, the Shiretoko Park and Foundation, and the power of open communities. All the cool kids are doing it. ^^

Related posts:

1. Attachment Ambiguity—or—when is the gyudon cheap?
2. Report from Mozilla Party JP 10!
3. Ten Grand Is Buried There

Related posts brought to you by Yet Another Related Posts Plugin.

• Goals for 0.5
  • Parser 2 (on by default)
  • underlying support for localization of commands
  • localization of standard feed commands for a few languages
  • Parser 2 language files for those same languages
• Nongoals for 0.5
  • distribution/sharing of localizations
  • localization of nountypes

The overall goal for this release of Ubiquity is to come up with a format and standard for localization. Localizations in Ubiquity 0.5 will only apply to commands bundled with Ubiquity, and the localization files themselves will be distributed with Ubiquity. In a future release we will tackle the problem of localizations for commands in the wild and truly croud-source¹ this process.

Localization Architecture

The localization of Ubiquity commands will use a gettext-style approach where localization files list key-value pairs for different properties and messages of the commands. For Ubiquity 0.5, where we only deal with the standard command feeds bundled Ubiquity, we can simply place all the localization files in ubiquity/standard-feeds/localization. Localization files are organized by source feed, with one localization file per source feed, per language.

The localizable components of commands will include the names, contributors, and help properties, as well as any localizable strings in the command’s preview() and execute() methods. To make strings localizable in preview() and execute(), they must be wrapped in the localize function, _().²

Other localizable components, like names, contributors, and help will not need to be wrapped in the _() function. In addition, as the localization files can only hold values of strings, for values such as names and contributors, the delimiter | can be used to delimit multiple values.

zoom.names=ズーム|ズームして|ズームする|ズームしろ

The Localization Experience

One tool we have planned to help kickstart the localization process is a tool that will automatically create a template of strings that need localization in a user’s commands. I took a first stab at this tool today. Clicking on the “get localization template” link next to each feed in the Ubiquity command list will give you a template which you can then copy into a text file:

Additionally, instructions will later be added to this page to specify how and where to save localizations to test them or perhaps we can add a button that will automatically save it in the right location.

Open Questions

Localization file formats

There are two kinds of file formats for localizations we are considering: .properties and .po, the native gettext format. As an example, here is the same key-value pair in the two formats:

.properties:

# This is a comment
welcomeMessage=Hello, world!

.po:

#. This is a comment (the . is actually optional)
msgid "welcomeMessage"
msgstr "Hello, world!"

The advantage of .properties over .po is that Mozilla natively supports this format with an XUL/XPCOM interface called stringbundle and it is what is used to localize JavaScript in Firefox itself. We actually already have the _() localization function working with the properties file format, following gomita’s great instructions (Japanese) on how to load properties files in using Mozilla’s native stringbundle tools via JavaScript.

The advantage of .po over .properties is that it is the de-facto standard in localization, particularly in the UNIX world. Lots of great tools have been written for it. The adoption of .po could make Ubiquity localization more accessible for more people. Another advantage is that .po files can have keys with spaces, as I note below.

If we do opt to work with .po files, the two libraries I see out in the wild for dealing with .po files are gettext-js (MIT) and jsgettext (LGPL). While I haven’t looked at the libraries in depth yet, so far jsgettext seems to be the winner, as some sections of gettext-js require the use of the prototype.js library.

A “key” question

In either file format, we need a unique way to refer to each localizable string—a key format. As each localization file refers to a command feed, the first collision we must avoid is the command name. With this in mind, we can come up with some trivial keys for the localizable properties: (here, consider the command hello)

• hello.names
• hello.contributors
• hello.help

However, we run into difficulty when we try to come up with keys for the arbitrary text in previews and executes. For example, for a message like “Hello world!” in the preview, we could simply make the key hello.preview.Hello world! but this may be unruly and be prone to typos. In addition, in .properties files keys cannot have certain characters in them, like spaces, so we would have to make the key something like hello.preview.Hello_world! or, stripping symbols and standardizing case, hello.preview.HELLO_WORLD.

Keys could also get very long with this type of key format, although here again .po files may have an advantage as they can stay relatively more legible even with long keys. One option to deal with this would be to optionally supply a key argument to _() so that it is used instead of the automatic key. For example, suppose the hello command’s preview() included this code:

_('This is a really long greeting message. Hello there!','longmessage')

then a localizer would only have to refer to hello.preview.longmessage, not hello.preview.THIS_IS_A_REALLY_LONG_GREETING_MESSAGE_HELLO_THERE.

satyr points out that some commands use another function to incorporate similar actions and messages in both preview() and execute(). In this case, he argues, it wouldn’t make sense to have to keep both localizations (hello.preview.… and hello.execute.…). He suggests that optional keys (mentioned above) could be used without the preview. or execute. infixes, as in hello.longmessage. By taking out the preview and execute namespacing in the localization keys, though, it becomes the command author’s responsibility to not accidentally use strings named “names”, “help”, etc. that will have unintended consequences.

Conclusion

I hope that this blog post gives people an idea of the progress we’ve made in the localization area and gets people thinking about the challenges we still face. We’d love to get your feedback on the localization format and process in Ubiquity, as well as the open problems of the file format and keys.

Related posts:

1. Big Issues and Small Issues with Parser 2
2. Localizing Ubiquity: commands and nountypes
3. Ubiquity Commands by The Numbers

Related posts brought to you by Yet Another Related Posts Plugin.

This past Monday I presented at Tokyo 2.0, Japan’s largest bilingual web/tech community. I presented as part of a session on The Web and Language, which I also helped organize. Other presenters included Junji Tomita from goo Labs, Shinjyou Sunao of Knowledge Creation, developers of the Voice Delivery System API, and Chris Salzberg of Global Voices Online on community translation.

I just put together a video of my Ubiquity presentation, mixing the audio recorded live at the presentation together with a screencast of my slides for better visibility. The presentation is 10 minutes long and is bilingual, English and Japanese.

Ubiquity: Command the Web with Language 言葉で操作する Web from mitcho on Vimeo.

The event also coincided with Dave McClure’s Geeks on a Plane Asia tour, attracting even more interest to the event. In the end it was the largest Tokyo 2.0 event ever.

As I leave Tokyo next month, I’ll be sad to not be able to continue to be a part of Tokyo 2.0. I’ve met a lot of fascinating people and learned a lot at the monthly events. I’ll definitely make sure to schedule them in in my future travels back to Japan and I highly recommend any of you who travel to Tokyo do so as well.

Related posts:

1. Changes to Ubiquity Parser 2 and the Playpen
2. Foxkeh demos Ubiquity Parser: The Next Generation
3. Notes from BarCamp Tokyo 2009

Related posts brought to you by Yet Another Related Posts Plugin.

Changes to Ubiquity Parser 2 + Playpen from mitcho on Vimeo.

All the features covered, as with all Parser 2 features, require that you get the latest Ubiquity code from our Mercurial repository.

Related posts:

1. Foxkeh demos Ubiquity Parser: The Next Generation
2. A Demonstration of Ubiquity Parser 2
3. Adding Your Language to Ubiquity Parser 2

Related posts brought to you by Yet Another Related Posts Plugin.

I’ve been framing this problem as two subproblems:

1. What will be the data structure of localized commands/nountypes within Ubiquity?
2. How do we distribute/share these localizations?

We’ve mostly been discussing the first problem, weighing the merits of unified objects (with different localized text as different JS properties) as opposed to a gettext-style approach, and noting that our requirements for commands and nountypes may be different. I hope we can discuss the second issue more in the coming week.

Should everything go through the command author? Should localization be centralized through some web tool? Should it be completely distributed like commands currently are? I invite you to join us in this conversation on the Google group. ^^

Related posts:

1. Ubiquity Commands by The Numbers
2. Big Issues and Small Issues with Parser 2
3. Localizing Ubiquity: an open letter to linguists

Related posts brought to you by Yet Another Related Posts Plugin.

Read about how this security hole works here. Hopefully flashy demos like this will bring more attention to this issue.

Related posts:

1. Foxkeh demos Ubiquity Parser: The Next Generation
2. Adding Your Language to Ubiquity Parser 2
3. Rolling out the Roles

Related posts brought to you by Yet Another Related Posts Plugin.

You’ve seen the video. You speak another language. And you’re wondering, “how hard is it to add my language to Ubiquity with Parser 2?” The answer: not that hard. With a little bit of JavaScript and knowledge of and interest in your own language, you’ll be able to get at least rudimentary Ubiquity functionality in your language. Follow along in this step by step guide and please submit your (even incomplete) language files!

As Ubiquity Parser 2 evolves, there is a chance that this specification will change in the future. Keep abreast of such changes on the Ubiquity Planet and/or this blog (RSS).

Set up your environment

If you’re new to Ubiquity core development, you’ll want to first read the Ubiquity 0.1 Development Tutorial to learn how to get a live copy of the Ubiquity repository using Mercurial. Once you’ve set up your Firefox profile to use this development version, make sure to try changing the extensions.ubiquity.parserVersion value to 2 in about:config (as seen in this demo video) to verify that Parser 2 is working for you.

As you read along, you may find it beneficial to follow along in the languages currently included in Parser 2: English, Japanese, Portuguese, and Swedish (and the incomplete Chinese and French).

The structure of the language file

Each language in Parser 2 gets its own file which acts as a JavaScript module. You’ll need to look up the ISO 639-1 code for your language… Here we’ll use English (code en) as an example here and the JavaScript language file would then be called en.js and go in the /ubiquity/modules/parser/new/ directory of the repository.

Here is the basic template for a Ubiquity Parser 2 language file:

1
2
3
4
5
6
7
8
9
10
11
12

var EXPORTED_SYMBOLS = ["makeEnParser"];
 
if ((typeof window) == 'undefined') // kick it chrome style
  Components.utils.import("resource://ubiquity/modules/parser/new/parser.js");
 
function makeEnParser() {
  var en = new Parser('en');
 
...
 
  return en;
};

After lines 1-4 which set up the JavaScript module, everything else is wrapped in a factory function called makeLaParser (for Latin) or makeEnParser (for English, en) or makeFrParser (for French, fr), etc. This function initializes the new Parser object (line 7) with the appropriate language code, sets a bunch of parameters (elided above) and returns it. That’s it!

Now let’s walk through some of the parameters you must set to get your language working. For reference, the properties the language parser object is required to have are: branching, anaphora, and roles.

Identifying your branching parameter

  en.branching = 'right'; // or 'left'

One of the first things you’ll have to set for your parser is the branching parameter. Ubiquity Parser 2 uses the branching parameter to decide which direction to look for an argument after finding a delimiter or “role marker” (most often, these are prepositions or postpositions. For example, in English “from” is a delimiter for the goal role and its argument is on its right.

So “John” is a possible argument for the source role, but “Mary” should not be. Ubiquity can figure this out because English has the property en.branching = 'right'.

In Japanese, on the other hand, the argument of a delimiter like から (“from”) is found on the left of that delimiter, so en.branching = 'left'.

In general, if your language has prepositions, you should use .branching = 'right' and if your language has postpositions, you can use .branching = 'left'.

For more info:

• see branching on Wikipedia.

Defining your roles

  en.roles = [
    {role: 'goal', delimiter: 'to'},
    {role: 'source', delimiter: 'from'},
    {role: 'position', delimiter: 'at'},
    {role: 'position', delimiter: 'on'},
    {role: 'alias', delimiter: 'as'},
    {role: 'instrument', delimiter: 'using'},
    {role: 'instrument', delimiter: 'with'}
  ];

The second required property is the inventory of semantic roles and their corresponding delimiters. Each entry has a role from the inventory of semantic roles and a corresponding delimiter. Note that this mapping can be many-to-many, i.e., each role can have multiple possible delimiters and different roles can have shared delimiters. Try to make sure to cover all of the roles in the inventory of semantic roles.

For more info:

• Writing commands with semantic roles
• the proposed inventory of semantic roles
• Wikipedia entry on thematic relations

Entering your anaphora (“magic words”)

  en.anaphora = ["this", "that", "it", "selection", "him", "her", "them"];

The final required property is the anaphora property which takes a list of “magic words”. Currently there is no distinction between all the different deictic anaphora which might refer to different things.

Special cases

Some special language features can be handled by overriding the default behavior from Parser. Many of these features are still in the works, however, so we’d love to get your comments!

Languages with no spaces

If your language does not delimit arguments (or words, more generally) with spaces, there will be a need to write a custom wordBreaker() function and set usespaces = false and joindelimiter = ''. For an example, please take a look at the Japanese or Chinese.

Case marking languages

If you have a strongly case-marked language, you’ll have to write some rules to identify those different cases in wordBreaker() and then add some extra roles for these case markers, but for a number of languages the current design does not allow an elegant solution for parsing such arguments. Updates to this issue will be posted to this trac ticket.

In the mean time, however, if you could write a parser even with only the prepositions/postpositions in your language, that would be a great benefit in getting started in your language. UPDATE: a proposal on how to deal with strongly case-marked languages has been written here: In Case of Case….

Stripping articles

Some languages have some delimiters which combine with articles. For example, in French, the preposition “à” combines with the masculine definite article “le” but not “la”:

1. à + la = à la
2. à + le = au

You can add both “à” and “au” as delimiters of the goal role, but then you will get feminine arguments back with the determiner (e.g. “la table”) while masculine arguments would be parsed without a determiner (e.g. “chat”).

1. “à la table” = “to the table”
2. “au chat” = “to the cat”

One possible solution to this is to write a custom cleanArgument() method. After arguments have been parsed and placed in their appropriate roles, each argument text (say, “la table” or “chat”) are passed to cleanArgument(). You can simply write a cleanArgument() to strip off any “la ” at the beginning of the input and return it and both example inputs will get normalized arguments: “table” and “chat”, respectively. UPDATE: For more up-to-date information on how to deal with these types of articles, please see Solving a Romance Problem.

Test your parser

Now you can go into about:config and change extensions.ubiquity.language to be your language code and restart. All the verbs and nountypes at this point will remain the same as in the English version, but it should obey the argument structure (the word order and delimiters) of your language.¹ If you run into any trouble, feel free to ask for help on the Ubiquity i18n listhost or find me on the Ubiquity IRC channel (mitcho @ irc.mozilla.org#ubiquity). Of course, once you’re at a good stopping point, please contribute your language file to Ubiquity!

More to come…

At this point, you’ve only localized the argument structure of your language… additional work will be required to localize the nountypes and verb names, which is the subject of ongoing discussion… join the Google Group to get in on the discussion!

Related posts:

1. Ubiquity Parser: The Next Generation Demo
2. Rolling out the Roles
3. Foxkeh demos Ubiquity Parser: The Next Generation

Related posts brought to you by Yet Another Related Posts Plugin.

Ubiquity, Oni, and Composability:

but I cannot instruct it to give me list of translated google results:

translate (google foo) to German  // doesn't work

Or email me the resulting list:

email(translate (google foo) to German) // doesn't work

…So how does Oni relate to this? Oni is a browser-based “embedded structured concurrency framework”. It allows you to write asynchronous code as if it was synchronous, adding back the kind-of composibility that is lost when juggling concurrent strands of execution (such as e.g. pending XMLHttpRequests) with ‘conventional’ sequential languages.

Related posts:

1. Ubiquity i18n: questions to ask
2. Where’s The Verb?
3. Count command for Ubiquity

Related posts brought to you by Yet Another Related Posts Plugin.

A while back I created a count command for Ubiquity to count HTML elements on a page, so I’ll share it here. The idea is super simple: select some text on your page and execute count p to get the number of paragraphs, or count a to get the number of links, or count tr to get the number of table rows. This is super useful when reading articles with charts or lists online and you want to know how many there are without doing something like copy-pasting into Excel.

The count command is built using jQuery so it can even understand targets like p.class or a[href=...]. Give it a try! ^^

Related posts:

1. Friendlier command feed subscription
2. selecting from Ubiquity
3. Rolling out the Roles

Related posts brought to you by Yet Another Related Posts Plugin.

The issue of ranking Ubiquity suggestions can be restated as predicting an optimal output given a certain input and various conflicting considerations. Ubiquity (1.8, as of this writing) computes four “scores” for each suggestion:

1. duplicateDefaultMatchScore: 100 by default—lowered if an unused argument gets multiple suggestions (in the words of the code: “reduce the match score so that multiple entries with the same verb are only shown if there are no other verbs.”)
2. frequencyMatchScore: a score from the suggestion memory of the frequency of the suggestion’s verb, given the input verb (currently the first word) or nothing, in the case of noun-first suggestions
3. verbMatchScore: float in [0,1]: (as described here)
   • 0.75 is returned in case there it is a noun-first suggestion (by virtue of the fact that String.indexOf('')==0)
   • 1 if the verb name is equivalent across input-output
   • in [0.75,1) if the input is a prefix of the suggestion verb name
   • in [0.5,0.75) if the input is a non-prefix substring of the suggestion verb
   • in [0.25,0.5] if the input is a prefix of one of the synonyms
   • in [0,0.25) if the input is a non-prefix substring of one of the synonyms
4. argMatchScore: the number of arguments with matching “specific” nountypes, where “specific” is designated by the nountype having property rankLast=false.

With the numeric scores for each of these criteria, a partial order of suggestions is constructed using a lexicographic order: that is, compare candidates first using duplicateDefaultMatchScore, break ties using frequencyMatchScore, if still tied break using verbMatchScore, and if still tied break using argMatchScore. This paradigm of constraints is called “strictly ranked” and a corollary of this is that lower constraints, no matter how well you score on them, can never overcome a loss at a higher constraint. A crucial corollary of this system is that lower constraints’ scores need not be computed if a higher constraint already dooms it to a lower position.¹

Ranking in The Next Generation

One of the goals of Parser The Next Generation is to make noun/argument-first input first-class citizens of Ubiquity, improving their suggestions in particular to the benefit of verb-final languages. Arguments will be split up and tested against different noun types before a verb is even entered into the input, in which case target verbs can be ranked according to the appropriateness of the input’s arguments. As such, I believe the argMatchScore criteria above should either be ranked higher in a strictly ranked model or be allowed to overtake lower scores for the higher constraints in a non-strictly ranked model.

The Parser The Next Generation proposal and demo currently orders using a product of various criteria’s scores, rather than a lexicographic order of strictly ranked constraints. The component factors are:

1. 0.5 for parses where the verb was suggested
2. 0.5 for each extra (>1) object argument (essentially “unused words” in the previous parser)
3. the score of each argument against that semantic role’s target noun type
4. 0.8 for each unset argument of that verb

Each component score is a value in [0,1], so the score is always non-decreasing across the derivation. This offers a natural way to optimize the candidate set creation: if a possible parse ever gets a score below a magic “threshold” value, it is immediately thrown away.

A possible problem with the current Parser TNG scoring model is that it will implicitly hinder verbs and parses with more arguments as it could have more sub-1 noun type score factors—this consideration may be great enough that a weighted additive model should be considered over a multiplicative one.

How do you think we can make Ubiquity’s suggestion ranking smarter? What other factors should be considered, and what factors could be left out?

Related posts:

1. Ubiquity Parser: The Next Generation Demo
2. Ubiquity in Firefox: Focus on Japanese
3. Ubiquity Commands by The Numbers

Related posts brought to you by Yet Another Related Posts Plugin.

Foxkeh demos Ubiquity Parser: The Next Generation from mitcho on Vimeo.

There are some details which are not covered in this introductory video, such as how to deal with case marking languages or languages without spaces. Hopefully this’ll inspire some people to play with the demo, though. I’d love to hear your comments! ^^

Related posts:

1. This week on Ubiquity Parser: The Next Generation
2. Ubiquity Parser: The Next Generation Demo
3. Localizing Ubiquity: an open letter to linguists

Related posts brought to you by Yet Another Related Posts Plugin.