mitcho Michael 芳貴 Erlewine

Linguist. Fifth year PhD student at MIT.


Posts Tagged ‘interface’

The Ubiquity Persistence Project: exploring a persistent Ubiquity in the toolbar

Thursday, August 20th, 2009

It’s often hard to remember Ubiquity’s presence and keystroke without a visual reminder—even I often forget that I could use Ubiquity and end up going to a search engine or using the search bar for some quick lookup task. What if the Ubiquity input were in the toolbar and always visible? How would that affect people’s use of Ubiquity? And what could we make that look like and how would it behave? Today we’re kicking off the Ubiquity Persistence Project, a new Ubiquity initiative to explore what a persistent Ubiquity might look like in the Firefox toolbar.


In order to facilitate this discussion, we created the Persistence tool. With the Persistence tool you can quickly try out new design and interaction ideas, mocking things up with some simple jQuery-powered JavaScript and CSS and see your changes live. The Persistence tool is bundled with our latest Ubiquity beta (install link).

The Ubiquity Persistence Project: exploring a persistent Ubiquity in the toolbar from mitcho on Vimeo.

I just put together a screencast introducing the initiative, demoing the Persistence tool, as well as talking about this project’s relation to the ongoing work on Taskfox. We’ll look forward to your comments and designs! :D

A Visual Guide to Community Command Localization

Monday, July 13th, 2009

A natural language interface is only “natural” if it’s in your natural language. With this mantra in mind, we’ve been making steady progress on the challenging problem of Ubiquity localization. The first fruit of this research is in the localization of the parser and bundled commands in Ubiquity 0.5. Here today is a visual guide on command localization in Ubiquity and different options we can take in attacking the community command localization problem. (more…)

Ubiquity 0.5 日本語紹介ビデオ

Thursday, July 2nd, 2009

今夜リリースされる Ubiquity の最新版、0.5 に備えて日本語で Ubiquity のスクリーンキャストを作ってみました。 Ubiquity 0.5 は特に多言語化を重視したリリースで、 Ubiquity 内蔵のコマンドが日本語とデンマーク語で使えるようになっています。是非インストールしてみてください!

追伸: 7月3日現在、 Ubiquity 0.5 のリリースを遅らせる方向になったので、残念ながら今日はリリースされません。是非リリース後インストールしてみてください。

Ubiquity 0.5 日本語紹介ビデオ from mitcho on Vimeo.

As Ubiquity 0.5 will be released soon (Thursday morning in Mountain View), I decided it was a good time to put together a screencast in Japanese demoing the use of the new Japanese parser and commands.

The Hit List: Better Software Through Less UI

Wednesday, March 25th, 2009

The Hit List is a to-do list app for Mac OS X with a beautiful interface and some nice features. Creator Andy Kim’s latest blog post (Better Software Through Less UI) is excellent reading for the Ubiquity community. He describes the thought process behind the design of a new clean and “frictionless” interface for specifying how tasks are repeated. After throwing out the regular combinations and templates of different input widgets, his solution was to implement a partial natural language input interface:

There is no myriad of buttons and fields to choose from. All the user has to do is directly type in what he wants.

Here are a couple other choice quotes which will ring true for the Ubiquity users and internationalization folks in the audience:

For this to work without driving the user mad, the natural language parser has to be near perfect. The last thing I want is for this to come out smelling like AppleScript.

This design isn’t perfect as it has two glaring problems. One is that the user has no easy way of discovering how complex the recurrence rules can be. This isn’t such a huge problem, but a way to solve this is to include a help button to show example rules or to include an accompanying iCal style UI to let the user setup the recurrence rule in a more typical fashion. I didn’t include these in the initial implementation though because I wanted to see how users would react to this kind of UI.
Another problem is localization. Even if I write parsers for a few more popular languages, it won’t accommodate the rest of the users in the world. Again, the solution is an accompanying traditional UI, but for now, I’m leaving it the way it is until I get some feedback.

There’s a trend in the wind, my friends: the incorporation of near-natural language for more humane interfaces.

Ubiquity Parser: The Next Generation Demo

Wednesday, March 18th, 2009


A week or two ago while visiting California, Jono and I had a productive charrette, resulting in a new architecture proposal for the Ubiquity parser, as laid out in Ubiquity Parser: The Next Generation. The new architecture is designed to support (1) the use of overlord verbs, (2) writing verbs by semantic roles, and (3) better suggestions for verb-final languages and other argument-first contexts. I’m happy to say that I’ve spent some time putting a proof-of-concept together.

I’ve implemented the basic algorithm of this parser for [[left-branching]] languages (like English) and also implemented some fake English verbs, noun types, and semantic roles. This demo should give you a basic sense of how this parser will attempt to identify different types of arguments and check their noun types even without clearly knowing the verb. This should make the suggestion ranking much smarter, particularly for verb-final contexts. (For a good example, try from Tokyo to San Francisco.)

➔ Check out the Ubiquity next-gen parser demo


User-Aided Disambiguation: a demo

Saturday, March 14th, 2009

A few weeks ago I made some visual mockups of how Ubiquity could look and act in Japanese. Part of this proposal was what I called “particle identification”: that is, immediate in-line identification of delimiters of arguments, which can be overridden by the user:

The inspiration for this idea came from Aza’s blog post “Solving the ‘it’ problem” which advocates for this type of quick feedback to the user in cases of ambiguity. Such a method would help both the user better understand what is being interpreted by the system, as well as offer an opportunity for the user to correct improper parses. I just tried mocking up such an input box using jQuery.

Try the User-Aided Disambiguation Demo

If you have any bugfixes to submit or want to play around with your own copy, the demo code is up on BitBucket. ^^ Let me know what you think!

Friendlier command feed subscription

Monday, February 23rd, 2009


If you’ve ever subscribed to a new Ubiquity command before, you know the red screen of doom. Ubiquity currently takes users to this page every time they wish to subscribe to a new command. The current design is meant to encourage users to be aware of the possible security implications of enabling and executing a command, to avoid getting a [[trojan horse]].

The current screen, however, does not make subscribing to commands foolproof. I personally know I’ve subscribed to a number of commands without reading through the code, defeating the purpose of the anti-trojan horse display. Moreover, the page doesn’t give you any information on how you can use this new command. Especially given the inherent limited discoverability of a natural language interface, taking a moment to help the user actually learn the command becomes key.

Today I did a quick mockup of what a friendlier command feed subscription page might look like. Take a look at this screenshot with some of the features marked:


You can also check out the page itself. If you’d like to visualize it without the “trust” warning, you can also view the trusted version.

This mockup here is but a first iteration. What do you think about this subscription page? What is missing? What should be changed?

Ubiquity in Firefox: Focus on Japanese

Friday, February 20th, 2009

One of the eventual goals of the Ubiquity project is to bring some of its functionality and ideas to Firefox proper. To this end, Aza has been exploring some possible options for what that would look like (round 1, round 2). All of his mockups, however, use English examples. I’m going to start exploring what Ubiquity in Firefox might look like in different kinds of languages. Let’s kick this off with my mother tongue, Japanese.1

今後多様な言語に対応したFirefox内のUbiquityを検討していきますが、その中でも今日は日本語をとりあげます。後日日本語で同じ内容を投稿するつもりです。^^ 日本語でのコメントも大歓迎です!


How natural should a natural interface be?

Monday, February 16th, 2009

I’m very happy to announce that, starting today, I will be working full-time on Ubiquity, a Mozilla Labs experiment to connect the web with language. I’ll be heading up research on different linguistic issues of import to a linguistic user interface and blogging about these topics here. If you’re interested, please subscribe to my blog’s RSS feed or the RSS feed for only Ubiquity-related items. Commenting is encouraged! ^^

Every day, more users are trying out Ubiquity, the Mozilla Labs experiment that lets users accomplish common Internet tasks faster through a natural language interface. As we live more and more of our lives on the web, there is a huge appeal to—and need for—a faster way to access and mashup our information.

But what exactly do we mean by a “natural language interface”? Is it just another programming language with lots of English keywords? Should the final goal be a computer that understands everything we tell it?

Ubiquity is not HAL

As we think about the future directions and possibilities of Ubiquity, we need to go back to our roots and understand the project’s motivations. With that in mind, here are some initial thoughts on the advantages of a natural language interface. The ultimate goal here is to refine the notion of natural language interface and to come up with a set of principles that we can follow in pushing Ubiquity further, into other languages and beyond.