As Ubiquity 0.5 will be released soon (Thursday morning in Mountain View), I decided it was a good time to put together a screencast in Japanese demoing the use of the new Japanese parser and commands.
As many of you know, earlier this week we released a preview of version 0.5 (0.5pre). We’re going to stress test and refine this release through the weekend and push the official 0.5 out next Tuesday. This release will have fully localized commands for Danish and Japanese, as well as parser settings for a number of other languages. Read this Labs blog post to learn more about the 0.5 release and how to test it.
It’s not too late to add localizations for other languages to 0.5, though. Localizations help make Ubiquity more “natural” for more users, offering a new level of ease and familiarity to the already powerful Ubiquity. We have a new tutorial to help you localize commands.
To help encourage command localization, we now have gettext-style po template files for all the bundled command feeds in the hg repository. You can find these files in the ubiquity/localization/templates directory of the repository, or on our online hg repository.
If you complete some localizations (even incomplete) for your language and would like to submit them into the repository, for the time being, you can post them on this trac ticket.1
As we get close to wrapping up Ubiquity 0.5 (currently planned to ship—fingers crossed—on Monday) one remaining issue is how to incorporate our cute new Cocoia-designed and community-produced icon, the Ubiquibot. The difficult decision is how to take this finely detailed icon and produce a 16 x 16 favicon.
I came up with three different options:
1. 2. 3.
Seeing them on my blog doesn’t quite compare to how they will be used, so here are some screenshots of them in context:[^1]
By now many of you have probably seen this new Microsoft Australia campaign, “Ten Grand Is Buried Here.com,”1 which calls Firefox “old” and Safari “boring”:
I’m not sure what this is saying about me, but my immediate reaction was to go check whether tengrandisburiedthere.com was available. To my surprise, Microsoft had yet to snatch it up! A few hours later, here’s the result:
Note: Not being a marketing guy, I just threw some text together to introduce Firefox. If someone has some better copy for this display, please let me know.
As of this writing, this domain actually has yet to serve anything. ↩
localization of standard feed commands for a few languages
Parser 2 language files for those same languages
Nongoals for 0.5
distribution/sharing of localizations
localization of nountypes
The overall goal for this release of Ubiquity is to come up with a format and standard for localization. Localizations in Ubiquity 0.5 will only apply to commands bundled with Ubiquity, and the localization files themselves will be distributed with Ubiquity. In a future release we will tackle the problem of localizations for commands in the wild and truly croud-source1 this process.
This past Monday I presented at Tokyo 2.0, Japan’s largest bilingual web/tech community. I presented as part of a session on The Web and Language, which I also helped organize. Other presenters included Junji Tomita from goo Labs, Shinjyou Sunao of Knowledge Creation, developers of the Voice Delivery System API, and Chris Salzberg of Global Voices Online on community translation.
I just put together a video of my Ubiquity presentation, mixing the audio recorded live at the presentation together with a screencast of my slides for better visibility. The presentation is 10 minutes long and is bilingual, English and Japanese.
Here’s a quick screencast highlighting some of the changes to Parser 2 and the updated Parser 2 Playpen. This video should be particularly useful to people hoping to add their language to Parser 2. It’s also a good reference for Ubiquity core developers.
Now that Parser 2 is in decent shape and a number of parsing problems in different languages have been tackled, the focus has now shifted to coming up with an approach for localizing Ubiquity commands and nountypes. At last week’s weekly Ubiquity meeting we had a great conversation on this subject, which then has continued on the Google group.
I’ve been framing this problem as two subproblems:
What will be the data structure of localized commands/nountypes within Ubiquity?
How do we distribute/share these localizations?
We’ve mostly been discussing the first problem, weighing the merits of unified objects (with different localized text as different JS properties) as opposed to a gettext-style approach, and noting that our requirements for commands and nountypes may be different. I hope we can discuss the second issue more in the coming week.
Should everything go through the command author? Should localization be centralized through some web tool? Should it be completely distributed like commands currently are? I invite you to join us in this conversation on the Google group. ^^
Jono and I had a good conversation this morning on IRC about the remaining Big Issues which are blocking the release of Parser 2 as the default parser for Ubiquity. Here are our Top 4 Big Issues:
Some commands’ preview’s and execute’s are not working properly (trac #652). This could be an underlying issue with some pipes not rerouted correctly in Parser 2, or it could be that the commands have not been rewritten correctly to take advantage of Parser 2.
Flesh out how to localize resources, like commands and nountypes. We started a conversation on this subject a few weeks ago but we never reached a resolution. This blocks issues 3 and 4 below.
We need to standardize a format for commands for Parser 2. As noted in last week’s meeting (among other places) Parser 2 will require at least some modification to all commands. Jono and I came up with a simple hybrid format for commands which specify takes and modifiers for Parser 1 and arguments for Parser 2, but until we figure out how exactly the localization of commands will work, we can’t write a definitive standard.
Enable nountype localization. While the most popular nountypes used are those that ship with Ubiquity, it is important to come up with a localization process which can apply to custom nountypes as well. Nountype localizations need the ability to either (1) replace the _name only, or (2) replace both the _name and the suggest() logic, as both cases will be necessary.
Given that Big Issue 3 and Big Issue 4 are both dependent on Big Issue 2, there clearly needs to be a continued public discussion of how we should make these resources localizable. I look forward to this discussion taking place at tomorrow’s joint (general + i18n) Ubiquity meeting.
In other news, here are some Small Issues:
Add a switch for parser version and language settings: Jono’s already made a space for this in the new “Settings and Skins” page in about:ubiquity. He’s on it. Like a bonnet.
Magic word (anaphor) substitution is not yet working properly. This needs to work both when there is an explicit magic word and when there are simply missing arguments.
The position of suggested verbs is always sentence-initial (trac #655). This also requires that we can specify whether verb name localizations are sentence-initial forms or sentence-final forms.1
Thanks to the great work of Sandro Della Giustina, we now have a preliminary Italian parser for use with Ubiquity Parser 2. Sandro brought up a good point, however, about Italian prepositions which contract with the article and the head noun. For example,
traduci dall'inglese al cinese
translate from=the=English to=the Chinese
One current solution is to add zero-width spaces after these contracted articles, all’ and dall’.1 The appropriate way to add this to the parser is by defining a custom wordBreaker() method.
As John Daggett pointed out to me, in the future we may have to add an intermediate shallow parse instead of adding characters (in this case, the zero-width space) to the modified input. ↩
Here’s a little picture of the different sections of text in a single parsed argument and which properties of the resultant argument object they are assigned to.
You’ll see, from left to right, outerSpace, modifier, innerSpace, inactivePrefix, input/data, inactiveSuffix.
The example text is from the Catalan example, “compra mitjons amb el Google,” meaning “buy socks with Google.” You’ll notice the argument “amb el Google” is literally “with the Google.” The normalizeArgument() method of the Catalan parser, as I described earlier this week, strips the article “el ” and puts it in the inactivePrefix property of the argument.
I’m going to spend the rest of the day updating Parser 2 design doc and related documentation so they match these and other recent developments in the parser.
Weak pronouns in romance languages (as well as some other languages) have a special property where they cliticize to the verb, moving from its regular argument position to a position next to the verb. For example, in French, we have an imperative like (1) with gloss as (2):
1
2
Envoyez le lettre à Pierre!
send.IMP the letter to Pierre
If we replace le lettre or à Pierre with a preposition (le, “it”, or lui, “to him”, respectively), those weak pronouns move next to the verb—in particular, (5) exemplifies the change in word order. Replacing both arguments with prepositions creates the stacked clitic form of (7).1
3
4
5
6
7
8
Envoyez-la à Pierre!
send -it to Pierre
Envoyez-lui la lettre!
send -him the letter
Envoyez-le-lui!
send -it-him
The fact that these weak pronouns are attached to the verb and lack separate delimiters mean that we will need a separate mechanism to parse these arguments: indeed, this functionality has been planned in Ubiquity Parser 2 as “step 3”. Here I’ll examine some data and discuss a strategy for the parsing of weak pronouns.
Note that the reverse order of “Envoyez-lui-le” is ungrammatical… fortunately we most likely will not have to deal with multiple clitics… see footnote two below. ↩
In many romance languages, prepositions and articles often form portmanteau morphs, combining to form a single word.1 Some examples include (French) à + le > au, de + le > du, (Catalan) a + el > al, de + les > dels, per + el > pel. Italian has a particularly productive system of portmanteau’ed prepositions and articles… I refer you to the contraction article on Wikipedia.
Thanks to Jeremy O’Brien for helping me figure out how to refer to this phenomenon. ↩
This also relates to the issue of parsing multi-word delimiters, though the argument normalization strategy covered here should reduce the necessity of multi-word delimiters. ↩
Grammatical case is a marking on nouns that express grammatical function. Not all languages exhibit case. In many of the Indo-European languages we hope to bring Ubiquity to, case is realized as a suffix.1
Here’s a classic example of case from Latin. (Line 2 is the gloss of 1, line 4 of 3.)
Example (1) is “the man bit the dog,” while example (3) is “the dog bit the man.” The only difference, as you see in the gloss, is that the nouns canis and vir are marked with different case endings in the two sentences. By marking the nouns with different cases (here, nominative and accusative), their semantic roles in the sentence—which is the the biter and which is the bitee—can be identified unambiguously. (Their positions are also switched in these examples but in reality Latin has a very free word order—the same sentences with other word orders including OSV or VSO are also common.)
At first glance, strongly case-marked languages may look like a godsend for identifying the semantic roles of arguments.2 If we can easily and unambiguously recognize arguments’ cases to put them in their appropriate semantic roles, this could simplify processing as well as make Ubiquity input follow a natural syntax for such languages. Unfortunately, there are some significant challenges which must be overcome in order to make the processing of case-markers worthwhile.
Note that when linguists talk about “case,” they could be referring to two different (though related) concepts: case (lowercase) is the observed pattern of affixes on nouns which indicate grammatical function, while Case (uppercase) refers to a theoretical (formal) feature of syntactic objects—certain lexical items “assign Case” or “receive Case” and its mismatches were ruled out in GB syntax by the Case Filter. You’ll find GB linguistics papers referring to “case” when discussing Mandarin Chinese, for example, a language that doesn’t have any overt case (lowercase) and you’ll know immediately that this usage is an uppercase Case case. In this blog post I’ll be dealing primarily with the former descriptive notion. ↩
When I refer to “strongly case-marking languages,” I am referring to languages with a non-trivial inventory of cases (not just nominative, accusative, and genitive) and where a noun phrase’s case is not reflected on determiners. For example, German is excluded by this definition as case is realized exclusively on articles and there is no need to find and parse the noun head itself to identify its case—more information on German is in the section “finding the edges.” ↩
Here’s a quick demonstration of Ubiquity Parser 2, aka “the new parser.” I’ll show you how you can use the parser yourself and point out some highlights of the new functionality.
2. 
3. 
