Everyone I know in linguistics uses the LINGUIST List website to a greater or lesser degree. Linguist List began as a mailing list in the 90’s, with book, job, and dissertation announcements, call-for-papers, and general academic discussions.

Nowadays many people follow the various announcements on Linguist List using an RSS feed reader such as Google Reader or my personal favorite NetNewsWire.

Unfortunately, the Linguist List RSS feeds (at least recently) don’t include the full text of the articles and have a few other quirks as well. It’s often hard to judge based on the title whether it’s really something I’m interested or not, so I’ve spent a lot of time frustratedly opening any possibly interesting-looking entry in a separate NetNewsWire tab. Today I decided enough was enough: I just wrote a script which parses each of the Linguist List RSS feeds, finds the actual descriptions and interleaves them.¹ It’s working remarkably well so far:

Here are all the RSS links for all of you to subscribe to. I plan on keeping this up and maintained for the foreseeable future (or, until Linguist List improves their own RSS feeds!) so feel free to subscribe to it. Please do let me know if you run into any issues or have a suggestion.

Most recent issues
All (announcements)
Book announcements
Call for papers
Conference announcements
Discussions
Dissertation announcements
FYI
Job announcements
Topics in the Media
Queries
Book reviews
Software announcements
Summaries of Query Responses
Support for Students

You can find descriptions of the content of each of these feeds on Linguist List’s RSS feeds page.

Perhaps in the future I’ll do a “how to” post as well with the code I use to make this happen. I’ll note that it’s just 70 lines of PHP (including whitespace) and could no doubt be tightened up.

Related posts:

1. Automating the Linguist’s Job
2. The Hit List: Better Software Through Less UI
3. Localizing Ubiquity: an open letter to linguists

Related posts brought to you by Yet Another Related Posts Plugin.

JSConf.us:

I’ll be putting my Mozilla Jetpack Ambassador hat on to represent Mozilla Labs’ Jetpack project at the premier Javascript conference in North America, JSConf.us, which this year will be April 17-18 in Washington D.C. and has a pirate theme.¹ I’ll be giving a short talk in the main session and will also lead a hands-on Jetpack workshop in the hacker lounge. I’ve heard that JSConf is a lot of fun and I’m really looking forward to it!

WordCamp San Francisco:

I’m honored to have been invited to give a talk at WordCamp San Francisco 2010. WordCamps are community-organized events for the WordPress community, and the San Francisco WordCamp is the original and biggest. WordCamp SF will be at the Mission Bay Conference Center on May 1st. Tickets available.

My talk is tentatively titled “Abstract Your Code.”² WordPress is a great platform to build amazing content-rich applications on, and many of us have written new functionality in the form of plugins. I hope to encourage developers to make their code more portable and reusable after the project is done—or, ideally, to even start with abstraction in mind—to add to the “life” of the code and to consider then open-sourcing that functionality.

Hope to see you there!

International Association of Chinese Linguistics (IACL) 18:

Finally, I’m thrilled to say that I got a paper accepted to the annual meeting of the International Association of Chinese Linguistics which this year is at Harvard on May 20-22. IACL is the big conference for Chinese linguistics, with about 180 papers presenting. I’ll be presenting Two Onlys in Mandarin Chinese, my recent work on the formal syntax/semantics of two only words in Chinese: zhǐ (只) and éryǐ (而已). I’ve put up a handout of some of this material in work-in-progress form which I recently presented at SNEWS.

Related posts:

1. WordCamp Boston 2010
2. Jetpacking in Boston
3. Mashing up the browser in Maine

Related posts brought to you by Yet Another Related Posts Plugin.

We see photographers on the street from time to time taking pictures of the building. In fact, the first day I got to Boston this fall, I walked into my hotel room and there was a framed photo of Stata on the wall.

The department basically occupies the eighth and ninth floors of the Dreyfoos tower side. There’s a nice common space with some red couches and a spiral staircase.

All of us first years share an office together on the ninth floor (32-D976). The room is elliptical, with our desks along the side with windows and the other “wall” being a series of wooden bookshelves. It’s great to share an office with the cohort, as we all take all the same classes the first year as well.

Here’s the room for “Meg Data” just across the hall from our office. I was really hoping it was a woman named “Meg Data”… marrying her would get me one step closer to becoming “Dr. Data.” Alas, “Meg Data” or “MEG Data” apparently is simply a label for some data store—possibly some servers.

The majority of the building is the Computer Science and Artificial Intelligence Lab (CSAIL) so there are posters up for fun-looking talks that I’m too busy for.

Here’s my desk (with Foxkeh!) and my new bike which I’ve been bringing up to the office. The final photo is a photo of me with an “I ♥ The Web” poster in commemoration of OneWebDay.

We’ve been getting pretty busy with the readings and work for our classes, as well as occasionally hanging out. I hope to write some more posts both linguistic and non-linguistic in the coming days on life at MIT.

Related posts:

1. I’m Busy to Die
2. Dinner with Barack and Hillary
3. Linguistics in 嘉義

Related posts brought to you by Yet Another Related Posts Plugin.

The keynote speaker was Ralf Steinberger of the European Commission’s Joint Research Centre, presenting the European Media Monitor family of applications. EMM is a suite of different applications all based on a core platform which aggregates news stories from a variety of sources around the world in a few dozen different languages. The system uses various CLIR techniques to then cluster stories by event, regardless of language or source country. Large news agencies and organizations, as well as the European Commission itself, use the system to track upcoming news stories as well as health concerns. In the European Union which has over a dozen different “official” languages, there is a great need for this type of service. The applications are available to the public so I invite you to play around with them at the EMM site.¹

The workshop’s paper presentations were quite diverse and all interesting. Elena Filatova presented an interesting approach to measuring the “trustworthiness” of statements in Wikipedia entries, by comparing the overlap in content statements between different language entries (more overlap => more trustworthy) which can be used to create inverted pyramid summaries. Perspectives from different use cases were also examined through presentations on news analysts and health organizations, patent searches, and medical information for personal use, as well as discussion of the need for CLIR for historical and religious texts.

One interesting thread throughout the day’s sessions was the issue of loanword processes and the possible use of romanization as an interlingua. Cross-language named entity recognition is a major problem in CLIR. Many novel words and names (common in news articles) go through processes of loanword adaptation and transliteration and are hard to identify and also are not in the systems’ dictionaries. A few of the talks touched on these problems, including a great talk by Kashif Riaz of the U of M on the salient (and great) differences between Hindi and Urdu. Frederic Gey described a number of different approaches to comparing strings via a romanization interlingua.

Another interesting thread was the idea of the target user. The needs of CLIR applications can vary greatly depending on its use case and the users’ savvy. The needs of a patent search office, where many professional searchers are already multilingual, is clearly different from the needs of an executive hoping to stay on top of the world news related to their organization. It was brought up, however, that in this age of open data and API’s, if a CLIR resource provides a good API, it need not necessarily supply the perfect interface for every type of user, as different third parties could also develop such targeted interfaces.

While most of the presentations and research interests in the room were on users accessing resources in various languages, I presented on Ubiquity. The talk was intended to highlight the idea that the opposite problem of users with different languages getting at the same kinds of information and getting equally powerful user experiences is also a different but worthwhile problem. Below are the slides from this talk. As a web archive of all the papers will be set up soon, I believe it’s safe to put up my paper as well, so please check it out. It’s a good short (four pages) overview of the innovative approaches we’ve taken to build a localizable natural language interface.

Related posts:

1. Presenting in Boston at SIGIR Workshop
2. The Hit List: Better Software Through Less UI
3. Talking Ubiquity in Japan: 拡張機能勉強会にて発表

Related posts brought to you by Yet Another Related Posts Plugin.

Of course, one of the best ways to learn about the new features is to experience them… try Ubiquity 0.5 now!

Install now!

Preface: What’s What

To give users a completely localized experience, there are many different components that need to be made to work with different languages. In a single Ubiquity input, like

1

translate hello from English to Spanish

there are actually many different components that need to all be localized in order to comprehend the equivalent sentence in a different language. The diagram below will give you a sense for the different components that need to be localized: the parser, verbs, and nountypes.

What’s New

Ubiquity 0.5’s improved language support can be thought of as the product of two more or less orthogonal developments: the brand-new parser, Parser 2, as well as local command localization support.

Parser 2

Parser 2 (née Parser: The Next Generation) is a completely new parser designed to support different languages easily. Taking a serious look at the similarities and differences between different languages, we created a universal parser design which takes a minimal set of settings for particular languages to “learn” that language’s grammar.

The key insight to Parser 2’s design is that, for the limited range of inputs Ubiquity should understand, languages deal with them in remarkably similar ways. The input we’re dealing with here are all commands or actions without quantification or negation. These are all comprised of a single verb and a series of arguments with certain markings to designate their roles in the sentence. For example, here’s our example Ubiquity input:

1

translate hello from English to Spanish

In this example, “translate” is the verb, which we recognize by looking at our bank of known verbs, and the rest of the input can be split up into three different arguments: “hello,” “from English,” and “to Spanish.” Of these, the markers “from” and “to” tell us that “English” is a source of some sort and “Spanish” is a goal, while the unmarked “hello” is simply an object—the target of the action. By identifying arguments by these abstract semantic roles, we’re able to quickly identify different kinds of arguments in different languages. For example, the following is the exact same example but using the Japanese syntax and markers:

2

helloをEnglishからSpanishにtranslate

Ubiquity knows what the different markers mean in Japanese, like “を” > object, “から” > source, “に” > goal, and can easily interpret this to mean the exact same command as (1). With just a few lines of code, you can teach Ubiquity how to recognize these different semantic roles in your language. This innovation also means that Ubiquity commands can be written once for one language and automatically used with another language’s parser, bringing us half-way to the goal of command localization.

Note also that Japanese (as in example (2)) is verb-final and uses no spaces between words. We’ve tried to make Parser 2 itself agnostic towards these types of different ways in which languages vary.

Parser 2 also adds better argument-first suggestions, inspired by some earlier thoughts on Ubiquity in Japanese. Ubiquity will now start to parse arguments in the input even if a verb isn’t found, and suggest verbs based on that input. For example, if you enter “hello to Spanish,” it’ll recognize that you have an object of “hello” and a goal of “Spanish” which can be understood as a language name, so it’ll suggest the verb “translate.” This is the way it should be.

For more information and background, feel free to check out some of my previous blog posts on the new parser and on the different linguistic considerations. I also have a four-page academic paper giving an overview of some innovations in the parser—email me at x@x.com where x=mitcho if you’d like to get a copy.

Internationalization of bundled commands

The move to use semantic roles in the new command API, described above, means that the same Ubiquity command code can be used with inputs in different languages. Two things are left, then, to make a completely localized input work: (1) translation (localization) of different strings in the commands and (2) localization of the nountypes.

In Ubiquity 0.5, we built a localization infrastructure for commands (1, above) but have not yet tackled the nountypes (2). Ubiquity 0.5 uses the gettext po (portable object) file format for localizations, which many localizers in the UNIX world are very familiar with. This choice of file format potentially opens Ubiquity localization up to many who are new to localization or are unfamiliar with other Mozilla localization. Ubiquity is able to produce localization templates by itself and we also have a great tool to check the completeness of different localizations.

A huge caveat, however, is that this localization support currently only works with the commands bundled with Ubiquity itself.

What’s Next

We’re going to continue working to make Ubiquity more natural for more users. The tasks we have ahead of us are the localization of nountypes and community commands.

Nountype localization

With the new semantic role argument specifications, command localization simply became a question of translating some strings, which many localizers are used to. After all, we want localizations to affect the presentation of commands, not the logic of the commands. When it comes to nountypes, however, it is quite possible that we would actually want the nountype localization to affect its behavior.

Consider, for example, an imaginary day_of_the_week nountype. In English, this nountype might accept or suggest strings like “Monday” or “Tuesday,” while a French localization would accept “lundi” or “mardi.” More complicated still, consider a date nountype. In English this nountype may have custom logic to parse strings like “June 1st” while another language may have to parse very different kinds of strings. These nountype localizations thus involve not just string translations, but actual changes in their logic, making the po format approach we took to command localization a poor fit.

Making nountypes localizable, however, will make Ubiquity significantly more “natural” for many users. In the coming weeks and months we’ll be discussing and debating different options to accomplish this.

Community command localization

Even though the file format and infrastructure for command localization itself has been fleshed out with Ubiquity 0.5, the distributed nature of all these community commands adds an additional complication. Do we want community command localizations to be completely distributed, or should they be centralized? If they’re distributed, how do you find them? These are the types of questions we’ll need to ask and answer. The ease of creating a new Ubiquity command and sharing it with the world is a huge asset of the platform, so we’ll definitely be thinking about how best to localize these community commands as well. In the next day or two I’ll be writing up a more detailed blog post on what we need from a good community command localization solution.

Summary

For the more visually inclined (including myself), here’s a handy diagram to summarize what components are localizable now, what will be in the future, and what this means for Ubiquity users of different languages.

Get Involved

Whether you’re a speaker of an as-yet unsupported language, a veteran localization contributor, or simply interested in seeing how we can offer this natural language interface to more languages and more users, there are lots of ways to get involved. If you have some JavaScript experience and want to teach Ubiquity your native languages’ grammar, read our parser localization tutorial. If you would like to contribute localizations for our built-in commands, there’s a command localization tutorial. To discuss how best to localize nountypes and community commands, or to ask questions about or discuss command and parser localization, join us on the Ubiquity-i18n mailing list.

Related posts:

1. Localizing Commands for Ubiquity 0.5
2. Localizing Ubiquity: commands and nountypes
3. Big Issues and Small Issues with Parser 2

Related posts brought to you by Yet Another Related Posts Plugin.

This past Monday I presented at Tokyo 2.0, Japan’s largest bilingual web/tech community. I presented as part of a session on The Web and Language, which I also helped organize. Other presenters included Junji Tomita from goo Labs, Shinjyou Sunao of Knowledge Creation, developers of the Voice Delivery System API, and Chris Salzberg of Global Voices Online on community translation.

I just put together a video of my Ubiquity presentation, mixing the audio recorded live at the presentation together with a screencast of my slides for better visibility. The presentation is 10 minutes long and is bilingual, English and Japanese.

Ubiquity: Command the Web with Language 言葉で操作する Web from mitcho on Vimeo.

The event also coincided with Dave McClure’s Geeks on a Plane Asia tour, attracting even more interest to the event. In the end it was the largest Tokyo 2.0 event ever.

As I leave Tokyo next month, I’ll be sad to not be able to continue to be a part of Tokyo 2.0. I’ve met a lot of fascinating people and learned a lot at the monthly events. I’ll definitely make sure to schedule them in in my future travels back to Japan and I highly recommend any of you who travel to Tokyo do so as well.

Related posts:

1. Changes to Ubiquity Parser 2 and the Playpen
2. Foxkeh demos Ubiquity Parser: The Next Generation
3. Notes from BarCamp Tokyo 2009

Related posts brought to you by Yet Another Related Posts Plugin.

In many romance languages, prepositions and articles often form portmanteau morphs, combining to form a single word.¹ Some examples include (French) à + le > au, de + le > du, (Catalan) a + el > al, de + les > dels, per + el > pel. Italian has a particularly productive system of portmanteau’ed prepositions and articles… I refer you to the contraction article on Wikipedia.

As I noted a couple weeks ago, however, some combinations do not form portmanteaus.²

French:

1. à + le > au
2. à + la > à la

The problem with this is that if we use both à and au as delimiters, we may end up passing the definite article to the verb as part of the argument in some cases, but not in other cases.

1. “à la table” = “to the table”
2. “au chat” = “to the cat”

The solution:

The solution is a new step in the Parser 2 process which normalizes the form of arguments. Each language’s parser can now optionally define a normalizeArgument() method which takes an argument and returns a list of normalized alternates. Normalized arguments are returned in the form of {prefix: '', newInput: '', suffix: ''}. For example, if you feed “la table” to the French normalizeArgument(), it ought to return

[{prefix: 'la ', newInput: 'table', suffix: ''}]

If there are no possible normalizations, normalizeArgument() should simply return []. Each alternative returned by normalizeArgument() is substituted into a copy of the possible parses just before nountype detection. The prefixes and suffixes are stored in the argument (as inactivePrefix and inactiveSuffix) so they can be incorporated into the suggestion display.

Here, for example, is how the inactive prefix “l’” is displayed in the parser demo. This way the user is told that the “l’” prefix is being ignored, and the nountype detection and verb action can act on the argument “English”.³ (In the future, of course, we could teach this nountype to accept the Catalan “anglès”.)

The easiest way to produce this output is to use the String.match() method. For example normalizeArgument() code, I refer you to the Catalan and French parser files.

I hope that this solution will help make Ubiquity with Parser 2 feel more natural for many romance languages.

Related posts:

1. Where’s The Verb?
2. Adding Your Language to Ubiquity Parser 2
3. Scoring and Ranking Suggestions

Related posts brought to you by Yet Another Related Posts Plugin.

Introduction

Grammatical case is a marking on nouns that express grammatical function. Not all languages exhibit case. In many of the Indo-European languages we hope to bring Ubiquity to, case is realized as a suffix.¹

Here’s a classic example of case from Latin. (Line 2 is the gloss of 1, line 4 of 3.)

1
2
3
4

canis      virum      momordit
dog=sg.NOM man=sg.ACC bite=3sg.perfect
vir        canem      momordit
man=sg.NOM dog=sg.ACC bite=3sg.perfect

Example (1) is “the man bit the dog,” while example (3) is “the dog bit the man.” The only difference, as you see in the gloss, is that the nouns canis and vir are marked with different case endings in the two sentences. By marking the nouns with different cases (here, nominative and accusative), their semantic roles in the sentence—which is the the biter and which is the bitee—can be identified unambiguously. (Their positions are also switched in these examples but in reality Latin has a very free word order—the same sentences with other word orders including OSV or VSO are also common.)

At first glance, strongly case-marked languages may look like a godsend for identifying the semantic roles of arguments.² If we can easily and unambiguously recognize arguments’ cases to put them in their appropriate semantic roles, this could simplify processing as well as make Ubiquity input follow a natural syntax for such languages. Unfortunately, there are some significant challenges which must be overcome in order to make the processing of case-markers worthwhile.

The case against case

There are broadly three different difficulties with dealing with strongly case-marking languages: (1) how to identify case correctly, (2) how to identify the boundaries of the arguments, and (3) what case to use when handing the arguments to the verb’s preview and execution.

Parsing for case

In some languages, it is very easy to recognize different case endings. For example, for Turkish it would be relatively easy to write a regular expression for each of the cases below, even with the vowel harmony as exhibited in the genitive and accusative cases between i and ü.³

(Example from Turkish language on Wikipedia.)

However, in many other languages identifying case affixes can be quite difficult as they vary greatly depending on the root noun, not to mention irregular declensions. For example, in Polish the nominative student becomes the studenta in the accusative which may look like a simple suffix, but the nominative pies (“dog”) becomes psa while stół (“table”) remains unchanged.⁴ Writing rules for these differing (and sometimes not unambiguous) case-marking paradigms without building in lexical information would be very difficult indeed.

Finding the edges

Recall that the current Ubiquity Parser 2 design identifies arguments by identifying known delimiters (most often some adposition) as a left or right edge of an argument. By not having to run the nountype detection over every substring of the input, we greatly reduce the processing time needed in each parse. This approach, however, relies on our being able to reliably identify some sort of boundary for each of our arguments.

In strongly case-marking languages, the case is realized on the noun itself, but this noun may be buried in the middle of the noun phrase. Even if we could reliably identify the case-marker, it would mark neither the left nor right edge of the argument, making our current parsing strategy worthless. For example, consider the following Arabic example of “the house of the man” in nominative and accusative cases:⁵

5
6
7
8

baytu 'r-rajuli
house=NOM of=man
bayta 'r-rajuli
house=ACC of=man

In these cases, we see that the only distinction between (5) (بَيتُ الرَّجُلِِ) and (7) (بَيتِ الرَّجُلِِ) is the case suffix on the head noun, “house,” which sits in the middle of the noun phrase. Even if we could properly identify this case ending, it would mark neither the left nor the right edge of the entire argument.

Contrast this with German where, even though arguments have case, the case is realized on the article, not on the noun head itself, so we can essentially deal with these articles as prepositions, using the article as the left edge of the argument.

5
6
7
8

den     großen Hund
the=ACC big    dog
dem     großen Hund
the=DAT big    dog

Believe it or not, things can actually get even worse than just not being able to find an edge of our arguments. The worst-case scenario comes from discontinuous constituents, in languages where case marking on both nouns and modifiers allow for very free word order. Latin is just such a language:⁶

From M. Tullius Cicero, “Against Catiline,” chapter 1:

quem        ad finem      sese effrenata                       iactabit         audacia?
what=sg.ACC to extent=ACC self unbridle=perf-past-part.3sg.NOM fling=future.3sg audacity=sg.NOM

In this example we see that effrenata is modifying audacia but the two do not form a unit in the linear order but their relationship can be recovered because both words carry the nominative case marking. While it would be unfair to expect Ubiquity to ever be able to properly parse such arguments, requiring a certain amount of discipline from the user, this is an illustration of how bad things could get if we took the processing of case-markers to the extreme.

The proper case for execution

The final difficulty in processing case-markings in Ubiquity comes from the preview and execution stages of a Ubiquity command’s usage. That is, after we parse the input, we must give the verb the arguments we found so that it can display a meaningful preview or behave correctly when executed. At this point, what case should the noun be when we hand the string of the argument to the verb?

photo credit: Platinatore

For example, consider the Latin expression “cave canem!” meaning “beware the dog!”

cave              canem!
beware=imperative dog=sg.ACC

Supposing for a moment that we’ve implemented the cavere (“beware”) verb in Ubiquity and properly parsed “cave canem,” should we pass the literal string “canem” in accusative case to the verb, or the nominative string “canis,” or the root “can-“? Which is more appropriate? If “canis” is the more appropriate choice, Ubiquity would then have to be responsible for declining the accusative into a nominative… for all case-marked languages. This is clearly a road we do not want to go down.

Proposal: only support determiners and adpositions

I’ve laid out three reasons why processing strongly case-marked languages in Ubiquity is a non-starter. Fortunately, languages often have multiple different strategies for accomplishing similar communicative tasks. One oft-used strategy for marking different roles of arguments is the use of adpositions (a fancy term for prepositions and postpositions). Unlike case-markers which often are affixes on nouns, prepositions mark the beginning of an argument and postpositions the end, as is used in the current parsing strategy.

From a formal/theoretical perspective, adpositions sit above the noun phrase proper, while modifiers like adjectives live within the noun phrase. This reflects the fact that, with few exceptions, adpositions mark an edge of the noun phrase, which is crucial to our parsing strategy. (Here, PP is a prepositional phrase and NP is a noun phrase.) Note also that for languages such as German which marks case on determiners (D), the same logic holds.

Note also that, as long as the case-marking is phrase-marking (i.e. marking the edge of the noun phrase) rather than just affixing to the head noun, our parsing strategy will work. This means we could possibly in the future write a simple RegExp to split off the Basque dative suffix, as it marks the end of the entire noun phrase. This can be seen in the following data from Tseng (2004), where the suffix -(r)i affixes to the last word in the noun phrase, no matter the type of speech of that last word. (Basque is crazy cool!)

Conclusion

In this blog I’ve outlined some reasons why it would be unreasonable or very difficult to incorporate case-marker processing into our current parser strategy. The case markers themselves are often hard to identify, the case markers do not align at the edge of arguments, and there is the question of what form of the argument should be passed to the verb for preview and/or execution. Luckily many languages allow for adpositions (prepositions and postpositions) as an alternative strategy to case as a means of marking the different grammatical functions of arguments. By limiting Ubiquity parsing to adpositions (and case-marked determiners), I believe we are able to reach a good compromise between each user’s natural language and an easily machine-processable form.

Related posts:

1. Three ways to argue over arguments
2. Contribute: how your language identifies its arguments
3. Writing commands with semantic roles

Related posts brought to you by Yet Another Related Posts Plugin.

From Dates in the Month of May that Are of Interest to Linguists by the late James D. McCawley:

May 6, 1939. The University of Chicago trades Leonard Bloomfield to Yale University for two janitors and an undisclosed number of concrete gargoyles.

May 23, 38,471 B.C. God creates language.

May 29, 1962. Angular brackets are discovered. Classes at M.I.T. are dismissed and much Latvian plum brandy is consumed.

May 31, 1951. Chomsky discovers Affix-hopping and is reprimanded by his father for discovering rules on shabas.

Unfortunately May 31, 1951 was a Thursday…

Related posts:

1. Localizing Ubiquity: an open letter to linguists
2. Jerry Sadock’s Automodular Grammar on iTunes
3. Bathroom Graffiti

Related posts brought to you by Yet Another Related Posts Plugin.

You’ve seen the video. You speak another language. And you’re wondering, “how hard is it to add my language to Ubiquity with Parser 2?” The answer: not that hard. With a little bit of JavaScript and knowledge of and interest in your own language, you’ll be able to get at least rudimentary Ubiquity functionality in your language. Follow along in this step by step guide and please submit your (even incomplete) language files!

As Ubiquity Parser 2 evolves, there is a chance that this specification will change in the future. Keep abreast of such changes on the Ubiquity Planet and/or this blog (RSS).

Set up your environment

If you’re new to Ubiquity core development, you’ll want to first read the Ubiquity 0.1 Development Tutorial to learn how to get a live copy of the Ubiquity repository using Mercurial. Once you’ve set up your Firefox profile to use this development version, make sure to try changing the extensions.ubiquity.parserVersion value to 2 in about:config (as seen in this demo video) to verify that Parser 2 is working for you.

As you read along, you may find it beneficial to follow along in the languages currently included in Parser 2: English, Japanese, Portuguese, and Swedish (and the incomplete Chinese and French).

The structure of the language file

Each language in Parser 2 gets its own file which acts as a JavaScript module. You’ll need to look up the ISO 639-1 code for your language… Here we’ll use English (code en) as an example here and the JavaScript language file would then be called en.js and go in the /ubiquity/modules/parser/new/ directory of the repository.

Here is the basic template for a Ubiquity Parser 2 language file:

1
2
3
4
5
6
7
8
9
10
11
12

var EXPORTED_SYMBOLS = ["makeEnParser"];
 
if ((typeof window) == 'undefined') // kick it chrome style
  Components.utils.import("resource://ubiquity/modules/parser/new/parser.js");
 
function makeEnParser() {
  var en = new Parser('en');
 
...
 
  return en;
};

After lines 1-4 which set up the JavaScript module, everything else is wrapped in a factory function called makeLaParser (for Latin) or makeEnParser (for English, en) or makeFrParser (for French, fr), etc. This function initializes the new Parser object (line 7) with the appropriate language code, sets a bunch of parameters (elided above) and returns it. That’s it!

Now let’s walk through some of the parameters you must set to get your language working. For reference, the properties the language parser object is required to have are: branching, anaphora, and roles.

Identifying your branching parameter

  en.branching = 'right'; // or 'left'

One of the first things you’ll have to set for your parser is the branching parameter. Ubiquity Parser 2 uses the branching parameter to decide which direction to look for an argument after finding a delimiter or “role marker” (most often, these are prepositions or postpositions. For example, in English “from” is a delimiter for the goal role and its argument is on its right.

So “John” is a possible argument for the source role, but “Mary” should not be. Ubiquity can figure this out because English has the property en.branching = 'right'.

In Japanese, on the other hand, the argument of a delimiter like から (“from”) is found on the left of that delimiter, so en.branching = 'left'.

In general, if your language has prepositions, you should use .branching = 'right' and if your language has postpositions, you can use .branching = 'left'.

For more info:

• see branching on Wikipedia.

Defining your roles

  en.roles = [
    {role: 'goal', delimiter: 'to'},
    {role: 'source', delimiter: 'from'},
    {role: 'position', delimiter: 'at'},
    {role: 'position', delimiter: 'on'},
    {role: 'alias', delimiter: 'as'},
    {role: 'instrument', delimiter: 'using'},
    {role: 'instrument', delimiter: 'with'}
  ];

The second required property is the inventory of semantic roles and their corresponding delimiters. Each entry has a role from the inventory of semantic roles and a corresponding delimiter. Note that this mapping can be many-to-many, i.e., each role can have multiple possible delimiters and different roles can have shared delimiters. Try to make sure to cover all of the roles in the inventory of semantic roles.

For more info:

• Writing commands with semantic roles
• the proposed inventory of semantic roles
• Wikipedia entry on thematic relations

Entering your anaphora (“magic words”)

  en.anaphora = ["this", "that", "it", "selection", "him", "her", "them"];

The final required property is the anaphora property which takes a list of “magic words”. Currently there is no distinction between all the different deictic anaphora which might refer to different things.

Special cases

Some special language features can be handled by overriding the default behavior from Parser. Many of these features are still in the works, however, so we’d love to get your comments!

Languages with no spaces

If your language does not delimit arguments (or words, more generally) with spaces, there will be a need to write a custom wordBreaker() function and set usespaces = false and joindelimiter = ''. For an example, please take a look at the Japanese or Chinese.

Case marking languages

If you have a strongly case-marked language, you’ll have to write some rules to identify those different cases in wordBreaker() and then add some extra roles for these case markers, but for a number of languages the current design does not allow an elegant solution for parsing such arguments. Updates to this issue will be posted to this trac ticket.

In the mean time, however, if you could write a parser even with only the prepositions/postpositions in your language, that would be a great benefit in getting started in your language. UPDATE: a proposal on how to deal with strongly case-marked languages has been written here: In Case of Case….

Stripping articles

Some languages have some delimiters which combine with articles. For example, in French, the preposition “à” combines with the masculine definite article “le” but not “la”:

1. à + la = à la
2. à + le = au

You can add both “à” and “au” as delimiters of the goal role, but then you will get feminine arguments back with the determiner (e.g. “la table”) while masculine arguments would be parsed without a determiner (e.g. “chat”).

1. “à la table” = “to the table”
2. “au chat” = “to the cat”

One possible solution to this is to write a custom cleanArgument() method. After arguments have been parsed and placed in their appropriate roles, each argument text (say, “la table” or “chat”) are passed to cleanArgument(). You can simply write a cleanArgument() to strip off any “la ” at the beginning of the input and return it and both example inputs will get normalized arguments: “table” and “chat”, respectively. UPDATE: For more up-to-date information on how to deal with these types of articles, please see Solving a Romance Problem.

Test your parser

Now you can go into about:config and change extensions.ubiquity.language to be your language code and restart. All the verbs and nountypes at this point will remain the same as in the English version, but it should obey the argument structure (the word order and delimiters) of your language.¹ If you run into any trouble, feel free to ask for help on the Ubiquity i18n listhost or find me on the Ubiquity IRC channel (mitcho @ irc.mozilla.org#ubiquity). Of course, once you’re at a good stopping point, please contribute your language file to Ubiquity!

More to come…

At this point, you’ve only localized the argument structure of your language… additional work will be required to localize the nountypes and verb names, which is the subject of ongoing discussion… join the Google Group to get in on the discussion!

Related posts:

1. Ubiquity Parser: The Next Generation Demo
2. Rolling out the Roles
3. Foxkeh demos Ubiquity Parser: The Next Generation

Related posts brought to you by Yet Another Related Posts Plugin.

Every day on the way to work I walk by a fine establishment known as Yoshinoya (吉野家), Japan’s largest gyudon (牛丼) chain restaurant. For those of you whose lives have yet to be graced by gyudon, it’s a bowl of rice topped with beef and onions stewed in a sweet-savory soy-based sauce. Loving gyudon and being a cheapskate, I naturally noticed the recent 50 yen off gyudon promotion at Yoshinoya. The above photo is a photo of part of that sign.

Part of this sign, though, made me think about our new Ubiquity parser. In particular, it was the attachment ambiguity in the end date of the promotion. The text in the photo above literally is “April 15th (Wed.) 8PM until”. (Note that Japanese is a strongly head-final language, and that the “until” is a postposition.) There are two possible readings for this expression, as illustrated by the two composition trees below.

The first tree, on the left, represents the reading “until (April 15th 8PM)”, while the second represents two arguments: “on April 15th” and “until 8PM”. In other words, in the first reading, the promotion begins at some earlier date and extends until April 15th at 8PM while, in the second reading, the promotion is one day only, on April 15th, until 8pm. Such syntactic ambiguities are called “attachment ambiguities” in linguistics as it is an ambiguity of where different arguments “attach” in a tree representation.

This attachment ambiguity was possible because there was no clear marker on “April 15th,” which may have disambiguated it as “on April 15th”. In fact, in many languages this time position argument comes with no case marker or preposition, or it’s optional, making parsing for them difficult. If such a sentence is entered with spaces, the Ubiquity Parser: The Next Generation would try a parse where “8PM” is the “until” or goal argument and “April 15th” is an object argument, but it will only check its noun type, not put it in the correct semantic role (position). Perhaps this is something to think about in the future.

These types of situations will surely come up as we continue work on the Ubiquity parser, making it essential to look at different languages. Are there certain kinds of arguments in your language that do not have any word-external markers such as case or prepositions/postpositions?

Related posts:

1. User-Aided Disambiguation: a demo
2. Ubiquity in Firefox: Focus on Japanese
3. Talking Ubiquity in Japan: 拡張機能勉強会にて発表

Related posts brought to you by Yet Another Related Posts Plugin.

The issue of ranking Ubiquity suggestions can be restated as predicting an optimal output given a certain input and various conflicting considerations. Ubiquity (1.8, as of this writing) computes four “scores” for each suggestion:

1. duplicateDefaultMatchScore: 100 by default—lowered if an unused argument gets multiple suggestions (in the words of the code: “reduce the match score so that multiple entries with the same verb are only shown if there are no other verbs.”)
2. frequencyMatchScore: a score from the suggestion memory of the frequency of the suggestion’s verb, given the input verb (currently the first word) or nothing, in the case of noun-first suggestions
3. verbMatchScore: float in [0,1]: (as described here)
   • 0.75 is returned in case there it is a noun-first suggestion (by virtue of the fact that String.indexOf('')==0)
   • 1 if the verb name is equivalent across input-output
   • in [0.75,1) if the input is a prefix of the suggestion verb name
   • in [0.5,0.75) if the input is a non-prefix substring of the suggestion verb
   • in [0.25,0.5] if the input is a prefix of one of the synonyms
   • in [0,0.25) if the input is a non-prefix substring of one of the synonyms
4. argMatchScore: the number of arguments with matching “specific” nountypes, where “specific” is designated by the nountype having property rankLast=false.

With the numeric scores for each of these criteria, a partial order of suggestions is constructed using a lexicographic order: that is, compare candidates first using duplicateDefaultMatchScore, break ties using frequencyMatchScore, if still tied break using verbMatchScore, and if still tied break using argMatchScore. This paradigm of constraints is called “strictly ranked” and a corollary of this is that lower constraints, no matter how well you score on them, can never overcome a loss at a higher constraint. A crucial corollary of this system is that lower constraints’ scores need not be computed if a higher constraint already dooms it to a lower position.¹

Ranking in The Next Generation

One of the goals of Parser The Next Generation is to make noun/argument-first input first-class citizens of Ubiquity, improving their suggestions in particular to the benefit of verb-final languages. Arguments will be split up and tested against different noun types before a verb is even entered into the input, in which case target verbs can be ranked according to the appropriateness of the input’s arguments. As such, I believe the argMatchScore criteria above should either be ranked higher in a strictly ranked model or be allowed to overtake lower scores for the higher constraints in a non-strictly ranked model.

The Parser The Next Generation proposal and demo currently orders using a product of various criteria’s scores, rather than a lexicographic order of strictly ranked constraints. The component factors are:

1. 0.5 for parses where the verb was suggested
2. 0.5 for each extra (>1) object argument (essentially “unused words” in the previous parser)
3. the score of each argument against that semantic role’s target noun type
4. 0.8 for each unset argument of that verb

Each component score is a value in [0,1], so the score is always non-decreasing across the derivation. This offers a natural way to optimize the candidate set creation: if a possible parse ever gets a score below a magic “threshold” value, it is immediately thrown away.

A possible problem with the current Parser TNG scoring model is that it will implicitly hinder verbs and parses with more arguments as it could have more sub-1 noun type score factors—this consideration may be great enough that a weighted additive model should be considered over a multiplicative one.

How do you think we can make Ubiquity’s suggestion ranking smarter? What other factors should be considered, and what factors could be left out?

Related posts:

1. Ubiquity Parser: The Next Generation Demo
2. Ubiquity in Firefox: Focus on Japanese
3. Ubiquity Commands by The Numbers

Related posts brought to you by Yet Another Related Posts Plugin.

Today I’ll examine highlight some disparate typological data to help us understand these questions: where’s the verb? and what does the verb look like? There are broadly three different verb forms taken in commands in different languages:¹

1. the infinitive,
2. subjunctive mood, or
3. a special verb form such as imperative, participial, or conjunctive (such as Japanese て form)

Let’s give an example of each:

Infinitive (English):²

1

Hit me!

Subjunctive mood (Modern Greek): “Eat it all!”

2
3

Na   to fas olo!
SUBJ it eat all

Imperative form (French): “Eat it!”

4
5

Mange   -le!
eat.IMP it

It’s important to note that some languages have multiple forms available for the same command. For example:

Dutch: three ways to say “watch out!” with the same verb

1. Infinitive: Oppassen!
2. Imperative: Pas op!
3. Participial: Opgepast!

Similarly, I received a great comment by PhiliKON on German and associated data by Robert Kaiser on my blog post yesterday:

German: “search hello with google”

1. Infinitive: hello mit google suchen
2. Imperative: suche hello mit google

In addition, German and Dutch are interesting as they are verb second (V2) languages, so the verb may surface at the beginning or the end of the sentence, depending on the form.

The new parser design (which you can demo) assumes for simplicity that the verb should be found at the beginning or the end of the input, which is consistent with the data I’ve seen (modulo clitics). Multiple verb forms could be accounted for by supporting “synonyms” of the verbs.

What are the different ways verbs are expressed in commands in your language? Is the verb always found at the beginning or the end of the sentence? Is it ever somewhere in the middle?

Related posts:

1. Ubiquity i18n: questions to ask
2. Ubiquity Parser: The Next Generation Demo
3. Contribute: how your language identifies its arguments

Related posts brought to you by Yet Another Related Posts Plugin.

In the future we ideally could build a web-based system to collect these “utterances.” We could … generate parser parameters based on those sentences. That would essentially reduce the parser-construction process to a more run-of-the-mill string translation process.

If we build this type of “command-bank” of common Ubiquity input translated into various languages, we could build a tool to learn various features of each language and generate each parser, essentially learning the language based on data. Today I’ll elaborate on how I believe this could be possible, by analogy to another language learning device: the human.

Step 1: learning words

How does a human learn language? Without getting into any details or theory, we can say that the input for a language learner is always a combination of linguistic input and a referent. In the case of a child, this could be a pairing of linguistic input with real world stimulus:

input	referent
“taiyaki!”	by makitani via creative commons.
“cat!”	by victoriachan via creative commons.

The human child will hear “cat” while looking at the cat and, with time and repetition, learn that that thing is called a “cat,” and some other thing is called “taiyaki.”

Similarly, we could take single-verb data points from our command-bank to match new words with a know referent—in this case, the base English string. Here’s an example from Jan’s comment on yesterday’s sample survey.

Step 2: deduction

Now suppose we know some single words like “taiyaki” and “cat.” Consider the two situations. Given the first sentence and referent “mitcho’s eating a taiyaki,” the child could intuit the appropriate linguistic representation for the latter situation.

input	referent
“mitcho’s eating a taiyaki!”
???

The process is simple. First note that there is only one variable changed between the two situations: the taiyaki has been replaced by a cat head. You can then construct the correct utterance by analogy, replacing “taiyaki” with “cat,” yielding “mitcho’s eating a cat!”¹

Similarly, we could build a tool to analyze the data in a translated command-bank to identify particular features of each language, generating at least basic parsers for each language. Such a task would require a number of minimal pairs in our data set—here’s one such example from yesterday’s survey (with Dutch data from Jan):

Parse {
  verb:      'search',
  arguments: {
    object:  ['HELLO'],
    service: 'Google'
  }
}

Parse {
  verb:      'search',
  arguments: {
    object:  ['this'],
    service: 'Google'
  }
}

A simple string analysis² would tell us that the text HELLO was replaced by dit in the latter Dutch sentence. Meanwhile, since the English reference sentence is chosen manually, we also know the appropriate parses for each of those sentences. An object difference operation would note that the object property was changed from a value of 'HELLO' to 'this'. We could then map dit to the English this. We’ve now learned one (of perhaps many) Dutch deictic pronouns (aka “magic words”).

Given an adequately universal but customizable parser design, we can then develop tests for various parameters by constructing appropriate minimal pairs in the base sentences and having them translated.³ As noted yesterday, such a system could reduce the laborious task of writing individual parsers to a task of string translation, which our community does exceedingly well. I’m eager to hear what others think of this approach. What concerns would you have for this approach? What potential benefits do you see?

Related posts:

1. Ubiquity i18n: questions to ask
2. Localizing Ubiquity: an open letter to linguists
3. Writing commands with semantic roles

Related posts brought to you by Yet Another Related Posts Plugin.

As a linguist, studying a language involves looking at instances of that language as data. To this end, we as Ubiquity internationalizers need to get at some examples of target utterances. Here’s an example survey which could be a good starting point for native speakers who want to contribute information on their language, based on Blair’s list of common Ubiquity verbs.

A survey for Ubiquity localization

Instructions

How would you express the following commands in your language? The words in CAPITAL LETTERS do not need to be translated. Feel free to give multiple possible answers for each command.

Try to express the same command rather than forcing a “literal translation”; for example, if there’s no “map” verb in your language, you could translate example (8) as lookup a map of PLACE. Please keep in mind that the addressee is a computer.

Basic word order / argument structure

1. search HELLO
2. search HELLO with google
3. translate HELLO from English to French
4. lookup the weather for PLACE
5. shop for SHOES with Amazon
6. email HELLO to Bill
7. email HELLO to ADDRESS
8. map PLACE
9. find HELLO
10. tab to HELLO or switch to HELLO tab

…

Pronominal/deictic arguments (aka “magic words”)

1. search this with google
2. translate this to French
3. bookmark this tab

…

How this data is used

Responses to these surveys would be used to identify certain salient features of the language, such as how the language codes for its arguments (for example using adpositions, case marking, or word order), whether the commands tend to be verb-inital or -final. Individual case markings, for example, can be identified by comparing minimal pairs—for example, by comparing item (1) and (2), we can learn how google in an instrumental role is marked, or by comparing example (2) and the “magic word” example (1), we can identify the appropriate “magic word” and determine whether the language uses any clitics or not.

Data collection

In the future we ideally could build a web-based system to collect these “utterances.” We could also use such a system to automatically test our parsers in different languages against the sentences in the command-bank, or ultimately even generate parser parameters based on those sentences. That would essentially reduce the parser-construction process to a more run-of-the-mill string translation process.

Related posts:

1. Contribute: how your language identifies its arguments
2. Writing commands with semantic roles
3. Ubiquity in Firefox: Focus on Japanese

Related posts brought to you by Yet Another Related Posts Plugin.

What would the challenges be for Ubiquity in your language? We’d love to see an increasing number of blog posts on this topic in different languages. Thanks Felipe! ^^

Related posts:

1. Localizing Ubiquity: an open letter to linguists
2. Ubiquity in Firefox: Focus on Japanese
3. Writing commands with semantic roles

Related posts brought to you by Yet Another Related Posts Plugin.

What caught my eye is the line “served with ご一緒に 配,” meant to be read as part of “Beef in BBQ sauce… served with Pepsi…”. The Chinese 配 (pèi) is fine here, meaning “with,” but the Japanese “ご一緒に” (goissho-ni) seemed awkward to me.

The issue is that this adverbial meaning “together” normally comes after the “what it’s with” in an order like (1) (glossed in (2)):

1
2

A B-と       ご一緒に
A B-and/with together

In other words, where English and Chinese both would say “A with B”, it is most natural in Japanese to say the equivalent of “A B with (together)”.¹ This is the reason why it seems unnatural to have anything between the “Beef in BBQ sauce…” line and “Pepsi…” line.

Looking at the rest of the menu, it’s clear that this isn’t a case where a native speaker wasn’t involved with the writing of the menu—the rest of the Japanese is perfect. The Japanese modifier was inserted there just for the sake of parallel design, to the detriment of the text’s naturalness. When have you seen design conflict with the structure of your language?

Related posts:

1. Three ways to argue over arguments
2. Testing Google’s Language Detection
3. Ubiquity in Firefox: Focus on Japanese

Related posts brought to you by Yet Another Related Posts Plugin.

Below is a transcript of this video. Please distribute this video far and wide to anyone who may be interested. ^^

Dear linguist,

My name is mitcho and I am currently involved with an exciting open source community project at Mozilla Labs called Ubiquity. The project’s goal is to empower the user to accomplish more on the web, and to do this, Ubiquity connects disparate services from across the web with language. Let me show you what I mean.

Example 1

Suppose you’re emailing a friend to tell them where to meet later. It would be great if you could insert a map in your email, but you would regularly have to go to a maps application and insert a link.

But with Ubiquity installed, you can just pull up Ubiquity with a keystroke, enter map Shinjuku, pick the right result and rearrange it, and click “insert map.” It’s that simple.

Example 2

Let’s say you’ve stumbled upon an article in French and you want to send a bit of it to a friend. Again, you could just send a link, but with Ubiquity we can do much better than that. We’ll first select the quote we want and translate it to English. There we go. Now I’ll select the bit I actually want to send, pull up Ubiquity and execute email this to Aza. Because Ubiquity interacts with my address book, it knows who I mean by “Aza” and composes my email for me.

As you can see, the Ubiquity interface makes it easy to accomplish tasks with minimal interruption to a user’s workflow. New verbs are easy to write and there’s already an active community writing new verbs and sharing them with users.

The localization of Ubiquity

Right now Ubiquity offers a basic English parser and an experimental Japanese one, but Firefox itself currently ships with 55 distinct localizations. We hope to localize Ubiquity into a number of these languages.

In order to accomplish this, we need support from viewers like you.

For each language—whether Basque, Polish, or Esperanto—we hope to build a parser to take the command and pick out its verb and the arguments and then identify the semantic roles of each argument. Ubiquity will then apply the verb to those arguments and execute.

We’ll need both data and expertise into the languages we hope to localize. If you’re a native speaker or researcher, what is the structure of commands in your language? How do you code for different kinds of arguments? What do your pronouns look like? We’d love to see blog posts, discussions, or quick mockups on these and other topics.

For those of you with NLP experience, we’d love to get more folks involved in writing these parsers. And in the near future, we’d appreciate anyone who wants to simply try out Ubiquity in their language and give us feedback.

The idea of using natural language to direct computers is far from new. But in this limited context of individual commands—single imperative clauses without complexities such as negation or quantification—I believe there is great potential for Ubiquity to support other languages and bring this interface—this new way of interacting with the web—to more people in more communities.

ubiquity.mozilla.com
bringing linguistics to a browser near you

wiki.mozilla.org/Labs/Ubiquity/i18n
help us localize the future

Related posts:

1. Ubiquity in Firefox: Focus on Japanese
2. Contribute: how your language identifies its arguments
3. Writing commands with semantic roles

Related posts brought to you by Yet Another Related Posts Plugin.

今後多様な言語に対応したFirefox内のUbiquityを検討していきますが、その中でも今日は日本語をとりあげます。後日日本語で同じ内容を投稿するつもりです。^^ 日本語でのコメントも大歓迎です！

What commands look like in Japanese

Japanese is not only just a verb-final language but it is strongly head-final, meaning it has postpositions instead of prepositions, direct objects come before verbs, and adjectives precede nouns. In terms of how it identifies its arguments, every argument has a postposition/case marker (called a particle in the Japanese literature) which marks its role in the sentence.

A couple common particles we’ll look at in this example include -を (-o) which marks the direct object (accusative case, you might say) and -に (-ni) which acts like English “to” (dative case). The example sentence we’ll look at today is:

(Note: ʔ is a glottal stop. ACC=accusative, DAT=dative, and IMP=imperative form.)

That final ください is often dropped in very casual speech and, as it adds no new information, we’ll assume today that the user will not enter it. Finally, Japanese doesn’t use spaces in their orthography, so the actual input would be “ケーキをブレアに送って”.

Mockup 1: Particle identification

One of the major hurdles in working with Japanese is that there are no spaces between the words. The natural first step is to split the sentence up into words, but this is a very difficult problem in NLP which big name research groups actively work on.

Fortunately, however, in “Solving the ‘It’ Problem” Aza suggests that, when we encounter ambiguity in our input, we can go ask the user. Great minds think alike, and computer scientist Jean E. Sammet suggested the same idea way back in 1953:

Using English [or any other natural language] definitely involves the requirement for the computer (or more accurately its programming system) to query the user about any possible ambiguity.

Parsing a sentence into words, in the limited context of Ubiquity, is really about identifying the particles which mark the end of each argument. Here’s a mockup of an application of the Sammet-Raskin Method to this problem:

Pros: This completely takes care of the word-breaking problem, with minimal arbitration from the user. The parser knows exactly what arguments it’s dealing with and the visual feedback means the user won’t be surprised by the parse.

Cons: Most of the particles/postpositions we’d have to deal with are a single character, so they may show up pretty often within words, in which case it would be quite annoying to have to press escape after each one.

An even smarter system, when wanting to mark a character as a particle, would first check to see that the argument (before the particle) is a valid argument type for that particle. If the check fails, it doesn’t have to bother with suggesting that character as a particle. This may cut down on the false positives.

Smart suggestions: what works, what doesn’t

One of the key suggestions in Aza’s mockups include a way to choose the prepositions while entering your arguments, based on the current verb.

For example, here, the translate command accepts a direct object, a to-object, and a from-object, so little to and from markers magically show up on the right side, making the appropriate prepositions (and by extension the appropriate arguments) discoverable. I think this line of thinking is a really good one, at least for English.

In a verb-final language, however, you enter the arguments first and then the verb, making this strategy of suggesting appropriate arguments impossible. Note that in the user-contributed spreadsheet of how languages identify their arguments we see that about a quarter of the languages we looked at are verb-final—that is, with Subject-Object-Verb canonical word order.

Instead of seeing this as a disadvantage, however, let’s see what verb-final order allows us to do.

Mockup 2: A different kind of suggestion

Not all verbs allow for every different kind of particle. For example, it doesn’t make sense to have a -に (-ni, “to” or dative) argument for a verb like 検索して (kensaku-shite, “search for”). In English we used this to suggest different types of arguments given a specific verb. In a verb-final language, we could do this backwards.

Pros: This makes verbs highly discoverable, given a certain argument structure. For example, if you enter a few arguments, like a direct object, a “to” argument, and a “from” argument, it’ll suggest verbs that will do something to an object from somewhere to somewhere else. This way, you can easily try out verbs you didn’t even know existed. It’ll only give you verbs appropriate for your arguments, reducing the chance of writing a an infelicitous command.

Cons: Without knowing what kinds of actions are available, it may be difficult to know what kinds of arguments to enter in the first place. If you have a specific verb or service you want to use it may be counterintuitive or downright tricky to start by guessing the right set of arguments.

In addition, from a technical point of view, this requires much of the prediction algorithms in English Ubiquity to run backwards. Ideally, there would be a closed (predetermined) class of particles and a predefined set of noun types. Verbs would not be able to define their own modifiers and noun classes as easily or freely as they can now.

Conclusion

The properties and challenges of Japanese grammar require that we not try to outright copy the English behavior but to think about what really makes sense in that language and that may be an important lesson as we move toward designing a localizable Ubiquity. Please post your questions and criticisms of this design or post your own mockups!

Related posts:

1. Three ways to argue over arguments
2. Contribute: how your language identifies its arguments
3. How natural should a natural interface be?

Related posts brought to you by Yet Another Related Posts Plugin.

I gathered some data from the fantastic World Atlas of Language Structures to put together a survey of many of the languages on the Internet. For each of the languages, I got the canonical word order and whether the language marks the role of its argument on the verb and/or the arguments themselves.

As you can see, there are a number of data points that are still missing. Please contribute information on the languages you speak! You can edit the spreadsheet on Google Docs. Thanks!

Related posts:

1. Three ways to argue over arguments
2. Testing Google’s Language Detection
3. Setting Language Research to Music

Related posts brought to you by Yet Another Related Posts Plugin.