Exploring Command Chaining in Ubiquity: Part 2

Aug 23, 2009

Introduction

I recently have begun giving serious thought to what command chaining might look like in Ubiquity and the various considerations which must be made to make it happen. The “command chaining,” or “piping,” described here always involves (at least) two verbs acting sequentially on a passed target—that is, the first command performs some action or lookup and the second command acts on the first command’s output.

A few days ago I penned some initial technical considerations regarding command chaining. In this post I’ll be point out some linguistic considerations involved in supporting a natural syntax for chaining.

Simple syntaxes: sequential vs embedding strategies

When it comes to creating a natural language interface, there’s always a decision to make between requiring a certain kind of input, or working a little harder to understand the user’s natural input. From an implementation point of view, adopting certain programmatic conventions is of course simpler and to this end, there have been a couple different “unnatural” command chaining syntaxes suggested. While these both go against Ubiquity’s basic tenet of natural syntax — that is, to not introduce rules which contradict the user’s natural language — which gives Ubiquity its strengths of usability and memorability, I’ll entertain them here as they illustrate two different structural relationships that we will want to consider.

The first suggestion is to adopt the [[Pipeline_(Unix)

shell pipe]] (

), which would lead to input such as

translate hello to Spanish | email to Jono

While this itself is pretty unnatural unless you speak shell, note that this syntax is similar to the more natural “, and” syntax, yielding translate hello to Spanish, and email to Jono, which we will consider below. I’ll refer to this strategy as the sequential strategy.

Another very interesting proposal by Alex Fritze is to embed each subordinate computation into an argument position, marked by parentheses. This could also be parsed relatively straightforwardly by writing a noun type which first checks for parentheses and then runs the content of the argument through another ParseQuery.

email (translate hello to Spanish) to Jono

I’ll refer to this pattern as the embedding strategy.

Sequential and embedding strategies in natural language

What’s interesting about the two proposals above is that both strategies are seen in natural language. The sequential strategy could correspond to the following linguistic phenomena:

1. [[coordination (linguistics)

   coordination]]: a non-hierarchical joining of two or more [[clauses (linguistics)

   clauses]], often marked by a [[conjunction]]. Here’s an example from English:

   • “[I made a sandwich] and [you will eat it]” where [] represent clause boundaries. Here, “and” is the conjunction.

2. [[serial verb construction

   serial verb]] and [[converb

   converb]] constructions: a joining of multiple verbs or verb phrases within a single clause, with shared subject and tense/aspect values, with no particular conjugation or delimiter between them. Such constructions are common in many African and east Asian languages. Here are two examples:1

• A converbal construction in Japanese:

  僕は     サンドイッチを 作って     食べる
  boku-wa sandiʔchi-o  tsuku-ʔte tabe-ru
  I-TOP   sandwich-ACC make-CON  eat
  

</pre>

“I (will) make a sandwich and eat [it].” (Here, \`TOP\` = topic, \`ACC\` = accusative, \`CON\` = converbal ending)[^3] *   A serial verb construction in Mandarin Chinese:  
<pre lang='zh' line='6'>我 作   三明治      吃 wǒ zùo  sānmíngzhì chī I  make sandwich   eat

</pre>

“I (will) make a sandwich and eat [it]” or “I (will) make a sandwich [in order to] eat [it].”

Note that in both the converb and serial verb construction, the second verb (eat) takes shares its object (sandwich) with the first verb and there is no need for a pronoun such as “it” to introduce that argument as it is with coordination, above.²

The embedding strategy is observed in natural language as well, in the form of the following phenomena:

1. [[embedded clauses]]: a sentence is itself the argument of another verb. Example:

• “John says [he likes sandwiches].”

Embedded clauses, however, clearly have no relation to command chaining and does not require our attention. 2. [[relative clauses]]: a partial sentence³ is attached to a noun in order to describe it or distinguish it from other possible referents. Example:

• “You ate the sandwich that I made” where “sandwich” is called the “head” of the relative clause, and “I made” is what I here call the “partial sentence” (see footnote). The “relative clause” is used here to distinguish “the sandwich that I made” from other sandwiches.

The natural syntax of chaining

So which strategy is used in complex natural language commands: the sequential strategy or the embedding strategy? Both the sequential strategy and embedding strategy can be involved with commands:

[Make a sandwich] and [eat it]!
Eat (the sandwich that I made)!

These two commands do not mean the same thing, though, and only (9) is the kind of command we would want to give Ubiquity. The problem with relative clauses, as in (10), is that it *[[presupposition

presupposes]] the existence of the sandwich in the context*. If we both know you just made a sandwich, saying (10) is perfectly appropriate, but out of the blue it doesn’t make sense. For this reason, only the sequential strategy is used in the natural syntax of chaining.

Parsing the sequential strategy

In natural language, unlike the initial simple proposals laid out above, there is often no clear delimiter marking the boundary between the two parts in a sequential relation (e.g. examples (3) and (6) above, particularly given that neither Japanese and Chinese normally break words with spaces). How would we parse a sequential string of commands?

Let’s assume for our purposes here that we can identify find all verbs within the input string.⁴ Parsing a sequential strategy string is not particularly difficult if we can also assume that the verb in any particular language is either always verb-initial or always verb-final. Let’s look at both cases:

• Always verb-initial: Mandarin Chinese:

  翻譯       hello 到  西班牙語   送    給  Juanito
  fānyì     hello dào xībānyáyǔ sòng gěi Juanito
  translate hello to  Spanish   send to  Juanito

  “Translate hello to Spanish [and] send [it] to Juanito”

  1. find every possible verb:
     翻譯hello到西班牙語送給Juanito
  2. as every verb marks the beginning of a sentence, we now have our two commands: “翻譯hello到西班牙語” (translate hello to Spanish) and “送給Juanito” (send to Juanito).
• Always verb-final: Japanese

  helloを スペイン語に 訳して Juanitoに 送って
  hello-o supeingo-ni yakus-ite Juanito-ni oku-ʔte
  hello-ACC Spanish-DAT translate-CON Juanito-DAT send-CON
  

</pre>

“Translate hello to Spanish [and] send [it] to Juanito” 
1.  find every possible verb:  
    helloをスペイン語に**訳して**Juanitoに**送って**
2.  as every verb marks the end of a sentence, we now have our two commands: “helloをスペイン語に**訳して**” (translate hello to Spanish) and “Juanitoに**送って**” (send to Juanito).

For languages where there is a clear conjunction between the two commands, such as English “and”, we can also use that conjunction as a delimiter as well. We then simply execute the first command and then execute the second with the first command’s output in its interpolation context. This way the output of the first command will be picked up both by an overt pronoun such as “it” in the second command and without it, such as in the Chinese and Japanese examples above.⁵

The only potential problem with this approach is in the case of languages where some commands are verb-initial while others are verb-final. I note that such languages do exist in a previous blog post, Where’s The Verb. In these languages, commands can be expressed by more than one verb form (such as infinitive, imperative, subjunctive, etc.) and some of those verb forms are sentence-initial while others are sentence-final. Here’s one such example from German:

“search hello with google” (German) 1. Infinitive: hello mit google suchen 2. Imperative: suche hello mit google

Here the verb for “search” is “suchen” (infinitive) or “suche” (imperative). I know that this same type of phenomena occurs in other Germanic languages such as Dutch with infinitive and imperative and also other languages such as Modern Greek with infinitive and subjunctive forms. If you are a speaker of one of these lanuages (German, Dutch, Greek, etc.) I would love to know whether you can chain verb-final and verb-initial commands together.

Conclusion

In this blog post I examined command chaining in natural language, focusing on data from English, Mandarin Chinese, and Japanese, which exhibit three linguistically different approaches to chaining. What we found is that the sequential strategy—that of listing the commands one by one, in order of execution—is what is used in natural languages, rather than any sort of embedding. This fact, combined with the fact that our parser can recognize every available verb, offers a simple approach to doing a naive parse of natural command chains in most languages, even without explicit delimiters.

In a final installation of this series on “exploring command chaining,” I hope to consider how the Ubiquity interface itself could present command chains and aid in its input.