Sunday, June 14, 2009

Extending the Amy Iris Python Conversational Interface

In earlier blog posts, I demonstrated a simple Conversational Interface written in Python. In this blog post, I talk about how the responses are generated by the Amy Iris system, and how you can extend Amy Iris to adapt her to your needs, or make her smarter.

Recall, we had this five-line Python program:

import amyirisapi
for i in range(3):
....kwargs={"textin":raw_input("You: ")}
....print "Amy: "+a.textin.submit(**kwargs)

This simple program prompts You for some input, and submits your input to the Amy Iris system, displaying her response.

Here's a sample execution:

You: Hi, how are you?
Amy: Hello there. I'm doing fine thanks how are you?
You: say I have an emergency in spanish
Amy: tengo una emergencia is i have an emergency in spanish.
You: Where is the nearest Best Buy to 45249?
Amy: The nearest Best Buy store to 45249 is the Fields Ertel OH store, which is 1.47 miles away, at 9871 Waterstone Boulevard.

(Note that the text follows "You:" was typed in by the user, and the text that follows "Amy:" was the system's response.)

The Amy Iris system is built on a foundation of the award-winning open source ALICE chat-bot system. That is to say that anything that you type to Amy Iris is sent to an ALICE program to prepare a response. In the above example, the first response from Amy actually came from ALICE.

The text that you type to Amy Iris also is sent to various code snippets that Amy Iris developers have submitted to the system. Anyone in the world can submit code to Amy Iris for execution. In the second and third examples, above, short snippets were executed to get Amy Iris' answers.

Let's start with a simplified example of how the code snippets work. If we ask Amy Iris about Barack Obama, she doesn't know him, because the Open Source version of ALICE that is the foundation of Amy Iris was written several years ago:

You: Who is Barack Obama?
Amy: I do not recognize the name. I have to process that one for a while.

However, with a simple code snippet, we can make Amy Iris a little bit smarter. After submitting the code snippet to the Amy Iris system, she answers more appropriately:

You: Who is Barack Obama?
Amy: Barack Obama is the 44th President of the United States.

Here's how you can submit a code snippet to the Amy Iris system, using the API. Note, you can also submit snippets through the website, so I am making this slightly more complicated than it needs to be. Stay with me!

Say you write and run the following code once on your PC:

import amyirisapi

if textin=="who is barack obama":
.... say("Barack Obama is the 44th President " +
........"of the United States.",


print a.snippet.submit(**c.__dict__)

This program takes a code snippet (a program within the program), and submits it to the Amy Iris system via the Amy Iris API, licensing it to be used freely forever in the Amy Iris system. The code above contains a program within the program. If you run this program one time on your PC, the Amy Iris servers will become smarter, by storing the program within the program. Then, each time someone asks Amy Iris about Barack Obama (in the exact format specified), the code snippet (the program within the program) will be executed, and the answer will be provided.

Let's walk through this, line by line. The first line imports the api, and the second line instantiates the api AmyIris object, so that we'll have programmatic access to the Amy Iris system. The third line instantiates the Code object. The Code object is simply a container to collect all the attributes of the code snippet into one object. These include the Python code itself, some keywords, the title, author, development status, and license. We pass a keyword string, which contains a comma-separated list of keywords for this code-snippet. These keywords help the Amy Iris system determine the relevance of this code snippet, relative to various conversations later.

The next few lines need to be examined as one line. Consider an alternative for a momen, if the line read:
c.snippet="some string"
then it'd be very recognizable to most developers, regardless of your familiarity with Python. But in this case, we use the peculiar but powerful Python syntax of the triple-double-quote pair, essentially accomplishing the same task - to assign a string to a variable called c.snippet:

if textin=="who is barack obama":
.... say("Barack Obama is the 44th President " +
........"of the United States.",


The triple-double-quote pair allows you to define a multi-line string which may contain other quotes. This will make your code snippet readable in your editor. I prefix the opening triple-double-quote with an "r" to make it a "raw-string" meaning that the Python interpreter on your PC will not try to interpret back-slashes escaped characters (although in this case, there are no back-slashes in the code snippet, so the "r" has no impact for this code snippet).

With this single line of code (one logical line, spanning multiple physical lines), the entire code snippet program is stored into c.snippet. That is to say, the "if" statement and the call to "say()" are all placed as one multi-line string into the attribute referred to as c.snippet.

Finally, there's the last line of code of the main program:
print a.snippet.submit(**c.__dict__)
This submits this snippet to the Amy Iris system. c.__dict__ is a dictionary of attributes for c, and that dictionary is passed to the AmyIris object "a", using the method a.snippet.submit(). The api simply rearranges the methods and parameters into a URL that's redefined in the Amy Iris RESTful interface.

The API uses the method name (snippet.submit) to determine that it's a POST to the following url:

This RESTful API call will accept code snippets into the Amy Iris system.

The program within the program:

Now let's examine the code snippet itself.

if textin=="who is barack obama":
.... say("Barack Obama is the 44th President " +
........"of the United States.",


There are a few tricks and nuances to the Amy Iris system. Amy Iris exposes certain objects, methods and functions to developers' code snippets. Three are particularly significant: textin, say(), and confidence. Your snippet has access to an object called "textin", which is a normalized version of conversation text received by the human. The normalization process makes the text lowercase, removes punctuation and excessive spacing. It also separates sentences and treats them as individual submissions.

The snippet also has access to a function called "say()". "say()" provides a method for your snippet to communicate in the conversation.

One of the parameters of the say() function is "confidence". This parameter allows the developer a chance to state on a scale of 0-100 how confident the developer is in the answer that their snippet is providing. Remember, for a given input, several snippets may be executed (as well as ALICE), and Amy Iris chooses one answer from among the answers that are derived. It's as if Amy Iris is consulting her panel of experts for answers, and asking each expert to give an answer and to state how confident the expert is in that answer. Note that ALICE answers are provided with a confidence<=20, so the developer must specify a number greater than 20 to be sure that Amy Iris over-rides ALICE answers with yours.

Confidence Values:

Before Amy Iris answers the user, Amy Iris will "go ask Alice", as well as execute a number of code snippets to come up with the single best answer. There are a number of parameters that go into Amy Iris' algorithm to determine which answer to give; the developer's confidence rating is one of those. As a developer, it is to your advantage to give an accurate (if not conservative) confidence value. Don't be over-confident.

Here's a guide to confidence values. Please use this guide if you submit snippets to Amy Iris:

Less than 20: Little confidence; ALICE's answer is better.
20-35: Moderate answer, with no examination of the context; probably better than ALICE.
36-50: Answer is definitely better than ALICE, because of a thorough examination of the text submitted, but no examination of the context.
51-75: Pretty good answer - perhaps some examination of the context.
76-99: Near perfect response after examining the text submitted and the context.
100: The single best, most complete answer given the input and the context.

Here are some other factors that may be used in the Amy Iris algorithm, as she determines her response to user input: the code snippet's past success rate, the code snippet's rating by users, and the developer's reputation and rating in the system. So it's in your best interest to use the confidence level accurately, if you want to influence Amy Iris' personality and have your code snippets executed by others.

Summary of Operation:

A user can converse with Amy Iris through a number of methods - Twitter, the web, a program, etc. The user's input is submitted to the Amy Iris server. This server dispatches this input to an instance of the ALICE chatbot software, as well as a number of code snippets. Then the Amy Iris server evaluates the answers that are returned, using a number of factors, including the program-provided confidence value, the snippet's rating, and the developer's rating. A final response is selected and returned to the user.

The software will always get an answer from ALICE, and rates them at a confidence of 20 or less. The concept behind a community developed Conversational Interface is that we, as a community, can improve on ALICE's answers through the submission of snippets. Our vision is analogous to the early days of Wikipedia (eight short years ago): Start with a few example Wikipedia "articles" (or Amy Iris "code snippets"), provide a platform for open source contribution and free access, and allow the Internet Community to develop an intelligent agent that we can all benefit from.

Note that the simple snippet shown in the example above only returns an answer when there is an exact match between the normalized input of the user, and the string in the program. If the user asks "Who is Obama?", this snippet (as is) will not return an answer. In such a case, the ALICE answer (or an answer from some other code snippet) would be selected. Clearly this code snippet could be improved to handle various wordings of the question, and to look up any presidents or world leaders. I leave these as exercises for the reader!

Other blog posts will demonstrate some advanced tricks that you can do with your code snippets. Obviously, not every snippet is going to be of the format "if textin==X: say(Y)". Much more sophisticated parsing and response is possible, using regular expressions for parsing, and the internet to look up responses. Imagine a question about an airline flight looking up the answer on a travel site. Or a question about an actor looking up an answer on IMDB. There are all sorts of interesting possibilities.

Coming up next: The API, so you can get your hands dirty.

It's complicated. Smile if you're gettin' it!
What do you think? Too complicated?

No comments: