Natural Language Processing - Class 3

Eliza

General Background on Eliza

Turing Test: is a computer program intelligent? (1954)
would a human find out that he speaks with a computer?

Eliza (Weizenbaum 1966, MIT)

A conversation of a Rogerian psychotherapist style: attempt to draw the patient out by reflecting the patient's statements back to him, and to encourage him to continue talking.

Men Are all alike.
IN WHAT WAY.
They're always bugging us about something or another.
CAN YOU BE MORE SPECIFIC.
Well, my boyfriend made me come here.
YOUR BOYFRIEND MADE YOU COME HERE.
He says I'm depressed much of the time.
I AM SORRY TO HEAR YOU ARE DEPRESSED.
It's true. I'm unhappy.
DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY.
I need some help, that much is certain.
WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP.
Perhaps I could learn to get along with my mother.
TELL ME MORE ABOUT YOUR FAMILY.

Amazingly:

Why conversation systems attracted interest in the 60s?
maybe because people tend to read much more meaning into what is said, than there is [Hutchers].

Now - the Program itself

Pattern Matching and Unification

Code if from Paradigms of Artificial Intelligence Programming, Peter Norvig, Morgan Kaufmann.

Specification:

  1. Read input.
  2. Find a pattern matching input.
  3. Transform input into a response.
  4. Print response.
To start, for 1 & 4 use READ and PRINT.

Pattern matching:

Pattern:
(i need a X)
Response:
(what would it mean to you if you got a X ?)
Match symbols with same symbols and variables with any symbol.
Generalization of equal:
(defun simple-equal (x y)
 "Simple recursive equal, not for inside strings"
 (if (or (atom x) (atom y))
     (eql x y)
     (and (simple-equal (first x) (first y))
          (simple-equal (rest x) (rest y)))))

(defun pat-match (pattern input)
  "Does pattern match input? Any variable can match anything."
  (if (variable-p pattern)
    t
    (if (or (atom pattern) (atom input))
      (eql pattern input)
      (and (pat-match (first pattern) (first input))
           (pat-match (rest pattern) (rest input))))))

Representing variables:

Use naming convention on symbols. Any symbol with name starting with '?'.
(defun variable-p (x)
  "Is x a variable (a symbol beginning with '?')?"
  (and (symbolp x) (equal (char (symbol-name x) 0) #\?)))

Extracting information from the match

Want to get out the bindings of variables when successful.
How to represent bindings? .
Take advantage of the SUBLIS primitive.
(sublis '((?X . vacation)) '(what would it mean to you if you got a ?X ?)) 
(WHAT WOULD IT MEAN TO YOU IF YOU GOT A VACATION ?)
So represent bindings as Association lists. Must compute bindings.
(defun pat-match (pattern input)
  "Does pattern match input? BUGGY VERSION."
  (if (variable-p pattern)
    (list (cons pattern input))
    (if (or (atom pattern) (atom input))
      (eql pattern input)
      (append (pat-match (first pattern) (first input))
              (pat-match (rest pattern) (rest input))))))

4 bugs:
  1. (eql pattern input) can return T and append will die.
  2. (eql pattern input) can return NIL meaning failure, and be interpreted as an empty binding (semi-predicate problem).
  3. We want the bindings of variables to agree: (?X ?X) (1 2) fails.
  4. Inefficient to check first and rest when we know that first fails.

Bindings Management

Need to be more careful about bindings: New abstraction:
(defconstant fail nil)
(defconstant no-bindings '((t . t)))

(defun get-binding (var bindings)
  "Find a (var . value) pair in a binding list."
  (assoc var bindings))

(defun binding-val (binding) 
  "Get the value part of a binding."
  (cdr binding))

(defun lookup (var bindings)
  "Get the value part (for var) from a binding list."
  (binding-val (get-binding var bindings)))

(defun extend-bindings (var val bindings)
  "Add a (var . value) pair to a binding list."
  (cons (cons var val) bindings))

A (first) correct version of pat-match with binding management

(defun pat-match (pattern input &optional (bindings no-bindings))
  "Match pattern against input in the context of the bindings."
  (cond ((eq bindings fail) fail)
        ((variable-p pattern) 
         (match-variable pattern input bindings))
        ((eql pattern input) bindings)
        ((and (consp pattern) (consp input))
         (pat-match (rest pattern) (rest input)
                    (pat-match (first pattern) (first input) bindings))) 
        (t fail)))

Lisp Note: The use of &optional: can also assign default value.

(defun match-variable (var input bindings)
  "Does VAR match input? Uses (or updates) and returns bindings."
  (let ((binding (get-binding var bindings)))
    (cond ((not binding) (extend-bindings var input bindings))
          ((equal input (binding-val binding)) bindings)
          (t fail))))

> (pat-match '(i need a ?X) '(i need a vacation))
((?X . vacation) (t . t))
Make extend-bindings smart (if not empty, remove 'no binding'):
(defun extend-bindings (var val bindings)
  "Add a (var . val) pair to a binding list."
  (cons (var val)
        (if (eq bindings no-bindings) nil bindings)))

> (pat-match '(1 2 3) '(1 2 3))
((t . t))
> (pat-match '(?X is ?X) '((2 + 2) is 4))
nil
> (pat-match '(?X is ?X) '((2 + 2) is (2 + 2)))
((?X 2 + 2))
> (pat-match '(?p need ?x) '(I need a long vacation))
((?x a long vacation) (?p I))

Segment pattern matching

(defun segment-pattern-p (pattern)
  (and (consp pattern) (starts-with (first pattern) '?*)))

> (pat-match '((?* ?x) is (?* ?p) ?) '(1 + 2 * 3 is 5 + 2 ?))
((?x 1 + 2 * 3) (?p 5 + 2))

(defun pat-match (pattern input &optional (bindings no-bindings))
  "Match pattern against input in the context of the bindings."
  (cond ((eq bindings fail) fail)
        ((variable-p pattern) 
         (match-variable pattern input bindings))
        ((eql pattern input) bindings)
        ((segment-pattern-p pattern)                       ;; ***
         (segment-pattern-match pattern input bindings))   ;; ***
        ((and (consp pattern) (consp input))
         (pat-match (rest pattern) (rest input)
                    (pat-match (first pattern) (first input) bindings))) 
        (t fail)))

Segment Pattern Matching and backtracking

How far should segment pattern match?

Assume segment pattern is followed by a non variable C (pattern starts with
((?* ?p) C p2...))
Look for first occurrence of C in input: index pos.
(... C i2 ...)
If not found, fail. Else match up to pos and check that (rest input) matches (rest pattern)
b2 = (pat-match (p2...) (r2 ...))
If b2 succeeds - > succeed and extend with binding ?p. Else? Maybe need to bind ?p to a longer segment.
> (pat-match '((?* ?p) is a (?* ?x)) '(what he is is a fool))
((?p what he is) (?x fool))
Need another argument to remember length of segment tried last.

Also, need to call match-variable before testing whether b2 fails
(for cases like
> (pat-match '((?* ?x) a b (?* ?x)) '(1 2 a b a b 1 2 a b)) 
otherwise, it will try ?x / (1 2), then (a b) matches, but 2nd call to ?x fails.
(defun segment-match (pattern input bindings &optional (start 0))
  "Match segment pattern ((?* var) . pat) against input."
  (let ((var (second (first pattern)))
        (pat (rest pattern)))
    (if (null pat)
      (match-variable var input bindings)
      ;; assume pat starts with a constant
      (let ((pos (position (first pat) input :start start :test #'equal))) 
        (if (null pos)
          fail
          (let ((b2 (pat-match pat (subseq input pos) 
                               (match-variable var (subseq input 0 pos)       
                                               bindings))))
            ;; If this one failed, try a longer segment
            (if (eq b2 fail)
              (segment-match pattern input bindings (+ pos 1))
              b2)))))))

> (pat-match '((?* ?x) a b (?* ?x)) '(1 2 a b a b 1 2 a b))
((?X 1 2 A B))

Lisp Note:
One of a set of "Searching Sequences for Items" functions.
Examples:
(find item sequence &key :from-end :test :test-not :start :end :key)
(find-if-not predicate sequence &key :from-end :start :end :key)
(position item sequence &key  :from-end  :test :test-not 
          :start :end :key)
(count-if predicate sequence &key :from-end :start :end :key )

(subseq sequence start &optional end)
This returns the subsequence of sequence specified by start and end. subseq always allocates a new sequence for a result; it never shares storage with an old sequence. The result subsequence is always of the same type as the argument sequence.

ELIZA: A Rule-based translator

(defun rule-pattern (rule) (first rule))

(defun rule-responses (rule) (rest rule))
Rule example:
((?* ?x) I want (?* ?y))
 (What would it mean if you got ?y)
 (Why do you want ?y)
 (Suppose you got ?y soon))
If pattern matches, select one of the answers randomly.

How to handle a set of rules?
Several rules match, what to do?

Use option 1.


(defparameter *eliza-rules*
  '((((?* ?x) hello (?* ?y))
     (How do you do. Please state you problem.))
    (((?* ?x) I want (?* ?y))
     (What would it mean if you got ?y)
     (Why do you want ?y) (Suppose you got ?y soon))
    (((?* ?x) if (?* ?y))
     (Do you really think it is likely that ?y) (Do you wish that ?y)
     (What do you think about ?y) (Really -- if ?y))))

A rule-based interpreter

(defun eliza ()
  "Respond to user input using pattern matching rules."
  (loop
    (print 'eliza>)
    (write (flatten (use-eliza-rules (read))) :pretty t)))


(defun use-eliza-rules (input)
  "Find some rule with which to transform the input."
  (some #'(lambda (rule)
            (let ((result (pat-match (rule-pattern rule) input)))
              (if (not (eq result fail))
                (sublis (switch-viewpoint result)
                        (random-elt (rule-responses rule))))))
        *eliza-rules*))


(defun switch-viewpoint (words)
  "Change I to you and vice-versa, and so on."
  (sublis '((I . you) (you . I) (me . you) (am . are)) words))


(defun flatten (the-list)
  "Append together elements or lists in the-list"
  (mappend #'mklist the-list))

(defun mklist (x)
  "Return x if it is a list, otherwise (x)"
  (if (listp x) x (list x)))

(defun random-elt (choices)
  (elt choices (random (length choices))))


Test ELIZA - http://www-ai.ijs.si/eliza/eliza.html
And another one - http://www.parnasse.com/drwww.cgi
its quite popular - http://calendarhome.com/eliza/
Other conversations - http://ciips.ee.uwa.edu.au/~hutch/hal/
Alan turing - http://www.abelard.org/turpap/turpap.htm turing - http://www.turing.org.uk/turing/scrapbook/test.html

For any question, contact me: yaeln@cs.bgu.ac.il
Back to course homepage

Last modified , 2000