General Intro to NLP

• What is NLP About?
• Ambiguity in Language
• Variability in Language
• Levels of Linguistic Description
• Words: Parts of Speech and Morphology
• The Penn Treebank Annotation Scheme
• Sentences: Syntax
• Argument Structure: Lexical properties of the verb
• Adjuncts and Disjuncts
• The Mood of Clauses
• Noun Phrase

What is NLP about

• Natural Language Processing
• Computational Linguistics
• Human Language Technologies

• Engineering: useful tasks involving text (Information Retrieval, Machine Translation, Summarization, Report Generation)
• Scientific: understand language (what it is, how it is used, how it is learned, how it evolves over time) using computational models

• Knowledge-based methods: rely on symbolic knowledge representation (lexicon, grammar, semantic structures and inference rules)
• Statistical methods: rely on statistical models, machine learning and large collections of data. Probabilistic inference.

• Ambiguity
• Variability
• Structure
• Conventionality: arbitrary agreement on sign/denoted relation
• Non-categorical phenomena (non-binary decisions) [what is grammatical, what is a word, what is meaningful]
• Relation to cognition: meaning, communication, experience [what is more likely to "make sense"]; related to other cognitive phenomena [memory, learning, planning]

Ambiguity in Language

• Lexical: bank (river/financial)
• Morphological: books (book(N)+plural) vs. (book(V)+present+singular+third person
• Syntactic: I see a man on the hill with a telescope / (I am on the hill vs. the man is on the hill)
• Pragmatic: Can you pass the salt (Do you have the ability of passing the salt (Yes-no question) vs. request to pass the salt)

• Global:

  Flying planes made her duck

  1. the airplanes made her change her position
  2. the act of piloting made her change her position
  3. piloting turned her into a duck
  4. the airplanes caused her duck (the animal) to exist
  5. the act of piloting made her duck exist
• Local:

  The company that bought [Scicon sold VIM].

  Is [Scicon sold VIM] a proposition? Implications on IR.

Variability in Language

• Lexical:
  • book, volume, document, best-seller, publication... (synonyms)
  • X causes Y vs X is the cause of Y vs. X leads to Y vs. X triggers Y
  • the book deals with Linguistics vs. the book is about linguistics
• Syntactic:
  • the cat eats the mouse vs. the mouse is eaten by the cat (active vs. passive)
  • John gives Mary a book vs. John gives a book to Mary (dative move)
  • John gives a book about Linguistics to Mary vs. John gives to Mary a book about linguistics (heavy NP shift)
• Referential: when refering to a table in a room:
  • The table
  • The piece of furniture in the left corner
  • The table next to which I am standing (if there are more than one table)
  • it
  • this
  • that
• Stylistic: Raymond Queneau: Exercises in Style (99 ways to retell the same story in different styles).

Levels of Linguistic Description

• Discourse: text structure, conversation structure
• Sentence: syntax
• Lexical: words, word structures, relations among words within a dictionary
• Phonologic: sound structure
• Letters

• Textual: what are the relations of different text spans with other textual spans (repetition, reference, citation, allusion) and within a given textual unit, how do sub-units relate to each other (syntax).
• Inter-personal: Communication, what is achieved by using language between speakers, writer/reader relation.
• Content: relation between text and the world language describes.

• Syntax, morphology
• Pragmatics
• Semantics

Words: Parts of Speech and Morphology

• What is a word? (basic terminal unit in the parse tree) / Issues in tokenization.
• What is a sentence? (top level unit in the parse tree) / Sentence delimitation.
• What are the basic properties of words

• Graphic: a sequence of letters surrounded by delimiters (issues: what are delimiters)
• Phonetic: a sequence of sounds surrounded by silence (issues: what is a phonetic delimiter, are all words really separated by silence)
• Semantic: an independent unit of meaning (issues: non-compositional word compounds, semantically empty words)
• Morphological: a structural unit composed of a least one base morphemes and decomposable into morphemes
• Syntactic: a constituent which can be the head of a lexical phrase
• Dictionary-based: a unit that is listed in the "lexicon" of a language

 A {large, small, blue, enormous, ...} box is in the room.  

• verb: action or state
• noun: entity (people, concepts, things)
• adjective: property

• Open-class (many members, new members often added)
• Closed-class (few members, functional use): article, preposition...

• Features of words: number, gender, tense, person, case
• Processes: Inflection, derivation, compounding.

• Inflection is the process of modifying a root form by combining prefixes and suffixes to indicate the presence of morphological features.
• Derivation is the process of creating a new lexical item from an existing, more basic one. For example, the derivation of the noun of action destruction from the verb destroy, or the chain of derivations luck/noun, lucky/adjective, luckily/adverb.
• Compounding is the process of combining several lexical items into a new one, whose properties can be derived from the compounded elements, or can be independent. For example, in Hebrew beit sefer is derived from bayit and sefer.

• Prefix and suffix attachment (for example, add an "s" at the end of nouns to mark plural)
• Internal change (for example: sing -> sang).

• Noun: number, gender, [possessive marker]
• Pronoun: number (he/they), gender (he/she), person (I/you/he), case (objective - him, subjective - he, possessive - his), reflexive (himself)
• Verb: number, gender, person, tense
• Adjective: number, gender

The Penn Treebank Annotation Scheme

Sentence: Syntax

• Graphic: a sequence of words surrounded by delimiters (issues: what are delimiters)
• Semantic: a complete unit of meaning expressing a proposition (predicate stated on its arguments).
• Syntactic: a toplevel syntactic constituent (not covered by other syntactic elements).

• Substitution: if one word is replaced by another with different morphological features (e.g., singular->plural), are other words affected by this change? (Agreement).
• Deletion: if one word is removed from the sentence, does the sentence remain syntactically correct? Must other words be deleted as a consequence? (Dependency)
• Movement: if one word is moved within the sentence, must other words follow it?

• Constituent-based: a constituent is a group of words, that is assigned a certain category. Basic-level constituents are called phrases.
• Dependency-based: describes a hierarchical structure of dependency among words within the sentence.

Phrases are groups of words that depend on a "central word" that we call the phrase head. Common Phrases include - depending on the part of speech of the head:

• Noun Phrase (NP): two-thirds of the boys
• Adjectival Phrase (AP): very large
• Verb Phrase (VP): walks quickly
• Prepositional Phrases (PP): in the room

• Definition of constituent. When is a group a constituent?

     [John] [   [eats] [an apple]]
     [NP  ] [VP [V   ] [NP      ]]  (embedded)
     [NP  ]     [V   ] [NP      ]   (flat)
  

• The definition of constituency depends on the goal of the analysis. Syntactic constituents, Informational constituents (given/new or theme/rheme constituents, intonation constituents. Questions raised:
  1. Is there a connection between different structural dimensions? ie, Must a theme/rheme constituent also be a proper syntactic constituent?
  2. How do you define empirical tests for the determination of constituents?
• Are there discontinuous constituents?

     [Do] you [have] a hammer?
  

• Are there limits to the complexity of the structure of constituents?

• Structure determines meaning.
• Structure impacts on form (for example, agreement).
• Syntactic structure helps us perform text transformations (simplify a sentence, combine 2 sentences).

S1: John eats
S2: John sleeps

S3: After eating, John slept

Descriptive Syntax of the Clause

1. Transitivity: determines the type of the main process and its participants. (This is often called the "argument structure" of the clause.)
2. Circumstantials: determines the structure of modifiers to the predicate and to the clause as a whole.
3. Mood: determines whether the clause is finite (declarative, interrogative or relative) or non-finite (imperative, infinitive, participial). For finite moods, tense and person are specified. For non-finite, they are not.
4. Voice: active, passive, causative etc.

• They can surface as subject in one syntactic alternation of the clause. For example:
  • John gives Mary a book.
  • Mary is given a book by John.
  • A book is given to Mary by John.
  This indicates that John, Mary, and a book are participants in the clauses.
• They cannot be moved around in the clause without affecting the other constituents. For example:
  • John eats a pie.
  • * John a pie eats.
  • ? A pie John eats. (would need a comma after pie).
  Whereas for non-participants, moving is easier:
  • John eats a pie on the sofa.
  • On the sofa John eats a pie.
• They cannot be omitted from the clause. For example,
  • John uses a car to travel.
  • *John uses to travel.
  • John uses a car.

Argument Structure: Lexical properties of the verb

While systemic linguists have developed complex transitivity systems to account for the structure of the clause, lexicalist approaches to grammar have developed the notion of subcategorization. The idea is that in any phrase, the lexical head determines the structure and order of the subconstituents or dependents in a dependency-basedgrammar like Meaning Text Theory (Mel’cuk and Pertsov, 1987) of the phrase. To capture this dependency, HPSG grammars describe each lexical item capable of governing arguments with a feature called the subcategorization list (short name: subcat). The subcat of a verb is a list of the constituents the verb expects to form a complete phrase. So forexample, the verb to deal as in "AI deals with logic" is represented in the lexicon with a subcat

deal[Subcat( [ NP ] [ PP : with ])]

Adjuncts and Disjuncts

• Predicate Adjuncts:

  

• Sentence Adjuncts:

  

• Disjuncts:

  

The Mood of Clauses

Noun Phrase