Introduction To Natural Language Processing

Natural Language Processing
Michael Collins, Columbia University
Overview
I
I
I
What is Natural Language Processing (NLP)?

Why is NLP hard?
What will this course be about?
What is Natural Language Processing?
computers using natural language as input and/or output

language
understanding
(NLU)
computer
language
generation
(NLG)
Machine Translation:
e.g., Google Translation from Arabic
Stock prices retreated in the stock markets again with increasing concern
about the circumstances surrounding the credit markets in the world, due
mostly to the problems it faces American mortgage lending market,
which raised concern among investors.
The index retreated Vuciji / 100 on the London Stock Exchange at the
beginning of a percentage point in the dealings of up to 6082 points,
while the Nikkei index retreated / 225 Japanese rate of 2.2% to close at
the lowest level in eight months.
The American Jones index has lost about 1.6 points Tuesday to reach
13029 points, the Nasdaq index had lost 1.7 of its value.
These declines came despite statements by the American Federal Reserve
Bank (Central Bank), in which he said that the process of pumping more
funds into capital markets when necessary.
Information Extraction
10TH DEGREE is a full service advertising agency specializing in direct and interactive marketing. Located in Irvine CA, 10TH DEGREE is looking for an Assistant
Account Manager to help manage and coordinate interactive marketing initiatives
for a marquee automative account. Experience in online marketing, automative
and/or the advertising field is a plus. Assistant Account Manager Responsibilities
Ensures smooth implementation of programs and initiatives Helps manage the delivery of projects and key client deliverables . . . Compensation: $50,000-$80,000
Hiring Organization: 10TH DEGREE
INDUSTRY
POSITION
LOCATION
COMPANY
SALARY
Advertising
Assistant Account Manager
Irvine, CA
10TH DEGREE
$50,000-$80,000
Information Extraction
I
I
Goal: Map a document collection to structured database

Motivation:
I
Complex searches (Find me all the jobs in advertising

paying at least $50,000 in Boston)
Statistical queries (How has the number of jobs in
accounting changed over the years?)
Text Summarization
Dialogue Systems
User: I need a flight from Boston to Washington, arriving by

10 pm.
System: What day are you flying on?
User: Tomorrow
System: Returns a list of flights
Basic NLP Problems: Tagging

TAGGING: Strings to Tagged Sequences
a b e e a f h j a/C b/D e/C e/C a/D f/C h/D j/C
Example 1: Part-of-speech tagging
Profits/N soared/V at/P Boeing/N Co./N ,/,
easily/ADV topping/V forecasts/N on/P Wall/N
Street/N ./.
Example 2: Named Entity Recognition
Profits/NA soared/NA at/NA Boeing/SC Co./CC
,/NA easily/NA topping/NA forecasts/NA on/NA
Wall/SL Street/CL ./.
Basic NLP Problems: Parsing

INPUT:
Boeing is located in Seattle.
OUTPUT:
S
NP
N
Boeing
VP
V
is
VP
V
located
PP
P
in
NP
N
Seattle
Overview
I
I
I

Why is NLP hard?
Why is NLP Hard?

[example from L.Lee]
At last, a computer that understands you like your mother
Ambiguity
At last, a computer that understands you like your mother

1. (*) It understands you as well as your mother understands
you
2. It understands (that) you like your mother
3. It understands you as well as it understands your mother
1 and 3: Does this mean well, or poorly?
Ambiguity at Many Levels
At the acoustic level (speech recognition):

1. . . . a computer that understands you like your mother
2. . . . a computer that understands you lie cured mother
At the syntactic level:

VP
V
understands
NP
you
VP
S
like your mother [does]
V
understands
S
[that] you like your mother
Different structures lead to different interpretations.
More Syntactic Ambiguity
VP
V
list
VP
NP
DET
V
N
list
all
NP PP
flights
PP
on Tuesday
all
on Tuesday
flights
At the semantic (meaning) level:

Two definitions of mother
I a woman who has given birth to a child
I a stringy slimy substance consisting of yeast cells and
bacteria; is added to cider or wine to produce vinegar
This is an instance of word sense ambiguity
More Word Sense Ambiguity
At the semantic (meaning) level:

I
They put money in the bank

= buried in mud?
I saw her duck with a telescope
At the discourse (multi-clause) level:

I
Alice says theyve built a computer that understands you

like your mother
But she . . .
. . . doesnt know any details
. . . doesnt understand me at all
This is an instance of anaphora, where she co-referees to some

other discourse entity
Overview
I
I
I

Why is NLP hard?
Course Coverage
NLP sub-problems: part-of-speech tagging, parsing,

word-sense disambiguation, etc.
Machine learning techniques: probabilistic context-free
grammars, hidden markov models, estimation/smoothing
techniques, the EM algorithm, log-linear models, etc.
Applications: information extraction, machine translation,
natural language interfaces...
A Syllabus
Language modeling, smoothed estimation
Tagging, hidden Markov models
Statistical parsing
Machine translation
Log-linear models, discriminative methods
Semi-supervised and unsupervised learning for NLP
Prerequisites
I
I
Basic linear algebra, probability, algorithms

Programming skills
Assessment
Questions during the lectures
3 homeworks
Books
Comprehensive notes for the course will be provided at

http://www.cs.columbia.edu/~mcollins
Additional useful background:
Jurafsky and Martin:
Speech and Language Processing (2nd Edition)

Introduction To Natural Language Processing

Uploaded by

Copyright:

Available Formats

Introduction To Natural Language Processing

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Natural Language Processing

Uploaded by

Copyright:

Available Formats

Natural Language Processing

Michael Collins, Columbia University

What is Natural Language Processing (NLP)?

What is Natural Language Processing?

computers using natural language as input and/or output

Goal: Map a document collection to structured database

Complex searches (Find me all the jobs in advertising

User: I need a flight from Boston to Washington, arriving by

Basic NLP Problems: Tagging

Basic NLP Problems: Parsing

What is Natural Language Processing (NLP)?

Why is NLP Hard?

At last, a computer that understands you like your mother

At last, a computer that understands you like your mother

Ambiguity at Many Levels

At the acoustic level (speech recognition):

Ambiguity at Many Levels

At the syntactic level:

Different structures lead to different interpretations.

More Syntactic Ambiguity

Ambiguity at Many Levels

At the semantic (meaning) level:

More Word Sense Ambiguity

At the semantic (meaning) level:

They put money in the bank

I saw her duck with a telescope

Ambiguity at Many Levels

At the discourse (multi-clause) level:

Alice says theyve built a computer that understands you

This is an instance of anaphora, where she co-referees to some

What is Natural Language Processing (NLP)?

NLP sub-problems: part-of-speech tagging, parsing,

Language modeling, smoothed estimation

Tagging, hidden Markov models

Log-linear models, discriminative methods

Semi-supervised and unsupervised learning for NLP

Basic linear algebra, probability, algorithms

Questions during the lectures

Comprehensive notes for the course will be provided at

You might also like