Coreference Resolution and Entity Linking

Understanding Referring Expressions

Let's dive into the world of how we use language to refer to things. It's not always as straightforward as naming something once and sticking to that name. We use different types of expressions, depending on whether we're introducing something new or talking about something we've already mentioned.

Types of Referring Expressions

Indefinite Noun Phrases: Think of these as introductions. They use words like "a" or "some" to bring new entities into the conversation. For example:
"I saw a cat in the garden." Here, the cat is likely being mentioned for the first time.
Definite Noun Phrases: These refer to something the listener can identify. This could be because it's already been mentioned, or because it's unique and well-known. Example:
"I read about it in the New York Times." Most people know what the New York Times is.

"Have you seen the car keys?" Assuming you own a car, 'the car keys' would refer to something that you would know and is identifiable to you.
Pronouns: These are used for entities that are already very obvious in the conversation. Think "he," "she," or "it."
"Emma smiled and chatted as cheerfully as she could." 'She' clearly refers to Emma.
Demonstrative Pronouns: "This" and "that" are used to point to specific items.
"I just bought a copy of Thoreau’s Walden. That one had been very tattered; this one was in much better condition."
Zero Anaphora: In some languages, you can leave out a pronoun entirely, but it's still understood.
"[John] went to visit some friends. On the way [he] bought some wine." The "he" is implied in the second sentence.
Names: These can refer to new or old entities (people, places, organizations)
"Miss Woodhouse certainly had not done him justice."

"International Business Machines sought patent compensation from Amazon; IBM had previously sued other companies."

Information Status: Given vs. New

The way we use these referring expressions tells us something about the "information status" of the entity. Is it new information? Is it old information that we both know about?

Brand new NPs: These introduce entities that are discourse-new and hearer-new like a fruit or some walnuts.
Unused NPs: These introduce entities that are discourse-new but hearer-old (like Hong Kong, Marie Curie, or the New York Times.
Old NPs: also called evoked NPs, these introduce entities that already in the discourse model, hence are both discourse-old and hearer-old, like it in “I went to a new restaurant. It was...”.
Inferrables: Sometimes, we can infer an entity's existence based on what's already been said.
"I went to a superb restaurant yesterday. The chef had just opened it." We can infer that restaurants have chefs.

"Mix flour, butter and water. Knead the dough until shiny." We can infer that mixing flour and liquid will result in 'the dough'.

Complications: Non-Referring Expressions

Not every noun phrase refers to a specific entity. Some act more like descriptions or fillers. Recognizing these is key for coreference resolution.

Appositives: These are phrases that describe another noun phrase, often set off by commas.
"Victoria Chen, CFO of Megabucks Banking, saw..." The CFO phrase describes Victoria Chen but isn't a separate entity.
Predicative and Prenominal NPs: These describe properties of a noun.
"United is a unit of UAL." "A unit of UAL" describes United, rather than referring to a distinct entity.
Expletives: Some pronouns, like "it" in "It is raining," don't refer to anything.
It was Emma Goldman who founded Mother Earth

It surprised me that there was a herring hanging on her wall.
Generics: These refer to a whole class of things, not a specific instance.
"I love mangos. They are very tasty." "They" refers to mangos in general.

"In July in San Francisco you have to wear a jacket." The "you" refers to anyone that's in San Francisco in July, and not a specific person.

Linguistic Properties of Coreference

Certain grammatical rules and common sense helps to determine if two entities are referring to the same thing. Some common rules are as follows:

Number Agreement: Singular things are usually referred to by singular pronouns (he, she, it), and plural things by plural pronouns (they, them).
"IBM announced a new machine translation product yesterday. They have been working on it for 20 years." IBM (the business entity) can be referred to as either 'it' or 'they'

"singular they has become much more common, in which they is used to describe singular individuals, often useful because they is gender neutral."
Person Agreement: You can't switch between "I," "you," and "he/she/it" without a good reason.
“I voted for Nader because he was most aligned with my values,” she said. I, my, and she are coreferent
Gender Agreement: Pronouns usually match the gender of the thing they refer to. (he for males, she for females, it for non-personal grammatical genders).
Maryam has a theorem. She is exciting. (she=Maryam, not the theorem)

Maryam has a theorem. It is exciting. (it=the theorem, not Maryam)
Binding Theory Constraints: Syntactic rules about how pronouns and their referents can relate in the same sentence.
Janet bought herself a bottle of fish sauce. [herself=Janet]

Janet bought her a bottle of fish sauce. [her≠Janet]
Recency: Things mentioned recently are more likely to be referred to again.
The doctor found an old map in the captain’s chest. Jim found an even older map hidden on the shelf. It described an island. 'It' is more likely to refer to Jim's map.
Grammatical Role: Subjects of sentences are more salient than objects.
Billy Bones went to the bar with Jim Hawkins. He called for a glass of rum. [ he = Billy ]

Jim Hawkins went to the bar with Billy Bones. He called for a glass of rum. [ he = Jim ]
Verb Semantics: Some verbs emphasize certain entities.
John telephoned Bill. He lost the laptop. 'He' is more likely to refer to John

John criticized Bill. He lost the laptop. 'He' is more likely to refer to Bill
Selectional Restrictions: Verbs limit what kinds of things can be their subjects or objects.
I ate the soup in my new bowl after cooking it for hours 'It' is more likely to refer to the soup, since you eat soup

Coreference Resolution: Tasks and Datasets

Coreference resolution aims to find all mentions of entities in a text and link them together. In other words, figuring out which words are referring to the same thing.

For example, imagine the following text:

"Victoria Chen, CFO of Megabucks Banking, saw her pay jump to $2.3 million, as the 38-year-old also became the company’s president. It is widely known that she came to Megabucks from rival Lotsabucks."

The coreference chains would be:

Victoria Chen, her, the 38-year-old, She
Megabucks Banking, the company, Megabucks
her pay
Lotsabucks

Datasets like OntoNotes are used to train and test these systems.

Exactly what counts as a mention and what links are annotated differs from task to task and dataset to dataset. For example some coreference datasets do not label singletons, making the task much simpler. Resolvers can achieve much higher scores on corpora without singletons, since singletons constitute the majority of mentions in running text, and they are often hard to distinguish from non-referential NPs. Some tasks use gold mention-detection (i.e. the system is given human-labeled mention boundaries and the task is just to cluster these gold mentions), which eliminates the need to detect and segment mentions from running text.

Mention Detection: Finding the Mentions

Mention detection is the first step: identifying the text spans that could be mentions. The algorithms are often very liberal, proposing many candidates and filtering them later.

Consider the sample text again:

A simple approach might identify these potential mentions:

Victoria Chen
$2.3 million
she
CFO of Megabucks Banking
the 38-year-old
Megabucks
Megabucks Banking
the company
Lotsabucks
her
the company’s president
her pay
It

The algorithm then filters out non-referring expression like predicate nominals, and expletives like "It".

More sophisticated algorithms use machine learning to do mention detection, anaphoricity and coreference jointly, in a single end-to-end model.

Architectures for Coreference Algorithms

The Mention-Pair Architecture

The mention-pair architecture is based around a classifier that is given a pair of mentions, a candidate anaphor and a candidate antecedent, and makes a binary classification decision: coreferring or not.

The Mention-Rank Architecture

The mention ranking model directly compares candidate antecedents to each other, choosing the highest-scoring antecedent for each anaphor.

Entity-based Models

Entity-based models link each mention not to a previous mention but to a previous discourse entity (cluster of mentions).

Classifiers Using Hand-Built Features

Given an anaphor mention and a potential antecedent mention, feature based classifiers make use of three types of features: (i) features of the anaphor, (ii) features of the candidate antecedent, and (iii) features of the relationship between the pair.

Feature	Description
First (last) word	First or last word (or embedding) of antecedent/anaphor
Head word	Head word (or head embedding) of antecedent/anaphor
Attributes	The number, gender, animacy, person, named entity type attributes of (antecedent/anaphor)
Length	Length in words of (antecedent/anaphor)
Mention type	Type: (P)roper, (D)eﬁnite, (I)ndeﬁnite, (Pr)onoun) of antecedent/anaphor
Entity shape	The ‘shape’ or list of types of the mentions in the antecedent entity (cluster), i.e., sequences of (P)roper, (D)eﬁnite, (I)ndeﬁnite, (Pr)onoun.
Entity attributes	The number, gender, animacy, person, named entity type attributes of the antecedent entity
Ant. cluster size	Number of mentions in the antecedent cluster
Sentence distance	The number of sentences between antecedent and anaphor
Mention distance	The number of mentions between antecedent and anaphor
Exact String Match	True if the strings of any two mentions from the antecedent and anaphor clusters are identical.
Head Word Match	True if any mentions from antecedent cluster has same headword as any mention in anaphor cluster
Word Inclusion	All words in anaphor cluster included in antecedent cluster

Entity Linking: Connecting to the Real World

Entity linking goes a step further: it associates a mention in the text with a specific entity in a knowledge base (like Wikipedia). This is also known as "wikification."

For example, the mention "Megabucks Banking" could be linked to its Wikipedia page, providing more information about the company.

The most common association for the span Yuan in the anchor dictionary is the name of the Chinese currency, i.e., the probability p(Yuan currency| yuan) is very high. Rarer Wikipedia associations for Yuan include the common Chinese last name, a language spoken in Thailand, and the correct entity in this case, the name of the Chinese dynasty.

Evaluation of Coreference Resolution

Coreference systems are evaluated by comparing the system-generated chains to human-annotated "gold standard" chains.

Common metrics include: MUC F-measure, B3, CEAF, BLANC, and LEA.

Winograd Schema Problems: Testing Understanding

These are tricky coreference problems designed to require world knowledge and common-sense reasoning.

Example:

"The city council denied the demonstrators a permit because they feared violence." Who feared violence? (The council)

"The city council denied the demonstrators a permit because they advocated violence." Who advocated violence? (The demonstrators)

Gender Bias in Coreference

Coreference models can exhibit gender bias, often reflecting societal stereotypes. WinoBias dataset can be used to test gender bias. Zhao et al. (2018a) show that coreference systems of all architectures show significant bias, performing on average 21 F1 points worse in the anti-stereotypical cases.

Data Coder

Search This Blog