Word Analysis of 2020 U.S. Presidential Debates
Mike Pence vs. Kamala Harris
7 October 2020
Introduction
After the first Trump vs Pence debate, this was delightfully boring, measured and evasive. I'll take it.
Speaking Turns and Interruptions
Here, I look at the length of each turn of uninterrupted speech.
Table 1
length of sections in words
The number of uninterrupted deliveries (sections), mode/median/mean length of sections in words, and the shortest section length in words that composed 10%, 50% and 90% of the debate.
Although Pence talked over both Harris and the moderator, both candidates took roughly the same number of turns speaking, which was about a third of what we saw in the Trump vs Pence debate. There, Trump's transcript had +177.0% (313 vs 113) more sections than Pence's and Biden's transcript had +164.9% (249 vs 94) more sections than Harris.
The median length of Pence's sections was –53.1% (19 vs 40.5) shorter in words than Harris', though their means are closer. This difference between median and mean indicates that the distribution of section lengths is quite asymmetric — a few very long sections skew the mean. Half of Pence's delivery was made in sections shorter than 126 words and this is slightly higher at 136 for Harris.
Flesch-Kincaid Reading Ease and Grade Level
The Flesch-Kincaid reading ease and grade level metrics are designed to indicate how difficult a passage in English is to understand.
This metric does not take repetition into account. A grade level 10 sentence that is repeated 100 times still generates the same metrics because the words per sentence and syllables per word remain constant. To measure how many times a speaker repeats themselves, I use my Windbag Index, below.
Reading ease ranges from 100 (easiest) down to 0 (hardest) and can be interpreted as follows
| Very easy to read. Easily understood by an average 11-year-old student. |
| Easy to read. Conversational English for consumers. |
| Fairly easy to read. |
| Plain English. Easily understood by 13- to 15-year-old students. |
| Fairly difficult to read. |
| Difficult to read. |
| Very difficult to read. Best understood by college/university graduates. |
| Extremely difficult to read. Best understood by college/university graduates. |
The grade level corresponds roughly to a U.S. grade level. It has a minimum value of –3.4 and no upper bound.
Two sets of readability scores are calculated. One for the entire debate and one that only considers section with at least 9 words.
Table 2a
readability — entire debate
Flesch-Kincaid reading ease and grade level.
Table 2b
readability — excluding short sections
Flesch-Kincaid reading ease and grade level for sections with at least 9 words.
After Trump's 3.60 and Biden's 4.27 grade level deliveries in the first debate, it was good to see discourse level elevated. Pence's grade level was +12.0% (8.61 vs 7.69) higher than Harris'. Excluding short sections are excluded (I added this option to exclude interruptions from the grade level calculation), grade level increases by about 0.3 for both candidates but the difference of +10.4% (8.94 vs 8.1) remains roughly the same.
It's interesting to see why the grade level is different between Pence and Harris. Pence delivered slightly more sentences (+5.6% (379 vs 359)) and slightly more words (+4.3% (6,053 vs 5,806)) but quite a few more syllables (+10.4% (9,218 vs 8,351)). And so while Pence's word per sentence count was actually –1.2% (15.97 vs 16.17) lower than Harris, his syllables per word was +5.9% (1.52 vs 1.44) higher.
Sentence Size
Table 3
sentence size
Number of sentences spoken by each speaker and sentence word count statistics. Number of words in a sentence is shown by average and 50%/90% cumulative values for all, stop and non-stop words.
Sentence size was roughly equal with Harris using +16.7% (9.1 vs 7.8) more stop words in a sentence, on average.
Word Statistics
Debate Word Count
Summary Word Count
The summary word count reports the total number of words and the
number of unique, non-stop words
used by each candidate. Word number is expressed as both absolute and
relative values.
Table 4a
all words
Number of all words and unique words used by each speaker.
Table 4b
exclusive and shared words
Words exclusive to speaker (e.g. speaker A
but not speaker B) and shared by speakers (speaker A
and B).
Pence used +4.0% (6,090 vs 5,855) more words and had +27.4% (1,147 vs 900) more exclusive words.
Harris used relatively Δrel=+4.4% (Δabs=+2.8%, 65.9% vs 63.1%) more unique words.
Stop Word Contribution
In the table below, the candidates' delivery is partitioned into stop and non-stop words. Stop words (full list) are frequently-used bridging words (e.g. pronouns and conjunctions) whose meaning depends entirely on context. The fraction of words that are stop words is one measure of the complexity of speech.
Table 5a
non-stop words
Counts of stop and non-stop words.
Table 5b
exclusive and shared non-stop words
Non-stop words exclusive to speaker (e.g. speaker A
but not speaker B) and shared by speakers (speaker A
and B).
Harris used proportionately Δrel=+13.7% (Δabs=+6.7%, 55.6% vs 48.9%) more stop words than Pence.
Word frequency
The word frequency table summarizes the frequency with which words
were used. I show the average word frequency and the weighted
cumulative frequencies at 50 and 90 percentile. The average word
frequency indicates how many times, on average, a word is used. For a
given fraction of the entire delivery, the weighted cumulative
frequency indicates the largest word frequency within this fraction
(details about weighted
cumulative distribution).
Table 6a
word use frequency
Average and 50%/90% percentile word frequencies.
Table 6b
exclusive and shared non-stop word use frequency
Average and 50%/90% cumulative percentile word frequencies. Non-stop words exclusive to speaker (e.g. speaker A
but not speaker B) and shared by speakers (speaker A
and B).
Harris repeated her words +8.3% (5.2 vs 4.8) more often. If we consider non-stop words, then both candidates average word frequency was nearly identical. Pence's 90% word frequency percentile was quite a bit higher than Harris at +31.6% (50 vs 38).
All further word use statistics represent content that has been filtered for stop words, unless explicitly indicated.
Part of Speech Analysis
In this section, word frequency is broken down by their part of
speech (POS). The four POS groups examined are nouns, verbs,
adjectives and adverbs. Conjunctions and prepositions are not
considered. The first category (n+v+adj+adv) is composed of all four
POS groups.
Part of Speech Count
Table 7
part of speech count
Count of words categorized by part of speech (POS).
Both candidates used proportionately roughly the same fraction of nouns and adjectives. Proportionately, Harris used Δrel=+12.7% (Δabs=+3.2%, 28.4% vs 25.2%) more verbs and Pence had Δrel=+47.2% (Δabs=+1.7%, 5.3% vs 3.6%) more adverbs.
In terms of proportion of unique parts of speech, both candidates had roughly the same fraction of unique verbs and adjectives. However, Harris repeated her nouns less often and had Δrel=+8.3% (Δabs=+2.9%, 37.7% vs 34.8%) proportionately more unique nouns. She also had proportionately more unique adverbs compared to Pence Δrel=+53.4% (Δabs=+20.2%, 58% vs 37.8%). Thus, while Pence did use more nouns and adverbs in total, he tended to repeat them more than Harris.
Part of Speech Frequency
Table 8
part of speech frequency
Frequency of words categorized by part of speech (POS).
We see Pence's repetition of nouns and adverbs here quite starkly. His 50% percentile noun and adverb frequency was +75.0% (7 vs 4) and +200.0% (6 vs 2) higher, respectively, than Harris.
Part of Speech Pairing
Through word pairing, I extract concepts from the text. The number of unique word pairs is a function of sentence length and is one of the measures of complexity.
Table 9a
part of speech pairing — Mike Pence
Word pairs (total and unique) categorized by part of speech (POS)
Table 9b
part of speech pairing — Kamala Harris
Word pairs (total and unique) categorized by part of speech (POS)
Table 9c
unique part of speech pairing — candidate comparison
Unique word pairs categorized by part of speech (POS)
With the exception for verb/verb pairs, Pence had more pairs unique to him than Harris.
You can really get into the weeds here. Parts of speech are counted more granularly in these tables — nouns and verbs are split into classes and many other word types are shown, such as conjunctions and prepositions.
Table 10a
detailed POS tags — nouns and verbs
Count by part of speech tag:
NN (noun, singular),
NNP (proper noun, singular),
NNPS (proper noun, plural),
NNS (noun plural),
VB (verb, base form),
VBD (verb, past tense),
VBG (verb, gerund/present participle),
VBN (verb, past participle),
VBP (verb, sing. present, non-3d),
VBZ (verb, 3rd person sing. present)
Table 10b
detailed POS tags — adjectives, pronouns, adverbs and wh-words
Count by part of speech tag:
JJ (adjective),
JJR (adjective, comparative),
JJS (adjective, superlative),
PRP (personal pronoun),
PRP$ (possessive pronoun),
RB (adverb),
RBR (adverb, comparative),
RBS (adverb, superlative),
WDT (wh-determiner),
WP (wh-pronoun),
WP$ (possessive wh-pronoun),
WRB (wh-abverb)
Table 10c
detailed POS tags — prepositions, conjunctions, determiners and others
Count by part of speech tag:
CC (coordinating conjunction),
CD (cardinal digit),
DT (determiner),
EX (existential there),
FW (foreign word),
IN (preposition/subordinating conjunction),
MD (modal),
PDT (predeterminer),
POS (possessive ending),
RP (particle),
TO (to),
UH (interjection)
Proportionately Pence had more proper nouns (Δrel=+51.6% (Δabs=+3.8%, 11.22% vs 7.4%)) and Harris used Δrel=+19.7% (Δabs=+2.1%, 12.89% vs 10.77%) more prepositions and subbordinating conjunctions.
Exclusive and Shared Usage
This section enumerates words that were exclusive to a
candidate (e.g. used by one candidate but not the other). This content
provides insight into what the candidates' priorities are and
reveals differences in perspective on similar topics.
For a given part of speech, the table breaks down the number of
words that were spoken by only one of the candidates or both
candidates (intersection). The last row includes words spoken by
either candidate (union).
Table 11
exclusive word usage
Total and unique words used exclusively by a candidate, or by both.
Pence had more words exclusive to him for each of the noun, verb, adjective and adverb categories. He had a Δrel=+30.1% (Δabs=+4.6%, 19.9% vs 15.3%) higher fraction of exclusive words than Harris.
Pronoun Usage
This section explores pronoun use in detail. Refer to the methods section for details.
Pronoun Count
Fraction of all words that were pronouns.
Table 12a
pronoun fraction
Fraction of words that were pronouns.
Table 12b
exclusive and shared pronouns
Pronouns exclusive to speaker (e.g. speaker A
but not speaker B) and shared by speakers (speaker A
and B).
Pronoun by Person, Gender and Count
Pronoun usage by person (1st, 2nd, 3rd), gender (masculine, feminine, neuter) and count (singular, plural).
Table 13a
Pronoun by person
Count of pronouns by first, second or third person.
Table 13b
Pronoun by gender
Count of pronouns by masculine, feminine or neuter gender.
Table 13c
Pronoun by number
Count of pronouns by singular or plural.
In this debate, pronoun use was quite a lot lower than in the first Trump vs Biden debate. There, Trump used 1,681 pronouns, which is +90.4% (1,681 vs 883) more than Pence (nearly twice as many) and Biden used +35.6% (1,376 vs 1,015) more pronouns than Harris.
Harris used Δrel=+19.3% (Δabs=+2.8%, 17.3% vs 14.5%) proportionately more pronouns than Pence.
Pronoun usage by person was Δrel=+10.7% (Δabs=+4.8%, 49.5% vs 44.7%) higher for Pence for 1st person pronouns but Δrel=+26.9% (Δabs=+7.7%, 36.3% vs 28.6%) higher for Harris for 3rd person pronouns.
Looking at gender, Harris used Δrel=+43.1% (Δabs=+10.7%, 35.5% vs 24.8%) more masculine pronouns but Pence used Δrel=+89.1% (Δabs=+8.2%, 17.4% vs 9.2%) more feminine pronouns. This makes sense as this indicates that they referred to one another. The differences aren't symmetric because, presumably, there were more reasons why Harris would use a masculine pronoun and not refer to Pence, whereas there were fewer reasons why Pence would use a feminine pronoun but not be referring to Harris. Basically, there are fewer women to refer to.
Both candidates used singular and plural pronouns in roughly the same proportion.
First and third person pronouns — a closer look
These tables break pronouns by interesting contrasts. For example, the ratio of singular to plural 1st person pronouns reveals the use of "I/my/myself" vs. "we/our/ours".
Table 14a
1st person pronouns, by count
Count of singular and plural first person pronouns. This table contrasts use of I/my/myself vs. we/our/ours.
Table 14b
3rd person pronouns, by count
Count of singular and plural third person pronouns. This table contrasts he/she/his/her/it vs. they/them/theirs.
Table 14c
Me and you — 1st person singular and second person pronouns
Count of 1st person singular and second person pronouns. This table contrasts me/my/myself vs you/yours/yourself.
Table 14d
I, me, myself and my — closer look at 1st person singular pronouns
Count of specific 1st person singular pronouns: I, me, myself and my.
Both Pence and Harris, despite their different word counts, used nearly the same number of 1st person pronouns and their proportion of singular and plural forms was nearly identical. Both candidates used roughly 50% more 1st person plural than singular pronouns.
Harris did use +35.8% (216 vs 159) more 3rd person pronouns. Here, the use of plural is about half of the singular.
Both had mostly the same fraction of me vs. you and the same fraction of various forms of 1st person singular pronouns.
The results of this section are quite balanced and nothing like the first Trump vs. Biden debatea>, where the differences were stark.
Pronouns by Category
This table tallies the use of pronoun by category. The categories are personal, demonstrative, indefinite, object, possessive, interrogative, others, relative, reflexive. Note that some pronouns that belong to multiple categories are counted in only one. For a list of pronouns for each category, see the pronoun methods section.
Table 15
Pronouns by cateogry
Count of pronouns by category.
The biggest difference here is Harris' more frequent use of interrogative pronouns, which she used +165.1% (114 vs 43) more often than Pence. In fact, proportionately, Harris' had Δrel=+128.6% (Δabs=+6.3%, 11.2% vs 4.9%) more interrogative pronouns. She was asking more questions.
Noun Phrase Usage
Noun phrases were extracted from the text and analyzed for
frequency, word count, unique word count and richness. Single-word phrases were not counted.
Top-level noun phrases are those without a parent noun phrase (a
parent phrase is one that a similar, longer phrase). Derived noun
phrases are those with a parent (more
details about noun phrase analysis).
The top-level noun phrases can be interpreted as independent
concepts. Derived noun phrases can be interpreted as variants on
concepts embodied by the top-level phrases.
Noun Phrase Count and length
This table reports the absolute number of noun phrases, which is
related to the number of nouns, and their length.
Table 16a
noun phrase count
Counts of noun phrases in words and per noun.
Table 16b
noun phrase length
Average and 50%/90% cumulative length of noun phrases, in words.
Pence delivered +37.4% (547 vs 398) more noun phrases than Harris, which he repeated sligthly more often.
Exclusive and Shared Noun Phrase Count and length
Table 17a
exclusive and shared noun phrase count
Counts of exclusive and shared noun phrases in words and per noun.
Table 17b
exclusive and shared noun phrase length
Average and 50%/90% cumulative length of noun phrases, in words.
Pence had +29.5% (369 vs 285) more exclusive noun phrases.
Windbag Index
The Windbag Index is a compound measure that characterizes the complexity of speech. A low index is indicative of succinct speech with low degree of repetition and large number of independent concepts.
Unlike the Flesch-Kincaid readability metrics, the Windbag Index does not take into account the length of sentences or complexity (e.g. number of syllables) of individual words.
Table 18
windbag index
Windbag Index for each speaker. The higher the value, the more repetitive the speech.
Pence's Windbag Index is +107.9% (555 vs 267) larger than Harris'. To put things in perspective, Trump's Windbag Index from the first debate was 1,200 and Biden's was 539.
The reason why Pence's index is so much higher is his much smaller fraction of unique adverbs and unique noun phrases, despite his larger fraction of non-stop words.
Word Clouds
In the word clouds below, the size of the word is proportional to
the number of times it was used by a candidate (method details).
Not all words from a group used to draw the cloud fit in the image
— less frequently used words for large word groups may fall
outside the image.
All Words for Each Candidate
Each candidate's debate portion was extracted and frequencies were
compiled for each part of speech (noun, verb, adjective, adverb), with
words colored by their part of speech category.
The distribution of sizes within a tag cloud follows the frequency
distribution of words. However, word size cannot be compared between
clouds, since the minimum and maximum size of the words is fixed.
Debate Word Cloud for Mike Pence - all words
Size proportional to word frequency. Color encodes part of speech: noun verb adjective adverb
Debate Word Cloud for Kamala Harris - all words
Size proportional to word frequency. Color encodes part of speech: noun verb adjective adverb
Both candidates used "American" very frequently. Recall that Trump in the first debate never said that word.
Pence also said "president" proportionately more often than Harris.
Exclusive Words for Each Candidate
The clouds below show words used exlusively by a candidate. For
example, if candidate A used the word "invest" (any number of times),
but candidate B did not, then the word will appear in the exclusive word
tag cloud for candidate A.
Words exclusive to Mike Pence
Size proportional to word frequency. Color encodes part of speech: noun verb adjective adverb
Words exclusive to Kamala Harris
Size proportional to word frequency. Color encodes part of speech: noun verb adjective adverb
Pence's unique words had quite a lot of adjectives in the center of the word cloud, such as "liberal", "new", "average" and "strong". Pence tended to repeat his unique adjectives.
Harris, on the other hand, repeated her unique adverbs "certainly", "almost", "clearly" and "incredibly" followed by nouns.
The difference in these distributions suggests more substantive concepts in Harris' unique words.
Pronouns for Each Candidate
Word clouds based on only pronouns.
Pronouns for Mike Pence
Size proportional to word frequency. Color encodes pronoun type: masculine feminine neuter 1st person 2nd person singular plural other
Pronouns for Kamala Harris
Size proportional to word frequency. Color encodes pronoun type: masculine feminine neuter 1st person 2nd person singular plural other
Pronoun usage was very similar.
Part of Speech Word Clouds
In these clouds, words from each major part of speech were colored
based on whether they were exclusive to a candidate or shared by the
candidates.
The size of the word is relative to the frequency for the candidate
— word sizes between candidates should not be used to indicate
difference in absolute frequency.
Cloud of noun words, by speaker
Words unique to each candidate (Pence, Harris) and those spoken by both.
Just as for the first Trump vs Biden debate, we see nouns dominating the exclusive word list for the Democractic candidate. This suggests more of a focus on concepts and ideas.
Cloud of verb words, by speaker
Words unique to each candidate (Pence, Harris) and those spoken by both.
Unique verb use was more balanced. Pence wants to "continue" and we all remember when Kamala said "speaking".
Cloud of adjective words, by speaker
Words unique to each candidate (Pence, Harris) and those spoken by both.
Unique adjectives by Pence were repeated a lot more than those by Harris.
Cloud of adverb words, by speaker
Words unique to each candidate (Pence, Harris) and those spoken by both.
Unique adverb use is a flip of the unique adjective use.
Cloud of all words, by speaker
Words unique to each candidate (Pence, Harris) and those spoken by both.
The distribution of color in this cloud is nearly the same as from the first Trump vs. Biden debate. Most of the words in the center were from the Democratic candidate (these are exclusive and often repeated).
Word Pair Clouds for Each Candidate
Pairs used only once during the debate are not shown.
word pairs for Mike Pence
▲
JJ/JJ by Mike Pence
▲
JJ/RB by Mike Pence
▲
JJ/N by Mike Pence
▲
JJ/V by Mike Pence
▲
RB/RB by Mike Pence
▲
RB/N by Mike Pence
▲
RB/V by Mike Pence
▲
N/N by Mike Pence
▲
N/V by Mike Pence
▲
V/V by Mike Pence
word pairs for Kamala Harris
▲
JJ/JJ by Kamala Harris
▲
JJ/RB by Kamala Harris
▲
JJ/N by Kamala Harris
▲
JJ/V by Kamala Harris
▲
RB/RB by Kamala Harris
▲
RB/N by Kamala Harris
▲
RB/V by Kamala Harris
▲
N/N by Kamala Harris
▲
N/V by Kamala Harris
▲
V/V by Kamala Harris
Pence drives "unleashing american", "american people", "back regulation" and "people deserve".
Meanwhile, Harris also repeats "american people" but also "president said" and "people know".
Downloads
Debate transcript
Parsed word lists and word clouds (word lists, part of speech lists, noun phrases, sentences) (word clouds)
Raw data structure
Please see the methods section for details about these files.
