30 Years of Perplexity

Computational linguists have been trying for decades to accurately predict the next word in running text. Why were they so determined to succeed at this strange task? How did they track their successes (and failures)? Why was the word-prediction task at the center of early statistical work in text compression, code-breaking, machine translation, and speech recognition? Will word prediction lead to artificial general intelligence (AGI)? I’ll attempt to answer these questions with formulas and anecdotes drawn from three decades of research in various fields.

by Kevin Knight

Dr. Kevin Knight served on the faculty of the University of Southern California (26 years), as Chief Scientist at Language Weaver, Inc. (9 years), and as Chief Scientist for Natural Language Processing at Didi Global (4 years). He received a PhD in computer science from Carnegie Mellon University and a bachelor’s degree from Harvard University. Dr. Knight’s research interests include machine translation, natural language generation, automata theory, decipherment of historical documents, and number theory. He has co-authored over 150 research papers on natural language processing, as well as the widely adopted textbook “Artificial Intelligence” (McGraw-Hill). Dr. Knight served as President of the Association for Computational Linguistics (ACL) in 2011, as General Chair for ACL in 2005, as General Chair for the North American ACL (NAACL) in 2016, and as co-program chair for the inaugural Asia-Pacific ACL (2020). He received an Outstanding Paper Award at NAACL 2018, and Test-of-Time awards at ACL 2022 and ACL 2023. He is a Fellow of the ACL and a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI).

Back to program