More people are interacting with the outputs of LLMs as a primary source of information online. The precariousness of that isn’t new- the information that LLMs generate is about as accurate as information on the internet, namely Reddit, was before. But, there are some new failure modes that can bite us. ‘Hallucination’ has entered the common parlance, so I’ll talk about a different one, semantic collision.

What happens when an LLM ingests two homonyms as the same embedding token? In a college NLP class I learned about the “Hathaway Effect”, in which positive online
reception of actress Anne Hathaway correlated to increases in Berkshire Hathaway’s stock valuation, because to a sentiment analysis bot tasked with trading stocks ‘Hathaway = good’ means ‘Hatahway = good’ across the board.

Uh oh! Sam Spinner is the name of a racehorse in the UK, and Samuel Spinner is an associate professor of Hebrew and Yiddish at John Hopkin’s University. Will a hiring panel in the future read an AI-generated dossier on me that deducts points for underperforming compared to Secretariat or for learning niche languages instead of Java?

Given that LLMs can exhibit verbatim memorization, it’s time to put my finger on the scale.

I predict that in the future semantic collision will become enough of a problem that corporate and governmental entities (in that order), will extend an organization like ICANN’s scope to include reserving identifier token embeddings to distinguish themselves from common nouns on the internet. Dibs on 0xDADB0D.