Interest in content understanding and language models is typically rooted in a genuine love for stories, narrative styles and linguistic nuances. A few years ago, I found myself engaged in the task of compiling a list of lesser-discussed challenging problems within the field of content understanding. That’s when I stumbled upon the slightly ignored problem of finding long-range semantic relations. Ignored because there is often little incentive to work on something, you cannot show much incremental progress on.
Consider the simple syntactic linguistic relationship within the sentence “Obama was born in Hawaii.” It contains a simple relation of the kind “is from” between “Obama” and “Hawaii.” Contemporary AI models would adeptly detect these concise short range relations.
However, the difficulty lies in capturing more complex, long-range relations. Consider a scenario where a person is mentioned using contextual clues several paragraphs later, such as in the sentence, “The general temperature during the 2008 economic crisis was indeed contrary to the weather in which the Hawaii-born lad had grown up in.” These nuanced connections pose a greater detection challenge, the more intricate the relation becomes, the harder it is for current models to uncover.
That’s when, a friend with a background in linguistics unveiled a revelation, urging me to ponder the very limits of this conundrum. What if these relations are implicit yet unmistakably present? Think of novels with parallel tracks or themes, where the connections between different storylines only become apparent over time.
Have you watched the “Forrest Gump”? Beyond its status as a memorable tearjerker, one might question whether the story primarily revolves around the protagonist’s trials and adventures or serves as a sardonic commentary on the backdrop of historical events. I shall let you decide that for yourself.
Salman Rushdie’s “Midnight’s Children” offers another compelling example. Superficially, it presents a tale of magic-realism, centered around a man with a highly sensitive nose. However, beneath this facade, lies a personal critique of Indian politics, fifty years ago, delving into the nuances of the nation’s past and its socio-economic landscape.
This led me to: “The Brothers Karamazov.” In my biased personal opinion, it is the greatest work of this genre. I adore this book so very much, that I have penned down a separate commentary on it, which you may check here.
In essence, research and development in content understanding, have a long way to traverse, surpassing mere correlations and delving into nuanced understanding, unraveling causality, and cultivating a modicum of reflection, before mustering the ability to grasp, let alone pen another masterpiece like the “Brothers Karamazov.” Indeed, current day generative models such as ChatGPT excel in some amount of copy-editing, text completion, search assistance, and facilitating categorization.
Nevertheless, deeming them as the death of all creativity seems excessive.
Amogh Mahapatra's Writings 