A balanced take of Generative AI
and why it's all about language
I recently read this excellent blog post by George Colony: The GenAI Imperative , that clearly underscores three pivotal facets of generative AI: End of the Web, death of Google, the renewed importance of trust.
Although I agree with these observations, I think he is missing the potential of LLMs and embeddings and to the existing list of three key points, I propose adding a fourth: The Power of Document Intelligence.
LLMs and embeddings will unlock an incredible amount of insights (granular insights), that have until now been impossible to discover and utilize at scale.
Embeddings
Let’s look at what embeddings are in more details. In NLP ( natural language processing), embeddings are mathematical representations of words, phrases, or even entire documents in a continuous vector space. This means that each word or document is transformed into a multi-dimensional point, where semantically similar words or documents are located closer to each other in this space.
These words can even be combined and you may build equations like: Queen+ Man - Women = King, or Paris+Spain = Madrid.
from: https://medium.com/@st3llasia/graph-embedding-techniques-7d5386c88c5
But starting with the blog post:
George Colony’s blog post
What I really liked about this article isn't just the elements it emphasizes but also the aspects it omits.
First and foremost, Colony refrains from making grandiose claims about Large Language Models (LLMs) being sentient or getting close to Artificial General Intelligence (AGI) – an intelligence that rivals or surpasses human cognition.
This contrasts sharply with the stories that companies like Microsoft have been promoting: “we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system”
Moreover, Colony doesn't scare us, cautioning against AI investments due to AGI risks and the end of the world.
Also he does not predict the end of labor and the massive layoffs like Goldman Sachs, predicting 300M jobs lost or degraded!
Instead, Colony offers a grounded perspective on the capabilities and implications of LLMs and Generative AI:
A Language-Rich Interface: LLMs usher in an era where individuals can engage in dynamic conversations with documents.
Redefining Search: The traditional search paradigm is evolving, and LLMs are at the forefront of this transformation.
The Imperative of Trust: With the advent of "context injection" in LLMs, the source and traceability of content are becoming critical.
Getting more intelligence from text
While Colony's insights are invaluable, I believe there's a fourth dimension he hasn't touched upon.
The essence of LLMs is intrinsically textual and for me the #1 benefit of LLMs (and embeddings) is to overlay text with a representation that enables connection, abstraction and manipulation at semantic level.
The Power of document intelligence: LLMs, with their ability to process fine-grain embeddings, will revolutionize business intelligence. They will empower us to extract knowledge from documents at a semantic level and learn from combining documents, comparing documents, predicting based on meta data …
To illustrate, here are a few examples:
Retail
Imagine a retail store enriching its customer database with insights from consumer locations.
By aggregating articles about various cities, the retailer can amass a huge amount of metadata about these cities.
This could range from conventional metrics like population density, average age of the population, average wealth … to nuanced attributes like, who knows: the city's proximity to a river, insights about the weather, the presence of a library, a swimming pool, sentiment about the housing market, local preferences from a specific category or even the prevailing political views.
Using LLMs, such knowledge can seamlessly integrate with existing business intelligence frameworks, offering unprecedented insights.
Maybe the retailer can correlate its books sale with these parameters and make rich/granular predictions.
HR
The vast majority of data handled in HR is textual.
For instance, consider the hiring process in HR.
Traditionally, HR matched keywords, potentially overlooking very good candidates with synonymous but not identical skills or job titles.
Embeddings in HR are game-changers, linking resumes, job postings, skills, and company values with unprecedented precision. They identify subtle skill correlations, bridging keyword gaps and making sure no promising candidate is missed.
Furthermore, embeddings could help evaluate cultural alignment by encoding company values into vectors and measuring the similarity between a candidate's values and those of the company.
Scope can also be extended to career plans, evaluations, organizational descriptions, company culture, company policies, labor law….
This transformative capability of embeddings empowers us to break down the silos between seemingly unrelated documents, thereby enhancing our ability to draw meaningful insights, make data-driven decisions, and harness the full potential of document intelligence. In essence, embeddings are the bridge that enables us to unlock a vast quantity of knowledge that was previously hidden by the limitations of traditional data analysis methods.
Personal Data & Social Data
Aggregating data about people - while respecting privacy laws- will also be possible at scale, starting with public social data that we all create on a daily basis.
To illustrate, I asked a question to ChatGPT: 'Given the profiles of people I follow on X/Twitter (for which I provided a few pages of data), what insights can you offer about me?'
Here's what it responded with:
It is clearly not perfect… but I doubt any retailer or recruiter I visit would have such fine grain insights.
The potential applications are vast.
In retail this translate to enhanced customer intelligence and personalized offering.
In healthcare, it could mean correlating specific lifestyles with certain diseases, leading to preventive recommendations.
In human resources, envision leveraging such insights during the recruitment process
Conclusion
Although George Colony’s insights are great and realistic, I think we are at a beginning of new era for data intelligence. The future of AI and the future of business, transformed by AI will come from harnessing the power of documents and texts.
This will enable businesses to gain deeper insights/intelligence and to refine their product and services with this newly uncovered intelligence.
I often compare GPT to a calculator or a spreadsheet for “text and ideas”. The introduction of LLMs will be as revolutionary as the invention of the calculator, reshaping our understanding and use of (unstructured) information.