Understanding the sweet spots for LLM use cases across all domains…
Made famous by the remarks of Donald Rumsfeld in 2002, the quad chart of below can provide valuable insight to the application of LLMs within any domain. There are two continual issues with LLM utilization that both functionally and technically are unavoidable.
The first is the stochastic nature of the models causing an avoidable level of possible variation in even the most well structured prompts, in addition to the models inability to innately say (without guard rails at least) some version of “I don’t know”. Unavoidable possible variation in response along with a functional requirement for a response, leads LLMs to not be “ideal” for simple fact or file based information retrieval. Databases, knowledge graphs, etc are all far better suited for these straight forward information retrieval tasks. This type of issue is represented by the Known-Knowns in the chart.
The second issue is the inability for LLMs to truely create “new” information. Currently there is no research, engineering, or other demonstration that indicates LLMs have any inate ability to extrapolate or create completely “new” information. Driven by multiple factors both technical and functional, neither the design nor the data lends to this “creative” ability to likely even become emergent in the future according to most experts. Not to say different technical designs might not lead to this capability in the future, but that current LLM technology simply doesn’t posses it today. This type of issue is represented by the Unknown-Unknowns in the chart.
So what does all this indicate LLMs would be best used for? The chart provides the answers. The Known-Unknowns being “things we are aware of but don’t understand”, is a perfect example of data or context translation. This would be, for example, LLMs converting between languages to included programming. This is why we see the best quality linguistic translation capability to date coming from LLMs, as well as LLMs providing quantum leaps in efficiency of write code in almost any program language. The Unknown-Knowns being “things we understand but are not aware of” represents all the various information that we have yet to uncover for ourselves individually or as a group. This would be, for example, LLMs researching/analyzing/extrapolating information either it was previously trained on or provided to it via a prompt. This is why we see so many people using LLMs to analyze, distill, and/or expand content via prompts into any number of highly variable subjective forms from homework assignments to social media content.
Ultimately there are some hard limits to LPM technology, and exploring those limits is both important and a worthwhile task for developers and engineers. For everyone else just looking to use the technology for maximum effect or simple to get the best “bang for their buck”…stick to the these two sweet spots for best success.
