Alright, let’s be honest about the current state of generative models. Right now they’re being discussed endlessly as a cure-all when there are many problems that LLMs can’t speak to. But that doesn’t stop vested interests from trying. It’s like the story of someone who only has a hammer in their toolbox – suddenly, every problem looks like a nail. LLMs are the new hammers.

• Need to write a poem? Hammer.
• Need a synonym for the word ‘hammer’? Hammer.
• Predicting the stock market? Big, fancy, digital hammer.

It’s like using a sledgehammer to swat a fly. Sure, it might get the job done, but it’s overkill. And let’s be honest, many times it doesn’t even come close to hitting the fly!

Yes this image was AI-generated. Irony is also something LLMs can’t recognize yet.

Is RAG All You Need?

Retrieval Augmented Generation attempts to fix some of the limitations of transformer models. In the right situations it can be pivotal. Here’s how it works: RAG combines the generative capabilities of LLMs with a retrieval system. Initially, when a query is made, the retrieval system fetches relevant information from a database, document collection or search engine/knowledge graph. That response is then provided to the LLM as additional context.

Throughout this process, the model may go back and forth between generating text and seeking additional information to refine its output further. At this point it’s really hard miss the metaphor

“Hey, we’ve got a sledgehammer here that’s too clunky to crack a nut, so let’s turn it into a jackhammer!”

This isn’t just about wait/lag times for a model to iterate on ever-growing lists of information. It’s also about the “sticker shock” that occurs after all of that ‘work’ is completed using a commerical model. The final tally on the bill can be – to borrow a phrase – eye-watering.

Every tool has its use – and we at Rower have worked with RAG-extended models. The key is to ensure that it’s utilized in situations to which it’s uniquely suited.

From RAGs To Riches

Depending on the specific situation, such as the need for precision, structured analysis, or handling specialized data types, other technologies can be dramatically more efficient and effective.

• Temporal Graph Neural Networks: You use this technology any time you use Google Maps. TGNNs allow for integration of loosely associated, time-sensitive data, offering a powerful tool for real-time context updates. They capture dynamic relationships within data over time.
• Support Vector Machines (SVMs): Highly effective for classification problems, especially narrow datasets. They are used in a range of cases from bioinformatics to credit scoring.
• Linear Programming: LP finds the best outcome (such as maximum profit or lowest cost) in a given model whose requirements are represented by linear relationships. This is ideal for optimizing resource allocation, scheduling, and manufacturing processes.
• Mathematical Planning: MP is often used in logistics and supply chain management for optimizing routes and inventory levels. Unlike LP, which is confined to linear objective functions and constraints, mathematical planning can handle more complex scenarios involving multiple objectives, varying constraints over time, and uncertainty in data.
• Data Lakehousing: Effective for business intelligence and analytics, offering structured storage and complex query capabilities. There’s a reason why this is often the starting point for many company’s ‘data estate’.
• Database Management Systems (DBMS): Don’t skip past it. Traditional SQL databases can get a targeted job done quickly and easily. No joking.

Anyone who has been working in data science and machine learning for the past few decades owes a debt of gratitude to those who have brought generative models into the mainstream. They have inspired everyone to reconsider the role of intelligent systems in their work and personal lives, and it’s a future that many in “the data community” have been envisioning for quite some time. As the work in this area continues, it’s worth noting the broad variety of highly effective technical solutions that companies can draw on today to solve a wide range of complex problems with high efficacy and high efficiency.

The Best of All Worlds

As aluded to above, when blending these technologies together in a focused strategy, particularly with a smaller locally deployed language model, the result can be surprisingly good. Orchestration tools like LangChain and Semantic Kernel are designed to leverage language models as part of a larger result, and their effectiveness can be greatly amplified when working with a constellation of efficient tooling. Here’s how these integrations could work:

• Time Series Analysis: Integrate time series analysis tools for forecasting market trends and analyzing historical data. Tools like LangChain and Semantic Kernel can use this data to provide context to the targeted language model which can return a summary of high-confidence estimations.
• Machine Learning Techniques: An orchestrator can utilize machine learning models like SVMs for classification to derive market segmentation or trace customer behavior patterns. The result of those models can be used to inform the language model, enabling the response from the transformer to be more targeted and accurate.
• Real-Time Data Integration: By integrating real-time data sources into the context window, systems can ensure that the information being processed and generated is up-to-date, surpassing the limitations of the original training data of the model.

The key is to identify the right tool for the job. Generative models may eventually surpass all of this specialized tooling in accuracy and efficiency. But until that time the ground truth is that businesses can’t run on hype alone. It’s today’s reality that large language models are just one piece of a broader, more complex solution mosaic.

Regardless of whether you’re on the LLM hype train don’t use a sledgehammer to swat a fly.

Begin Your Journey With Rower Consulting

Here at Rower Consulting, we equip our clients with the tools they need to right-size the solution to their business challenges. If you’re ready to continue the conversation, contact us to learn more about how we can help you move your business forward in the current technology landscape!