By Paul Shearer, Solution Architecture, VP

When the topic of generative AI arises in boardrooms, there's often a sense of enchantment. To many in the C-Suite, it feels like magic—a transformative technology that promises to revolutionize operations and more importantly, skyrocket their company’s valuation. Yet, the reality of deploying Gen-AI in an enterprise context is far more mundane. Is it best thought of as integrating flexible new microservices that often trade speed and accuracy than flexibility.  

For example, consider building a calculator using a large language model (LLM). It would be orders of magnitude more expensive and slower than the traditional approach. What do you get in return for using this shiny new toy? Expensive answers that are right “most of the time.” This is because, in most cases, LLMs mirror human performance, both our strengths and our weaknesses but at scale. Most humans suck at mental math. The LLM calculator would be an example of scaling “human-weakness”.

Starting with the Use Case and Using a Balanced Scorecard

When considering the deployment of generative AI in an enterprise setting, it's essential to start with a clear use case. What specific problem are you trying to solve? Understanding this is the first step to determining whether traditional approaches or generative AI is the right tool for the job.

Let’s consider a practical example such as automating document management. This involves handling and processing large volumes of varied documents. The traditional approach might involve rule-based systems based on data labeling, whereas the generative AI approach would leverage a multi-modal large language model (LLM) to classify and extract useful data from these documents returning the data in a structured format.

To evaluate the effectiveness of each approach, we can use a balanced scorecard, which considers three key factors: Speed, Accuracy, and Cost. These factors can be quantified and weighted according to their importance in the specific use case.

Matrix: Traditional vs. LLM Approach

Scoring Each Factor

Each factor is scored from 1 to 5, with 5 being the best possible score.

Weighting the Importance of Each Factor

Now, assign a weight to each factor based on its importance in your specific use case. These weights should total 1.

Calculating the Balanced Scorecard

Multiply the score for each factor by its weight to determine the weighted score for each approach.

From this balanced scorecard, we can see that the common traditional approaches might score the highest in this use case mainly due to their superior accuracy and balanced costs. LLMs, while offering substantial improvements in speed over current human-based methods, may not always justify their cost depending on the volume of documents processed.

Conclusion: Solve for the Use Case

Deploying generative AI within an enterprise context should be driven by a clear understanding of the use case and a methodical evaluation of the approach’s performance across key factors. Using a balanced scorecard helps in making an informed decision by comparing current methods, traditional approaches, and LLM-based solutions in a structured manner. This approach ensures that the adoption of generative AI delivers tangible benefits aligned with the enterprise’s strategic goals. In the end, generative AI is a powerful tool, but it's not magic. Approach it with the right mindset: as a valuable microservice within a well-engineered system, rather than a silver bullet.

Want to see your question answered in the series, or just want to subscribe for alerts on future issues? Simply fill out the form below!