My experience of strategic decision-making when implementing AI

By Toby Davidson, Chief Product and Technology Officer at Proactis

Implementing Artificial Intelligence (AI) and Machine Learning (ML) solutions involves key decisions. One of the more pertinent decisions is such as whether to utilise commercially available Large Language Models (LLMs) or build an in-house model. While hybrid models exist, this discussion will focus on these two primary approaches. There are a number of critical factors in this decision, including the specific goals of the initiative, the importance of AI/ML to its success, available skills and resources, projected growth and usage, and budget considerations.

So, considering these factors, when is a commercial LLM the correct route to take?

Time constraints: The immediate deployment capability of a commercial LLM can significantly speed up turnaround time. If you need quick results or want to conduct a proof of concept (POC) to assess AI/ML's value, a commercially available LLM can expedite the process. It supports a 'Fail Fast' approach to innovation by enabling rapid testing without the need for infrastructure setup or model tuning.
Resource limitations: Whether enhancing existing AI/ML projects or starting from scratch, leveraging AI and ML offers exponentially growing possibilities. However, resourcing for all future opportunities is impractical. A commercially available LLM can simplify this journey and reduce resource overhead. It can reduce your need for: Technical Expertise: If you lack expertise in machine learning and natural language processing, using a commercial LLM can reduce the technical load and bridge the gap until you can hire or train in-house resources. Computational Resources: While training LLMs requires substantial computational power, commercial LLMs are hosted on robust infrastructure, providing the ability to scale on demand. Maintenance and Updates: Commercial LLM providers usually handle updates, maintenance, and improvements, saving you both time and resources, and ultimately, money.
Scalability and cost management: When starting a new project, predicting growth is challenging. Few businesses achieve perfect growth projections, so minimising capital expenditure and cash flow impact is crucial to avoid risking poor customer experience. Leveraging a commercial LLM allows for a low-cost start, reducing exposure until the capability is proven and adoption accelerates.

Matching scale to cost can be challenging, especially for new initiatives without a proven market. Leveraging a commercial LLM mitigates risk by aligning costs with current demands and simplifies handling significant scale changes without managing infrastructure.

At Proactis, we've used commercial LLMs to accelerate several POCs and bring a solution to market about 12 months ahead of schedule compared to building an in-house document LLM. This approach allows us to fine-tune infrastructure capacity and scale model capabilities in line with product growth and customer adoption.

It seems so simple based on what has been discussed so far to look only at commercial LLMs as the go to, especially in the early stages. However, they are not always the perfect or viable option in every case.

So, when should you consider creating your own Document LLM?

If returning to the initial set of factors to consider when deciding which way to go, look at what you are ultimately trying to achieve.

Within Proactis, for example, we have a number of projects under way which look at leveraging knowledge that is part of our DNA and formulated over the last 20 years. It is part of the ‘secret sauce’ we use for serving our customers in a way that exceeds the competition.

At a previous company, we explored using AI and machine learning to enhance advanced pricing algorithms for initiatives such as new market entry and determining contract price breakpoints or negotiation thresholds for long-term agreements. These points aim to protect suppliers from substantial market fluctuations (e.g., a rise in raw material costs, such as nitrogen fertilisers post-pandemic) and ensure that customers benefit from price adjustments if production costs significantly decrease.

Neither of these use cases suit commercial LLMs, whether document-focused or MathLLMs. The requirements are too specific, and the outputs from available LLMs fall short of delivering the necessary benefits to support the commercial objectives.

Some of the key factors:

Specialised data requirements Both examples contain data and process which is highly specialised, therefore building a specific model and investing the time training is going to be required to maximise its potential. With examples like this which then are also key elements of a commercial strategy to achieve a goal, the time and investment required are a direct factor in the ultimate success of the initiative.
Customisation and control Whilst commercial LLMs can and do have capabilities to adapt or to some extent customise the process or output, these are by design more limited. This is often done because a commercial LLM is looking to attract a broad base for adoption, maximising the return the original provider has invested into the model. By building your own LLM you maintain complete control and have the flexibility to fine tune everything from the architecture, through the process and the training data you use. You can continue to adapt and optimise the model without the constraints around upgrade and release or designed limitations for factors such as performance, as you get with a commercial LLM.
Cost considerations: We used initial start up or set up cost as a Commercial LLM plus point, so how can it also be in this section? While the initial investment is higher when creating your own LLM, developing your own model can be more cost-effective in the long run. This is especially if you anticipate heavy usage and throughput, or you look to leverage a broad set of capabilities provided by a commercial subscription model. For example, if you were to take a basic level of service from a commercial LLM at one price per page for 1000 pages, to leverage custom elements in the same model, the price is approximately 4x the base price for each page for the same 1000 pages. It is a mix of volume and capability which drives the pricing with commercial LLMs, so if you run high volume or look to leverage a broad cross section of the capability provided, you may be better off with your own.
Data privacy and security: Managing your own model ensures that sensitive data is kept in-house, potentially reducing the risk of data breaches or misuse. In many cases the data processed by a commercial LLM is not kept and the information is removed either immediately or after a set period, however the data has been transported outside of your organisation. There is data held within organisations where the risk of a breach would have serious implications, or at best have significant impact on the company's reputation. The nature of the data you are looking to process is a key factor in the security approach to your decision.

Summary

There are times when it is a very simple decision as to whether to go with a commercial LLM or to self-build.

Use a Commercial LLM if you need a quick, reliable, and scalable solution without the need for deep customisation, and if you want to leverage existing expertise and infrastructure.

Create your own LLM if you require specific customisations, need full control over the model and data, want to invest in long-term savings, or have specific data governance requirements. You may also want to breed innovation within your company and have that as a factor which either set’s you apart or helps drive the valuation of your business.

In my last three companies we have used both approaches to help drive the overall strategy within the business, and within Proactis we have current initiatives which are following each of these paths. By carefully considering these factors, you can make an informed decision that aligns with your strategic goals and operational capabilities of your own business.

One final note to consider: One path may not fit for the entire life-cycle of a capability or product. Within Proactis we have a scenario where we have produced a POC and very fast initial market launch by leveraging a best-of-breed commercial document LLM. However, the expectation is that we may want to consider switching to our own model at a later point. As the volumes of data and the capabilities we look to provide in that area of our business grow, it is likely a combination of the cost profile and the ability to cater to our needs will drive a switch.

We identified this early in the solution design process and have taken an architectural approach where we can switch model and source relatively easily. We have given ourselves the ability to leverage the benefits brought by each approach, and therefore continue to drive the customer experience and value on an upwards trajectory.

Learn more about how Proactis is using AI