How to Choose an AI Vendor (Who Can Actually Deliver)
Companies of all sizes are actively exploring how emerging AI technologies can overcome longstanding business challenges. Inevitably, they run up against the reality that weathered AI pros like myself have long known: AI ain’t easy. Rather than going it alone, many businesses choose to partner with firms that specialize in building solutions with LLMs. The good news? There are a growing number of AI vendors to pick from, with more popping up all the time. The bad? Discerning if a vendor can deliver what you need isn’t always so straightforward.
It seems like everyone and their little cousin touts the ability to “wrap” custom applications around one of the big-name LLMs. If that’s all they bring to the table, they might help you address simple use cases, but probably won’t have the chops to build and manage complex solutions in heavily regulated industries like insurance. That’s a whole different thing.
So, how can you tell if a prospective vendor can meet your business's needs? In this blog post, I’ll explore some key areas along the AI value chain and propose some questions to ask so you can make an informed decision.
So, how can you tell if a prospective vendor can meet your business's needs? In this blog post, I’ll explore some key areas along the AI value chain and propose some questions to ask so you can make an informed decision.
Input preparation
What you put into your AI system is what you get out of it. Make sure a prospective vendor prioritizes clean data, stored & handled in a secure compliant manner.
You can think of data like a commodity that powered the previous century: oil. You don’t just dig some oil out of the ground and pour it into your gas tank. (Or, I guess you could, but you wouldn’t get far before your engines seized up.) Like oil, data requires multiple rounds of preparation before it can be used.
The value of the output your AI produces is directly related to the quality of the input. Before moving forward with any prospective vendor, ensure they have the means—and indeed, the knowledge—to help you build compliant, secure data workflows from beginning to end.
The value of the output your AI produces is directly related to the quality of the input. Before moving forward with any prospective vendor, ensure they have the means—and indeed, the knowledge—to help you build compliant, secure data workflows from beginning to end.
Here are some points to consider to ensure this is a vendor for you.
Questions to consider:
- How will the data be collected?
Data must be carefully collected to protect privacy and prevent bias. Ensuring that data has been ethically obtained and correctly governed is a point of emphasis for regulators. - How will the data be “cleaned”?
Data needs to be refined and structured in a way that an AI solution can use and interpret. Make sure a prospective vendor understands what types of data are appropriate for your use case and how to prepare it at scale. - How will the data be transferred, stored, and secured?
When developing solutions for complex, highly regulated industries, proof of certification for things like SOC2 and HIPAA are table stakes. Additionally, you’ll want to verify that the vendor uses secure data transfer methods, such as encryption during transit and at rest, to prevent unauthorized access. Also, ensure they effectively track the status of the data over time via robust version control and data lineage systems.
Prompt development
LLMs work best when you make it difficult for them to make mistakes. An AI vendor should understand how to craft prompts to generate business value.
For an AI solution to generate value, it must surface useful information with as little human intermediation as possible. This is achieved by ensuring that every prompt to an LLM includes all guidance, data access, and guardrails necessary to generate a high-quality return. Things like:
1. Industry-specific content to guide results
2. Phrasing that reflects informed insight into the domain
3. Precise instructions on the structure of the result being sought
Your vendor will need to demonstrate they understand the capabilities and limitations of AI and can provide insights on how to structure LLM conversations to extract maximum value. Here are some points to review with a prospective partner to ensure they have the means—and better yet, a history—of value-oriented prompt engineering.
Questions to consider:
- How do they build prompts, and what domain-specific knowledge do they have?
Technical acumen is one thing, but does the vendor understand the specific needs of your industry? It’s one thing to ask an LLM to plan out a fun afternoon at the beach, it’s another thing to have it understand if, for example, family-owned restaurants align with a home insurer's risk appetite, or not. - What methods are used to select material included in the context window?
You should understand the vendor’s criteria for selecting contextually relevant information and how they ensure this information is timely and accurate. Ask what processes they use to filter and prioritize the most pertinent data for inclusion in prompts. - How often, and in what ways are prompts updated over time? Are these changes tracked?
Learn about their schedule for reviewing and updating prompts to keep them aligned with the latest industry trends and data. Ensure they have a system for tracking changes to prompts, including version history and impact analysis, to maintain transparency and continuous improvement. - What methods are used to evaluate the results of prompts, and to compare the results to prior versions when changes are made?
Ask about their evaluation metrics and benchmarks for assessing prompt performance, including accuracy, relevance, and consistency. Understand their process for A/B testing new prompt versions and how they compare the results to previous versions to ensure improvements.
Output control
Non-deterministic AI systems act in unpredictable ways. A quality vendor should know how to measure misaligned behaviors, as well as how to address them.
The value of the output your AI produces is directly related to the quality of the input. Before moving forward with any prospective vendor, ensure they have the means—and indeed, the knowledge—to help you build compliant, secure data workflows from beginning to end. Ask an LLM the same question 10 times and you might get 10 different responses. The goal is to generate 10 accurate, useful answers. Achieving this requires putting as much care into reviewing the system’s output as you do into preparing the input.
Ask an LLM the same question 10 times and you might get 10 different responses. The goal is to generate 10 accurate, useful answers. Achieving this requires putting as much care into reviewing the system’s output as you do into preparing the input.
Continuous monitoring and tweaking are necessary to adapt your system to accommodate new data and evolving requirements. Here are some questions to explore when evaluating a vendor’s approach to scaled output control.
Questions to consider:
- What evals will you run?
Inquire about their evaluation frameworks, including both automated and manual assessments, to ensure outputs meet quality standards. Learn about the specific metrics they use to evaluate outputs, such as precision, recall, and F1 score, as well as checks for hallucinations and biases. - What role will human experts play in this process?
Verify that human subject matter experts are involved in reviewing and validating AI outputs to ensure they are contextually appropriate and accurate. Ask about their process for incorporating expert feedback into continuous improvement cycles for the AI system. - How often will you review overall results, and what metrics will you use to guide refinement and improvement?
Get a handle on their schedule for regular reviews and audits of AI outputs to ensure ongoing quality and relevance. Inquire about the key performance indicators (KPIs) and metrics they use to monitor and refine the AI system, such as user satisfaction scores, error rates, and feedback loops.
Transparency
Not only does visibility allow you to properly evaluate an AI’s performance, it’s increasingly required by regulators as a means to address system bias.
Transparency is crucial for every step from data preparation to prompt development and output review. You cannot evaluate what you cannot see. To maintain the highest possible standards, every AI vendor should be prepared to provide a window into every step under their control.
Questions to consider:
- Can you provide clear documentation of your processes and methods?
Ensure that the vendor offers comprehensive and understandable documentation covering all aspects of their AI processes, from data collection to output generation. Ask for examples of their documentation to assess its clarity and completeness. - Can you demonstrate every point at which they interact with an LLM, and provide a complete trail of what information was exchanged?
Verify that the vendor maintains detailed logs and records of interactions with the LLM, including data inputs, prompts, and outputs. Ensure they can provide audit trails that detail the flow of information through their systems, which is crucial for regulatory compliance and troubleshooting. - Will you provide a routine report about their evaluations and measuring for potential bias?
Inquire about their regular reporting practices, including how often they produce reports on AI performance, bias detection, and mitigation efforts. Ask to see examples of these reports to evaluate their thoroughness and transparency in addressing potential biases and other issues.
At Sixfold, we’ve created a Responsible AI framework for prospects and customers to showcase our ongoing transparency work.
In Summary
AI has the potential to overcome challenges that have been holding businesses back for decades. If you haven’t started your AI journey, now’s the time to get started. Partnering with an AI vendor can help you identify use cases ripe for transformation and provide the skills to get you there.
I hope that this checklist helped you identify which vendor has the right combination of technical know-how, industry expertise, and regulatory awareness to get your business where it needs to be.
