AI Vendor Due Diligence Checklist for CFOs and CTOs

Author: Derek Adams

Published:

AI software, tools, and platforms are everywhere, nowadays, taking over tasks that used to require significant human effort and time. With the right tech stack, even small guys can compete in the same league as the big brands without incurring any major cost increases. 

As a result, the competition is fiercer than ever. Who wants to stay relevant, must invest in and integrate AI into the company’s inner flow. There’s no other way around it. 

Yet, how do you find the right AI vendor in a market that’s becoming more saturated by the day? You do your due diligence, of course! Here’s a quick checklist for CFOs and CTOs to follow when deciding who to let manage their processes and data for the foreseeable future.

The AI Legal Experts

When hiring or buying software from an AI vendor, companies use a bundle of standard legal agreements. It’s just like when buying a car: besides the purchase agreement, you also sign a warranty, a financing contract, and proof of insurance.

In the case of AI, there are four main legal contracts you need your legal team to review before giving an AI vendor the green light:

  • Master Services Agreement: sets the ground rules for the entire relationship
  • Statement of Work: defines the project scope, user licenses, or API token limits
  • Service Level Agreement: outlines technical promises guaranteed by the vendor
  • Data Processing Addendum: governs what the vendor is allowed to do with the data

Each of these agreements, statements, and addendums are important as they set up the rules that will govern a company’s relationship with the vendor. Since AI tools and platforms are quite new, in terms of legal steps, it’s highly recommended to bring in a legal AI expert before signing anything. 

The good news? You don’t have to dig around too much to find the right team. Simply go online and browse experienced artificial intelligence attorneys in your area. You should find the right one(s) in just a few minutes of searching. 

Technical Architecture and Security

Generative AI can be involved in almost anything, from product development to documentation and promotion, but before you let an AI model work with your data, you need to make sure it’s safe.

During due diligence, evaluate the AI vendor’s technical architecture and security, ask for clear documentation of foundational models used, weight sources, fine-tuning methodologies, and dependency chains.

The scope here is to ensure the model is built safely, won’t break down under stress, and won’t leak your private information.

Model Governance and Operational Integrity

AI models act as assistants that work tirelessly in the background, automating various processes. For consistency’s sake (and company reputation), you need an AI that behaves predictably, gives trustworthy answers, and doesn’t go off the rails over time.

Good operational integrity means the vendor actively tests and measures the AI’s error rates using real-world scenarios. They should also connect the AI to verified, up-to-date business documents so the system grounds its answers in real facts rather than guessing.

The AI can also degrade over time, a phenomenon known as “model drift.” As real-world conditions, language, or business needs evolve, an AI’s performance can subtly decline if it isn’t monitored. 

Model governance means the vendor regularly monitors output quality, notifies you before rolling out major updates, and provides clear version control so an unexpected system update doesn’t suddenly disrupt your day-to-day operations.

AI Doesn’t Work Alone

Even though AI models feel like a stand-alone entity sometimes, there’s still a lot of tinkering going around in the background. This is why companies must choose an AI vendor with experience in monitoring and maintaining their tools. Also, complete transparency and good data organization are a good sign when looking for your next intelligent virtual assistant.