Assessing an AI provider and comparing up to three offerings

What the grid for assessing an AI provider contains

• A “How to use” tab: purpose, rating scale, eliminatory criteria rule, reading of the result, update frequency.
• A “Grid” tab: twenty criteria in six families, data (4), location (3), contract (4), security (4), use (3), cost (2).
• Location criteria: place of processing stated, processing possible in a State on the list of the National Commission for the Control of Personal Data Protection (CNDP), sub-processors published.
• A score from 0 to 3 per criterion and an adjustable weight from 1 to 3, selected from drop-down lists.
• Three eliminatory criteria marked “E”: no use of exchanges for training, place of processing stated, data processing agreement.
• Finally, three calculated results per offering: weighted score as a percentage, number of criteria completed, eliminatory alert.

When to use the grid

The grid is used before choosing an AI offering intended to process data of the organisation. It also makes it possible to assess an AI provider against two other offerings, for the same use. Moreover, provider terms change frequently; the assessment is therefore renewed at each change to the published or contractual terms. It is aimed in particular at organisations selecting an AI offering for personal or confidential data.

Instructions for use

First, the “Grid” tab receives, for each offering, its name, the source consulted and its date. Next, each criterion receives a score from 0, unfavourable or undocumented, to 3, highly satisfactory. Then the weightings in column C are adjusted, if necessary, according to the intended use. Finally, the reading focuses on the weighted score, the number of criteria completed and the eliminatory alert.

Expected outcome

Assessing an AI provider with the grid produces a weighted score per offering, expressed as a percentage. The score appears once all twenty criteria have been rated; an eliminatory criterion rated 0, however, triggers an alert. Finally, the number of criteria completed is displayed for each offering.

Key facts

Limitations

The grid does not designate a preferable provider: the score depends on the intended use and the weightings selected. In addition, each assessment is valid as of its date, since provider terms change frequently. A decision-support tool, to be adapted to the organisation and the intended use.

Further reading: Information Systems & Digital Transformation; List of States ensuring adequate protection of personal data (CNDP).