Artificial Intelligence

Criteria’s Guiding Principles for AI 

Criteria’s goal is to help our customers achieve powerful results when it comes to hiring and developing their talent, while providing a positive and fair experience for every job candidate. AI offers extremely effective ways to help organizations achieve these results. 

At Criteria, we are thoughtful, transparent, and ethical in our use of AI. This means that we don’t regard AI as the best solution to every problem or as the right tool for every client. We take the time to determine how AI can advance our goal of helping our customers achieve talent outcomes, as well as the risks that AI might present. We implement solutions that use AI where it’s appropriate to do so, and we ensure that we have guardrails in place to minimize and control any risks.  

We follow these core guiding principles for ethical use of AI in hiring:

1. Criteria’s AI is backed by our stringent scientific standards. 

Ever since our founding, Criteria has been guided by rigorous scientific standards. These standards, developed by our experts in Industrial / Organizational Psychology, apply to our AI-based products just as they have always applied to our traditional products.  

Our standards ensure that we take an evidence-based view of the risks and benefits associated with any specific application of AI. They ensure that we evaluate each product in terms of its reliability, validity, and fairness. And they ensure that we only release a product after thorough testing and documentation.  

2. We ensure our AI systems are trained on relevant data. 

AI systems can only provide output based on the information that’s provided to them, and the quality of that information is key to the AI’s performance. It’s a simple case of “garbage in, garbage out.” If you feed an AI system irrelevant or biased inputs, you’ll get inaccurate and biased outputs.  

At Criteria, we tightly control the data that gets passed to our AI products. When we train our AI models, we leverage our unique data asset, encompassing job-relevant data on over 10 million candidates each year, across more than 100 countries, in over 600 occupational categories. The size, scope, and diversity of this data ensures that we train our AI on large, representative, and relevant datasets.  

3. We validate our AI solutions to ensure they do what we say they do. 

Validation at Criteria is an ongoing process in which we collect evidence to evaluate whether our products do what we say they do and deliver the outcomes we say they deliver. We undertake extensive research studies, often involving tens of thousands of individuals, before we launch a product. And we work with our clients and partners to ensure that the output produced by our products aligns with the results that clients are experiencing.

For products associated with evaluating job candidates, our validation efforts follow the criteria outlined in the EEOC’s Uniform Guidelines on Employee Selection Procedures, the Society for Industrial and Organizational Psychology’s Principles for the Validation and Use of Personnel Selection Procedures, and the American Psychological Society’s Standards for Educational and Psychological Testing.  

4. We test for and mitigate biased AI output. 

Testing for biased output is a standard component of the evaluations that we’ve always conducted across all our products since we were founded. When we find that the inputs to our AI are producing biased outcomes, we remove, modify, or mitigate the impact of those inputs.  

We embed diversity into our AI testing and evaluation process by engaging large, multinational, demographically diverse groups of candidates and clients in the testing process. We seek feedback from those who take part in our testing program to help us identify potentially biased content.  

In addition to our own internal testing, we also subject our systems to independent third-party audits for bias and disparate impact. The results of our third-party bias audits are summarized on our public website and are available to our clients to review.  

5. We build human oversight and monitoring into our AI solutions. 

AI works best when it supports and enables human decision making, as opposed to replacing it. At Criteria, we ensure that human insight and oversight remain at the core of our AI throughout the development process.  

When we build an AI product, every step of the process is scrutinized and evaluated by our product experts. We obsess over every detail: the questions we write for our assessments, the contextual information we expose to our chatbots, and the guardrails we build into the products.  

We make sure that our clients have the tools they need to manage and monitor our AI products once they start using them. Clients can choose when and how they use our AI, including customizing the product to meet their unique needs.  

We monitor and evaluate the products ourselves once they are released, through our own research studies, user feedback, and detailed logging. The data we get through these processes means we’re always aware of how our products are performing, and ensures that we can update and enhance our products as required.  

6. We are transparent in our use of AI. 

We are transparent with clients about which of our products are powered by AI, and we provide accessible explanations of how these products function. This enables clients to make informed decisions about when and how to use AI-based solutions.  

7. We take the privacy and security of our AI solutions seriously. 

We’re committed to safeguarding the privacy and security of our candidate and client data, and we adhere to industry leading standards in data protection and cyber security.  

Our Criteria AI core is a centrally managed AI hub where we locate the models that power our AI-based products. Our AI core gives us fine-grained control over the data that is passed to our AI and strictly limits the AI to the minimum amount of information necessary to do its job.  

We are also ISO-27001 certified, ensuring safe and secure data storage and processing. We are GDPR compliant, and we comply with privacy regulations in other jurisdictions, such as California’s CPRA.  

8. We keep up to date with AI regulations, legislation, and best practice guidelines. 

The regulatory landscape for AI is constantly evolving. We track AI legislation and regulations at the national and sub-national level. Our constant monitoring ensures that we can comply with relevant legislation and remain a trusted partner to our clients.  

Beyond simply focusing on legislation compliance, we also benchmark our AI development, testing, and monitoring processes against best-practice standards, including those published by the National Institute of Standards and Technology (NIST), the Institute of Electrical and Electronics Engineers (IEEE), the International Standards Organization (ISO), and the EU’s Expert Group on AI.    

9. We deploy our AI to deliver reliably and at scale. 

Criteria's technology infrastructure is designed to deliver robust, scalable, and reliable AI solutions. Our state-of-the-art AI Core ensures that we can manage, deploy, and scale AI models efficiently, maintaining high performance and uptime guaranteed by our rigorous Service Level Agreements (SLAs). We prioritize staying ahead of industry developments, integrating new technologies rapidly to provide our customers with solutions allowing them to confidently rely on our AI-driven applications to meet their evolving needs.