Book an exam
Menu

ISTQB Certified Tester · CT-AI

AI Testing

  • Questions40
  • Duration60 min
  • Pass mark65%
  • PrerequisiteFoundation Level

Overview

The ISTQB® AI Testing (CT-AI) certification extends understanding of artificial intelligence and/or deep (machine) learning, most specifically testing AI-based systems and using AI in testing.

Audience

The Certified Tester AI Testing certification is aimed at anyone involved in testing AI-based systems and/or AI for testing. This includes people in roles such as testers, test analysts, data analysts, test engineers, test consultants, test managers, user acceptance testers, and software developers. This certification is also appropriate for anyone who wants a basic understanding of testing AI-based systems and/or AI for testing, such as project managers, quality managers, software development managers, business analysts, operations team members, IT directors, and management consultants.

To gain this certification, candidates must hold the Certified Tester Foundation Level certificate.

Individuals who hold the ISTQB® Certified Tester- AI Testing certification should be able to accomplish the following business outcomes:

  • Understand the current state and expected trends of AI;
  • Experience the implementation and testing of a ML model and recognize where testers can best influence its quality;
  • Understand the challenges associated with testing AI-Based systems, such as their self-learning capabilities, bias, ethics, complexity, non-determinism, transparency and explainability;
  • Contribute to the test strategy for an AI-Based system;
  • Design and execute test cases for AI-based systems;
  • Recognize the special requirements for the test infrastructure to support the testing of AI-based systems
  • Understand how AI can be used to support software testing

ISTQB Specialist Level AI Testing Content

  1. Introduction to AIDefinition of AI and AI Effect
    Narrow, General and Super AI
    AI-Based and Conventional Systems
    AI Technologies
    AI Development Frameworks
    Hardware for AI-Based Systems
    AI as a Service (AIaaS)
    Pre-Trained Models
    Standards, Regulations and AI
  2. Quality Characteristics for AI-Based SystemsFlexibility and Adaptability
    Autonomy
    Evolution
    Bias
    Ethics
    Side Effects and Reward Hacking
    Transparency, Interpretability and Explainability
    Safety and AI
  3. Machine Learning (ML)Forms of ML
    ML Workflow
    Selecting a Form of ML
    Factors involved in ML Algorithm Selection
    Overfitting and Underfitting
  4. ML - DataData Preparation as part of the ML Workflow
    Training, Validation and Test Datasets in the ML Workflow
    Dataset Quality Issues
    Data quality and its effect on the ML model
    Data Labelling for Supervised Learning
  5. ML Functional Performance MetricsConfusion Matrix
    Additional ML Functional Performance Metrics for Classification, Regression and
    Clustering
    Limitations of ML Functional Performance Metrics
    Selecting ML Functional Performance Metrics
    Benchmark Suites for ML
  6. ML - Neural Networks and TestingNeural Networks
    Coverage Measures for Neural Networks
  7. Testing AI-Based Systems OverviewSpecification of AI-Based Systems
    Test Levels for AI-Based Systems
    Test Data for Testing AI-Based Systems
    Testing for Automation Bias in AI-Based Systems
    Documenting an ML Model
    Testing for Concept Drift
    Selecting a Test Approach for an ML System
  8. Testing AI-Specific Quality CharacteristicsChallenges Testing Self-Learning Systems
    Testing Autonomous AI-Based Systems
    Testing for Algorithmic, Sample and Inappropriate Bias
    Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
    Challenges Testing Complex AI-based Systems
    Testing the Transparency, Interpretability and Explainability of AI-based Systems
    Test Oracles for AI-Based Systems
    Test Objectives and Acceptance Criteria
  9. Methods and Techniques for the Testing of AI-Based SystemsAdversarial Attacks and Data Poisoning
    Pairwise Testing
    Back-to-Back Testing
    A/B Testing
    Metamorphic Testing
    Experience-Based Testing of AI-Based Systems
    Selecting Test Techniques for AI-Based Systems
  10. Test Environments for AI-Based SystemsTest Environments for AI-Based Systems
    Virtual Test Environments for Testing AI-Based Systems
  11. Using AI for TestingAI Technologies for Testing
    Using AI to Analyze Reported Defects
    Using AI for Test Case Generation
    Using AI for the Optimization of Regression Test Suites
    Using AI for Defect Prediction
    Using AI for Testing User Interfaces

ISTQB Specialist Level AI Testing Exam Structure

The Specialist Level AI Testing certification exam is held in a digital form, as well as remotely (online) in English. The exam is comprised of 40 multiple choice questions, with a pass mark grade of 65% to be completed within 60 minutes. Participants that take the exam not in their spoken language, will receive additional 25% time, and will have 15 minutes more, or a total of 75 min.