Which ONE of the following tests is LEAST likely to be performed during the ML model testing phase? SELECT ONE OPTION
Correct Answer: A
The question asks which test is least likely to be performed during the ML model testing phase. Let's consider each option: * Testing the accuracy of the classification model (A): Accuracy testing is a fundamental part of the ML model testing phase. It ensures that the model correctly classifies the data as intended and meets the required performance metrics. * Testing the API of the service powered by the ML model (B): Testing the API is crucial, especially if the ML model is deployed as part of a service. This ensures that the service integrates well with other systems and that the API performs as expected. * Testing the speed of the training of the model (C): This is least likely to be part of the ML model testing phase. The speed of training is more relevant during the development phase when optimizing and tuning the model. During testing, the focus is more on the model's performance and behavior rather than how quickly it was trained. * Testing the speed of the prediction by the model (D): Testing the speed of prediction is important to ensure that the model meets performance requirements in a production environment, especially for real- time applications. References: * ISTQB CT-AI Syllabus Section 3.2 on ML Workflow and Section 5 on ML Functional Performance Metrics discuss the focus of testing during the model testing phase, which includes accuracy and prediction speed but not the training speed.
Question 57
You have been developing test automation for an e-commerce system. One of the problems you are seeing is that object recognition in the GUI is having frequent failures. You have determined this is because the developers are changing the identifiers when they make code updates. How could AI help make the automation more reliable?
Correct Answer: A
Object recognition issues in test automation often arise whendevelopers frequently change object identifiers in the GUI. AI can enhance the stability of GUI automation by: * Using multiple criteria for object identification * AI cantrack UI elements using multiple attributessuch asXPath, label, ID, class, and screen coordinatesrather than relying on a single identifier that may change over time. * This approach makes the automationless brittle and more adaptive to changes in the UI. * Why other options are incorrect? * B (Ignore unrecognizable objects to avoid failures): Ignoring objects instead of identifying them properly wouldlead to incomplete or incorrect test execution. * C (Dynamically name objects and alter source code): AI-based testing tools donot modify application source code; they work byadjusting the recognition strategy. * D (Anticipate developer changes and pre-alter automation code): While AI can adapt,it does not predict future changes to the GUI, making this option unrealistic. Thus,Option A is the best answer, as AI tools enhance object recognitionby dynamically selecting the most stable and persistent identification methods, improving test automation reliability. Certified Tester AI Testing Study Guide References: * ISTQB CT-AI Syllabus v1.0, Section 11.6.1 (Using AI to Test Through the Graphical User Interface (GUI)) * ISTQB CT-AI Syllabus v1.0, Section 11.6.2 (Using AI to Test the GUI).
Question 58
The training of an ML model... What type of bias is LEAST important to look for when testing the model? Choose ONE option (1 out of 4)
Correct Answer: B
The ISTQB CT-AI syllabus distinguishes between several types of bias relevant in AI testing, including sample bias,algorithmic bias, andinappropriate bias. In Section3.3 - Bias in AI-Based Systems, the syllabus stresses the importance of identifying biases that originate fromtraining data,model development, anddecision logic. Sample bias occurs when the training data does not adequately represent the population; algorithmic bias arises when the model produces systematically skewed results due to learned patterns; inappropriate bias involves ethically or socially problematic distortions in the outcomes. All three of these bias types directly affect theoutputs of the AI modeland are therefore highly relevant when testing an industrial inspection system intended to reliably detect defects. These biases can lead to defective items being missed or false alarms being raised, which impacts quality assurance significantly . Automation bias, however, is fundamentally different. It refers to ahuman cognitive bias, where users (e.g., inspectors) overly trust or rely on the AI system's output. While important in user-interaction testing, it isnota biaswithin the ML model itself. Since the question asks which bias isleast important when testing the model, automation bias can be legitimately deprioritized duringmodel-level testing. Therefore, OptionBis correct.
Question 59
Which of the following is an example of an input change where it would be expected that the AI system should be able to adapt?
Correct Answer: B
AI systems, particularly machine learning models, need to exhibit adaptability and flexibility to handle slight variations in input data without requiring retraining. The ISTQB CT-AI syllabus outlines adaptability as a crucial feature of AI systems, especially when the system is exposed to variations in its operational environment. * Option A:"It has been trained to recognize cats and is given an image of a dog." * This scenario introduces an entirely new class (dogs), which is outside the AI system's expected scope. If the AI was only trained to recognize cats, it would not be expected to recognize dogs correctly without retraining. This does not demonstrate adaptability as expected from an AI system. * Option B:"It has been trained to recognize human faces at a particular resolution and it is given a human face image captured with a higher resolution." * This is an example of an AI system encountering a variation of its training data rather than entirely new data. Most AI-based image processing models can adapt to different resolutions by applying downsampling or other pre-processing techniques. Since the data remains within the domain of human faces, the model should be able to process the higher-resolution image without significant issues. * Option C:"It has been trained to analyze mathematical models and is given a set of landscape pictures to classify." * This represents a complete shift in the data type from structured numerical data to unstructured image data. The AI system is unlikely to adapt effectively, as it has not been trained on image classification tasks. * Option D:"It has been trained to analyze customer buying trend data and is given information on supplier cost data." * This introduces a significant domain shift. Customer buying trends focus on consumer behavior, while supplier cost data relates to pricing structures and logistics. The AI system would likely require retraining to process the new data meaningfully. * Adaptability Requirements:The syllabus discusses that AI-based systems must be able to adapt to changes in their operational environment and constraints, including minor variations in input quality (such as resolution changes). * Autonomous Learning & Evolution:AI systems are expected to improve and handle evolving inputs based on prior experience. * Challenges in Testing Self-Learning Systems:AI systems should be tested to ensure they function correctly when encountering new but related data, such as different resolutions of the same object. Analysis of the Answer Options:ISTQB CT-AI Syllabus References:Thus,option Bis the best choice as it aligns with the adaptability characteristics expected from AI-based systems.
Question 60
Which statement regarding data preparation in the ML workflow is correct? Choose ONE option (1 out of 4)
Correct Answer: C
The ISTQB CT-AI syllabus describes theML data preparation workflowin Section2.2 - Data Preparation. Data preparation consists ofdata gathering,cleaning,transformation, andsampling. The syllabus emphasizes that one significant challenge duringdata gatheringis combining data frommultiple heterogeneous sources, which often differ in structure, quality, and format. Ensuring the resulting dataset is accurate, complete, and representative can be complex, making this a critical challenge in the ML workflow. This aligns directly with OptionC. Option A is incorrect because erroneous data correction is part ofcleaning, not transformation. Option B contradicts the syllabus: while automation can help,not all steps should be automateddue to the need for expert oversight, especially in detecting subtle data quality issues. Option D is incorrect because sampling continues to involve risk-particularly around representativeness-and the syllabus emphasizes caution, not complacency. Thus, OptionCis the only statement that accurately reflects the syllabus.