Which of the following statements regarding experience-based testing for AI-based systems is correct? Choose ONE option (1 out of 4)
Correct Answer: C
The ISTQB CT-AI syllabus explains inSection 4.4 - Experience-Based Testing for AI Systemsthat AI- based systems frequently suffer frominsufficient specifications, unpredictable model behavior, andtest oracle problems, especially when outputs depend on probabilistic or learned patterns. The syllabus explicitly states thatexploratory testingis especially valuable in such contexts because it allows testers to investigate the system interactively, observe unexpected behavior, and evaluate system responses that cannot be fully predicted beforehand. Thus, OptionCaccurately reflects the role and justification of exploratory testing for AI systems. Option A describes data analysis rather than intuitive test design. Option B is incorrect because checklist- based testing does not dynamically adapt test cases; instead, it follows predetermined checklists. Option D incorrectly defines "tour-based testing"; tours refer to structured exploratory approaches, not biased datasets. Therefore,Option Cis the syllabus-aligned correct statement.
Question 47
A transportation company operates three types of delivery vehicles in its fleet. The vehicles operate at different speeds (slow, medium, and fast). The transportation company is attempting to optimize scheduling and has created an AI-based program to plan routes for its vehicles using records from the medium-speed vehicle traveling to selected destinations. The test team uses this data in metamorphic testing to test the accuracy of the estimated travel times created by the AI route planner with the actual routes and times. Which of the following describes the next phase of metamorphic testing?
Correct Answer: A
Metamorphic Testing (MT)is a testing technique that verifies AI-based systems by generatingfollow-up test casesbased on existing test cases. These follow-up test cases adhere to aMetamorphic Relation (MR), ensuring that if the system is functioning correctly, changes in input should result in predictable changes in output. * Metamorphic testing works by transforming source test cases into follow-up test cases * Here, thesource test caseinvolves testing themedium-speed vehicle'stravel time. * Thefollow-up test casesare derived byextrapolating travel times for fast and slow vehiclesusing predictable relationships based on speed differences. * MR states that modifying input should result in a predictable change in output * Since the speed of the vehicle is a known factor, it is possible to predict the new arrival times and verify whether they follow expected trends. * This is a direct application of metamorphic testing principles * Inroute optimization systems, metamorphic testing often applies transformations tospeed, distance, or conditionsto verify expected outcomes. * (B) Decomposing each route into traffic density and vehicle power# * While useful for statistical analysis, this approach does not generate follow-up test cases based on a definedmetamorphic relation (MR). * (C) Selecting dissimilar routes and transforming them into a fast or slow route# * Thisdoes not follow metamorphic testing principles, which require predictable transformations. * (D) Running fast vehicles on long routes and slow vehicles on short routes# * This methoddoes not maintain a controlled MRand introduces too manyuncontrolled variables. * Metamorphic testing generates follow-up test cases based on a source test case."MT is a technique aimed at generating test cases which are based on a source test case that has passed.One or more follow- up test cases are generated by changing (metamorphizing) the source test case based on a metamorphic relation (MR)." * MT has been used for testing route optimization AI systems."In the area of AI, MT has been used for testing image recognition, search engines, route optimization and voice recognition, among others." Why Option A is Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as it aligns with the principles ofmetamorphic testing by modifying input speeds and verifying expected results.
Question 48
Which ONE of the following statements is a CORRECT adversarial example in the context of machine learning systems that are working on image classifiers. SELECT ONE OPTION
Correct Answer: D
A . Black box attacks based on adversarial examples create an exact duplicate model of the original. Black box attacks do not create an exact duplicate model. Instead, they exploit the model by querying it and using the outputs to craft adversarial examples without knowledge of the internal workings. B . These attack examples cause a model to predict the correct class with slightly less accuracy even though they look like the original image. Adversarial examples typically cause the model to predict the incorrect class rather than just reducing accuracy. These examples are designed to be visually indistinguishable from the original image but lead to incorrect classifications. C . These attacks can't be prevented by retraining the model with these examples augmented to the training data. This statement is incorrect because retraining the model with adversarial examples included in the training data can help the model learn to resist such attacks, a technique known as adversarial training. D . These examples are model specific and are not likely to cause another model trained on the same task to fail. Adversarial examples are often model-specific, meaning that they exploit the specific weaknesses of a particular model. While some adversarial examples might transfer between models, many are tailored to the specific model they were generated for and may not affect other models trained on the same task. Therefore, the correct answer is D because adversarial examples are typically model-specific and may not cause another model trained on the same task to fail.
Question 49
Pairwise testing can be used in the context of self-driving cars for controlling an explosion in the number of combinations of parameters. Which ONE of the following options is LEAST likely to be a reason for this incredible growth of parameters? SELECT ONE OPTION
Correct Answer: C
Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options isleast likelyto be a reason for the explosion in the number of parameters. * Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters. * Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered. * ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle. * Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters. Hence, theleast likelyreason for the incredible growth in the number of parameters isC. ML model metrics to evaluate the functional performance. : ISTQB CT-AI Syllabus Section 9.2 on Pairwise Testing discusses the application of this technique to manage the combinations of different variables in AI-based systems, including those used in self-driving cars. Sample Exam Questions document, Question #29 provides context for the explosion in parameter combinations in self-driving cars and highlights the use of pairwise testing as a method to manage this complexity.
Question 50
The stakeholders of a machine learning model have confirmed that they understand the objective and purpose of the model, and ensured that the proposed model aligns with their business priorities. They have also selected a framework and a machine learning model that they will be using. What should be the next step to progress along the machine learning workflow?
Correct Answer: A
Themachine learning (ML) workflowfollows a structured sequence of steps. Once stakeholders have agreed on theobjectives, business priorities, and the framework/model selection, the next logical step is to prepare and pre-process the databefore training the model. * Data Preparationis crucial becausemachine learning models rely heavily on the quality of input data. Poor data can result in biased, inaccurate, or unreliable models. * The process involvesdata acquisition, cleaning, transformation, augmentation, and feature engineering. * Preparing the dataensures it is in the right format, free from errors, and representative of the problem domain, leading to better generalization in training. * A (Tune the ML Algorithm):Hyperparameter tuning occursafter the model has been trainedand evaluated. * C (Agree on Acceptance Criteria):Acceptance criteria should already have been defined in theinitial objective-setting phasebefore framework and model selection. * D (Evaluate the Framework and Model):The selection of the framework and ML model has already been completed. The next step isdata preparation, not reevaluation. * ISTQB CT-AI Syllabus (Section 3.2: ML Workflow - Data Preparation Phase) * "Data preparation comprises data acquisition, pre-processing, and feature engineering. Exploratory data analysis (EDA) may be performed alongside these activities". * "The data used to train, tune, and test the model must be representative of the operational data that will be used by the model". Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Since the model selection is complete, thenext step in the ML workflow is to prepare and pre-process the datato ensure it is ready for training and testing. Thus, thecorrect answer is B.