There is a growing backlog of unresolved defects for your project. You know the developers have an ML model that they have created which has learned which developers work on which type of software and the speed with which they resolve issues. How could you use this model to help reduce the backlog and implement more efficient defect resolution?
Correct Answer: B
The syllabus explains that ML models can be used to analyze reported defects and suggest which developers are best suited to fix them based on historical data about defect assignment and resolution speed: "Assignment: ML models can suggest which developers are best suited to fix particular defects, based on the defect content and previous developer assignments." (Reference: ISTQB CT-AI Syllabus v1.0, Section 11.2, page 78 of 99)
Question 22
A neural network has been designed and created to assist day-traders improve efficiency when buying and selling commodities in a rapidly changing market. Suppose the test team executes a test on the neural network where each neuron is examined. For this network the shortest path indicates a buy, and it will only occur when the one-day predicted value of the commodity is greater than the spot price by 0.75%. The neurons are stimulated by entering commodity prices and testers verify that they activate only when the future value exceeds the spot price by at least 0.75%. Which of the following statements BEST explains the type of coverage being tested on the neural network?
Correct Answer: A
Threshold coverageis a specific type of coverage measure used in neural network testing. It ensures that each neuron in the network achieves an activation value greater than a specified threshold. This is particularly relevant to the scenario described, where testers verify that neurons activate only when the future value of the commodity exceeds the spot price by at least0.75%. * Threshold-based activation:The test case in the question isexplicitly verifying whether neurons activate only when a certain threshold (0.75%) is exceeded.This aligns perfectly with the definition ofthreshold coverage. * Common in Neural Network Testing:Threshold coverage is used to measurewhether each neuron in a neural network reaches a specified activation value, ensuring that the neural network behaves as expected when exposed to different test inputs. * Precedent in Research:TheDeepXplore frameworkused a threshold of0.75%to identify incorrect behaviors in neural networks, making this coverage criterion well-documented in AI testing research. * (B) Neuron Coverage# * Neuron coverageonly checks whether a neuron activates (non-zero value)at some point during testing. It does not consider specific activation thresholds, making it less precise for this scenario. * (C) Sign-Change Coverage# * This coverage measures whether each neuron exhibitsboth positive and negative activation values, which isnot relevant to the given scenario(where activation only matters when exceeding a specific threshold). * (D) Value-Change Coverage# * This coverage requires each neuron to producetwo activation values that differ by a chosen threshold, but the question focuses onwhether activation occurs beyond a fixed threshold, not changes in activation values. * Threshold coverage ensures that neurons exceed a given activation threshold"Full threshold coverage requires that each neuron in the neural network achieves an activation value greater than a specified threshold. The researchers who created the DeepXplore framework suggested neuron coverage should be measured based on an activation value exceeding a threshold, changing based on the situation." Why is Threshold Coverage Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, asthreshold coverage ensures the neural network's activation is correctly evaluated based on the required condition (0.75%).
Question 23
The activation value output for a neuron in a neural network is obtained by applying computation to the neuron. Which ONE of the following options BEST describes the inputs used to compute the activation value? SELECT ONE OPTION
Correct Answer: A
In a neural network, the activation value of a neuron is determined by a combination of inputs from the previous layer, the weights of the connections, and the bias at the neuron level. Here's a detailed breakdown: * Inputs for Activation Value: * Activation Values of Neurons in the Previous Layer:These are the outputs from neurons in the preceding layer that serve as inputs to the current neuron. * Weights Assigned to the Connections:Each connection between neurons has an associated weight, which determines the strength and direction of the input signal. * Individual Bias at the Neuron Level:Each neuron has a bias value that adjusts the input sum, allowing the activation function to be shifted. * Calculation: * The activation value is computed by summing the weighted inputs from the previous layer and adding the bias. * Formula: z=#(wi#ai)+bz = \sum (w_i \cdot a_i) + bz=#(wi#ai)+b, where wiw_iwi are the weights, aia_iai are the activation values from the previous layer, and bbb is the bias. * The activation function (e.g., sigmoid, ReLU) is then applied to this sum to get the final activation value. * Why Option A is Correct: * Option A correctly identifies all components involved in computing the activation value: the individual bias, the activation values of the previous layer, and the weights of the connections. * Eliminating Other Options: * B. Activation values of neurons in the previous layer, and weights assigned to the connections between the neurons: This option misses the bias, which is crucial. * C. Individual bias at the neuron level, and weights assigned to the connections between the neurons: This option misses the activation values from the previous layer. * D. Individual bias at the neuron level, and activation values of neurons in the previous layer: This option misses the weights, which are essential. References: * ISTQB CT-AI Syllabus, Section 6.1, Neural Networks, discusses the components and functioning of neurons in a neural network. * "Neural Network Activation Functions" (ISTQB CT-AI Syllabus, Section 6.1.1).
Question 24
A bank wants to use an algorithm to determine which applicants should be given a loan. The bank hires a data scientist to construct a logistic regression model to predict whether the applicant will repay the loan or not. The bank has enough data on past customers to randomly split the data into a training data set and a test /validation data set. A logistic regression model is constructed on the training data set using the following independent variables: Gender Marital status Number of dependents Education Income Loan amount Loan term Credit score The model reveals that those with higher credit scores and larger total incomes are more likely to repay their loans. The data scientist has suggested that there might be bias present in the model based on previous models created for other banks. Given this information, what is the best test approach to check for potential bias in the model?
Correct Answer: A
Bias in an AI system occurs when the training data contains inherent prejudices that cause the model to make unfair predictions. Experience-based testing, particularlyExploratory Data Analysis (EDA), helps uncover these biases by analyzing patterns, distributions, and potential discriminatory factors in the training data. * Option A:"Experience-based testing should be used to confirm that the training data set is operationally relevant. This can include applying exploratory data analysis (EDA) to check for bias within the training data set." * This is the correct answer. EDA involves examining the dataset for bias, inconsistencies, or missing values, ensuring fairness in ML model predictions. * Option B:"Back-to-back testing should be used to compare the model created using the training data set to another model created using the test data set. If the two models significantly differ, it will indicate there is bias in the original model." * Back-to-back testing is used for regression testing and to compare versions of an AI system but is not primarily used to detect bias. * Option C:"Acceptance testing should be used to make sure the algorithm is suitable for the customer. The team can re-work the acceptance criteria such that the algorithm is sure to correctly predict the remaining applicants that have been set aside for the validation data set ensuring no bias is present." * Acceptance testing focuses on meeting predefined business requirements rather than detecting and mitigating bias. * Option D:"A/B testing should be used to verify that the test data set does not detect any bias that might have been introduced by the original training data. If the two models significantly differ, it will indicate there is bias in the original model." * A/B testing is used for evaluating variations of a model rather than for explicitly identifying bias. * Bias Testing Methods:"AI-based systems should be tested for algorithmic bias, sample bias, and inappropriate bias. Experience-based testing and EDA are useful for detecting bias". * Exploratory Data Analysis (EDA):"EDA helps uncover potential bias in training data through visualization and statistical analysis". Analysis of the Answer Options:ISTQB CT-AI Syllabus References:Thus,Option A is the best choice for detecting bias in the loan applicant model.
Question 25
Pairwise testing can be used in the context of self-driving cars for controlling an explosion in the number of combinations of parameters. Which ONE of the following options is LEAST likely to be a reason for this incredible growth of parameters? SELECT ONE OPTION
Correct Answer: C
Pairwise testing is used to handle the large number of combinations of parameters that can arise in complex systems like self-driving cars. The question asks which of the given options isleast likelyto be a reason for the explosion in the number of parameters. * Different Road Types (A): Self-driving cars must operate on various road types, such as highways, city streets, rural roads, etc. Each road type can have different characteristics, requiring the car's system to adapt and handle different scenarios. Thus, this is a significant factor contributing to the growth of parameters. * Different Weather Conditions (B): Weather conditions such as rain, snow, fog, and bright sunlight significantly affect the performance of self-driving cars. The car's sensors and algorithms must adapt to these varying conditions, which adds to the number of parameters that need to be considered. * ML Model Metrics to Evaluate Functional Performance (C): While evaluating machine learning (ML) model performance is crucial, it does not directly contribute to the explosion of parameter combinations in the same way that road types, weather conditions, and car features do. Metrics are used to measure and assess performance but are not themselves variable conditions that the system must handle. * Different Features like ADAS, Lane Change Assistance, etc. (D): Advanced Driver Assistance Systems (ADAS) and other features add complexity to self-driving cars. Each feature can have multiple settings and operational modes, contributing to the overall number of parameters. Hence, theleast likelyreason for the incredible growth in the number of parameters isC. ML model metrics to evaluate the functional performance. References: * ISTQB CT-AI Syllabus Section 9.2 on Pairwise Testing discusses the application of this technique to manage the combinations of different variables in AI-based systems, including those used in self- driving cars. * Sample Exam Questions document, Question #29 provides context for the explosion in parameter combinations in self-driving cars and highlights the use of pairwise testing as a method to manage this complexity.