A/B testing, also known as split testing, is a systematic empirical method used to compare two versions of a prompt (Version A and Version B) to determine which one performs better based on predefined evaluation metrics. In the realm of LLMs, where outputs can be stochastic (probabilistic), A/B testing is essential for mitigating inconsistency. When a team encounters vague or varying results for a user story, simply modifying the prompt iteratively (Option B) may improve the result but does not provide a statistical or objective basis for why one version is superior. Byrunning A/B tests, testers can evaluate prompts against specific KPIs such as accuracy, relevance, format adherence, or the absence of hallucinations. This process involves sending the same input data through both prompt versions multiple times and scoring the outputs. The version that consistently yields the "stronger wording" or more precise testware is then selected as the production standard. This data-driven approach is a cornerstone of prompt engineering in professional environments, ensuring that the most effective linguistic structures are utilized to maximize the model's performance and reliability.

This scenario describes a failure in the model's ability to apply logical weight to specific domain concepts, specifically in the context of Risk-Based Testing (RBT). When an LLM ranks a low-impact UI element (a tooltip) higher than a critical functional failure (payment processing), it demonstrates a "Reasoning error in risk calculation logic." While LLMs can follow formulas like $Risk = Likelihood \times Impact$, they may lack the deep semantic understanding of "Impact" within a specific business domain unless explicitly guided.
This is not necessarily a hallucination (Option C), as the model isn't necessarily inventing facts, but rather misapplying the logic of prioritization. It is also distinct from dataset bias (Option D), which would involve a systematic skewing across all outputs. In professional testing, this type of error highlights the necessity of
"human-in-the-loop" verification. Testers must review AI-generated prioritizations to ensure that the logical deductions align with the actual business risk and technical criticality of the features being tested.

Differentiating between prompting techniques is essential for a tester to select the right tool for the task.Few- shot promptingis characterized by providing the model with a few examples of inputs and desired outputs, allowing it to learn the pattern and format.Prompt Chaininginvolves breaking a complex task into a sequence of smaller, interconnected prompts, where the output of one step becomes the input for the next (e.g., first extract requirements, then generate test cases from those requirements).Meta-promptingis a more advanced technique where the user asks the LLM to help design, write, or refine the prompt itself-essentially using the AI as a "prompt engineer" to optimize the instructions. Option D correctly identifies these core characteristics.
Options A, B, and C contain fundamental mischaracterizations: for instance, Few-shotrequiresexamples (contradicting A), and Chaining is theoppositeof a single prompt (contradicting A). Mastering these distinctions allows testers to move from simple "chatting" to sophisticated AI orchestration that can handle complex, multi-stage testing workflows with high reliability.

In the evaluation of GenAI outputs for testing, metrics must align with the specific nature of the task. For anomaly identification, the goal is to correctly identify true issues without an overwhelming number of false positives; therefore,Precisionis the critical metric (the ratio of true anomalies to the total flagged).
Conversely, forpartition testing(identifying valid and invalid input classes), the goal is thoroughness and ensuring no significant category is missed.Recallis the most appropriate metric here, as it measures the model's ability to "call back" or cover all possible relevant partitions from the requirement set. As highlighted in the CT-GenAI syllabus, evaluating AI effectiveness often requires a combination of these model- performance metrics. While "Accuracy" (Option D) provides a general view, it is often misleading in imbalanced testing scenarios (like anomaly detection where anomalies are rare). By using Precision and Recall together, a test organization can quantitatively assess if the AI is both trustworthy in its alerts and comprehensive in its test design coverage.