Explanation This is the correct answer because it describes integration testing. Integration testing is a type of testing that validates interactions between subsystems of your application, such as modules, components, or services. Integration testing ensures that the subsystems work together as expected and produce the correct outputs or results. Integration testing can be done at different levels of granularity, such as component integration testing, system integration testing, or end-to-end testing. Integration testing can help detect errors or bugs that may not be found by unit testing, which only validates behavior of individual elements of your application. Verified References: [Databricks Certified Data Engineer Professional], under "Testing" section; Databricks Documentation, under "Integration testing" section.
Question 22
You would like to build a spark streaming process to read from a Kafka queue and write to a Delta table every 15 minutes, what is the correct trigger option
Correct Answer: D
Explanation The answer is trigger(processingTime = "15 Minutes") Triggers: *Unspecified This is the default. This is equivalent to using processingTime="500ms" *Fixed interval micro-batches .trigger(processingTime="2 minutes") The query will be executed in micro-batches and kicked off at the user-specified intervals *One-time micro-batch .trigger(once=True) The query will execute a single micro-batch to process all the available data and then stop on its own *One-time micro-batch.trigger .trigger(availableNow=True) -- New feature a better version of (once=True) Databricks supports trigger(availableNow=True) in Databricks Runtime 10.2 and above for Delta Lake and Auto Loader sources. This functionality combines the batch processing approach of trigger once with the ability to configure batch size, resulting in multiple parallelized batches that give greater control for right-sizing batches and the resultant files.
Question 23
Suppose there are three events then which formula must always be equal to P(E1|E2,E3)?
Correct Answer: B
Explanation This is an application of conditional probability: P(E1,E2)=P(E1|E2)P(E2). so P(E1|E2) = P(E1.E2)/P(E2) P(E1,E2,E3)/P(E2,E3) If the events are A and B respectively, this is said to be "the probability of A given B" It is commonly denoted by P(A|B):or sometimes PB(A). In case that both "A" and "B" are categorical variables, conditional probability table is typically used to represent the conditional probability.
Question 24
A Databricks job has been configured with 3 tasks, each of which is a Databricks notebook. Task A does not depend on other tasks. Tasks B and C run in parallel, with each having a serial dependency on Task A. If task A fails during a scheduled run, which statement describes the results of this run?
Correct Answer: D
Explanation When a Databricks job runs multiple tasks with dependencies, the tasks are executed in a dependency graph. If a task fails, the downstream tasks that depend on it are skipped and marked as Upstream failed. However, the failed task may have already committed some changes to the Lakehouse before the failure occurred, and those changes are not rolled back automatically. Therefore, the job run may result in a partial update of the Lakehouse. To avoid this, you can use the transactional writes feature of Delta Lake to ensure that the changes are only committed when the entire job run succeeds. Alternatively, you can use the Run if condition to configure tasks to run even when some or all of their dependencies have failed, allowing your job to recover from failures and continue running. References: transactional writes: https://docs.databricks.com/delta/delta-intro.html#transactional-writes Run if: https://docs.databricks.com/en/workflows/jobs/conditional-tasks.html
Question 25
What is the best way to query external csv files located on DBFS Storage to inspect the data using SQL?
Correct Answer: C
Explanation Answer is, SELECT * FROM CSV. 'dbfs:/location/csv_files/' you can query external files stored on the storage using below syntax SELECT * FROM format.`/Location` format - CSV, JSON, PARQUET, TEXT