Online Access Free Associate-Developer-Apache-Spark Practice Test

Which of the following statements about stages is correct?

• A.Stages ephemerally store transactions, before they are committed through actions.
• B.Stages consist of one or more jobs.
• C.Stages may contain multiple actions, narrow, and wide transformations.
• D.Tasks in a stage may be executed by multiple machines at the same time.
• E.Different stages in a job may be executed in parallel.

Which of the following code blocks returns a new DataFrame with only columns predError and values of every second row of DataFrame transactionsDf?
Entire DataFrame transactionsDf:
1.+-------------+---------+-----+-------+---------+----+
2.|transactionId|predError|value|storeId|productId| f|
3.+-------------+---------+-----+-------+---------+----+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.| 4| null| null| 3| 2|null|
8.| 5| null| null| null| 2|null|
9.| 6| 3| 2| 25| 2|null|
10.+-------------+---------+-----+-------+---------+----+

• A.transactionsDf.filter(col("transactionId").isin([3,4,6])).select([predError, value])
• B.transactionsDf.select(col("transactionId").isin([3,4,6]), "predError", "value")
• C.transactionsDf.filter("transactionId" % 2 == 0).select("predError", "value")
• D.transactionsDf.filter(col("transactionId") % 2 == 0).select("predError", "value") (Correct)
• E.1.transactionsDf.createOrReplaceTempView("transactionsDf")
  2.spark.sql("FROM transactionsDf SELECT predError, value WHERE transactionId % 2 = 2")
• F.transactionsDf.filter(col(transactionId).isin([3,4,6]))

Which of the following describes a valid concern about partitioning?

• A.No data is exchanged between executors when coalesce() is run.
• B.Short partition processing times are indicative of low skew.
• C.A shuffle operation returns 200 partitions if not explicitly set.
• D.The coalesce() method should be used to increase the number of partitions.
• E.Decreasing the number of partitions reduces the overall runtime of narrow transformations if there are more executors available than partitions.

Which of the following describes the difference between client and cluster execution modes?

• A.In cluster mode, the driver runs on the edge node, while the client mode runs the driver in a worker node.
• B.In cluster mode, the driver runs on the worker nodes, while the client mode runs the driver on the client machine.
• C.In cluster mode, each node will launch its own executor, while in client mode, executors will exclusively run on the client machine.
• D.In client mode, the cluster manager runs on the same host as the driver, while in cluster mode, the cluster manager runs on a separate node.
• E.In cluster mode, the driver runs on the master node, while in client mode, the driver runs on a virtual machine in the cloud.

Which of the following describes characteristics of the Dataset API?

• A.In Python, the Dataset API mainly resembles Pandas' DataFrame API.
• B.The Dataset API does not provide compile-time type safety.
• C.The Dataset API does not support unstructured data.
• D.In Python, the Dataset API's schema is constructed via type hints.
• E.The Dataset API is available in Scala, but it is not available in Python.