Databricks Certified Machine Learning Professional Practice Questions

Databricks Certified Machine Learning Professional exam preparation

Databricks Certified Machine Learning Professional practice questions in English: 180 questions with explanations. The mock exam has 59 questions and 120 minutes.

Databricks Certified Machine Learning Professional free question 4 / 10 · Domain 1: Model Development

A Databricks GPU cluster runs a Spark preprocessing job with UDFs and then a Ray GPU training job. The team wants Spark transformations and UDFs to leave the GPUs to Ray. What does the Databricks guide recommend?

  1. Set num_gpus_per_node to 0 in the setup_ray_cluster call.
  2. Start two clusters and copy the data between them by hand.
  3. Set spark.executor.memory to 4g in the cluster configuration.
  4. Set the Spark configuration spark.task.resource.gpu.amount to 0.
Show the answer and explanation

Answer: D

Why this is correct
The Ray cluster runs on top of a Databricks Spark cluster. A typical scenario is to use a Spark job and Spark UDFs for simple preprocessing that does not need GPU resources and then use Ray for the complex machine learning tasks that benefit from GPUs. Databricks recommends setting the Spark cluster-level configuration parameter spark.task.resource.gpu.amount to 0, so that Spark DataFrame transformations and UDF executions do not use GPU resources, which leaves them to the Ray tasks.

Why the other options are wrong
"Set spark.executor.memory to 4g in the cluster configuration." … Reducing spark.executor.memory is the guidance for hybrid workloads that compete for memory, not for GPU use by Spark tasks.
"Set num_gpus_per_node to 0 in the setup_ray_cluster call." … num_gpus_per_node gives GPUs to the Ray workers. Setting it to 0 would take the GPUs away from Ray instead of from Spark.
"Start two clusters and copy the data between them by hand." … The guide describes a setting for a single cluster in which Spark does the simple preprocessing and Ray runs the GPU tasks.

Keep GPUs for Ray
The Ray GPU cluster itself is created with num_gpus_per_node and num_gpus_head_node.

Go to the Databricks Certified Machine Learning Professional practice app (10 free questions; purchase unlocks all questions)

See the full practice set