Databricks Certified Machine Learning Associate free question 9 / 10 · Domain 4: Model Deployment
A custom model runs a simulation for each input record, and one computation takes about 15 minutes. A few thousand records arrive each day, and the results only need to be ready by the next morning. A teammate proposes putting the model on a Model Serving endpoint and querying it over REST one record at a time. How should this plan be judged against the Databricks documentation?
- Skip the endpoint and use Spark batch inference, because the endpoint times out a request whose model computation runs past its limit
- Use the endpoint with a GPU workload type, because a GPU endpoint has no time limit on a single computation
- Use the endpoint with scale to zero disabled, because an endpoint that stays warm keeps waiting for a long computation to finish
- Skip the endpoint and use Spark batch inference, because a served entity handles requests one at a time, in order
Show the answer and explanation
Answer: A
Why this is correct
The custom models page lists a request timeout among the things to expect from an endpoint: if model computation takes longer than 597 seconds, the request times out. One computation of about 15 minutes (900 seconds) can never return through the endpoint. The scenario also needs no low-latency answers, since the results only have to exist by the next morning. Scoring the whole table at once with Spark batch inference on a cluster fits this workload, and it does not depend on the per-request limit.
Why the other options are wrong
"Skip the endpoint and use Spark batch inference, because a served entity handles requests one at a time, in order" … The documentation does not describe a served entity as processing requests one at a time. It describes provisioned concurrency as the maximum number of parallel requests the endpoint can handle. The reason to move to batch is the timeout, not serial handling.
"Use the endpoint with scale to zero disabled, because an endpoint that stays warm keeps waiting for a long computation to finish" … Scale to zero is described as letting an endpoint shrink to zero after 30 minutes of inactivity, with a cold-start delay on the first request after that. Disabling it removes the cold start, but the 597-second request timeout still applies to a 15-minute computation.
"Use the endpoint with a GPU workload type, because a GPU endpoint has no time limit on a single computation" … The documented GPU limitations concern slower container image creation, deployment timeouts for very large models, and slower autoscaling. Nothing lifts the request timeout for GPU endpoints.
Key point
Real-time serving is built for answers returned one request at a time with low latency. A long per-record computation or an overnight job belongs in batch inference. The per-request time limit is the first condition to check when deciding whether an endpoint can handle the model at all.