Databricks Certified Machine Learning Associate free question 3 / 10 · Domain 1: Databricks Machine Learning
In the production stage of the recommended MLOps workflow, the model training pipeline tunes hyperparameters. Which statement about how that tuning is done matches the documentation?
- The production training code uses the hyperparameters that scored best in the staging integration tests, because those tests train on the full production dataset
- The production training code skips tuning, and the best model artifact from development is registered to the prod catalog exactly as it was produced
- The production training code tunes over a wider search space than development used, and searches again for the best combination on production data at each retraining
- The production training code considers only the top-performing options from development, which saves time and reduces tuning variance in automated retraining
Show the answer and explanation
Answer: D
Why this is correct
The page says that during development data scientists may test many algorithms and hyperparameters, but in the production training code it is common to consider only the top-performing options. Limiting tuning in this way saves time and can reduce the variance from tuning in automated retraining. If data scientists have read-only access to the production catalog, they may be able to determine the optimal set of hyperparameters, and the production pipeline can then run with that set, typically included as a configuration file.
Why the other options are wrong
"The production training code tunes over a wider search space than development used, and searches again for the best combination on production data at each retraining" … A wider search is the opposite of what the page describes. It says production code commonly considers only the top-performing options, because limiting tuning saves time and reduces variance in automated retraining.
"The production training code skips tuning, and the best model artifact from development is registered to the prod catalog exactly as it was produced" … The production stage of the workflow trains and tunes the model on production data and registers the resulting version in the prod catalog. The page does not describe promoting a development artifact without training, and the recommended approach promotes code rather than models.
"The production training code uses the hyperparameters that scored best in the staging integration tests, because those tests train on the full production dataset" … The integration tests in staging check that the pipelines work together, and to save time they may use small subsets of data or fewer training iterations. Values picked on that basis are not described as the source of the production hyperparameters.
Key point
Search broadly in development, narrow the search in production. Chosen hyperparameters typically travel with the pipeline as a configuration file.