Databricks Certified Generative AI Engineer Associate free question 5 / 10 · Domain 3: Application Development
A developer is building a LangChain chain in a Databricks notebook. The experiment is set with mlflow.set_experiment(), and the chain is called with invoke. When the notebook runs on serverless compute, no trace of the chain calls is saved to the experiment. How should this be fixed?
- LangChain autologging is not enabled automatically on serverless compute, so call mlflow.langchain.autolog() in the notebook
- Log the chain once with mlflow.langchain.log_model(), and later invoke calls will be traced automatically even on serverless compute
- Remove the mlflow.set_experiment() call and go back to the notebook default experiment, which turns on automatic tracing on serverless compute
- Serverless compute cannot write traces to an experiment, so record the inputs and outputs of each step to the run with mlflow.log_param()
Show the answer and explanation
Answer: A
Why this is correct
MLflow Tracing provides automatic tracing for LangChain. When you call mlflow.langchain.autolog(), nested traces are automatically logged to the active MLflow Experiment upon invocation of chains. But as a note in the Docs says, on serverless compute clusters autologging is not automatically enabled. To trace LangChain on serverless compute, you must explicitly call mlflow.langchain.autolog() in the notebook. Once you do, each invoke leaves the chain, LLM and retrieval steps as spans.
Why the other options are wrong
"Serverless compute cannot write traces to an experiment, so record the inputs and outputs of each step to the run with mlflow.log_param()" … Traces can be recorded on serverless compute. What is missing is turning on automatic tracing, and writing inputs and outputs as run parameters does not produce a trace with nested spans and latency.
"Log the chain once with mlflow.langchain.log_model(), and later invoke calls will be traced automatically even on serverless compute" … log_model saves the chain as a model, and it does not turn on automatic tracing. Even after logging, an invoke in the notebook is not traced unless autolog is called.
"Remove the mlflow.set_experiment() call and go back to the notebook default experiment, which turns on automatic tracing on serverless compute" … Changing the experiment it is recorded to does not change the fact that automatic tracing remains off. The cause of the missing traces is not the destination but the missing autolog call.
Key point
To stop automatic tracing, call mlflow.langchain.autolog(disable=True) or mlflow.autolog(disable=True). For production, the Docs show a setup that installs the lightweight mlflow-tracing package for production use instead of the full MLflow.