Databricks Certified Generative AI Engineer Associate Practice Questions

Databricks Certified Generative AI Engineer Associate exam preparation

Databricks Certified Generative AI Engineer Associate practice questions in English: 162 questions with explanations. The mock exam has 45 questions and 90 minutes.

Databricks Certified Generative AI Engineer Associate free question 6 / 10 · Domain 3: Application Development

A LangChain chain is traced automatically with mlflow.langchain.autolog(). The developer wants to keep automatic tracing but add the prompt version number as a custom attribute on chat model spans only. Which approach do the Docs show?

  1. Wrap the chain in a function decorated with @mlflow.trace(attributes=...) and pass the version so it is copied to the inner chat model spans
  2. Tag the experiment with the version using mlflow.set_experiment_tags(), so it is copied to the attributes of each span logged to it
  3. Pass a dictionary of attributes to the arguments of mlflow.langchain.autolog() so that it attaches them to chat model spans
  4. Create a handler that inherits from MlflowLangchainTracer, override on_chat_model_start, and add the attribute there
Show the answer and explanation

Answer: D

Why this is correct
To change what automatic tracing for LangChain logs, you create a callback handler that inherits from mlflow.langchain.langchain_tracer.MlflowLangchainTracer. MlflowLangchainTracer is a callback handler injected into the LangChain inference flow: it starts a span on events such as on_chain_start and on_llm_start, and closes the span when the event ends. If you override on_chat_model_start, which is called when a chat model starts, add the version number to the attributes, and then call _start_span, the custom attribute is attached to that span only. The span type, name, inputs, outputs and latency are still recorded automatically as before.

Why the other options are wrong
"Pass a dictionary of attributes to the arguments of mlflow.langchain.autolog() so that it attaches them to chat model spans" … autolog is the entry point that turns automatic tracing on or off. The way the Docs show to change the contents is to inherit the handler and override the start method, and autolog does not take arguments that add attributes by span type.
"Wrap the chain in a function decorated with @mlflow.trace(attributes=...) and pass the version so it is copied to the inner chat model spans" … @mlflow.trace creates one span for the wrapped function, and the attributes you pass go on that span. The inner chat model spans are created separately by automatic tracing, and the outer attributes are not copied to them.
"Tag the experiment with the version using mlflow.set_experiment_tags(), so it is copied to the attributes of each span logged to it" … Tags set with mlflow.set_experiment_tags() are information for classifying the experiment. There is no setting that copies them to the attributes of each span, so the chat model spans would not get the version number.

Key point
The LangChain calls that automatic tracing follows are invoke, batch and stream, their async versions (ainvoke, abatch, astream), the retriever's get_relevant_documents, and __call__ of Chain and AgentExecutor. Work called through any other entry point does not show up in the trace unless you create a span yourself.

Go to the Databricks Certified Generative AI Engineer Associate practice app (10 free questions; purchase unlocks all questions)

See the full practice set