Databricks-Machine-Learning-Professional Practice Exam Questions with Answers Databricks Certified Machine Learning Professional Certification

The correct line of code is “MODEL_VERSION_TRANSITIONED_STAGE”, which is the name of the event that occurs when a model version transitions to any stage in the MLflow Model Registry. This event can be used to trigger a webhook that will execute a Databricks Job based on the model version and the stage it transitions to. The other options are either invalid event names or too specific for the task.

References: MLflow Model Registry Webhooks, MLflow Model Registry Events

The context parameter is an instance of mlflow.pyfunc.model.ModelContext, which is a wrapper around the model metadata and artifacts. The context parameter allows the user to provide the model access to objects like preprocessing models or custom configuration files that are needed for the model inference. For example, the context parameter can be used to load a scikit-learn preprocessing model that was logged as an artifact with the MLflow model, and then apply it to the input data before passing it to the main model. The context parameter can also be used to access the model signature and the input example, if they were logged with the model. The context parameter is passed to the load_context and predict methods of the PythonModel class, which is asubclass of mlflow.pyfunc.PythonModel that defines the custom logic for loading and scoring the model. References:

• mlflow.pyfunc Documentation, p. 1-2
• mlflow.pyfunc.model Documentation, p. 1-2
• Understanding PyFunc in MLflow, Part 2: Pyfunc Components, p. 1-2
• python - How to load model from mlflow with custom predict without local file?, p. 1-2

MLflow Model flavors are a key concept that makes MLflow Models powerful: they are a convention that deployment tools can use to understand the model, which makes it possible to write tools that work with models from any ML library without having to integrate each tool with each library. MLflow defines a set of built-in model flavors for popular ML frameworks, such as scikit-learn, TensorFlow, PyTorch, etc. Each flavor defines a standard format for model data and a common interface for model inference. For example, the python_function flavor allows any model to be loaded as a Pythonfunction that can be applied to input data. MLflow also supports custom model flavors for user-defined models or community-developed flavors. MLflow Model flavors enable model portability and interoperability across different platforms and environments12 References:

• MLflow Models — MLflow 2.9.1 documentation
• Models, Flavors, and PyFuncs in MLflow

To restore the original scikit-learn model object, the correct method is mlflow.sklearn.load_model, which takes the model URI as an argument and returns a scikit-learn model. This method can load models with the sklearn flavor, which preserves the original attributes and methods of the model. The mlflow.load_model method is a generic method that can load models with any flavor, but it returns a pyfunc model, which does not have the feature_importances_ attribute. The client.list_artifacts method is used to list theartifacts associated with a run, not to load a model. The client.pyfunc.load_model method is similar to mlflow.load_model, but it takes a local file path instead of a model URI. The MLflow Experiments UI can show the logged parameters, metrics, and artifacts of a run, but it cannot show the model object itself. References:

• MLflow Models
• MLflow Python API

Data type tests and checks are particularly important for streaming deployments because of the following reasons:

• Because the streaming deployment is always on, all types of data must be handled without producing an error. Streaming data may come from various sources and have different formats, schemas, or quality. If the streaming deployment cannot handle the data properly, it may result in errors, failures, or inaccurate outputs. Therefore, data type tests and checks can help validate the data and ensure the streaming deployment can process it correctly.
• Because the streaming deployment is always on, there is no practitioner to debug poor model performance. Streaming data may also change over time, affecting the model performance and accuracy. If the streaming deployment does not have a mechanism to monitor and evaluate the model performance, it may not be able to detect or correct any issues. Therefore, data type tests and checks can help identify any data drift or anomalies that may impact the model performance and trigger alerts or actions.
• Because the streaming deployment is always on, there is a need to confirm that the deployment can autoscale. Streaming data may have varying volumes and velocities, requiring the streaming deployment to scale up or down accordingly. If the streaming deployment does not have a way to adjust the resources and throughput based on the data demand, it may result in bottlenecks, delays, or inefficiencies. Therefore, data type tests and checks can help measure the data load and ensure the streaming deployment can autoscale effectively. References: Streaming Data, Data Validation, Model Monitoring, Autoscaling

Label drift is a type of data drift that occurs when there is a change in the probability of a label P(Y). This means that the output distribution of the model’s data source has changed over time, which can affect the performance and accuracy of the model. For example, if a model is trained to classify images of animals, and the data source starts to provide more images of cats than dogs, this would be an example of label drift. Label drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc. Label drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc123 References:

• Monitoring Feature Drift - Databricks
• Drift Metrics: How to Select the Right Metric to Analyze Drift
• Detect data drift on datasets (preview) - Azure Machine Learning

Batch deployment is the most common paradigm for machine learning projects, as it accounts for 80-90% of deployments1. Batch deployment leverages databases and object storage to process a large batch of input data at once, rather than processing each input data point individually in real time. Batch deployment is well suited for large data volume scenarios that need efficient processing but response time isn’t critical. For example, you might use batch deployment to generate recommendations for customers based on their purchase history, or to detect anomalies in sensor data collected over a period of time. Batch deployment is less expensive and more efficient than real-time deployment, as the underlying compute runs only during the batch job and can be scaled up or down as needed. Batch deployment can be implemented using Databricks Machine Learning endpoints, which provide a unified interface to invoke and manage model deployments across compute types2. References:

• Databricks Machine Learning Professional Exam Guide, Section 3: Model Deployment, p. 4
• Databricks Machine Learning Documentation, Deploying ML Models, Batch Inference, p. 1
• Machine Learning in Production - Databricks, The Four Deployment Paradigms, p. 9
• Deploy machine learning models in production environments, Deployment methods, p. 1

MLflow Model Serving allows you to deploy MLflow models for real-time serving via REST API. When you enable Model Serving for a registered model, all model versions in the Staging and Production stages are automatically deployed. You can also manually deploy model versions in other stages, such as None or Archived, by using the UI or the API. However, only the model versions in the Staging and Production stages are automatically deployed by default. References: MLflow Model Serving, Deploy MLflow models

Feature drift is a change in the distribution of the input data over time, which can affect the performance and accuracy of the model. Monitoring feature drift is important to ensure that the model is still valid and reliable for the current data. One simple and low-cost method of monitoring numeric feature drift is to track the summary statistics trends of the input features, such as mean, standard deviation, minimum, maximum, etc. These statistics can be computed easily and efficiently, and can provide a quick overview of the changes in the data distribution. If the summary statistics deviate significantly from the baseline values, it may indicate a feature drift. However, summary statistics trends may not capture all the nuances of the data distribution, such as outliers, multimodality, skewness, etc. Therefore, other methods, such as statistical tests or visualizations, may be needed to complement the summary statistics trends and provide a more comprehensive analysis of the feature drift123 References:

• Monitoring Feature Drift - Databricks
• Monitor feature attribution skew and drift | Vertex AI | Google Cloud
• Monitoring Model Drift - 6 Different Methodologies - Qualdo

MLflow Autologging is a feature that automatically logs parameters, metrics, and artifacts for machine learning frameworks, such as Hyperopt, without requiring any code changes. When using MLflow Autologging with Hyperopt, MLflow will automatically create a parent run for the hyperparameter tuning process and a child run for each unique combination of hyperparameter values. This way, the user can easily compare and visualize the results of different hyperparameter settings in the MLflow UI. MLflow Autologging also supports nested runs, which means that if the user starts a manual run before calling fmin, the parent run will be nested under the manual run. This allows the user to organize and group related runs together. References: MLflow Autologging, Hyperparameter Tuning with MLflow, Apache Spark MLlib and Hyperopt, Using MLFlow with Hyperopt for Automated Machine Learning, Parallelize hyperparameter tuning with scikit-learn and MLflow, [Optimizing Model Performance with MLflow and Hyperopt: From Hyperparameter Tuning to Serving]

To create nested runs in MLflow, you need to specify the nested=True argument in the parent run, and omit it in the child runs. This will set a special tag (mlflow.utils.mlflow_tags.MLFLOW_PARENT_RUN_ID) in the child runs that indicates the parent run ID. This way, the UI will show the child runs under the parent run in a collapsible hierarchy. The indentation of the code blocks is not relevant, as long as the child runs are created within the context of the parent run. The run name parameter is also not relevant, as it does not affect the nesting of the runs. References: Understanding Parent and Child Runs in MLflow, Nested runs using MLflowClient, creating-child-runs-in-mlflow

The client.delete_registered_model method of the MlflowClient class can be used to delete a model from the MLflow Model Registry. The method takes the following parameter:

• name: The name of the registered model to delete.

The method deletes the registered model and all its versions from the Model Registry. The method does not return any value1

The other options are incorrect because:

• A. The client.transition_model_version_stage method can be used to transition a model version to a different stage, such as None, Staging, Production, or Archived. The method does not delete the model or the model version from the Model Registry2
• B. The client.delete_model_version method can be used to delete a specific model version from the Model Registry. The method does not delete the registered model or all its versions from the Model Registry3
• C. The client.update_registered_model method can be used to update the metadata of a registered model, such as its name or description. The method does not delete the model or its versions from the Model Registry4
• D. There is no method called client.delete_model. To delete a model from the Model Registry, the client.delete_registered_model method should be used.

References:

• mlflow.tracking.client.MlflowClient.delete_registered_model — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.transition_model_version_stage — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.delete_model_version — MLflow 2.9.1 documentation
• mlflow.tracking.client.MlflowClient.update_registered_model — MLflow 2.9.1 documentation

The correct code block is the one that matches the event type and the model name specified in the webhook. The event type is MODEL_VERSION_TRANSITIONED_TO_STAGING, which means the webhook will be triggered when a model version is transitioned to the staging stage. The model name is model, which means the webhook will only apply to the registered model with that name. Therefore, the code block that satisfies both conditions is:

client.transition_model_version_stage (name=model, version=model_version, stage=“Staging”)

This code block will transition the model version to the staging stage and invoke the webhook to run the associated job.

References: MLflow Model Registry Webhooks on Databricks, MLflow Python API