Databricks-Machine-Learning-Professional Practice Exam Questions with Answers Databricks Certified Machine Learning Professional Certification

To create nested runs in MLflow, you need to specify the nested=True argument in the parent run, and omit it in the child runs. This will set a special tag (mlflow.utils.mlflow_tags.MLFLOW_PARENT_RUN_ID) in the child runs that indicates the parent run ID. This way, the UI will show the child runs under the parent run in a collapsible hierarchy. The indentation of the code blocks is not relevant, as long as the child runs are created within the context of the parent run. The run name parameter is also not relevant, as it does not affect the nesting of the runs. References: Understanding Parent and Child Runs in MLflow, Nested runs using MLflowClient, creating-child-runs-in-mlflow

MLflow Model Serving is a feature that allows users to host MLflow models as REST endpoints on Databricks1. To query a model version that is in the Production stage of the MLflow Model Registry, the user can use the following model URI format: https:// /model-serving///invocations2. In this case, the model name is recommender and the stage is Production, so the correct model URI is https:// /model-serving/recommender/Production/invocations. The other options are incorrect because they either use the wrong prefix (model instead of model-serving), the wrong stage name (stage-production instead of Production), or require the version number of the model version in Production, which is not necessary for querying the model by stage. References: Announcing MLflow Model Serving on Databricks, [MLflow Model Serving], [MLflow Model Registry]

Feature drift is a change in the distribution of the input data over time, which can affect the performance and accuracy of the model. Monitoring feature drift is important to ensure that the model is still valid and reliable for the current data. One simple and low-cost method of monitoring numeric feature drift is to track the summary statistics trends of the input features, such as mean, standard deviation, minimum, maximum, etc. These statistics can be computed easily and efficiently, and can provide a quick overview of the changes in the data distribution. If the summary statistics deviate significantly from the baseline values, it may indicate a feature drift. However, summary statistics trends may not capture all the nuances of the data distribution, such as outliers, multimodality, skewness, etc. Therefore, other methods, such as statistical tests or visualizations, may be needed to complement the summary statistics trends and provide a more comprehensive analysis of the feature drift123 References:

• Monitoring Feature Drift - Databricks
• Monitor feature attribution skew and drift | Vertex AI | Google Cloud
• Monitoring Model Drift - 6 Different Methodologies - Qualdo

MLflow Autologging is a feature that automatically logs parameters, metrics, and artifacts for machine learning frameworks, such as Hyperopt, without requiring any code changes. When using MLflow Autologging with Hyperopt, MLflow will automatically create a parent run for the hyperparameter tuning process and a child run for each unique combination of hyperparameter values. This way, the user can easily compare and visualize the results of different hyperparameter settings in the MLflow UI. MLflow Autologging also supports nested runs, which means that if the user starts a manual run before calling fmin, the parent run will be nested under the manual run. This allows the user to organize and group related runs together. References: MLflow Autologging, Hyperparameter Tuning with MLflow, Apache Spark MLlib and Hyperopt, Using MLFlow with Hyperopt for Automated Machine Learning, Parallelize hyperparameter tuning with scikit-learn and MLflow, [Optimizing Model Performance with MLflow and Hyperopt: From Hyperparameter Tuning to Serving]

To obtain run-level results for an experiment ID in a Spark DataFrame, you can use the mlflow.search_runs(exp_id) function. This function takes a filter string, which acts as a filter to the query, and returns a pandas.DataFrame of runs, where each metric, parameter, and tag are expanded into their own columns named metrics., params., and tags.* respectively1. You can then convert the pandas.DataFrame to a Spark DataFrame using the spark.createDataFrame() method2. The other options are incorrect because:

• Option A: client.list_run_infos(exp_id) will return a list of RunInfo objects, not a DataFrame3.
• Option B: spark.read.format(“delta”).load(exp_id) will not work for MLflow experiments, as it expects a Delta table path, not an experiment ID4.
• Option C: There is a way to programmatically return row-level results from an MLflow experiment using the mlflow.search_runs(exp_id) function1.
• Option E: spark.read.format(“mlflow-experiment”).load(exp_id) is not a valid format for reading MLflow experiments, as there is no such format registered in Spark5. References: Search Runs, Create a DataFrame from an RDD, a list or a pandas.DataFrame, MlflowClient.list_run_infos, Delta Lake Quickstart, spark.read.format

 To read data from a stream source, such as Kafka, socket, or rate, the spark.readStream method should be used instead of spark.read. The spark.readStream method returns a streaming DataFrame that represents the unbounded input data stream. The spark.readStream method supports the same options and formats as the spark.read method, such as schema, delta, csv, json, etc. The spark.readStream method can also read from a Delta table as a stream source, by specifying the format("delta") and the path or table name of the Delta table123

The other options are incorrect because:

• A. Replacing “inference” with the path to the location of the Delta table does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the path or table name of the Delta table can be specified as an option or argument.
• B. Replacing schema(schema) with option("maxFilesPerTrigger", 1) does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the schema can be specified as an option or argument. The option("maxFilesPerTrigger", 1) is an optional configuration that limits the number of files processed in each trigger for file-based stream sources, such as delta, csv, json, etc. It does not affect the reading of data from a stream source4
• D. Replacing format("delta") with format("stream") does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the format can be specified as an option or argument. The format("stream") is not a valid format for reading data from a stream source. The supported formats are delta, kafka, socket, rate, etc1
• E. Replacing predict with a stream-friendly prediction function does not change the fact that spark.read is used to read from a stream source, which is not supported. The spark.readStream method should be used instead, and the prediction function can be applied to the streaming DataFrame as usual. The predict function does not need to be changed, as long as it can accept a streaming DataFrame as input and return a column of predictions as output5

References:

• Input Sources - Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Delta Lake - Databricks
• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Configuration - Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks

Jensen-Shannon (JS) distance is a symmetric measure of the distance between two probability distributions. It is based on the Kullback-Leibler (KL) divergence, but unlike KL, it does not depend on which distribution is considered the reference and which is considered the comparison. JS distance is also bounded between 0 and 1, which makes it easier to interpret and compare across different features.

Kolmogorov-Smirnov (KS) test is a non-parametric test that compares the cumulative distribution functions (CDFs) of two samples. It calculates the maximum difference between the CDFs and compares it to a critical value to determine if the samples are from the same distribution. KS test is sensitive to the shape, location, and scale of the distributions, but it can also be too sensitive for large datasets, where small changes can result in significant differences.

JS distance is more robust than KS test for numeric feature drift detection because it can handle large datasets without being too sensitive to minor fluctuations. It can also capture changes in the shape and spread of the distributions, not just the location. JS distance does not require any manual threshold or cutoff determinations, unlike KS test, which requires choosing a significance level and a critical value. JS distance can also be generalized to multivariate cases, while KS test is limited to univariate cases. References:

• Which test is the best? We compared 5 methods to detect data drift on large datasets
• The Jensen-Shannon Divergence: A Measure of Distance Between Probability Distributions
• Kullback–Leibler vs Kolmogorov-Smirnov distance

The mlflow.register_model operation can be used to add a new version of an MLflow Model to an existing MLflow Model Registry model. This operation takes the URI of the model source (such as an MLflow run or a local path) and the name of the registered model as arguments, and returns a ModelVersion object that contains metadata about the registered model version. The mlflow.register_model operation can also create a new registered model if the name does not exist in the Model Registry.

The other options are incorrect for the following reasons:

• B. MlflowClient.update_registered_model is used to update the metadata of an existing registered model, such as the name or description, not to add a new version of the model.
• C. mlflow.add_model_version does not exist as a valid MLflow operation.
• D. MlflowClient.get_model_version is used to fetch the metadata of a specific version of a registered model, not to add a new version of the model.
• E. The machine learning engineer does not need to create an entirely new MLflow Model Registry model, as they can use the mlflow.register_model operation to add a new version of the model to the existing “project” model.

References: Adding an MLflow Model to the Model Registry, mlflow.register_model

Real-time deployment is a paradigm that can centrally compute predictions for a single record with exceedingly fast results. Real-time deployment involves exposing a model as a REST API endpoint that can receive requests and return predictions in milliseconds. This paradigm is suitable for scenarios where low latency and high throughput are required, such as fraud detection, recommendation systems, or chatbots. The other deployment paradigms are not as fast or as centralized as real-time deployment. Streaming deployment involves processing data streams in micro-batches, which can introduce some delay. Batch deployment involves processing large batches of data at scheduled intervals, which can take hours or days. Edge or on-device deployment involves deploying models to edge devices, such as mobile phones or IoT sensors, which can reduce the network overhead but also limit the computational power and model size. https://docs.databricks.com/en/machine-learning/ml-tutorials.html

The correct way to log a scikit-learn model using MLflow is to use the mlflow.sklearn.log_model function, which logs the model as an MLflow artifact for the current run1. This function takes the scikit-learn model object and an artifact path as required arguments, and optionally accepts other arguments such as conda environment, code paths, serialization format, etc2. The other options are incorrect because they either use the wrong flavor (spark instead of sklearn), the wrong function (load_model or track_model instead of log_model), or do not exist (track_model is not a valid MLflow function). References: MLflow Models, mlflow.sklearn

To view all of the active MLflow Model Registry Webhooks for a specific model, the machine learning engineer needs to replace POST with GET in the call to http request. This is because the HTTP method for listing webhooks is GET, not POST. The GET method is used to retrieve information from the server, while the POST method is used to send data to the server1. The syntax of the http request function is as follows:

Python

http_request(host_creds, endpoint, method, json=None, headers=None)

AI-generated code. Review and use carefully. More info on FAQ.

The other options are incorrect because:

• Option A: There are no necessary changes is incorrect, as the code block will not work with the POST method.
• Option B: Replace list with view in the endpoint URL is incorrect, as the endpoint URL for listing webhooks is “/api/2.0/mlflow/registry/webhooks/list?model_name=model”, not “/api/2.0/mlflow/registry/webhooks/view?model_name=model”. There is no view endpoint for webhooks2.
• Option D: Replace list with webhooks in the endpoint URL is incorrect, as the endpoint URL for listing webhooks is “/api/2.0/mlflow/registry/webhooks/list?model_name=model”, not “/api/2.0/mlflow/registry/webhooks/webhooks?model_name=model”. The list keyword is required to specify the list operation2.
• Option E: Replace POST with PUT in the call to http request is incorrect, as the HTTP method for listing webhooks is GET, not PUT. The PUT method is used to update data on the server, not to retrieve information1. References: HTTP request methods, List webhooks

A two-way Chi-squared test is a statistical test that can be used to compare the frequencies of categorical variables across two or more groups. In this case, the engineer can use a two-way Chi-squared test to compare the frequencies of the categorical input variable values across different time periods (e.g., months, quarters, etc.) and see if there is a significant difference in the proportion of missing values. A significant difference would indicate that the missing values are not randomly distributed and may affect the performance of the machine learning model. References:

• Databricks Machine Learning Professional Exam Guide, Section 4: Solution and Data Monitoring, p. 5
• Databricks Machine Learning Documentation, Monitoring ML Models, Data Drift Detection, p. 3

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It allows users to express streaming computations the same way as batch computations on static data, using DataFrame and Dataset APIs. Structured Streaming can handle both unbounded and bounded data sources, and can process data in micro-batches or continuously. Structured Streaming can be used to provide continuous processing of data for machine learning applications, such as data ingestion, preprocessing, feature engineering, and model inference. StructuredStreaming can also integrate with MLflow and Delta Lake to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance123 References:

• Structured Streaming Programming Guide - Spark 3.2.0 Documentation
• Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 3.2.0 Documentation
• Machine Learning with Structured Streaming - Databricks
• [Continuous Machine Learning with Structured Streaming and MLflow - Databricks]