Tutorial: Use automated machine learning to predict taxi fares

In this tutorial, you use automated machine learning in Azure Machine Learning to create a regression model to predict NYC taxi fare prices. This process accepts training data and configuration settings, and automatically iterates through combinations of different feature normalization/standardization methods, models, and hyperparameter settings to arrive at the best model.

Flow diagram

In this tutorial you learn the following tasks:

  • Download, transform, and clean data using Azure Open Datasets
  • Train an automated machine learning regression model
  • Calculate model accuracy

If you don’t have an Azure subscription, create a free account before you begin. Try the free or paid version of Azure Machine Learning today.

Prerequisites

  • Complete the setup tutorial if you don't already have an Azure Machine Learning workspace or notebook virtual machine.
  • After you complete the setup tutorial, open the tutorials/regression-automated-ml.ipynb notebook using the same notebook server.

This tutorial is also available on GitHub if you wish to run it in your own local environment. Run pip install azureml-sdk[automl] azureml-opendatasets azureml-widgets to get the required packages.

Download and prepare data

Import the necessary packages. The Open Datasets package contains a class representing each data source (NycTlcGreen for example) to easily filter date parameters before downloading.

from azureml.opendatasets import NycTlcGreen
import pandas as pd
from datetime import datetime
from dateutil.relativedelta import relativedelta

Begin by creating a dataframe to hold the taxi data. When working in a non-Spark environment, Open Datasets only allows downloading one month of data at a time with certain classes to avoid MemoryError with large datasets.

To download taxi data, iteratively fetch one month at a time, and before appending it to green_taxi_df randomly sample 2,000 records from each month to avoid bloating the dataframe. Then preview the data.

green_taxi_df = pd.DataFrame([])
start = datetime.strptime("1/1/2015","%m/%d/%Y")
end = datetime.strptime("1/31/2015","%m/%d/%Y")

for sample_month in range(12):
    temp_df_green = NycTlcGreen(start + relativedelta(months=sample_month), end + relativedelta(months=sample_month)) \
        .to_pandas_dataframe()
    green_taxi_df = green_taxi_df.append(temp_df_green.sample(2000))

green_taxi_df.head(10)
vendorID lpepPickupDatetime lpepDropoffDatetime passengerCount tripDistance puLocationId doLocationId pickupLongitude pickupLatitude dropoffLongitude ... paymentType fareAmount extra mtaTax improvementSurcharge tipAmount tollsAmount ehailFee totalAmount tripType
131969 2 2015-01-11 05:34:44 2015-01-11 05:45:03 3 4.84 None None -73.88 40.84 -73.94 ... 2 15.00 0.50 0.50 0.3 0.00 0.00 nan 16.30 1.00
1129817 2 2015-01-20 16:26:29 2015-01-20 16:30:26 1 0.69 None None -73.96 40.81 -73.96 ... 2 4.50 1.00 0.50 0.3 0.00 0.00 nan 6.30 1.00
1278620 2 2015-01-01 05:58:10 2015-01-01 06:00:55 1 0.45 None None -73.92 40.76 -73.91 ... 2 4.00 0.00 0.50 0.3 0.00 0.00 nan 4.80 1.00
348430 2 2015-01-17 02:20:50 2015-01-17 02:41:38 1 0.00 None None -73.81 40.70 -73.82 ... 2 12.50 0.50 0.50 0.3 0.00 0.00 nan 13.80 1.00
1269627 1 2015-01-01 05:04:10 2015-01-01 05:06:23 1 0.50 None None -73.92 40.76 -73.92 ... 2 4.00 0.50 0.50 0 0.00 0.00 nan 5.00 1.00
811755 1 2015-01-04 19:57:51 2015-01-04 20:05:45 2 1.10 None None -73.96 40.72 -73.95 ... 2 6.50 0.50 0.50 0.3 0.00 0.00 nan 7.80 1.00
737281 1 2015-01-03 12:27:31 2015-01-03 12:33:52 1 0.90 None None -73.88 40.76 -73.87 ... 2 6.00 0.00 0.50 0.3 0.00 0.00 nan 6.80 1.00
113951 1 2015-01-09 23:25:51 2015-01-09 23:39:52 1 3.30 None None -73.96 40.72 -73.91 ... 2 12.50 0.50 0.50 0.3 0.00 0.00 nan 13.80 1.00
150436 2 2015-01-11 17:15:14 2015-01-11 17:22:57 1 1.19 None None -73.94 40.71 -73.95 ... 1 7.00 0.00 0.50 0.3 1.75 0.00 nan 9.55 1.00
432136 2 2015-01-22 23:16:33 2015-01-22 23:20:13 1 0.65 None None -73.94 40.71 -73.94 ... 2 5.00 0.50 0.50 0.3 0.00 0.00 nan 6.30 1.00

10 rows × 23 columns

Now that the initial data is loaded, define a function to create various time-based features from the pickup datetime field. This will create new fields for the month number, day of month, day of week, and hour of day, and will allow the model to factor in time-based seasonality. Use the apply() function on the dataframe to iteratively apply the build_time_features() function to each row in the taxi data.

def build_time_features(vector):
    pickup_datetime = vector[0]
    month_num = pickup_datetime.month
    day_of_month = pickup_datetime.day
    day_of_week = pickup_datetime.weekday()
    hour_of_day = pickup_datetime.hour

    return pd.Series((month_num, day_of_month, day_of_week, hour_of_day))

green_taxi_df[["month_num", "day_of_month","day_of_week", "hour_of_day"]] = green_taxi_df[["lpepPickupDatetime"]].apply(build_time_features, axis=1)
green_taxi_df.head(10)
vendorID lpepPickupDatetime lpepDropoffDatetime passengerCount tripDistance puLocationId doLocationId pickupLongitude pickupLatitude dropoffLongitude ... improvementSurcharge tipAmount tollsAmount ehailFee totalAmount tripType month_num day_of_month day_of_week hour_of_day
131969 2 2015-01-11 05:34:44 2015-01-11 05:45:03 3 4.84 None None -73.88 40.84 -73.94 ... 0.3 0.00 0.00 nan 16.30 1.00 1 11 6 5
1129817 2 2015-01-20 16:26:29 2015-01-20 16:30:26 1 0.69 None None -73.96 40.81 -73.96 ... 0.3 0.00 0.00 nan 6.30 1.00 1 20 1 16
1278620 2 2015-01-01 05:58:10 2015-01-01 06:00:55 1 0.45 None None -73.92 40.76 -73.91 ... 0.3 0.00 0.00 nan 4.80 1.00 1 1 3 5
348430 2 2015-01-17 02:20:50 2015-01-17 02:41:38 1 0.00 None None -73.81 40.70 -73.82 ... 0.3 0.00 0.00 nan 13.80 1.00 1 17 5 2
1269627 1 2015-01-01 05:04:10 2015-01-01 05:06:23 1 0.50 None None -73.92 40.76 -73.92 ... 0 0.00 0.00 nan 5.00 1.00 1 1 3 5
811755 1 2015-01-04 19:57:51 2015-01-04 20:05:45 2 1.10 None None -73.96 40.72 -73.95 ... 0.3 0.00 0.00 nan 7.80 1.00 1 4 6 19
737281 1 2015-01-03 12:27:31 2015-01-03 12:33:52 1 0.90 None None -73.88 40.76 -73.87 ... 0.3 0.00 0.00 nan 6.80 1.00 1 3 5 12
113951 1 2015-01-09 23:25:51 2015-01-09 23:39:52 1 3.30 None None -73.96 40.72 -73.91 ... 0.3 0.00 0.00 nan 13.80 1.00 1 9 4 23
150436 2 2015-01-11 17:15:14 2015-01-11 17:22:57 1 1.19 None None -73.94 40.71 -73.95 ... 0.3 1.75 0.00 nan 9.55 1.00 1 11 6 17
432136 2 2015-01-22 23:16:33 2015-01-22 23:20:13 1 0.65 None None -73.94 40.71 -73.94 ... 0.3 0.00 0.00 nan 6.30 1.00 1 22 3 23

10 rows × 27 columns

Remove some of the columns that you won't need for training or additional feature building.

columns_to_remove = ["lpepPickupDatetime", "lpepDropoffDatetime", "puLocationId", "doLocationId", "extra", "mtaTax",
                     "improvementSurcharge", "tollsAmount", "ehailFee", "tripType", "rateCodeID",
                     "storeAndFwdFlag", "paymentType", "fareAmount", "tipAmount"
                    ]
for col in columns_to_remove:
    green_taxi_df.pop(col)

green_taxi_df.head(5)

Cleanse data

Run the describe() function on the new dataframe to see summary statistics for each field.

green_taxi_df.describe()
vendorID passengerCount tripDistance pickupLongitude pickupLatitude dropoffLongitude dropoffLatitude totalAmount month_num day_of_month day_of_week hour_of_day
count 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00 48000.00
mean 1.78 1.37 2.87 -73.83 40.69 -73.84 40.70 14.75 6.50 15.13 3.27 13.52
std 0.41 1.04 2.93 2.76 1.52 2.61 1.44 12.08 3.45 8.45 1.95 6.83
min 1.00 0.00 0.00 -74.66 0.00 -74.66 0.00 -300.00 1.00 1.00 0.00 0.00
25% 2.00 1.00 1.06 -73.96 40.70 -73.97 40.70 7.80 3.75 8.00 2.00 9.00
50% 2.00 1.00 1.90 -73.94 40.75 -73.94 40.75 11.30 6.50 15.00 3.00 15.00
75% 2.00 1.00 3.60 -73.92 40.80 -73.91 40.79 17.80 9.25 22.00 5.00 19.00
max 2.00 9.00 97.57 0.00 41.93 0.00 41.94 450.00 12.00 30.00 6.00 23.00

From the summary statistics, you see that there are several fields that have outliers or values that will reduce model accuracy. First filter the lat/long fields to be within the bounds of the Manhattan area. This will filter out longer taxi trips or trips that are outliers in respect to their relationship with other features.

Additionally filter the tripDistance field to be greater than zero but less than 31 miles (the haversine distance between the two lat/long pairs). This eliminates long outlier trips that have inconsistent trip cost.

Lastly, the totalAmount field has negative values for the taxi fares, which don't make sense in the context of our model, and the passengerCount field has bad data with the minimum values being zero.

Filter out these anomalies using query functions, and then remove the last few columns unnecessary for training.

final_df = green_taxi_df.query("pickupLatitude>=40.53 and pickupLatitude<=40.88")
final_df = final_df.query("pickupLongitude>=-74.09 and pickupLongitude<=-73.72")
final_df = final_df.query("tripDistance>=0.25 and tripDistance<31")
final_df = final_df.query("passengerCount>0 and totalAmount>0")

columns_to_remove_for_training = ["pickupLongitude", "pickupLatitude", "dropoffLongitude", "dropoffLatitude"]
for col in columns_to_remove_for_training:
    final_df.pop(col)

Call describe() again on the data to ensure cleansing worked as expected. You now have a prepared and cleansed set of taxi, holiday, and weather data to use for machine learning model training.

final_df.describe()

Configure workspace

Create a workspace object from the existing workspace. A Workspace is a class that accepts your Azure subscription and resource information. It also creates a cloud resource to monitor and track your model runs. Workspace.from_config() reads the file config.json and loads the authentication details into an object named ws. ws is used throughout the rest of the code in this tutorial.

from azureml.core.workspace import Workspace
ws = Workspace.from_config()

Split the data into train and test sets

Split the data into training and test sets by using the train_test_split function in the scikit-learn library. This function segregates the data into the x (features) data set for model training and the y (values to predict) data set for testing.

The test_size parameter determines the percentage of data to allocate to testing. The random_state parameter sets a seed to the random generator, so that your train-test splits are deterministic.

from sklearn.model_selection import train_test_split

y_df = final_df.pop("totalAmount")
x_df = final_df

x_train, x_test, y_train, y_test = train_test_split(x_df, y_df, test_size=0.2, random_state=223)

The purpose of this step is to have data points to test the finished model that haven't been used to train the model, in order to measure true accuracy.

In other words, a well-trained model should be able to accurately make predictions from data it hasn't already seen. You now have data prepared for auto-training a machine learning model.

Automatically train a model

To automatically train a model, take the following steps:

  1. Define settings for the experiment run. Attach your training data to the configuration, and modify settings that control the training process.
  2. Submit the experiment for model tuning. After submitting the experiment, the process iterates through different machine learning algorithms and hyperparameter settings, adhering to your defined constraints. It chooses the best-fit model by optimizing an accuracy metric.

Define training settings

Define the experiment parameter and model settings for training. View the full list of settings. Submitting the experiment with these default settings will take approximately 5-10 min, but if you want a shorter run time, reduce the iterations parameter.

Property Value in this tutorial Description
iteration_timeout_minutes 2 Time limit in minutes for each iteration. Reduce this value to decrease total runtime.
iterations 20 Number of iterations. In each iteration, a new machine learning model is trained with your data. This is the primary value that affects total run time.
primary_metric spearman_correlation Metric that you want to optimize. The best-fit model will be chosen based on this metric.
preprocess True By using True, the experiment can preprocess the input data (handling missing data, converting text to numeric, etc.)
verbosity logging.INFO Controls the level of logging.
n_cross_validations 5 Number of cross-validation splits to perform when validation data is not specified.
import logging

automl_settings = {
    "iteration_timeout_minutes": 2,
    "iterations": 20,
    "primary_metric": 'spearman_correlation',
    "preprocess": True,
    "verbosity": logging.INFO,
    "n_cross_validations": 5
}

Use your defined training settings as a **kwargs parameter to an AutoMLConfig object. Additionally, specify your training data and the type of model, which is regression in this case.

from azureml.train.automl import AutoMLConfig

automl_config = AutoMLConfig(task='regression',
                             debug_log='automated_ml_errors.log',
                             X=x_train.values,
                             y=y_train.values.flatten(),
                             **automl_settings)

Note

Automated machine learning pre-processing steps (feature normalization, handling missing data, converting text to numeric, etc.) become part of the underlying model. When using the model for predictions, the same pre-processing steps applied during training are applied to your input data automatically.

Train the automatic regression model

Create an experiment object in your workspace. An experiment acts as a container for your individual runs. Pass the defined automl_config object to the experiment, and set the output to True to view progress during the run.

After starting the experiment, the output shown updates live as the experiment runs. For each iteration, you see the model type, the run duration, and the training accuracy. The field BEST tracks the best running training score based on your metric type.

from azureml.core.experiment import Experiment
experiment = Experiment(ws, "taxi-experiment")
local_run = experiment.submit(automl_config, show_output=True)
Running on local machine
Parent Run ID: AutoML_1766cdf7-56cf-4b28-a340-c4aeee15b12b
Current status: DatasetFeaturization. Beginning to featurize the dataset.
Current status: DatasetEvaluation. Gathering dataset statistics.
Current status: FeaturesGeneration. Generating features for the dataset.
Current status: DatasetFeaturizationCompleted. Completed featurizing the dataset.
Current status: DatasetCrossValidationSplit. Generating individually featurized CV splits.
Current status: ModelSelection. Beginning model selection.

****************************************************************************************************
ITERATION: The iteration being evaluated.
PIPELINE: A summary description of the pipeline being evaluated.
DURATION: Time taken for the current iteration.
METRIC: The result of computing score on the fitted pipeline.
BEST: The best observed score thus far.
****************************************************************************************************

 ITERATION   PIPELINE                                       DURATION      METRIC      BEST
         0   StandardScalerWrapper RandomForest             0:00:16       0.8746    0.8746
         1   MinMaxScaler RandomForest                      0:00:15       0.9468    0.9468
         2   StandardScalerWrapper ExtremeRandomTrees       0:00:09       0.9303    0.9468
         3   StandardScalerWrapper LightGBM                 0:00:10       0.9424    0.9468
         4   RobustScaler DecisionTree                      0:00:09       0.9449    0.9468
         5   StandardScalerWrapper LassoLars                0:00:09       0.9440    0.9468
         6   StandardScalerWrapper LightGBM                 0:00:10       0.9282    0.9468
         7   StandardScalerWrapper RandomForest             0:00:12       0.8946    0.9468
         8   StandardScalerWrapper LassoLars                0:00:16       0.9439    0.9468
         9   MinMaxScaler ExtremeRandomTrees                0:00:35       0.9199    0.9468
        10   RobustScaler ExtremeRandomTrees                0:00:19       0.9411    0.9468
        11   StandardScalerWrapper ExtremeRandomTrees       0:00:13       0.9077    0.9468
        12   StandardScalerWrapper LassoLars                0:00:15       0.9433    0.9468
        13   MinMaxScaler ExtremeRandomTrees                0:00:14       0.9186    0.9468
        14   RobustScaler RandomForest                      0:00:10       0.8810    0.9468
        15   StandardScalerWrapper LassoLars                0:00:55       0.9433    0.9468
        16   StandardScalerWrapper ExtremeRandomTrees       0:00:13       0.9026    0.9468
        17   StandardScalerWrapper RandomForest             0:00:13       0.9140    0.9468
        18   VotingEnsemble                                 0:00:23       0.9471    0.9471
        19   StackEnsemble                                  0:00:27       0.9463    0.9471

Explore the results

Explore the results of automatic training with a Jupyter widget. The widget allows you to see a graph and table of all individual run iterations, along with training accuracy metrics and metadata. Additionally, you can filter on different accuracy metrics than your primary metric with the dropdown selector.

from azureml.widgets import RunDetails
RunDetails(local_run).show()

Jupyter widget run details Jupyter widget plot

Retrieve the best model

Select the best model from your iterations. The get_output function returns the best run and the fitted model for the last fit invocation. By using the overloads on get_output, you can retrieve the best run and fitted model for any logged metric or a particular iteration.

best_run, fitted_model = local_run.get_output()
print(best_run)
print(fitted_model)

Test the best model accuracy

Use the best model to run predictions on the test data set to predict taxi fares. The function predict uses the best model and predicts the values of y, trip cost, from the x_test data set. Print the first 10 predicted cost values from y_predict.

y_predict = fitted_model.predict(x_test.values)
print(y_predict[:10])

Calculate the root mean squared error of the results. Convert the y_test dataframe to a list to compare to the predicted values. The function mean_squared_error takes two arrays of values and calculates the average squared error between them. Taking the square root of the result gives an error in the same units as the y variable, cost. It indicates roughly how far the taxi fare predictions are from the actual fares.

from sklearn.metrics import mean_squared_error
from math import sqrt

y_actual = y_test.values.flatten().tolist()
rmse = sqrt(mean_squared_error(y_actual, y_predict))
rmse

Run the following code to calculate mean absolute percent error (MAPE) by using the full y_actual and y_predict data sets. This metric calculates an absolute difference between each predicted and actual value and sums all the differences. Then it expresses that sum as a percent of the total of the actual values.

sum_actuals = sum_errors = 0

for actual_val, predict_val in zip(y_actual, y_predict):
    abs_error = actual_val - predict_val
    if abs_error < 0:
        abs_error = abs_error * -1

    sum_errors = sum_errors + abs_error
    sum_actuals = sum_actuals + actual_val

mean_abs_percent_error = sum_errors / sum_actuals
print("Model MAPE:")
print(mean_abs_percent_error)
print()
print("Model Accuracy:")
print(1 - mean_abs_percent_error)
Model MAPE:
0.14353867606052823

Model Accuracy:
0.8564613239394718

From the two prediction accuracy metrics, you see that the model is fairly good at predicting taxi fares from the data set's features, typically within +- $4.00, and approximately 15% error.

The traditional machine learning model development process is highly resource-intensive, and requires significant domain knowledge and time investment to run and compare the results of dozens of models. Using automated machine learning is a great way to rapidly test many different models for your scenario.

Clean up resources

Do not complete this section if you plan on running other Azure Machine Learning tutorials.

Stop the notebook VM

If you used a cloud notebook server, stop the VM when you are not using it to reduce cost.

  1. In your workspace, select Notebook VMs.
  2. From the list, select the VM.
  3. Select Stop.
  4. When you're ready to use the server again, select Start.

Delete everything

If you don't plan to use the resources you created, delete them, so you don't incur any charges.

  1. In the Azure portal, select Resource groups on the far left.
  2. From the list, select the resource group you created.
  3. Select Delete resource group.
  4. Enter the resource group name. Then select Delete.

You can also keep the resource group but delete a single workspace. Display the workspace properties and select Delete.

Next steps

In this automated machine learning tutorial, you did the following tasks:

  • Configured a workspace and prepared data for an experiment.
  • Trained by using an automated regression model locally with custom parameters.
  • Explored and reviewed training results.

Deploy your model with Azure Machine Learning.