Table of Contents

1. Save / Load the best model
2. Put it into a Webservice → Flask
3. Add requirements, model dependencies
4. Add system dependencies, Docker
5. Deploy via AWS Elastic Beanstalk
6. Additional Methods(i.e. deploy via GCP, Azure, Heroku, etc, or use FastAPI to create webservices)

1. Saving / Loading the Model — Model Persistence

• Use pickle to save / load the model after building it on sklearn / pytorch / tensorflow

import pickle 

# Train a Random Forest Classifier
rf_classifier = RandomForestClassifier(n_estimators=100, random_state=42)
rf_classifier.fit(X_train, y_train)

# Make predictions on the test set
y_pred = rf_classifier.predict(X_test)

# Save the model using pickle 
# give the file a name
model_filename = 'rf_classifier_iris_version1.pkl'
try:
    with open(model_filename, 'wb') as file:
        pickle.dump(rf_classifier, file)
    print(f"Model saved successfully to {model_filename}")
except Exception as e:
    print(f"Error saving model: {e}")

# Load the model using pickle
try:
    with open(model_filename, 'rb') as file:
        loaded_model = pickle.load(file)
    print(f"Model loaded successfully from {model_filename}")
except Exception as e:
    print(f"Error loading model: {e}")

• You can also use MLflow to save/load models

import mlflow
import mlflow.sklearn
import shutil

model_path = "models/logit_games_v1"
#shutil.rmtree(model_path)
mlflow.sklearn.save_model(model, model_path)

loaded = mlflow.sklearn.load_model(model_path)
print(loaded.predict_proba(x))

# You can also do this for deep learning models 
import tensorflow as tf
import keras
from keras import models, layers

# define the network structure 
model = models.Sequential()
model.add(layers.Dense(64,activation='relu',input_shape=(10,)))
model.add(layers.Dropout(0.1))
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

def auc(y_true, y_pred):
    auc = tf.metrics.auc(y_true, y_pred)[1]
    keras.backend.get_session().run(
                  tf.local_variables_initializer())
    return auc
    
model.compile(optimizer='rmsprop',
                 loss='binary_crossentropy', metrics=[tf.keras.metrics.AUC()])
history = model.fit(x, y, epochs=100, batch_size=100, 
                 validation_split = .2, verbose=0)
                 
# SAVE & LOAD the deep learning model in .h5 format                  
from keras.models import load_model
model.save("games.h5")

model = load_model('games.h5', custom_objects={'auc': auc})
model.evaluate(x, y, verbose = 0)

2. Creating a Webservice w/ Flask

• What is a Webservice?
  • Essentially it’s just a subset of an API which allows machines to communicate w/ each other over a network using a Protocol (HTPP, SOAP, etc. )
  • Thankfully most of it is covered by exsisting libraries, like Flask, FastAPI, etc for development purposes.
  • These libraries contain a PART of a webserver, which allows us to develop with it in development. When they’re integrated into your WebAPplication, we move onto serving with Gunicorn, and use other web-servers (like Nginx, Apache, etc) which handle security, load balancing, scalability, handling multiple requests, etc. More information on this below.

# Define a simple webservice: Ping -> Pong

from flask import Flask
from flask import request

# Define a simple Flask web application with the name "ping"
app = Flask("ping")

# Create a GET route for '/ping' that returns "PONG"
@app.route('/ping', methods=["GET"])
def ping():
    return "PONG"

# Create a POST route for '/pingme' that echoes back the JSON data sent in the request
@app.route('/pingme', methods=["POST"])
def pingme():
    data = request.get_json()
    return data

# Run the Flask application if this script is executed directly
if __name__ == "__main__":
		# Run the app on localhost, port 1010.
		# Exposes both endpoints, /ping and /pingme
		app.run(debug=True, host='0.0.0.0', port=1010)

• You can access the endpoints via localhost, or via curl

# See whats up at local host ping
curl <http://localhost:1010/ping>

# Send some data to our app
curl -X POST -H "Content-Type: application/json" -d '{"message": "Hello, Flask!"}' <http://localhost:1010/pingme>

Lets try this with a real-world example.

Suppose you have trained your model and are now ready to make it live to make inference.

Well, you save your model from Step1, then download Flask and create your predict.py file.