Flask

Flask is a lightweight web framework written in python which makes it easier to get started with a web application and also supports extensions to build complex applications..

Let’s divide the entire process into 2 steps:

1. Train a model


2. Deploy the trained model using flask

1. Train a model

Let us build an image classification model using keras to identify a specific type of cactus in aerial imagery. Our model should be able to identify whether a given image contains cactus plant. More details about this dataset can be found at the research paper: https://doi.org/10.1016/j.ecoinf.2019.05.005

Getting Started

Keras is a high-level neural network library that runs on top of tensorflow. It was developed with a focus on enabling fast experimentation. ( https://keras.io/ )

The dataset has 2 folders training and validation. Each of the folders contain a folder with cactus images and other folder with non-cactus images. There are 17500 images for training and 4000 images for testing. Keras has ImageDataGenerator function to load images in batches from the source folders and do the necessary  transformations. This is more useful when there is not enough memory to load all the images at once.

from keras.preprocessing.image import ImageDataGenerator



 



batch_size = 30



# this is the augmentation configuration we will use for training

train_datagen = ImageDataGenerator(

           rescale=1./255

           )



# this is the augmentation configuration we will use for testing:

test_datagen = ImageDataGenerator(

           rescale=1./255

           )



# this is a generator that will read pictures found in

# subfolers of 'data/train', and indefinitely generate

# batches of augmented image data

train_generator = train_datagen.flow_from_directory(

           'training_set',  # this is the target directory

           target_size=(100, 100),  # all images will be resized to 150x150

           color_mode="rgb",

           batch_size=batch_size,

           class_mode='binary')  

# this is a similar generator, for validation data

validation_generator = test_datagen.flow_from_directory(

           'validation_set',

           target_size=(100, 100),

           color_mode="rgb",

           batch_size=batch_size,

           class_mode='binary')

flow_from_directory:

• The flow_from_directory takes a source folder as an argument and categorizes all the sub-folders as an individual class. Sub-folders in our training data are ‘cactus’ and no_cactus. So, all the images inside ‘cactus’ folder are given a label 0 and all the images inside ‘no_cactus’ are given a label of 1.


• The class_mode is ‘binary’ as there are only 2 classes here.


• Target_size converts all the images to single desired size


• Batch_size is how many images we want to load at once.


• Color_mode is ‘rgb’ as our images are colour images.

CNN

Convolutional Neural Network(CNN) is a type of deep neural network that is most commonly used for image classification. Let’s create a CNN in keras and train on the images.

from keras.models import Sequential

from keras.layers import Dense, Dropout, Flatten

from keras.layers import Conv2D, MaxPooling2D

import keras





model = Sequential()

model.add(Conv2D(32, kernel_size=(3, 3),

                    activation='relu',

                    input_shape=train_generator.image_shape))

model.add(Conv2D(64, (3, 3), activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2), padding='valid'))

model.add(Dropout(0.25))

model.add(Flatten())

model.add(Dense(128, activation='relu'))

model.add(Dropout(0.5))

model.add(Dense(1, activation='sigmoid'))



model.compile(loss=keras.losses.binary_crossentropy,

                 optimizer=keras.optimizers.Adam(),

                 metrics=['accuracy'])

 Sequential: The sequential model allows to create a model in a layer by layer structure. Each layer is connected to the previous layer and the next layer.

Conv2D: This layer creates a convolution kernel that is convolved with the layer input to produce a tensor of outputs. If it is the first layer of a model, it takes an argument input_shape which is the image shape and format. Some other arguments are

• filters: Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution)


• kernel_size: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window


• activation: Activation function to use

MaxPooling2D: The role of max pooling is to down sample and reduce the dimensionality of features. The argument pool_size is an integer or tuple of 2 integers, factors by which to downscale (vertical, horizontal)

Dropout: The dropout is a technique used to address the problem of overfitting. Few units are randomly dropped from the network during training to prevent complex co-adaptations.

Flatten: The flatten layer converts entire data to a one dimensional array.

Dense: Each neuron of a dense layer receives input from all the neurons in the previous layer.

Activation: The activation function of a node defines the output of the node. It is important for non-linear properties in the network. Generally, in the last layer of a network, sigmoid activation is used for binary classification and softmax for multi-class classification.

Loss function: A loss function evaluates how well a model is performing by comparing predicted and actual values. During training, the model improves by trying to minimize the loss function.

Optimizer: An optimizer is used to decide how fast the weights of a network can be updated (learning rate) during training. Some of the optimizers are sgd (Stochastic Gradient Descent), Adam, Adagrad, RMSprop.

Metrics: The metrics is used to observe how model loss or accuracy changes with each epoch.

model_history = model.fit_generator(

                   train_generator,

                   steps_per_epoch= train_generator.labels.size//batch_size,

                   epochs=5,

                   validation_data=validation_generator,

                   validation_steps=validation_generator.labels.size//batch_size

                                       )

fit_generator: The train_generator is passed to the fit_generator to train the model and the accuracy is measured on validation_generator after each epoch.

steps_per_epoch: The steps per epoch should be total no. of images divided by batch_size so that each image is passed once to the model per each epoch. Suppose we have 200 training images and batch size = 50, then steps per epoch = 200/50 = 4. So, first 1-50 images are passed, then 51-100, 101-150, 151-200 which completes one epoch.

After training model for 5 epochs, an accuracy of 92.2% is achieved, it can be further improved by fine tuning the model.

Save the model so that it can be used in the flask application we are going to create.

model.save('model1.h5')

2. Create Flask App

Now, let’s create a flask application in windows to allow users to identify whether an image contains a cactus plant using the model we built above.

Virtual environment

Open terminal, go to the desired location and create a folder to host the app

C:\> mkdir aerial_cactus

C:\> cd aerial_cactus

Create a virtual environment, activate it. Virtual environment keeps the versions of flask and other libraries independent of the other projects. It is not mandatory to create a virtual environment but it is a good practice.

C:\aerial_cactus> python -m venv venv

C:\aerial_cactus> venv\Scripts\activate

(venv) C:\aerial_cactus> 

NOTE: In linux/mac os, instead of ‘venv\Scripts\activate’, use the following,

source venv/bin/activate

Creating virtual environment is only for the first time, later it is activated directly by venv\Scripts\activate.

Libraries

We need to install all the required packages/libraries before running flask by using ‘pip install <pkg name>’. An easy way to install all the required packages at once is

C:\aerial_cactus>pip install -r requirements.txt

Where requirements.txt file is provided in the project. If you created a project and want to make it easier for the others who use the project, create requirements.txt file which consists of all the necessary packages by using the below command.

C:\aerial_cactus>pip freeze > requirements.txt

If there is an error like “no module named <library name>“, it means the library is missing, it can be installed by “pip install <library name>”

Templates

Create templates directory to store the html files

(venv) C:\aerial_cactus> mkdir templates

Flask has flask_bootstrap and jinja2 template support which makes it easier to create and style html pages. Most of the pages of a website have portions which are similar, like the top bar and bottom bar of all pages of a website are always the same. So, instead of writing code for that similar parts in every html page, we can create a base page and extend the remaining pages from this base page.

Create base.html and index.html file inside templates folder to provide the users a form to upload image and allow our model to predict.

base.html

{% extends 'bootstrap/base.html' %}



{% block title %}

        {{ Welcome to Cactus Prediction }}

{% endblock %}



{% block navbar %}

        <nav class="navbar navbar-default">

           <div class="container">

               <div class="navbar-header">

                   <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false">

                       <span class="sr-only">Toggle navigation</span>

                       <span class="icon-bar"></span>

                       <span class="icon-bar"></span>

                       <span class="icon-bar"></span>

                   </button>

                   <a class="navbar-brand" href="{{ url_for('predict') }}">Cactus Prediction</a>

               </div>

           </div>

        </nav>

{% endblock %}

 index.html

{% extends "base.html" %}

{% import 'bootstrap/wtf.html' as wtf %}



{% block content %}

           

           {% if form %}

                   <div class="col-md-4 col-lg-offset-4">

                               {{ wtf.quick_form(form) }}

                   </div>

           {% endif %}

   

{% endblock %}

Create result.html to display the result of the prediction.

result.html

{% extends "base.html" %}



{% block content %}

           

           <div class="col-md-4 col-lg-offset-4">

               {% if result == "CACTUS" %}

                           <h4 style="color:green;">The image uploaded has a cactus plant</h1>

                   {% elif result == "NOT CACTUS" %}

                           <h4 style="color:red;">The image uploaded does not have a cactus plant</h1>

                   {% endif %}

           </div>

           

           <div class="col-md-4 col-lg-offset-4">

                   <img src="data:image/jpg;base64, {{encoded_photo | safe}}" width="50%">

           </div>

   

{% endblock %}

Main.py

Create a new directory ‘models’ and store the trained model in it. Create main.py file in the project root folder (aerial_cactus). This is the file which initializes and runs the flask application.

from flask import Flask, render_template, request

from io import BytesIO

from keras.preprocessing import image

from keras.preprocessing.image import array_to_img, img_to_array

from keras.models import load_model

import os

from PIL import Image

import numpy as np

from base64 import b64encode



from flask_bootstrap import Bootstrap

from flask_wtf import FlaskForm

from flask_wtf.file import FileField, FileRequired, FileAllowed

from wtforms import SubmitField



# code which helps initialize our server

app = Flask(__name__)

app.config['SECRET_KEY'] = 'any secret key'



bootstrap = Bootstrap(app)



saved_model = load_model("models/model1.h5")

saved_model._make_predict_function()



class UploadForm(FlaskForm):

   photo = FileField('Upload an image',validators=[FileAllowed(['jpg', 'png', 'jpeg'], u'Image only!'), FileRequired(u'File was empty!')])

   submit = SubmitField(u'Predict')



def preprocess(img):

   width, height = img.shape[0], img.shape[1]

   img = image.array_to_img(img, scale=False)



   # Crop 48x48px

   desired_width, desired_height = 100, 100



   if width < desired_width:

           desired_width = width

   start_x = np.maximum(0, int((width-desired_width)/2))



   img = img.crop((start_x, np.maximum(0, height-desired_height), start_x+desired_width, height))

   img = img.resize((100, 100))



   img = image.img_to_array(img)

   return img / 255.



@app.route('/', methods=['GET','POST'])

def predict():

   form = UploadForm()

   if form.validate_on_submit():

           print(form.photo.data)

           image_stream = form.photo.data.stream

           original_img = Image.open(image_stream)

           img = image.img_to_array(original_img)

           img = preprocess(img)

           img = np.expand_dims(img, axis=0)

           prediction = saved_model.predict_classes(img)



           if (prediction[0][0]==0):

                   result = "CACTUS"

           else:

                   result = "NOT CACTUS"



           byteIO = BytesIO()

           original_img.save(byteIO, format=original_img.format)

           byteArr = byteIO.getvalue()

           encoded = b64encode(byteArr)



           return render_template('result.html', result=result, encoded_photo=encoded.decode('ascii'))



   return render_template('index.html', form=form)



if __name__ == '__main__':

   app.run(debug=True)

Bootstrap(app): This enables the html pages to use the flask bootstrap extension for styling.

_make_predict_function: This builds the predict() function ahead of time and makes it ready to work when it is called from several threads. (https://github.com/keras-team/keras/issues/6124)

UploadForm: Instead of creating our own html forms, flask provides Flask_Form which can be passed to the template. Here we are creating a form to get an image from the user.

Preprocess: The preprocess(image) function is required as the user provided image should be resized and scaled in the same way as the training set images

@app.route('/', methods=['GET','POST']): This indicates that the homepage of our app has both get, post methods. The user enters the homepage url, the predict() function runs and loads the index.html template. When the user uploads an image and clicks predict, the form is validated and the remaining steps are executed.

if __name__ == '__main__':         

app.run(debug=True)

The above command makes sure that the file main.py is run only when it is explicitly called but not when it is called as an import in other file. The ‘debug=True’ is useful when the app is in development mode, it should be False when it is deployed to production server.

Run the application

(venv) C:\aerial_cactus> set FLASK_APP=main.py

(venv) C:\aerial_cactus> flask run

NOTE: In linux/mac os, instead of the above commands, you should execute

(venv) C:\aerial_cactus> export FLASK_APP=main.py

(venv) C:\aerial_cactus> flask run

(venv) C:\aerial_cactus>flask run

* Serving Flask app "main.py"

* Environment: production

  WARNING: Do not use the development server in a production environment.

  Use a production WSGI server instead.

* Debug mode: off

Using TensorFlow backend.

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

As shown above, the website can be accessed now at http://127.0.0.1:5000/ 

When we enter the URL, the app goes to the homepage route (app.route(‘/)). As this is a GET request, the predict function at this route renders index.html page. So, the index.html is loaded with the help of base.html and bootstrap.

Browse and select an image and click predict to get the prediction.

What happens when we click predict?

• The app first checks whether the URL has a ‘POST’ method. If it has that method, then it proceeds to the next step, otherwise it throws an error.


• Next, it validates the form. We made sure that the jpg, jpeg, png are the required file formats, so if a file with other format is uploaded or no file is uploaded, it throws an error.

• The image from the local file system is converted to data stream. Image.open() from PIL library is used to open the image and then the image is converted to an array


• The preprocess function is applied on the image to resize and scale the image and dimensions are extended in the way the model expects its input.


• The predict function is used to get the prediction from the saved model and the result is stored in the variable ‘prediction’. If the prediction[0][0] == 0, the variable ‘result’ is assigned a value ‘CACTUS’,         else it is assigned a value ‘NOT CACTUS’.


• In order to display the image uploaded by the user, it should be sent to the result.html. So, the image is converted to base64 encoding form by using BytesIO, b64encode and then passed to the result.html along with the ‘result’ variable.


• When the result.html is rendered, relevant text is displayed based on the value of ‘result’ variable. The image is generated from the encoded string and displayed to the user.

The application can be further extended so that instead of uploading an image, a url of an image is provided.

The code is also available at Github: https://github.com/DatascienceAuthority/Deploy-keras-model-with-flask