Masks are a key measure to suppress transmission and save lives.
Masks should be used as part of a comprehensive ‘Do it all!’ approach including physical distancing, avoiding crowded, closed and close-contact settings, good ventilation, cleaning hands, covering sneezes and coughs, and more.
Depending on the type, masks can be used for either protection of healthy persons or to prevent onward transmission.
Why should we wear masks?
The World Health Organization (WHO) reports suggesting that the two main routes of transmission of the COVID-19 virus are respiratory droplets and physical contact.
Respiratory droplets are generated when an infected person coughs or sneezes. Any person in close contact (within 1 m) with someone who has respiratory symptoms (coughing, sneezing) is at risk of being exposed to potentially infective respiratory droplets.
Droplets may also land on surfaces where the virus could remain viable; thus, the immediate environment of an infected individual can serve as a source of transmission (contact transmission). Wearing a medical mask is one of the prevention measures that can limit the spread of certain respiratory viral diseases, including COVID-19.
Can masks prevent the transmission of Covid-19?
One category of evidence comes from laboratory studies of respiratory droplets and the ability of various masks to block them. An experiment using high-speed video found that hundreds of droplets ranging from 20 to 500 micrometres were generated when saying a simple phrase, but that nearly all these droplets were blocked when the mouth was covered by a damp washcloth. Another study of people who had influenza or the common cold found that wearing a surgical mask significantly reduced the amount of these respiratory viruses emitted in droplets and aerosols.
Another study looked at coronavirus deaths across 198 countries and found that those with cultural norms or government policies favouring mask-wearing had lower death rates.
Facts
In India, the first case of coronavirus was diagnosed on January 30. The Health Ministry recommended homemade face masks on April 4, 2020. However, mask wear was high both before and after the recommendation. According to one poll, masks were worn by 60% of the public from March 12-14, 67% from March 19-21, and then from 73% to 76% between March 26 through April 12. According to another poll, masks were worn by 43% of the public on March 16, 46% on March 20, 65% on March 27, 71% on April 3, 79% on April 10, and 81-84% between April 17 and May 1. A survey conducted in March 2020 found that 75% of the public believed that masks should be worn even by asymptomatic people, and 77% of respondents indicated that the N95 mask was most protective. By May 9, the per-capita mortality was 1.5 per million.
Objective & Who can use it?
To identify people in an image/video if they are wearing a mask or not using computer vision and deep learning algorithms.
What is the dataset used?
This dataset contains 853 images belonging to the 3 classes, and their bounding boxes in the PASCAL VOC format. The classes are:
Dataset credits: Kaggle
Preview of the dataset
mask worn incorrectly
without mask
with mask
PASCAL VOC format :
These files contain the coordinates of the bounding boxes, Labels present in the image.
Given an image, we can make bounding boxes and annotate them here and export them to Pascal Voc format.
Which model did we use?
Before we begin the setup, make sure to change the runtime type in Colab to GPU so that we can make use of the free GPU provided.
There are many models ready to download from the Tensorflow Model Zoo.
Be careful in choosing which model to use as some are not made for Object Detection. For this tutorial we will be using the following model:
SSD MobileNet V2 FPNLite 640x640.
Download it into your Colab Notebook.
Let's understand the code
Let's Implement all the required Libraries
Here, we are trying to automate the whole process so that when a user uploads the images with their annotations they will receive the frozen graph.
Here is my approach in order to automate the process.
How to upload and unzip the dataset?
We are uploading the dataset in a zip format to google colab and then unzip the folder so that we can use it.
dataset.zip contains 2 folders train and test.
We create some temporary folders in order to store the configuration file, the tfrecords, weights and everything required.
Defining The Paths
We define the paths of everything that we use in this project.
Downloading the model to google colab
Installing The Required Libraries
Let's install the required libraries.
Creating CSV and LabelMap
xml_to_csv.py contains:
"""
Usage:
# Create train data:
python xml_to_csv.py -i [PATH_TO_IMAGES_FOLDER]/train -o [PATH_TO_ANNOTATIONS_FOLDER]/train_labels.csv
# Create test data:
python xml_to_csv.py -i [PATH_TO_IMAGES_FOLDER]/test -o [PATH_TO_ANNOTATIONS_FOLDER]/test_labels.csv
"""
import os
import glob
import pandas as pd
import argparse
import xml.etree.ElementTree as ET
def xml_to_csv(path):
"""Iterates through all .xml files (generated by labelImg) in a given directory and combines them in a single Pandas datagrame.
Parameters:
----------
path : {str}
The path containing the .xml files
Returns
-------
Pandas DataFrame
The produced dataframe
"""
classes_names = []
xml_list = []
for xml_file in glob.glob(path + "/*.xml"):
tree = ET.parse(xml_file)
root = tree.getroot()
for member in root.findall("object"):
classes_names.append(member[0].text)
value = (
root.find("filename").text,
int(root.find("size")[0].text),
int(root.find("size")[1].text),
member[0].text,
int(member[4][0].text),
int(member[4][1].text),
int(member[4][2].text),
int(member[4][3].text),
)
xml_list.append(value)
column_name = [
"filename",
"width",
"height",
"class",
"xmin",
"ymin",
"xmax",
"ymax",
]
xml_df = pd.DataFrame(xml_list, columns=column_name)
classes_names = list(set(classes_names))
classes_names.sort()
return xml_df, classes_names
def main():
# Initiate argument parser
parser = argparse.ArgumentParser(
description="Sample TensorFlow XML-to-CSV converter"
)
parser.add_argument(
"-i",
"--inputDir",
help="Path to the folder where the input .xml files are stored",
type=str,
)
parser.add_argument(
"-o", "--outputFile", help="Name of output .csv file (including path)", type=str
)
parser.add_argument(
"-l",
"--labelMapDir",
help="Directory path to save label_map.pbtxt file is specified.",
type=str,
default="",
)
args = parser.parse_args()
if args.inputDir is None:
args.inputDir = os.getcwd()
if args.outputFile is None:
args.outputFile = args.inputDir + "/labels.csv"
assert os.path.isdir(args.inputDir)
os.makedirs(os.path.dirname(args.outputFile), exist_ok=True)
xml_df, classes_names = xml_to_csv(args.inputDir)
xml_df.to_csv(args.outputFile, index=None)
print("Successfully converted xml to csv.")
if args.labelMapDir:
os.makedirs(args.labelMapDir, exist_ok=True)
label_map_path = os.path.join(args.labelMapDir, "label_map.pbtxt")
print("Generate `{}`".format(label_map_path))
# Create the `label_map.pbtxt` file
pbtxt_content = ""
for i, class_name in enumerate(classes_names):
pbtxt_content = (
pbtxt_content
+ "item {{\n id: {0}\n name: '{1}'\n}}\n\n".format(
i + 1, class_name
)
)
pbtxt_content = pbtxt_content.strip()
with open(label_map_path, "w") as f:
f.write(pbtxt_content)
if __name__ == "__main__":
main()
Output of xml to csv:
Now from the csv file we take the unique class values and use them to create LabelMap.
Output of Label map:
{'name': 'with_mask', 'id': 1}
{'name': 'without_mask', 'id': 2}
{'name': 'mask_weared_incorrect', 'id': 3}Now let's create the tfrecords:
generate_tfrecord.py contains:
""" Sample TensorFlow XML-to-TFRecord converter
usage: generate_tfrecord.py [-h] [-x XML_DIR] [-l LABELS_PATH] [-o OUTPUT_PATH] [-i IMAGE_DIR] [-c CSV_PATH]
optional arguments:
-h, --help show this help message and exit
-x XML_DIR, --xml_dir XML_DIR
Path to the folder where the input .xml files are stored.
-l LABELS_PATH, --labels_path LABELS_PATH
Path to the labels (.pbtxt) file.
-o OUTPUT_PATH, --output_path OUTPUT_PATH
Path of output TFRecord (.record) file.
-i IMAGE_DIR, --image_dir IMAGE_DIR
Path to the folder where the input image files are stored. Defaults to the same directory as XML_DIR.
-c CSV_PATH, --csv_path CSV_PATH
Path of output .csv file. If none provided, then no file will be written.
"""
import os
import glob
import pandas as pd
import io
import xml.etree.ElementTree as ET
import argparse
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' # Suppress TensorFlow logging (1)
import tensorflow.compat.v1 as tf
from PIL import Image
from object_detection.utils import dataset_util, label_map_util
from collections import namedtuple
# Initiate argument parser
parser = argparse.ArgumentParser(
description="Sample TensorFlow XML-to-TFRecord converter")
parser.add_argument("-x",
"--xml_dir",
help="Path to the folder where the input .xml files are stored.",
type=str)
parser.add_argument("-l",
"--labels_path",
help="Path to the labels (.pbtxt) file.", type=str)
parser.add_argument("-o",
"--output_path",
help="Path of output TFRecord (.record) file.", type=str)
parser.add_argument("-i",
"--image_dir",
help="Path to the folder where the input image files are stored. "
"Defaults to the same directory as XML_DIR.",
type=str, default=None)
parser.add_argument("-c",
"--csv_path",
help="Path of output .csv file. If none provided, then no file will be "
"written.",
type=str, default=None)
args = parser.parse_args()
if args.image_dir is None:
args.image_dir = args.xml_dir
label_map = label_map_util.load_labelmap(args.labels_path)
label_map_dict = label_map_util.get_label_map_dict(label_map)
def xml_to_csv(path):
"""Iterates through all .xml files (generated by labelImg) in a given directory and combines
them in a single Pandas dataframe.
Parameters:
----------
path : str
The path containing the .xml files
Returns
-------
Pandas DataFrame
The produced dataframe
"""
xml_list = []
for xml_file in glob.glob(path + '/*.xml'):
tree = ET.parse(xml_file)
root = tree.getroot()
for member in root.findall('object'):
value = (root.find('filename').text,
int(root.find('size')[0].text),
int(root.find('size')[1].text),
member[0].text,
int(member[4][0].text),
int(member[4][1].text),
int(member[4][2].text),
int(member[4][3].text)
)
xml_list.append(value)
column_name = ['filename', 'width', 'height',
'class', 'xmin', 'ymin', 'xmax', 'ymax']
xml_df = pd.DataFrame(xml_list, columns=column_name)
return xml_df
def class_text_to_int(row_label):
return label_map_dict[row_label]
def split(df, group):
data = namedtuple('data', ['filename', 'object'])
gb = df.groupby(group)
return [data(filename, gb.get_group(x)) for filename, x in zip(gb.groups.keys(), gb.groups)]
def create_tf_example(group, path):
with tf.gfile.GFile(os.path.join(path, '{}'.format(group.filename)), 'rb') as fid:
encoded_jpg = fid.read()
encoded_jpg_io = io.BytesIO(encoded_jpg)
image = Image.open(encoded_jpg_io)
width, height = image.size
filename = group.filename.encode('utf8')
image_format = b'jpg'
xmins = []
xmaxs = []
ymins = []
ymaxs = []
classes_text = []
classes = []
for index, row in group.object.iterrows():
xmins.append(row['xmin'] / width)
xmaxs.append(row['xmax'] / width)
ymins.append(row['ymin'] / height)
ymaxs.append(row['ymax'] / height)
classes_text.append(row['class'].encode('utf8'))
classes.append(class_text_to_int(row['class']))
tf_example = tf.train.Example(features=tf.train.Features(feature={
'image/height': dataset_util.int64_feature(height),
'image/width': dataset_util.int64_feature(width),
'image/filename': dataset_util.bytes_feature(filename),
'image/source_id': dataset_util.bytes_feature(filename),
'image/encoded': dataset_util.bytes_feature(encoded_jpg),
'image/format': dataset_util.bytes_feature(image_format),
'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
'image/object/class/label': dataset_util.int64_list_feature(classes),
}))
return tf_example
def main(_):
writer = tf.python_io.TFRecordWriter(args.output_path)
path = os.path.join(args.image_dir)
examples = xml_to_csv(args.xml_dir)
grouped = split(examples, 'filename')
for group in grouped:
tf_example = create_tf_example(group, path)
writer.write(tf_example.SerializeToString())
writer.close()
print('Successfully created the TFRecord file: {}'.format(args.output_path))
if args.csv_path is not None:
examples.to_csv(args.csv_path, index=None)
print('Successfully created the CSV file: {}'.format(args.csv_path))
if __name__ == '__main__':
tf.app.run()Or, there is another way to create Tf Records and apply Agumentation so that we can increase the size of the data.
We can do the following:
Creating the TFRecords ourselves is a bit tedious as the XML created after annotating may sometimes vary, so for the sake of ease, I suggest using Roboflow to perform the above task. They also provide an option to perform additional Data Augmentation which will increase the size of the dataset.
How to make changes in pipeline.config file?
Copying the pipeline.config file of SSD MobileNet V2 FPNLite 640x640 to our model's folder so that we can change the configurations with respect to our use case and without making changes to our original pipeline config file.
Now, let's make changes to our pipeline_config file.
The most important ones we will need to change are -
batch_size is the number of batches the model will train in parallel. A suitable number to use is 8. It could be more/less depending on the computing power available.
A good suggestion given on StackOverflow is:
Max batch size= available GPU memory bytes / 4 / (size of tensors + trainable parameters)
fine_tune_checkpoint **is the last trained checkpoint (a checkpoint is how the model is stored by Tensorflow).
If you are starting the training for the first time, set this to the pre-trained-model.
If you want to continue training on a previously trained checkpoint, set it to the respective checkpoint path. (This will continue training, building upon the features and loss instead of starting from scratch).
Let's Train!
Now, let's train the model
Now, you will have a checkpoint folder created with 8 checkpoints in it.
Now we freeze the weights
Freezing is the process to identify and save all of the required things(graph, weights etc) in a single file that you can easily use.
Now we will have a folder named saved model which contains
├── saved_model
├──assets
├── variable
├── saved_model.pb
How did the model Perform?
Let's check on some crowd images
‘Wear A Mask. Save Lives: Help Stop Coronavirus’
