A summary of the steps for optimizing and deploying a model that was trained with the TensorFlow* framework:
Supported Non-Frozen Topologies with Links to the Associated Slim Model Classification Download Files
Detailed information on how to convert models from the TensorFlow*-Slim Image Classification Model Library is available in the Converting TensorFlow*-Slim Image Classification Model Library Models chapter. The table below contains list of supported TensorFlow*-Slim Image Classification Model Library models and required mean/scale values. The mean values are specified as if the input image is read in BGR channels order layout like Inference Engine classification sample does.
|Model Name||Slim Model Checkpoint File||--mean_values||--scale|
|Inception ResNet v2||inception_resnet_v2_2016_08_30.tar.gz||[127.5,127.5,127.5]||127.5|
|MobileNet v1 128||mobilenet_v1_0.25_128.tgz||[127.5,127.5,127.5]||127.5|
|MobileNet v1 160||mobilenet_v1_0.5_160.tgz||[127.5,127.5,127.5]||127.5|
|MobileNet v1 224||mobilenet_v1_1.0_224.tgz||[127.5,127.5,127.5]||127.5|
Supported Frozen Topologies from TensorFlow Object Detection Models Zoo
Detailed information on how to convert models from the Object Detection Models Zoo is available in the Converting TensorFlow Object Detection API Models chapter. The table below contains models from the Object Detection Models zoo that are supported.
|Model Name||TensorFlow Object Detection API Models (Frozen)|
|SSD MobileNet V1 COCO*||ssd_mobilenet_v1_coco_2018_01_28.tar.gz|
|SSD MobileNet V1 0.75 Depth COCO||ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync_2018_07_03.tar.gz|
|SSD MobileNet V1 PPN COCO||ssd_mobilenet_v1_ppn_shared_box_predictor_300x300_coco14_sync_2018_07_03.tar.gz|
|SSD MobileNet V1 FPN COCO||ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz|
|SSD ResNet50 FPN COCO||ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz|
|SSD MobileNet V2 COCO||ssd_mobilenet_v2_coco_2018_03_29.tar.gz|
|SSD Lite MobileNet V2 COCO||ssdlite_mobilenet_v2_coco_2018_05_09.tar.gz|
|SSD Inception V2 COCO||ssd_inception_v2_coco_2018_01_28.tar.gz|
|RFCN ResNet 101 COCO||rfcn_resnet101_coco_2018_01_28.tar.gz|
|Faster R-CNN Inception V2 COCO||faster_rcnn_inception_v2_coco_2018_01_28.tar.gz|
|Faster R-CNN ResNet 50 COCO||faster_rcnn_resnet50_coco_2018_01_28.tar.gz|
|Faster R-CNN ResNet 50 Low Proposals COCO||faster_rcnn_resnet50_lowproposals_coco_2018_01_28.tar.gz|
|Faster R-CNN ResNet 101 COCO||faster_rcnn_resnet101_coco_2018_01_28.tar.gz|
|Faster R-CNN ResNet 101 Low Proposals COCO||faster_rcnn_resnet101_lowproposals_coco_2018_01_28.tar.gz|
|Faster R-CNN Inception ResNet V2 COCO||faster_rcnn_inception_resnet_v2_atrous_coco_2018_01_28.tar.gz|
|Faster R-CNN Inception ResNet V2 Low Proposals COCO||faster_rcnn_inception_resnet_v2_atrous_lowproposals_coco_2018_01_28.tar.gz|
|Faster R-CNN NasNet COCO||faster_rcnn_nas_coco_2018_01_28.tar.gz|
|Faster R-CNN NasNet Low Proposals COCO||faster_rcnn_nas_lowproposals_coco_2018_01_28.tar.gz|
|Mask R-CNN Inception ResNet V2 COCO||mask_rcnn_inception_resnet_v2_atrous_coco_2018_01_28.tar.gz|
|Mask R-CNN Inception V2 COCO||mask_rcnn_inception_v2_coco_2018_01_28.tar.gz|
|Mask R-CNN ResNet 101 COCO||mask_rcnn_resnet101_atrous_coco_2018_01_28.tar.gz|
|Mask R-CNN ResNet 50 COCO||mask_rcnn_resnet50_atrous_coco_2018_01_28.tar.gz|
|Faster R-CNN ResNet 101 Kitti*||faster_rcnn_resnet101_kitti_2018_01_28.tar.gz|
|Faster R-CNN Inception ResNet V2 Open Images*||faster_rcnn_inception_resnet_v2_atrous_oid_2018_01_28.tar.gz|
|Faster R-CNN Inception ResNet V2 Low Proposals Open Images*||faster_rcnn_inception_resnet_v2_atrous_lowproposals_oid_2018_01_28.tar.gz|
|Faster R-CNN ResNet 101 AVA v2.1*||faster_rcnn_resnet101_ava_v2.1_2018_04_30.tar.gz|
Supported Frozen Quantized Topologies
The topologies hosted on the TensorFlow* Lite site. The frozen model file (
.pb file) should be fed to the Model Optimizer.
|Model Name||Frozen Model File|
|Mobilenet V1 0.25 128||mobilenet_v1_0.25_128_quant.tgz|
|Mobilenet V1 0.25 160||mobilenet_v1_0.25_160_quant.tgz|
|Mobilenet V1 0.25 192||mobilenet_v1_0.25_192_quant.tgz|
|Mobilenet V1 0.25 224||mobilenet_v1_0.25_224_quant.tgz|
|Mobilenet V1 0.50 128||mobilenet_v1_0.5_128_quant.tgz|
|Mobilenet V1 0.50 160||mobilenet_v1_0.5_160_quant.tgz|
|Mobilenet V1 0.50 192||mobilenet_v1_0.5_192_quant.tgz|
|Mobilenet V1 0.50 224||mobilenet_v1_0.5_224_quant.tgz|
|Mobilenet V1 0.75 128||mobilenet_v1_0.75_128_quant.tgz|
|Mobilenet V1 0.75 160||mobilenet_v1_0.75_160_quant.tgz|
|Mobilenet V1 0.75 192||mobilenet_v1_0.75_192_quant.tgz|
|Mobilenet V1 0.75 224||mobilenet_v1_0.75_224_quant.tgz|
|Mobilenet V1 1.0 128||mobilenet_v1_1.0_128_quant.tgz|
|Mobilenet V1 1.0 160||mobilenet_v1_1.0_160_quant.tgz|
|Mobilenet V1 1.0 192||mobilenet_v1_1.0_192_quant.tgz|
|Mobilenet V1 1.0 224||mobilenet_v1_1.0_224_quant.tgz|
|Mobilenet V2 1.0 224||mobilenet_v2_1.0_224_quant.tgz|
It is necessary to specify the following command line parameters for the Model Optimizer to convert some of the models from the list above:
--input input --input_shape [1,HEIGHT,WIDTH,3]. Where
WIDTH are the input images height and width for which the model was trained.
Other supported topologies
There are three ways to store non-frozen TensorFlow models and load them to the Model Optimizer:
In this case, a model consists of two files:
If you do not have an inference graph file, refer to Freezing Custom Models in Python.
To convert such TensorFlow model:
mo_tf.pyscript with the path to the checkpoint file to convert a model:
In this case, a model consists of three or four files stored in the same directory:
model_name.data-00000-of-00001(digit part may vary)
To convert such TensorFlow model:
mo_tf.pyscript with a path to the MetaGraph
.metafile to convert a model:
SavedModel format of TensorFlow 1.x and 2.x versions:
In this case, a model consists of a special directory with a
.pb file and several subfolders:
assets.extra. For more information about the SavedModel directory, refer to the README file in the TensorFlow repository.
To convert such TensorFlow model:
mo_tf.pyscript with a path to the SavedModel directory to convert a model:
You can convert TensorFlow 1.x SavedModel format in the environment that has a 1.x or 2.x version of TensorFlow. However, TensorFlow 2.x SavedModel format strictly requires the 2.x version of TensorFlow. If a model contains operations currently unsupported by OpenVINO, prune these operations by explicit specification of input nodes using the
--input option. To determine custom input nodes, display a graph of the model in TensorBoard. To generate TensorBoard logs of the graph, use the
--tensorboard_logs option. TensorFlow 2.x SavedModel format has a specific graph due to eager execution. In case of pruning, find custom input nodes in the
StatefulPartitionedCall/* subgraph of TensorFlow 2.x SavedModel format.
When a network is defined in Python* code, you have to create an inference graph file. Usually graphs are built in a form that allows model training. That means that all trainable parameters are represented as variables in the graph. To be able to use such graph with Model Optimizer such graph should be frozen. The graph is frozen and dumped to a file with the following code:
sessis the instance of the TensorFlow* Session object where the network topology is defined.
["name_of_the_output_node"]is the list of output node names in the graph;
frozengraph will include only those nodes from the original
sess.graph_defthat are directly or indirectly used to compute given output nodes.
'name_of_the_output_node'here is an example of possible output node name. You should derive the names based on your own graph.
./is the directory where the inference graph file should be generated.
inference_graph.pbis the name of the generated inference graph file.
as_textspecifies whether the generated file should be in human readable text format or binary.
To convert a TensorFlow model:
mo_tf.pyscript to simply convert a model with the path to the input model
Two groups of parameters are available to convert your model:
NOTE: The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the
RGB<->BGRconversion specifying the command-line parameter:
--reverse_input_channels. Otherwise, inference results may be incorrect. For more information about the parameter, refer to When to Reverse Input Channels section of Converting a Model Using General Conversion Parameters.
The following list provides the TensorFlow*-specific parameters.
NOTE: Models produces with TensorFlow* usually have not fully defined shapes (contain
-1in some dimensions). It is necessary to pass explicit shape for the input using command line parameter
-bto override just batch dimension. If the shape is fully defined, then there is no need to specify either
transform.jsonwith information about input and output nodes of the matched sub-graph. For more information about this feature, refer to Sub-Graph Replacement in the Model Optimizer.
transform.jsonfor model conversion. For more information about this feature, refer to Sub-Graph Replacement in the Model Optimizer.
my_custom_op.so. Model Optimizer falls back to TensorFlow to infer output shape of operations implemented in the library if a custom TensorFlow operation library is provided. If it is not provided, a custom operation with an inference function is needed. For more information about custom operations, refer to the Extending the Model Optimizer with New Primitives.
TensorFlow* 2.X officially supports two model formats: SavedModel and Keras H5 (or HDF5). Below are the instructions on how to convert each of them.
A model in the SavedModel format consists of a directory with a
saved_model.pb file and two subfolders:
assets. To convert such a model:
mo_tf.pyscript with a path to the SavedModel directory:
TensorFlow* 2 SavedModel format strictly requires the 2.x version of TensorFlow installed in the environment for conversion to the Intermediate Representation (IR).
If a model contains operations currently unsupported by OpenVINO™, prune these operations by explicit specification of input nodes using the
--output options. To determine custom input nodes, visualize a model graph in the TensorBoard.
To generate TensorBoard logs of the graph, use the Model Optimizer
--tensorboard_logs command-line option.
TensorFlow* 2 SavedModel format has a specific graph structure due to eager execution. In case of pruning, find custom input nodes in the
If you have a model in the HDF5 format, load the model using TensorFlow* 2 and serialize it in the SavedModel format. Here is an example of how to do it:
Then follow the above instructions for the SavedModel format.
NOTE: Do not use other hacks to resave TensorFlow* 2 models into TensorFlow* 1 formats.
NOTE: Currently, OpenVINO™ support for TensorFlow* 2 models is in preview (aka Beta), which means limited and not of production quality yet. OpenVINO™ does not support models with Keras RNN and Embedding layers.
Internally, when you run the Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list of known layers. If your topology contains any layers that are not in this list of known layers, the Model Optimizer classifies them as custom.
See Custom Layers in the Model Optimizer for information about:
Refer to Supported Framework Layers for the list of supported standard layers.
The Model Optimizer provides explanatory messages if it is unable to run to completion due to issues like typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the Model Optimizer FAQ. The FAQ has instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
In this document, you learned: