NOTE: It is a preview version of the Inference Engine Python* API for evaluation purpose only. Module structure and API itself may be changed in future releases.
This API provides a simplified interface for Inference Engine functionality that allows to:
Currently the Inference Engine Python* API is supported on Ubuntu* 16.04, Microsoft Windows* 10 and CentOS* 7.3 OSes. Supported Python* versions:
To configure the environment for the Inference Engine Python* API, run:
source <INSTALL_DIR>/bin/setupvars.sh .
call <INSTALL_DIR>\deployment_tools\inference_engine\python_api\setenv.bat
The script automatically detects latest installed Python* version and configures required environment if the version is supported. If you want to use certain version of Python*, set the environment variable PYTHONPATH=<INSTALL_DIR>/deployment_tools/inference_engine/python_api/<desired_python_version>
after running the environment configuration script.
This class stores main information about the layer and allow to modify some layer parameters
name
- Name of the layertype
- Layer typeprecision
- Layer base operating precision. Provides getter and setter interfaces.layout
- Returns the layout of shape of the layer.shape
- Return the list of the shape of the layer.parents
- Returns a list, which contains names of layers preceding this layer.children
- Returns a list, which contains names of layers following this layer.affinity
- Layer affinity set by user or a default affinity set by the IEPlugin.set_initial_affinity()
method. The affinity attribute provides getter and setter interfaces, so the layer affinity can be modified directly. For example: To correctly set affinity for the network, you must first initialize and properly configure the HETERO plugin. set_config({"TARGET_FALLBACK": "HETERO:FPGA,GPU"})
function configures the plugin fallback devices and their order. plugin.set_initial_affinity(net)
function sets affinity parameter of model layers according to its support on specified devices.
After default affinity is set by the plugin, override the default values by setting affinity manually how it's described in example above
To understand how default and non-default affinities are set:
net.layers
function right after model loading and check that layer affinity parameter is empty.plugin.set_default_affinity(net)
.net.layers
and check layer affinity parameters to see how plugin set a default affinitynet.layers
again and check layer affinity parameters to see how it was changed after manual affinity settingPlease refer to affinity_setting_demo.py
to see the full usage pipeline.
weights
- Dictionary with layer weights, biases or custom blobs if anyparams
- Layer specific parameters. Provides getter and setter interfaces to get and modify layer parameters. Please note that some modifications can be ignored and overwriten by target plugin (e.g. modification of convolution kernel size will be reflected in layer parameters but finally the plugin will ignore it and will use initial kernel size)This class contains the information about the network model read from IR and allows you to manipulate with some model parameters such as layers affinity and output layers.
__init__(model: str, weights: str)
.xml
file of the IR.bin
file of the IRname
- Name of the loaded networkinputs
- A dictionary that maps input layer names to InputInfo objects. For example, to get a shape of the input layer: outputs
- A dictionary that maps output layer names to OutputInfo objects For example, to get a shape of the output layer: batch_size
- Batch size of the network. Provides getter and setter interfaces to get and modify the network batch size. For example: layers
- Return dictionary that maps network layer names to IENetLayer
objects containing layer properties in topological order. For example, to list all network layers: stats
- Returns LayersStatsMap
object containing dictionary that maps network layer names to calibration statistics represented by LayerStats
objects. LayersStatsMap
class inherited from built-in python dict
and overrides default update()
method to allow to set or modify layers calibration statistics. from_ir(model: str, weights: str)
NOTE: The function is deprecated. Please use
IENetwork()
class constructor to create valid instance ofIENetwork
.xml
and .bin
files of the IR..xml
file of the IR.bin
file of the IRIENetwork
classadd_outputs(outputs)
:outputs
- List of layer names to be set as model outputs. In case of setting one layer as output, string with one layer can be provided.NOTE: The last layers (nodes without successors in graph representation of the model) are set as output by default. In the case above,
prob
layer is a default output andconv5_1/dwise
,conv2_1/expand
are user-defined outputs.
reshape(input_shapes: dict)
:Note: Before using this method, make sure that the target shape is applicable for the network. Changing the network shape to an arbitrary value may lead to unpredictable behaviour.
input_shapes
- The dictionary that maps input layer names to tuples with the target shapeserialize(path_to_xml, path_to_bin)
:path_to_xml
- path to a file, where a serialized model will be stored.path_to_bin
- path to a file, where serialized weights will be stored.Layer calibration statistic container.
__init__(min: tuple = (), max: tuple = ())
This class contains the information about the network input layers
precision
- Precision of the input data provided by user. Provides setter and getter interfaces to get and modify input layer precision. List of applicable precisions: FP32 FP16, I32, I16, I8, U32, U16 NOTE: Support of any calculation precision depends on the target plugin.
layout
- Layout of the input data provided by user. Provides setter and getter interfaces to get and modify input layer layout. List of applicable layouts: NCHW, NHWC, OIHW, C, CHW, HW, NC, CN, BLOCKEDshape
- input layer data shapeThis class contains the information about the network input layers
precision
- Precision of the output data. Provides setter and getter interfaces to get and modify output layer precision.layout
- Layout of the output data provided by usershape
- Input layer data shapeThis class is the main plugin interface and serves to initialize and configure the plugin.
__init__(device: str, plugin_dirs=None)
device
- Target device name. Supported devices: CPU, GPU, FPGA, MYRIAD, HETEROplugin_dirs
- List of paths to plugin directoriesdevice
- a name of the device that was specified to initialize IEPluginversion
- a version of the pluginload(network: IENetwork, num_requests: int=1, config=None)
network
- A valid IENetwork
instancenum_requests
- A positive integer value of infer requests to be created. Number of infer requests may be limited by device capabilities.config
- A dictionary of plugin configuration keys and their valuesset_initial_affinity(net: IENetwork)
net
- A valid instance of IENetworkaffinity
attribute of the IENetLayer
class.add_cpu_extension(extension_path: str)
extension_path
- A full path to CPU extensions libraryReturn value: None
set_config(config: dict)
SetConfig()
in Inference Engine C++ documentation for acceptable keys and values list.config
- A dictionary of keys and values of acceptable configuration parametersset_affinity
method of the IENetwork
class.get_supported_layers(net: IENetwork)
add_cpu_extenstion()
methodnet
- A valid instance of IENetworkaffinity
attribute of the IENetLayer
class.This class represents a network instance loaded to plugin and ready for inference.
There is no explicit class constructor. To make a valid instance of ExecutableNetwork
, use load()
method of the IEPlugin
class.
requests
- A tuple of InferRequest instancesinfer(inputs=None)
infer()
method of the InferRequest
classinputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layernumpy.ndarray
objects with output data of the layerclassification_sample.py
).start_async(request_id, inputs=None)
async_infer()
method of the InferRequest
classrequest_id
- Index of infer request to start inferenceinputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerInferRequest
class.For more details about infer requests processing, see classification_sample_async.py
(simplified case) and object_detection_demo_ssd_async.py
(real asynchronous use case) samples.
This class provides an interface to infer requests of ExecutableNetwork
and serves to handle infer requests execution and to set and get output data.
There is no explicit class constructor. To make a valid InferRequest
instance, use load()
method of the IEPlugin
class with specified number of requests to get ExecutableNetwork
instance which stores infer requests.
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layeroutputs
- A dictionary that maps output layer names to numpy.ndarray
objects with output data of the layerIt is not recommended to run inference directly on InferRequest
instance. To run inference, please use simplified methods infer()
and start_async()
of ExecutableNetwork
.
infer(inputs=None)
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerasync_infer(inputs=None)
inputs
- A dictionary that maps input layer names to numpy.ndarray
objects of proper shape with input data for the layerwait(timeout=-1)
NOTE: There are special values of the timeout parameter:
timeout
- Time to wait in milliseconds or special (0, -1) cases described above. If not specified, timeout
value is set to -1 by default.async_infer()
method of the the InferRequest
class.get_perf_counts()
NOTE: Performance counters data and format depends on the plugin
set_batch(size)
NOTE: Support of dynamic batch size depends on the target plugin.
batch
- new batch size to be used by all the following inference calls for this request.dynamic_batch_demo.py
to see the full usage example.