nGraph is the OpenVINO™ graph manipulation library, used to represent neural network models in the form of a computational graph. With nGraph Python APIs, you can create, inspect, and modify computational graphs.
nGraph is able to represent a large set of mathematical operations, chosen to cover most operations used in popular neural network models. The
ngraph module is the basic namespace used to expose commonly used API methods. You can inspect the
ngraph module or call
help on any method for usage information.
You can use nGraph's Python API to create computational models. For example, to create a model that represents the formula
y=ReLU(x + b), you can use the following code to create parameters
b and create the computational graph:
nGraph operations are organized into sets, which are released with new versions of OpenVINO. As operation signatures can change between operation sets, it is a good practice to use operations from a specific operation set in your scripts. This ensures that your code is compatible with future versions of OpenVINO.
The example, the code above can be rewritten like this to use only operations from opset 4:
You can combine a graph together with its input parameters into an nGraph function object. An nGraph function can be passed to other parts of OpenVINO to perform analysis and inference.
In order to run inference on an nGraph function, convert it to an Inference Engine network and call its
You can load a model from a file using the Inference Engine
network = ie_core.read_network(model=model_path, weights=weights_path)
An nGraph function can be extracted from an Inference Engine network using the
ngraph.function_from_cnn method. The following example shows how an ONNX model can be loaded to an nGraph Function:
You can use the nGraph function's
get_ordered_ops method to get a topologically sorted list of its graph
Node has a unique
name property, assigned at creation time. User-provided names can be retrieved using the
get_type_name returns the operation type of the node and the
shape property returns the shape of the node's output tensor.
Node's parents can be retrieved using the
inputs() method, which returns
Input objects. Each
Input is a connection to an upstream
Output, which is connected to a
Node. The upstream node can be retrieved using
Node's children can be retrieved using the
outputs() method, which returns a list of
Output objects. By contrast to inputs, each output can be connected to multiple downstream nodes.
Output has an associated tensor shape, which describes what data it can accept or produce. Shapes can be dynamic, which means that they are only partially known. You can get information about this shape by calling the
get_partial_shape method. If the returned
is_dynamic property returns
True, then the shape is known partially. Otherwise, you can convert the shape to a fully known static shape (list of integers).
Nodes may also have additional attributes. For example, a
Softmax node has an
axis attribute. Each attribute may be retrieved using a dedicated method, for example,
You can also set attribute values using corresponding setter methods, for example:
Currently, you can get all attributes of a node using the
_get_attributes method. Please note that this is an internal API method and may change in future versions of OpenVINO.
The following code displays all attributes for all nodes in a function:
Attributes are properties of nodes in the computational graph, which will be stored when the model is serialized to a file. However, there can be additional properties of nodes, which are only important during the execution of a graph. An example of such a property is
affinity, which determines which operation will be executed on which hardware in a heterogeneous environment. You can get and set runtime information by using the
get_rt_info method of a node.
NOTE: If you set affinity manually, be careful at the current moment Inference Engine plugins don't support constant (
Result) and empty (
Result) networks. Please avoid such subgraphs when you set affinity manually.
Please note that the
rt_info API may change in future versions of OpenVINO.
You can retrieve the data tensor associated with a
Constant node by calling its
nGraph supports a growing number of transformation passes, which can be applied to a graph. For example, to perform constant folding of applicable subgraphs, you can use the
ConstantFolding pass. To do this, create an optimization pass
Manager, register the passes you want to execute, and feed an nGraph function to the configured pass manager.
The following code example illustrates the process:
Please note that the list of available transformations may change in future versions of OpenVINO.