Extension Library

Inference Engine provides an InferenceEngine::IExtension interface, which defines the interface for Inference Engine Extension libraries. Inherit all extension libraries from this interface. The example below contains an implementation of two operations: Template used as an example in this document and FFT used as a more complex example from the Custom Operations Guide.

NOTE: FFT operation is implemented using the OpenCV library functions cv::dft and cv::idft.

Based on that, the declaration of an extension class can look as follows:

namespace TemplateExtension {
class Extension : public InferenceEngine::IExtension {
void GetVersion(const InferenceEngine::Version*& versionInfo) const noexcept override;
void Unload() noexcept override {}
std::map<std::string, ngraph::OpSet> getOpSets() override;
std::vector<std::string> getImplTypes(const std::shared_ptr<ngraph::Node>& node) override;
InferenceEngine::ILayerImpl::Ptr getImplementation(const std::shared_ptr<ngraph::Node>& node, const std::string& implType) override;
} // namespace TemplateExtension
This class is the main extension interface.
Definition: ie_iextension.h:147
std::shared_ptr< ILayerImpl > Ptr
A shared pointer to the ILayerImpl interface.
Definition: ie_iextension.h:90
Represents version information that describes plugins and the inference engine runtime library.
Definition: ie_version.hpp:39

The extension library should contain and export the InferenceEngine::CreateExtension method, which creates an Extension class:

// Generate exported function
Generates extension creation function.
Definition: ie_iextension.h:233

Also, an Extension object should implement the following methods:

void Extension::GetVersion(const InferenceEngine::Version*& versionInfo) const noexcept {
static InferenceEngine::Version ExtensionDescription = {
{1, 0}, // extension API version
"template_ext" // extension description message
versionInfo = &ExtensionDescription;

Implement the InferenceEngine::IExtension::getOpSets method if the extension contains custom layers. Read Custom nGraph Operation for more information.

To understand how to integrate execution kernels to the extension library, read the documentation about development of custom CPU kernels.

To understand how to register custom ONNX operator to the extension library, read the documentation about custom ONNX operators.