GPU Plugin

The GPU plugin uses the Intel® Compute Library for Deep Neural Networks (clDNN) to infer deep neural networks. clDNN is an open source performance library for Deep Learning (DL) applications intended for acceleration of Deep Learning Inference on Intel® Processor Graphics including Intel® HD Graphics and Intel® Iris® Graphics. For an in-depth description of clDNN, see: cLDNN sources and Accelerate Deep Learning Inference with Intel® Processor Graphics.

Optimizations

The plugin supports the following optimizations:

CPU Executed Layers

The following layers are not accelerated on the GPU and executed on the host CPU instead:

Known Layers Limitations

Supported Configuration Parameters

The plugin supports the configuration parameters listed below. All parameters must be set before calling InferenceEngine::IInferencePlugin::LoadNetwork() in order to take effect.

Parameter Name Parameter Values Default Description
KEY_PERF_COUNT YES / NO NO Collect performance counters during inference
KEY_CONFIG_FILE "<file1> [<file2> ...]" "" Load custom layer configuration files
KEY_DUMP_KERNELS YES / NO NO Dump the final kernels used for custom layers
KEY_TUNING_MODE TUNING_DISABLED
TUNING_CREATE
TUNING_USE_EXISTING
TUNING_DISABLED Disable inference kernel tuning
Create tuning file (expect much longer runtime)
Use an existing tuning file
KEY_TUNING_FILE "<filename>" "" Tuning file to create / use
KEY_CLDNN_PLUGIN_PRIORITY <0-3> 0 OpenCL queue priority
KEY_CLDNN_PLUGIN_THROTTLE <0-3> 0 OpenCL queue throttling
KEY_CLDNN_GRAPH_DUMPS_DIR "<dump_dir>" "" clDNN graph optimizer stages dump output directory (in GraphViz format)
KEY_CLDNN_SOURCES_DUMPS_DIR "<dump_dir>" "" Final optimized clDNN OpenCL sources dump output directory

Note on Debug Capabilities of the GPU Plugin

Inference Engine GPU plugin provides possibility to dump the user custom OpenCL™ kernels to a file to allow you to properly debug compilation issues in your custom kernels.

The application can use the SetConfig() function with the key PluginConfigParams::KEY_DUMP_KERNELS and value: PluginConfigParams::YES. Then during network loading, all custom layers will print their OpenCL kernels with the JIT instrumentation added by the plugin. The kernels will be stored in the working directory under files named the following way: clDNN_program0.cl, clDNN_program1.cl.

This option is disabled by default. Additionally, the application can call the SetConfig() function with the key PluginConfigParams::KEY_DUMP_KERNELS and value: PluginConfigParams::NO before network loading.

How to verify that this option is disabled:

  1. Delete all clDNN_program*.cl files from the current directory
  2. Run your application to load a network
  3. Examine the working directory for the presence of any kernel file (for example, clDNN_program0.cl)

See Also