The GPU plugin uses the Intel® Compute Library for Deep Neural Networks (clDNN) to infer deep neural networks. clDNN is an open source performance library for Deep Learning (DL) applications intended for acceleration of Deep Learning Inference on Intel® Processor Graphics including Intel® HD Graphics and Intel® Iris® Graphics. For an in-depth description of clDNN, see: cLDNN sources and Accelerate Deep Learning Inference with Intel® Processor Graphics.
The plugin supports the following optimizations:
The following layers are not accelerated on the GPU and executed on the host CPU instead:
The plugin supports the configuration parameters listed below. All parameters must be set before calling InferenceEngine::IInferencePlugin::LoadNetwork()
in order to take effect.
Parameter Name | Parameter Values | Default | Description |
---|---|---|---|
KEY_PERF_COUNT |
YES / NO |
NO |
Collect performance counters during inference |
KEY_CONFIG_FILE |
"<file1> [<file2> ...]" |
"" |
Load custom layer configuration files |
KEY_DUMP_KERNELS |
YES / NO |
NO |
Dump the final kernels used for custom layers |
KEY_TUNING_MODE |
TUNING_DISABLED TUNING_CREATE TUNING_USE_EXISTING |
TUNING_DISABLED |
Disable inference kernel tuning Create tuning file (expect much longer runtime) Use an existing tuning file |
KEY_TUNING_FILE |
"<filename>" |
"" |
Tuning file to create / use |
KEY_CLDNN_PLUGIN_PRIORITY |
<0-3> |
0 |
OpenCL queue priority |
KEY_CLDNN_PLUGIN_THROTTLE |
<0-3> |
0 |
OpenCL queue throttling |
KEY_CLDNN_GRAPH_DUMPS_DIR |
"<dump_dir>" |
"" |
clDNN graph optimizer stages dump output directory (in GraphViz format) |
KEY_CLDNN_SOURCES_DUMPS_DIR |
"<dump_dir>" |
"" |
Final optimized clDNN OpenCL sources dump output directory |
Inference Engine GPU plugin provides possibility to dump the user custom OpenCL™ kernels to a file to allow you to properly debug compilation issues in your custom kernels.
The application can use the SetConfig()
function with the key PluginConfigParams::KEY_DUMP_KERNELS
and value: PluginConfigParams::YES
. Then during network loading, all custom layers will print their OpenCL kernels with the JIT instrumentation added by the plugin. The kernels will be stored in the working directory under files named the following way: clDNN_program0.cl
, clDNN_program1.cl
.
This option is disabled by default. Additionally, the application can call the SetConfig()
function with the key PluginConfigParams::KEY_DUMP_KERNELS
and value: PluginConfigParams::NO
before network loading.
How to verify that this option is disabled:
clDNN_program*.cl
files from the current directoryclDNN_program0.cl
)