Profile on Remote Machine

DL Workbench can collect performance data not only on the machine on which you run it, but also on other machines in your local network.

NOTES:

  • Learn more about profiling on a remote machine securely.
  • When using a target machine for remote profiling, make sure other users are not working on it at the same time.
  • Remote profiling takes more time than profiling on a local machine due to exchanging models, datasets, the script that starts inference, and performance data.

To profile on a remote machine, follow the steps below:

  1. Set up the remote machine.
  2. Register the remote machine in the DL Workbench.
  3. Add a model to a configuration.
  4. Add the remote machine to the configuration.
  5. Add an environment to the configuration.
  6. Add a dataset to the configuration.
  7. Run baseline inference with the configuration.

After these steps, you can run single and group inference and compare performance between models on local and remote machines. Features like optimization, accuracy measurements, and deployment package creation are disabled for remote machines:

disabled_optimization-b.png

Add Remote Machine to Configuration

On the Create Configuration page, select a model and move on to the Environment table.

Machines registered in the DL Workbench are added to the Environment table. You can only select a machine that indicates Available state:

available-b.png

Configuring and Connecting states mean that you need to wait for the machine to set up:

configuring-env-b.png
connecting-b.png

The remote machine is in Busy state when you run inference on it:

busy-env-b.png

If the machine you want to use indicates Configuration Failure, click Review and follow the instructions in Manipulate Remote Machines:

config-failure-b.png

Security Aspects of Remote Profiling

SSH keys are sensitive data and using them in the DL Workbench is at your own risk.

DL Workbench takes control of a remote machine to perform a limited set of tasks, like installing dependencies and OpenVINO™ tools. Connect a machine at your own risk and according to the security policy of your organization.

Run the DL Workbench with Transport Layer Security (TLS) to ensure data protection.

Sensitive data is removed when you remove a machine from the DL Workbench or remove the Docker container with the DL Workbench.

DL Workbench installs dependencies on a target machine and therefore takes additional space on it. DL Workbench data is stored in the .workbench folder in your user directory. To remove the new data from your target machine, remove the .workbench folder. DL Workbench transfers models and datasets to the target machine with the SSH protocol and removes them after completing experiments there.