Skip to content

IPU ToolKit Program Description

1. Feature Scenarios

IPU ToolKit consists of three independent tools designed for IPU model debugging, remote inference, and performance monitoring.

Tool Description Source Directory
dla_show_img_info Model Information Viewer - Offline viewing of IPU model structure and performance parameters sdk/verify/release_feature/source/dla/dla_show_img_info
ipu_server Remote Inference Server - Starts an RPC server for clients to remotely invoke IPU hardware for model inference sdk/verify/release_feature/source/dla/ipu_server
ipu_utilization IPU Utilization Monitor - Real-time statistics of IPU core working time and utilization percentage sdk/verify/release_feature/source/dla/ipu_utilization

2. Build Environment

2.1 Build Environment Setup

Set up the arm64 build environment from the project root directory:

export PATH=/tools/toolchain/aarch64-unknown-linux-gcc-12.4.0-glibc-2.37-gnu/bin:$PATH
export CROSS_COMPILE=aarch64-unknown-linux-gnu-12.4.0-
export ARCH=arm64
cd project
make linux-comake_mhera.emmc.glibc-12.4.0-arm64-ext4.d3.4096.fccsp16_lpddr4x_defconfig

If using 2GB DDR, the defconfig can use linux-comake_mhera.emmc.glibc-12.4.0-arm64-ext4.d3.2048.fccsp16_lpddr4x_defconfig

2.2 Build Commands

# Build the entire project
cd project
make clean;make image -j16

# Build IPU ToolKit
cd sdk/verify/release_feature/source
make dla

# After building, three tools are generated in the out/arm64/app directory:
# - prog_dla_show_img_info  (dla_show_img_info)
# - prog_dla_server         (ipu_server)
# - prog_ipu_utilization    (ipu_utilization)

2.3 Build Artifacts

  • prog_dla_show_img_info executable located in the out/arm64/app directory
  • prog_dla_server executable located in the out/arm64/app directory
  • prog_ipu_utilization executable located in the out/arm64/app directory

3. Runtime Environment

All IPU ToolKit tools run on SGS series chip platforms and depend on the MI (Module Interface) system driver. The IPU hardware module must be correctly loaded.

3.1 Hardware Requirements

  • Development Board: Comake PI D3

4. Usage Instructions

4.1 Model Information Viewer - dla_show_img_info

4.1.1 Command

# Basic usage
./prog_dla_show_img_info -m <model_path> [options]

# Command-line arguments
  -h, --help          Show help message
  -m, --model         Model file path (required)
                      Separate multiple models with commas
      --details_info  Show detailed model info including /proc/mi_modules/mi_ipu/mi_ipu0 content (optional, disabled by default)
      --batch         Batch size (optional, default 1)
      --subnet        Sub-network index (optional, default 0), for multi-subnet models
      --ipu_log       IPU Log output path (optional), exports IPU Log files when specified
      --ipu_log_size  IPU Log Buffer size (optional, default 0x800000)

4.1.2 Usage Example

# Example
./prog_dla_show_img_info -m ./model.img

Terminal Output Verification:

After successful execution, the terminal prints information in the following format:

model.img(0):
Invoke Time:   12.345 ms
IPU Time:      10.123 ms
BandWidth:     15.678 MB / frame
Variable size: 2.345 MB
Work mode:     single_core
Batch mode:    n_buf
Setting batch: 1
Suggest batch: [1, 2, 4, 8, 16, 32]
Model type:    AICOMMON
Input(0):
    name:   input_1
    dtype:  UINT8
    shape:  [1, 3, 640, 352]
    size:   675840
    layout: NCHW
Output(0):
    name:   output_1
    dtype:  INT16
    shape:  [1, 255, 20, 11]
    size:   112200
    layout: NCHW
    quantization:   (0.0039, -128)
  • Invoke Time: Elapsed time of the Invoke API on the board. This includes CPU overhead for driving the IPU, IPU Time, model output data synchronization to DRAM, etc.
  • IPU Time: IPU hardware time.
  • BandWidth: Amount of DRAM data accessed during a single model inference.
  • Variable size: Runtime memory required for model execution. Multiple models can share this memory, so the maximum Variable size among all models should be selected.
  • Work mode: Model single-core / dual-core mode information.
  • Batch mode: Model batch mode information.
  • Setting batch: Batch value set when converting the offline model.
  • Suggest batch: Batch values included within the offline model.

IPU Log Export Verification:

If --ipu_log is specified, the following files are generated in the target directory: - {model_name}_log_core0.bin - IPU core log - {model_name}_log_corectrl0.bin - IPU core controller log

These two files can be used with the IPU SDK analysis tool. Convert them to JSON and open in Chrome browser to view per-layer performance time ratios.

4.2 Remote Inference Server - ipu_server

4.2.1 Command

# Basic usage
./prog_dla_server -p <port>

# Command-line arguments
  -h, --help     Show help message
  -p, --port     Server listening port number (required)

4.2.2 Usage Example

  1. Run ipu_server on the board to start the RPC service (PORT is the designated port number)
# Example
./prog_dla_server -p 9000
  1. Run simulator.py on the PC
python3 SGS_IPU_Toolchain/Scripts/calibrator/simulator.py \
-i /path/to/input_data \
-m /path/to/offline.img \
-n /path/to/preprocess.py \
--host <board_ip_address> \
--port PORT \
--soc_version CHIP
  1. For detailed steps, refer to the Model Inference section of the IPU Toolchain User Manual.

Terminal Output Verification:

IPU Service established on port 9000

After server startup, the terminal displays the above message, indicating the service is ready and waiting for client RPC calls.

4.3 IPU Utilization Monitor - ipu_utilization

4.3.1 Command

# Basic usage
./prog_ipu_utilization -t <time_interval>

# Command-line arguments
  -t      Interval for statistics and printing, unit: seconds

4.3.2 Usage Example

# Example
# Print IPU utilization every second
./prog_ipu_utilization -t 1

Terminal Output Verification:

core0: 0.05%
core0: 0.03%
core0: 85.20%
core0: 87.15%
  • Idle: Utilization close to 0%
  • Under inference: Utilization rises significantly; the value depends on inference frequency and model complexity

Additional Information: - Check current IPU clock frequency: cat /proc/mi_modules/mi_ipu/debug_hal/freq - The current chip has only one IPU core; only core0 needs attention

5. FAQ

5.1 IPU Driver Issues

Unable to open /proc/mi_modules/mi_ipu/ directory:

  • Symptom: Tools report errors, unable to access IPU-related proc files
  • Cause: IPU driver not correctly loaded
  • Solution: 
    # Check if driver is loaded
ls /proc/mi_modules/mi_ipu/
# If not loaded, reload the MI system driver

5.2 Model Loading Issues

Model loading failure:

  • Symptom: MI_IPU_CreateCHN returns an error code
  • Cause: Incorrect model file path, file format mismatch, or insufficient Variable Buffer
  • Solution:
  • Verify the model file path is correct
  • Check model file integrity
  • Verify model matches the chip platform
  • Ensure sufficient Variable Buffer size

Batch size not supported:

  • Symptom: Specified batch size not found in the model's supported list
  • Cause: The model was compiled without that batch size
  • Solution: Use a batch size supported by the model, or recompile the model to include the required batch size

5.3 Utilization Issues

Utilization close to 100%:

  • Cause: Unlike CPU, IPU has no system-level time-sharing. When inference tasks are intensive, utilization approaches maximum
  • Note: Due to microsecond-level software overhead between inferences, utilization cannot reach 100%. Values in the 95%-99% range are normal
  • Check IPU frequency: cat /proc/mi_modules/mi_ipu/debug_hal/freq

Utilization always 0%:

  • Cause: No inference tasks are currently running
  • Verification: When running dla_show_img_info or other inference tasks, utilization should rise significantly

5.4 Network Issues

Client unable to connect to ipu_server:

  • Symptom: Client connection fails or times out
  • Cause: Server port not open, network unreachable, or firewall blocking
  • Solution:
  • Confirm the server has started and prints IPU Service established on port xxx
  • Check the board-side network interface status
  • Test port connectivity

RPC call returns error:

  • Cause: Calling ipu_binding_input/output when not in binding mode, invalid Channel ID, or Tensor not ready
  • Solution: Check the RPC call sequence; ensure ipu_set_io_binding is called before ipu_binding_input/output

5.5 IPU Log Export Issues

IPU Log not generated:

  • Symptom: --ipu_log is specified but no bin files are generated
  • Cause: IPU Log capture is not enabled
  • Solution: Enable IPU Log capture via debug commands first, then execute dla_show_img_info