Skip to content

Benchmark Demo Introduction

1 Introduction to Benchmark Demo

Benchmark demo is a tool provided for users to quickly view the model benchmark when deploying models on the SGS platform. It can run open-source models on the board, including: IPU inference speed, bandwidth, flash, and model runtime memory.

2 How to Obtain the Tool

The tool is located in the Linux SDK-alkaid, with the storage directory as follows: Linux_SDK/sdk/verify/opendla/source/benchmark

3 Board Deployment

3.1 Program Compilation

The Benchmark tool needs to be compiled after selecting the deconfig according to the board (nand/nor/emmc, DDR model, etc.) for SDK full-package compilation. For details, please refer to the Alkaid SDK SIGDOC "Development Environment Setup" document.

  • Compilation commands: $cd sdk/verify/opendla $make clean && make source/benchmark -j8
  • The final executable file can be found at: sdk/verify/opendla/out/${AARCH}/app/prog_benchmark

3.2 Running Files

When running the program, the following files need to be copied to the board: - prog_benchmark - The offline model to be tested

4 Running Instructions

  • Usage: ./prog_benchmark -p models/ (Command to execute the file)

  • Required Input:

    • -p: Path to the folder containing the offline models

  • Typical Output:

    ./prog_benchmark -p models/

    modelPath = models/
    model                          invoke time(ms)   bandwidth(KB)    bandwidth(KB)/s   ROM(KB)    RAM(KB)
    clip_image_encode.img          36.832            84300            2.28877e+06       38630      40626
    clip_text_encode.img           27.333            99330            3.6341e+06        89952      90468
    conformer_400x80.img           95.837            270181           2.81918e+06       84638      90780
    yolov8n_pose_640x640.img       40.616            95014            2.33934e+06       4432       7959
    yolov8n_seg_640x640.img        50.328            112223           2.22984e+06       4538       12097
    yolov8_world_image_encode.img  101.662           189103           1.86012e+06       16241      23877
    yolov8_world_text_encode.img   25.703            87787            3.41544e+06       89422      89904