Detection Algorithm

REVISION HISTORY

Revision No. Description Date
1.0 First version 04/25/2023
1.1 Second version 10/30/2023
3.0 Third version 04/11/2024
3.1 Remove deprecated models introduction 10/31/2024
3.2 Add ALGO_DET_SortResult API 12/03/2024
3.3 Adjust models description 12/06/2024

1. General Description

1.1. Algorithm Description

The mainly specific detection labels are described as follows:

Model Function Input size(w*h) Input Format
sbdy36.img Bag detection(Small), outputs one label, i.e. bag(class_id=0) 640*352 yuvsp420_nv12
sbdy36l.img Bag detection(Large), outputs one label, i.e. bag(class_id=0) 640*352 yuvsp420_nv12
sbdy48.img Bag detection(Small), outputs one label, i.e. bag(class_id=0) 800*480 yuvsp420_nv12
sbdy48l.img Bag detection(Large), outputs one label, i.e. bag(class_id=0) 800*480 yuvsp420_nv12
sdy36.img Person + vehicle + pet + head + face detection(Small), outputs ten labels, i.e. person(class_id=0), bicycle(class_id=1), cat(class_id=2), motorcycle(class_id=3), bus(class_id=4), truck(class_id=5), cat(class_id=6), dog(class_id=7), head(class_id=8), face(class_id=9) 640*352 yuvsp420_nv12
sdy36l.img Person + vehicle + pet + head + face detection(Large) , outputs ten labels, i.e. person(class_id=0), bicycle(class_id=1), cat(class_id=2), motorcycle(class_id=3), bus(class_id=4), truck(class_id=5), cat(class_id=6), dog(class_id=7), head(class_id=8), face(class_id=9) 640*352 yuvsp420_nv12
sdy48.img Person + vehicle + pet + head + face detection(Small) , outputs ten labels, i.e. person(class_id=0), bicycle(class_id=1), cat(class_id=2), motorcycle(class_id=3), bus(class_id=4), truck(class_id=5), cat(class_id=6), dog(class_id=7), head(class_id=8), face(class_id=9) 800*480 yuvsp420_nv12
sdy48l.img Person + vehicle + pet + head + face detection(Large), outputs ten labels, i.e. person(class_id=0), bicycle(class_id=1), cat(class_id=2), motorcycle(class_id=3), bus(class_id=4), truck(class_id=5), cat(class_id=6), dog(class_id=7), head(class_id=8), face(class_id=9) 800*480 yuvsp420_nv12
sfsdy36.img Fire and smoke detection, outputs two labels, i.e. fire(class_id=0), smoke(class_id=1) 640*352 yuvsp420_nv12
sfsdy48.img Fire and smoke detection, outputs two labels, i.e. fire(class_id=0), smoke(class_id=1) 800*480 yuvsp420_nv12
spdy36.img Person + head + face detection(Small), outputs three labels, i.e. person(class_id=0), head(class_id=1), face(class_id=2) 640*352 yuvsp420_nv12
spdy36l.img Person + head + face detection(Large), outputs three labels, i.e. person(class_id=0), head(class_id=1), face(class_id=2) 640*352 yuvsp420_nv12
spdy48.img Person + head + face detection(Small), outputs three labels, i.e. person(class_id=0), head(class_id=1), face(class_id=2) 800*480 yuvsp420_nv12
spdy48l.img Person + head + face detection(Large), outputs three labels, i.e. person(class_id=0), head(class_id=1), face(class_id=2) 800*480 yuvsp420_nv12

2. API Reference

This function module provides the following APIs:

API Name Function
ALGO_DET_CreateHandle Create handle
ALGO_DET_InitHandle Initialize handle
ALGO_DET_SetParams Set configurable parameters
ALGO_DET_GetInputAttr Get model attribute information
ALGO_DET_Run Run detection algorithm
ALGO_DET_SortResult Sort detection results
ALGO_DET_DeinitHandle Deinitialize detection algorithm
ALGO_DET_ReleaseHandle Release handle

2.1 ALGO_DET_CreateHandle

  • Function

    Create handle

  • Syntax

    MI_S32 ALGO_DET_CreateHandle(void **handle);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.2 ALGO_DET_InitHandle

  • Function

    Initialize handle

  • Syntax

    MI_S32 ALGO_DET_InitHandle(void *handle, DetInit_t *init_info);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input
    init_info Detection algorithm configuration option, see DetInit_t for details Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.3 ALGO_DET_SetParams

  • Function

    Set configurable parameters

  • Syntax

    MI_S32 ALGO_DET_SetParams(void *handle, const DetParams_t *params);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input
    params Configurable parameters Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.4 ALGO_DET_GetInputAttr

  • Function

    Get model attribute information, including model input resolution and input data type

  • Syntax

    MI_S32 ALGO_DET_GetInputAttr(void *handle, DetInputAttr_t *input_attr);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input
    input_attr Pointer to the saved attribute information, see DetInputAttr_t for details Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.5 ALGO_DET_Run

  • Function

    Run detection algorithm

  • Syntax

    MI_S32 ALGO_DET_Run(void *handle, const DetInput_t *algo_input, DetBox_t bboxes[MAX_DET_OBJECT], MI_S32 *num_bboxes);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input
    algo_input Input image buffer information Input
    bboxes Used to save the array of detection result boxes Input
    num_bboxes Pointer to the saved number of detection result boxes Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.6. ALGO_DET_SortResult

  • Function

    Sort detection results

  • Syntax

    MI_S32 ALGO_DET_SortResult(void *handle, const DetSortInput_t *sort_input, DetBox_t bboxes[MAX_DET_OBJECT], MI_S32 *num_bboxes);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input
    sort_input sort input structure Input
    bboxes Used to save the array of sorted result boxes Input/Output
    num_bboxes Pointer to the saved number of sorted result boxes Input/Output

  • Return Value

    Return Value Description
    0 Successful. NOTEļ¼šthis api will change bboxes and num_bboxes
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.7 ALGO_DET_DeinitHandle

  • Function

    Deinitialize detection algorithm

  • Syntax

    MI_S32 ALGO_DET_DeinitHandle(void *handle);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

2.8 ALGO_DET_ReleaseHandle

  • Function

    Release the resources occupied by the handle

  • Syntax

    MI_S32 ALGO_DET_ReleaseHandle(void *handle);

  • Parameter

    Parameter Name Description Input/Output
    handle Handle Input

  • Return Value

    Return Value Description
    0 Successful.
    Other Failed, see Error Code for details

  • Dependency

    • Header file: sgs_det_api.h
    • Library file: libsgsalgo_det.a / libsgsalgo_det.so

3. Structure Type

The detection related data types are defined as follows:

Data Type Definition
DetInputAttr_t Algorithm input structure
DetBox_t Algorithm output structure
DetInit_t Algorithm initial parameter structure
DetInput_t Algorithm input image data information
DetParams_t Algorithm configurable parameter structure
DetSortInput_t Sort input related structure

3.1 DetInputAttr_t

  • Description

    Define the resolution size and pixel format

  • Definition

    typedef struct
{
    MI_U32 width;
    MI_U32 height;
    MI_IPU_ELEMENT_FORMAT format;
} DetInputAttr_t;

  • Member

    Member Name Description
    width Model input data width
    height Model input data height
    format Model input pixel format

  • Related data type and interface

    ALGO_DET_GetInputAttr

3.2 DetBox_t

  • Description

    Algorithm output structure

  • Definition

    typedef struct
{
    MI_U32 x;
    MI_U32 y;
    MI_U32 width;
    MI_U32 height;
    MI_U32 class_id;
    MI_FLOAT score;
    MI_U64 pts;
}DetBox_t;

  • Member

    Member Name Description
    x, y, width, height Detection result box position
    class_id Detection result box class ID
    score Detection result box score
    pts Timestamp of the incoming detection frame

  • Related data type and interface

    ALGO_DET_Run

3.3 DetInit_t

  • Description

    Detection algorithm initial parameter structure

  • Definition

    typedef struct
{
    char ipu_firmware_path[MAX_DET_STRLEN]; // ipu_firmware.bin path
    char model_path[MAX_DET_STRLEN];        // detect model path
    MI_BOOL create_device;                  // set false to create ipu device outside algo lib
    MI_BOOL destroy_device;                 // set false to destroy ipu device outside algo lib
    void *model_buffer;                     // set it when load model from memory
    MI_U32 model_buffer_len;                // set it when load model from memory
}DetInit_t;

  • Member

    Member Name Description
    ipu_firmware_path Ipu firmware path
    model_path Model path
    create_device Whether to create IPU device in detection algorithm,default is true; You can set false when create IPU device outside
    destroy_device Whether to destory IPU device in detection algorithm,default is true; You can set false when destory IPU device outside
    model_buffer Set model_path to NULL when using model_buffer; set model_buffer and model_buffer_len to load model from RAM to initialize detection algorithm
    model_buffer_len Set model_path to NULL when using model_buffer_len; set model buffer length in RAM

  • Related data type and interface

    ALGO_DET_InitHandle

3.4 DetInput_t

  • Description

    Algorithm input image data information

  • Definition

    typedef struct
{
    void *p_vir_addr;
    MI_PHY phy_addr;
    MI_U32 buf_size;
    MI_U64 pts;
    MI_U16 width;
    MI_U16 height;
}DetInput_t;

  • Member

    Member Name Description
    p_vir_addr Input buffer virtual address
    phy_addr Input buffer physical address
    buf_size Input buffer size
    pts Input buffer timestamp
    width Input buffer width
    height Input buffer height

  • Related data type and interface

    ALGO_DET_Run

3.5 DetParams_t

  • Description

    Algorithm configurable parameter structure

  • Definition

    typedef struct
{
    MI_FLOAT conf_threshold;         // default=0.5
    MI_S32 disp_width;               // default=1920
    MI_S32 disp_height;              // default=1080
    MI_BOOL ignore_static_objects;   // default=false
    MI_FLOAT static_sensitive;       // default=0.85
    MI_BOOL stable_bbox;             // default=false
    MI_FLOAT stable_sensitive;       // default=0.63
    MI_S32 ignore_frame_number;      // [0-5] ignore the id first number box of detect,default=0
    MI_BOOL strict_mode;             // default=false
}DetParams_t;

  • Member

    Member Name Description
    conf_threshold Detection threshold (0.0~1.0), default is 0.5. Larger threshold will reduce false alarm and recall; Smaller threshold will increase false alarm and recall
    disp_width Display width (used for mapping the location of detection boxes), default is 1920
    disp_height Display height (used for mapping the location of detection boxes), default is 1080
    ignore_static_objects Whether to filter static objects, default is false, i.e. do not filter static objects
    static_sensitive Static detecton threshold(0.0~1.0), default is 0.85. Larger threshold will keep boxes exhibiting minor movement
    stable_bbox Whether to enable stable box, default is false, i.e. disable stable box
    stable_sensitive Stable box threshold (0.0~1.0), default is 0.63. Smaller threshold will get better stable boxes
    ignore_frame_number Filter the first frames, using to filter occasional false alarm, (0~5), default is 0, i.e. do not filter the first frames
    strict_mode Whether to run algorithm in strict mode, which can reduce false alarm when enable, default is false

  • Related data type and interface

    ALGO_DET_SetParams

3.6 DetSortInput_t

  • Description

    Sort input related structure

  • Definition

    typedef struct
{
    MI_S32 class_indexs[MAX_DET_CLASSES];
    MI_S32 class_num;
    DetSortType_e sort_type;
}DetSortInput_t;

  • Member

    Member Name Description
    class_indexs Index of the classes to be sorted
    class_num The number of classes to sort
    sort_type Sorting method type

  • Related data type and interface

    ALGO_DET_SortResult

4. Enumeration Type

The detection related data types are defined as follows:

Data Type Definition
Label_FD_Face_e Correspondence between model type and class_id of face detection
Label_FSD_e Correspondence between model type and class_id of fire and smoke detection
Label_SD_e Correspondence between model type and class_id of person + vehicle + pet + head + face detection
Label_SPD_e Correspondence between model type and class_id of person + head + face detection
Label_BD_e Correspondence between model type and class_id of bag detection

4.1 Label_FD_Face_e

  • Description

    Correspondence between model type and class_id of face detection

  • Definition

    typedef enum
{
    E_FD_FACE = 0,
}Label_FD_Face_e;

  • Member

    Member Name Description
    E_FD_FACE Face(class_id=0)

  • Related data type and interface

    DetBox_t

4.2 Label_FSD_e

  • Description

    Correspondence between model type and class_id of fire and smoke detection

  • Definition

    typedef enum
{
    E_FSD_FIRE,
    E_FSD_SMOKE
}Label_FSD_e;

  • Member

    Member Name Description
    E_FSD_FIRE Fire(class_id=0)
    E_FSD_SMOKE Smoke(class_id=1)

  • Related data type and interface

    DetBox_t

4.3 Label_SD_e

  • Description

    Correspondence between model type and class_id of person + vehicle + pet + head + face detection

  • Definition

    typedef enum
{
    E_SD_PERSON = 0,
    E_SD_BICYCLE,
    E_SD_CAR,
    E_SD_MOTOCYCLE,
    E_SD_BUS,
    E_SD_TRUCK,
    E_SD_CAT,
    E_SD_DOG,
    E_SD_HEAD,
    E_SD_FACE,
}LABEL_SD_e;

  • Member

    Member Name Description
    E_SD_PERSON Person(class_id=0)
    E_SD_BICYCLE Bicycle(class_id=1)
    E_SD_CAR Car (class_id=2)
    E_SD_MOTOCYCLE Motocyle(class_id=3)
    E_SD_BUS Bus(class_id=4)
    E_SD_TRUCK Truck(class_id=5)
    E_SD_CAT Cat(class_id=6)
    E_SD_DOG Dog(class_id=7)
    E_SD_HEAD Head(class_id=8)
    E_SD_FACE Face(class_id=9)

  • Related data type and interface

    DetBox_t

4.4 Label_SPD_e

  • Description

    Correspondence between model type and class_id of person + head + face detection

  • Definition

    typedef enum
{
    E_SPD_PERSON = 0,
    E_SPD_HEAD,
    E_SPD_FACE
}LABEL_SPD_e;

  • Member

    Member Name Description
    E_SPD_PERSON Person(class_id=0)
    E_SPD_HEAD Head(class_id=1)
    E_SPD_FACE Face(class_id=2)

  • Related data type and interface

    DetBox_t

4.5 Label_BD_e

  • Description

    Correspondence between model type and class_id of bag detection

  • Definition

    typedef enum
{
    E_BD_BAG = 0,
}LABEL_BD_e;

  • Member

    Member Name Description
    E_BD_BAG Bag(class_id=0)

  • Related data type and interface

    DetBox_t

5. Error code

Error Code Value Description
E_ALGO_SUCCESS 0 Operation successful
E_ALGO_HANDLE_NULL 1 Algorithm handle is null
E_ALGO_INVALID_PARAM 2 Invalid input parameter
E_ALGO_DEVICE_FAULT 3 Hardware error
E_ALGO_LOADMODEL_FAIL 4 Failed to load model
E_ALGO_INIT_FAIL 5 Algorithm initialization failed
E_ALGO_NOT_INIT 6 Algorithm not initialized
E_ALGO_INPUT_DATA_NULL 7 Algorithm input data is null
E_ALGO_INVALID_INPUT_SIZE 8 Invalid algorithm input data dimension
E_ALGO_INVALID_LICENSE 9 Invalid license
E_ALGO_MEMORY_OUT 10 Out of memory
E_ALGO_FILEIO_ERROR 11 File read and write operation error
E_ALGO_INVALID_OUTPUT_SIZE 12 Invalid algorithm output data dimension
E_ALGO_INVALID_DECODE_MODE 13 Invalid decode mode
E_ALGO_MODEL_INVOKE_ERROR 14 Invoke fail