Mobile Detection and Tracking Algorithm

REVISION HISTORY

Revision No. Description Date
1.0 First version 11/21/2023
2.0 Sync with latest api 08/06/2024

1. Overview

1.1. Algorithm Description

The mobile detection and tracking algorithm implements the detection and tracking of moving objects in the scene.

2. API Reference

This functional module provides the following APIs:

API Name Function
ALGO_MDT_CreateHandle Create handle
ALGO_MDT_InitHandle Initialize handle
ALGO_MDT_SetParams Set customizable parameters for the algorithm
ALGO_MDT_Run Run mobile detection and tracking algorithm
ALGO_MDT_Track Run tracking algorithm
ALGO_MDT_GetMdMask Get mask of the moving area
ALGO_MDT_DeinitHandle Deinitialize handle
ALGO_MDT_ReleaseHandle Release handle
ALGO_MDT_Detect Run mobile detection algorithm

2.1. ALGO_MDT_CreateHandle

  • Function

    Create algorithm handle.

  • Syntax

    MI_S32 ALGO_MDT_CreateHandle(void **handle);

  • Parameters

    Parameter Name Description Input/Output
    handle Pointer to the handle Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.2. ALGO_MDT_InitHandle

  • Function

    Initialize algorithm handle.

  • Syntax

    MI_S32 ALGO_MDT_InitHandle(void *handle, const MdtInit_t *init_info);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input
    init_info Algorithm initialization parameters (see MdtInit_t) Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.3. ALGO_MDT_SetParams

  • Function

    Set customizable parameters for the algorithm.

  • Syntax

    MI_S32 ALGO_MDT_SetParams(void *handle, const MdtParams_t *params);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input
    params Customizable algorithm parameters (see MdtParams_t) Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.4. ALGO_MDT_Run

  • Function

    Run mobile object detection and tracking algorithm.

  • Syntax

    MI_S32 ALGO_MDT_Run(void *handle, const MdtInput_t* prev_frame, const MdtInput_t* current_frame, MdtBox_t boxes[MAX_NUM_MDT_OBJS], MI_S32 *num_boxes);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input
    prev_frame Pointer to the structure of the previous frame input image (see MdtInput_t) Input
    current_frame Pointer to the structure of the current frame input image (see MdtInput_t) Input
    boxes Array to hold output result boxes (see MdtBox_t) Output
    num_boxes Pointer to save the number of output result boxes Output

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.5. ALGO_MDT_Track

  • Function

    Run tracking algorithm.

  • Syntax

    MI_S32 ALGO_MDT_Track(void *handle, const MdtInput_t* current_frame, MdtBox_t boxes[MAX_NUM_MDT_OBJS], MI_S32 *num_boxes, MdtStage_e *mdt_stage);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input
    current_frame Pointer to the structure of the current frame input image (see MdtInput_t) Input
    boxes Input and output result boxes, input is detection result, output is tracking result (see MdtBox_t) Input/Output
    num_boxes Input and output, input is the number of detection result boxes, output is the number of tracking result boxes pointer Input/Output
    mdt_stage Pointer to algorithm mode (see MdtStage_e) Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.6. ALGO_MDT_GetMdMask

  • Function

    Get mask of the moving area.

  • Syntax

    MI_S32 ALGO_MDT_GetMdMask(void *handle, const MdtInput_t *prev_frame, const MdtInput_t *current_frame, MI_U8* mask, MI_S32 mask_w, MI_S32 mask_h);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input
    prev_frame Pointer to the structure of the previous frame input image (see MdtInput_t) Input
    current_frame Pointer to the structure of the current frame input image (see MdtInput_t) Input
    mask Pointer to store mask data (must be allocated externally and length must be mask_w*mask_height), moving area value is 255, stationary area value is 0 Output
    mask_w Width of the mask Input
    mask_h Height of the mask Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.7. ALGO_MDT_DeinitHandle

  • Function

    Deinitialize handle.

  • Syntax

    MI_S32 ALGO_MDT_DeinitHandle(void *handle);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.8. ALGO_MDT_ReleaseHandle

  • Function

    Release handle.

  • Syntax

    MI_S32 ALGO_MDT_ReleaseHandle(void *handle);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

2.9. ALGO_MDT_Detect

  • Function

    Run mobile object detection algorithm.

  • Syntax

    MI_S32 ALGO_MDT_Detect(void *handle, const MdtInput_t *prev_frame, const MdtInput_t *current_frame, MdtBox_t boxes[MAX_NUM_MDT_OBJS], MI_S32 *num_boxes);

  • Parameters

    Parameter Name Description Input/Output
    handle Handle Input
    prev_frame Pointer to the structure of the previous frame input image (see MdtInput_t) Input
    current_frame Pointer to the structure of the current frame input image (see MdtInput_t) Input
    boxes Array to hold output result boxes (see MdtBox_t) Output
    num_boxes Pointer to save the number of output result boxes Output

  • Return Value

    Return Value Description
    0 Success
    Other Failure (see Error Codes)

  • Dependencies

    • Header file: sgs_mdt_api.h
    • Library: libsgsalgo_mdt.a / libsgsalgo_mdt.so

3. Structure Definition

The relevant structure definitions for detection are as follows:

Data Type Definition
MdtInit_t Algorithm initialization parameter structure
MdtParams_t Customizable algorithm parameter structure
MdtInput_t Algorithm input image frame structure
MdtBox_t Algorithm output box structure
MdtStage_e Algorithm mode

3.1 MdtInit_t

  • Description

    Algorithm initialization parameter structure.

  • Definition

    typedef struct
{
    MI_S32 ive_handle;
    MI_U32 in_width;
    MI_U32 in_height;
    MI_U32 in_stride;
} MdtInit_t;

  • Members

    Member Name Description
    ive_handle Handle of the MI_IVE module
    in_width Width of the output image
    in_height Height of the output image
    in_stride Row length after memory alignment of the output image buffer

  • Related Data Types and Interfaces

    ALGO_MDT_InitHandle

3.2 MdtParams_t

  • Description

    Customizable algorithm parameter structure.

  • Definition

    typedef struct
{
    MI_FLOAT track_threshold;   // 0.0 ~ 1.0 default 0.3
    MI_U8 detect_threshold;     // 0 ~ 255, default 8
    MI_FLOAT min_width;         // 0.0 ~ 1.0 default 0.06
    MI_FLOAT min_height;        // 0.0 ~ 1.0 default 0.1
    MI_FLOAT max_width;         // 0.0 ~ 1.0 default 0.3
    MI_FLOAT max_height;        // 0.0 ~ 1.0 default 1.0
    MI_FLOAT max_frame_gap;     // 0.0 ms ~ INF ms default 100ms
    MI_U32 max_num_boxes;       // max num detected boxes, default 1
} MdtParams_t;

  • Members

    Member Name Description
    track_threshold Threshold for the tracking algorithm; smaller value means better continuity but lower accuracy, range 0-1, default value 0.3
    detect_threshold Threshold for the detection algorithm (minimum frame difference for determining moving objects); smaller value means higher sensitivity, range 0-255, default value 8
    min_width Minimum target width (normalized, relative to the width of the entire scene), range 0-1, default value 0.06
    min_height Minimum target height (normalized, relative to the height of the entire scene), range 0-1, default value 0.1
    max_width Maximum target width (normalized, relative to the width of the entire scene), range 0-1, default value 0.3
    max_height Maximum target height (normalized, relative to the height of the entire scene), range 0-1, default value 1.0
    max_frame_gap Maximum frame interval time (if two frames exceed this interval, they will be ignored; can be set to 0.0 if frame interval filtering is not needed, unit ms), default value 100.0
    max_num_boxes Maximum allowed number of detected boxes, default value 1

  • Related Data Types and Interfaces

    ALGO_MDT_SetParams

3.3 MdtInput_t

  • Description

    Structure that contains information about the input image frame.

  • Definition

    typedef struct
{
    void *p_vir_addr;
    MI_PHY phy_addr;
    MI_U32 buf_size;
    MI_U64 pts;
} MdtInput_t;

  • Members

    Member Name Description
    p_vir_addr Virtual address of the input image frame
    phy_addr Physical address of the input image frame
    buf_size Buffer length of the input image frame
    pts Timestamp of the input image frame

  • Related Data Types and Interfaces

    ALGO_MDT_Run

    ALGO_MDT_Track

    ALGO_MDT_GetMdMask

3.4 MdtBox_t

  • Description

    Structure of algorithm output boxes.

  • Definition

    typedef struct
{
    MI_U32 x;
    MI_U32 y;
    MI_U32 width;
    MI_U32 height;
    MI_FLOAT score;
    MI_U64 track_id;
    MI_U64 pts;
} MdtBox_t;

  • Members

    Member Name Description
    x, y, width, height Coordinates of the top-left point and width/height of the result box
    score Score of the tracking result
    track_id Tracking ID
    pts Timestamp of the corresponding frame

  • Related Data Types and Interfaces

    ALGO_MDT_Run

    ALGO_MDT_Track

3.5 MdtStage_e

  • Description

    Algorithm modes.

  • Definition

    typedef enum
{
    E_MDT_STAGE_INIT = 0,
    E_MDT_STAGE_DETECT,
    E_MDT_STAGE_TRACK,
} MdtStage_e;

  • Members

    Member Name Description
    E_MDT_STAGE_INIT Algorithm initialization
    E_MDT_STAGE_DETECT Moving object detection
    E_MDT_STAGE_TRACK Moving object tracking

  • Related Data Types and Interfaces

    ALGO_MDT_Track

4. Error Codes

Error Code Value Description
E_ALGO_SUCCESS 0 Operation successful
E_ALGO_HANDLE_NULL 1 Algorithm handle is null
E_ALGO_INVALID_PARAM 2 Invalid input parameter
E_ALGO_DEVICE_FAULT 3 Hardware error
E_ALGO_LOADMODEL_FAIL 4 Failed to load model
E_ALGO_INIT_FAIL 5 Algorithm initialization failed
E_ALGO_NOT_INIT 6 Algorithm not initialized
E_ALGO_INPUT_DATA_NULL 7 Algorithm input data is null
E_ALGO_INVALID_INPUT_SIZE 8 Invalid algorithm input data dimensions
E_ALGO_INVALID_LICENSE 9 Invalid license
E_ALGO_MEMORY_OUT 10 Insufficient memory
E_ALGO_FILEIO_ERROR 11 File read/write operation error
E_ALGO_INVALID_OUTPUT_SIZE 12 Invalid algorithm output data dimensions
E_ALGO_INVALID_DECODE_MODE 13 Invalid decode mode
E_ALGO_MODEL_INVOKE_ERROR 14 Model invoke error
E_ALGO_INVALID_FILE 15 Invalid file