Human Pose Recognition Algorithm

---

REVISION HISTORY

Revision No.	Description	Date
101	Initial release	03/22/2024

---

1. Algorithm Description

The human pose recognition algorithm primarily recognizes 7 types of human poses:

• Static Poses (6 types): bend, cross_arms, standing, wave1, wave2, lie_down
• Dynamic Pose (1 type): walk

• Please adhere to the standard human poses described below when using this algorithm to ensure recognition effectiveness:

  

Model Introduction:

model	Function	Resolution (w*h)	Input Format
humanpose_det36l.img	Human Detection	640*352	yuvsp420_nv12
humanpose_det48l.img	Human Detection	800*480	yuvsp420_nv12
humanpose_cls.img	Human Pose Classification	224*224	yuvsp420_nv12

2. API Calling Process

The algorithm API calling process is ALGO_HumanPose_CreateHandle->ALGO_HumanPose_InitHandle->ALGO_HumanPose_GetInputAttr->ALGO_HumanPose_SetParams->ALGO_HumanPose_Detect->ALGO_HumanPose_Cls->ALGO_HumanPose_DeinitHandle->ALGO_HumanPose_ReleaseHandle.

3. Functional Module APIs

API Name	Function
ALGO_HumanPose_CreateHandle	Create Handle
ALGO_HumanPose_InitHandle	Initialize Handle
ALGO_HumanPose_GetInputAttr	Get Input Attributes
ALGO_HumanPose_SetParams	Set Parameters
ALGO_HumanPose_Detect	Human Pose Detection
ALGO_HumanPose_Cls	Human Pose Recognition
ALGO_HumanPose_DeinitHandle	Handle De-initialization
ALGO_HumanPose_ReleaseHandle	Release Handle

3.1. ALGO_HumanPose_CreateHandle

• Function

  Create Handle

• Syntax

  MI_S32 ALGO_HumanPose_CreateHandle(void** handle);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.2. ALGO_HumanPose_InitHandle

• Function

  Initialize Handle

• Syntax

  MI_S32 ALGO_HumanPose_InitHandle(void *handle, const PoseInit_t *init);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  init	Initialization Parameters	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.3. ALGO_HumanPose_GetInputAttr

• Function

  Get Input Attributes

• Syntax

  MI_S32 ALGO_HumanPose_GetInputAttr(void *handle, PoseInputAttr_t *det_input_attr, PoseInputAttr_t *cls_input_attr);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  det_input_attr	Detection Model Input Attributes (Width, Height, Image Format)	Output
  cls_input_attr	Recognition Model Input Attributes (Width, Height, Image Format)	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.4. ALGO_HumanPose_SetParams

• Function

  Set Parameters

• Syntax

  MI_S32 ALGO_HumanPose_SetParams(void *handle, const PoseParams_t* params);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  params	Parameters	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.5. ALGO_HumanPose_Detect

• Function

  Human Pose Detection

• Syntax

  MI_S32 ALGO_HumanPose_Detect(void *handle, const PoseInput_t *input, PoseBox_t boxes[MAX_POSE_OBJECT], MI_S32 *num_boxes);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  input	Detection Model Input Data 800*480 yuv420_nv12	Input
  boxes	Detection Box Results	Output
  num_boxes	Number of Detection Boxes	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.6. ALGO_HumanPose_Cls

• Function

  Human Pose Recognition, identifying which pose among the 7 types

• Syntax

  MI_S32 ALGO_HumanPose_Cls(void *handle,  const PoseInput_t *input, PoseBox_t* box, PoseCls_t* cls);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  input	Recognition Model Input Data 224*224 yuv420_nv12	Input
  box	Human Pose Detection Box	Input
  cls	Human Pose Results	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.7. ALGO_HumanPose_DeinitHandle

• Function

  Handle De-initialization

• Syntax

  MI_S32 ALGO_HumanPose_DeinitHandle(void *handle);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.8. ALGO_HumanPose_ReleaseHandle

• Function

  Release Handle

• Syntax

  MI_S32 ALGO_HumanPose_ReleaseHandle(void *handle);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

4. Data Types

Data Type	Definition
PoseInit_t	Initialization Parameter Structure
PoseInput_t	Input Image Attribute Structure
PoseInputAttr_t	Model Input Data Structure
PoseCls_t	Human Pose Recognition Result Structure
PoseBox_t	Human Pose Detection Box Structure
PoseParams_t	Input Parameter Structure
HumanPoseClass_e	Human Pose Category Information
HumanKeyPoint_e	Human Pose Keypoint Category Information

4.1. PoseInit_t

• Description

  Initialization Parameter Structure

• Definition

  typedef struct
  {
      char ipu_firmware_path[MAX_POSE_STRLEN];
      char pose_model[MAX_POSE_STRLEN];
      char cls_model[MAX_POSE_STRLEN];
      MI_BOOL create_device;
      MI_BOOL destroy_device;
  } PoseInit_t;
  

• Members

  Member Name	Description
  ipu_firmware_path	IPU firmware binary path
  pose_model	Detection Model Path (e.g.: ./models/humanpose_det48y_20231023.img)
  cls_model	Recognition Model Path (e.g.: ./models/humanpose_cls_20240312.img)
  create_device	Whether to create device: if false, need to create the device externally, if true, device will be created automatically inside the algorithm library
  destroy_device	Whether to destroy device: if false, need to destroy the device externally, if true, device will be destroyed automatically inside the algorithm library

• Related Interfaces

  ALGO_HumanPose_InitHandle

4.2. PoseInput_t

• Description

  Input Image Data Structure

• Definition

  typedef struct
  {
      void *p_vir_addr;
      MI_PHY phy_addr;
      MI_U32 buf_size;
      MI_U64 pts;
      MI_U16 width;
      MI_U16 height;
  } PoseInput_t;
  

• Members

  Member Name	Description
  p_vir_addr	Virtual Address
  phy_addr	Physical Address
  buf_size	Data Size
  pts	Timestamp (can be left unassigned)
  width	Image Width
  height	Image Height

• Related Data Types and Interfaces

  ALGO_HumanPose_Detect

  ALGO_HumanPose_Cls

4.3. PoseInputAttr_t

• Description

  Model Input Attributes Structure

• Definition

  typedef struct
  {
      MI_U32 width;
      MI_U32 height;
      MI_IPU_ELEMENT_FORMAT format;
  } PoseInputAttr_t;
  

• Members

  Member Name	Description
  width	Input Image Width
  height	Input Image Height
  format	Input Image Format

• Related Data Types and Interfaces

  ALGO_HumanPose_GetInputAttr

4.4. PoseCls_t

• Description

  Human Pose Recognition Result Structure

• Definition

  typedef struct
  {
      MI_S32 cls;
      MI_FLOAT score;
  } PoseCls_t;
  

• Members

  Member Name	Description
  cls	Human Pose Category (see correspondence in HumanPoseClass_e)
  score	Human Pose Category Score

• Related Data Types and Interfaces

  ALGO_HumanPose_Cls

4.5. PoseBox_t

• Description

  Human Pose Detection Box Structure

• Definition

  typedef struct
  {
      MI_U32 x;
      MI_U32 y;
      MI_U32 width;
      MI_U32 height;
      MI_S32 cls;
      MI_FLOAT score;
      MI_U64 pts;
      MI_FLOAT keypts[MAX_POSE_NUM_KEYPTS][2];
      MI_U64 track_id;
  } PoseBox_t;
  

• Members

  Member Name	Description
  x	Top left X coordinate
  y	Top left Y coordinate
  width	Box Width
  height	Box Height
  cls	Box Category
  score	Box Score
  pts	Timestamp (can be left unassigned)
  keypts	Keypoints Sequence
  track_id	Tracking ID

• Related Data Types and Interfaces

  ALGO_HumanPose_Detect

  ALGO_HumanPose_Cls

4.6. PoseParams_t

• Description

  Input Parameter Structure

• Definition

  typedef struct
  {
      MI_S32 disp_width;
      MI_S32 disp_height;
      MI_FLOAT min_width;
      MI_FLOAT min_height;
      MI_FLOAT det_threshold;
      MI_FLOAT cls_threshold;
      MI_FLOAT attr_threshold[MAX_POSE_NUM_ATTR];
  } PoseParams_t;
  

• Members

  Member Name	Description
  disp_width	Display Width
  disp_height	Display Height
  min_width	Minimum Width for Detection
  min_height	Minimum Height for Detection
  det_threshold	Detection Threshold
  cls_threshold	Recognition Threshold (Higher threshold means stricter recognition)
  attr_threshold	Attribute Threshold (not used in human pose recognition)

• Related Data Types and Interfaces

  ALGO_HumanPose_SetParams

4.7. HumanPoseClass_e

• Description

  Human Pose Category Information

• Definition

  typedef enum
  {
      E_HUMAN_POSE_NONE = 0,
      E_HUMAN_POSE_BEND,
      E_HUMAN_POSE_CROSS_ARMS,
      E_HUMAN_POSE_STANDING,
      E_HUMAN_POSE_WAVE1,
      E_HUMAN_POSE_WAVE2,
      E_HUMAN_POSE_LIE_DOWN,
      E_HUMAN_POSE_WALK,
      E_NUM_HUMAN_POSES
  } HumanPoseClass_e;
  

• Related Data Types and Interfaces

  PoseCls_t

  ALGO_HumanPose_Cls

4.8. HumanKeyPoint_e

• Description

  Human Pose Keypoint Category Information

• Definition

  typedef enum
  {
      E_NOSE = 0,
      E_LEFT_EYE,
      E_RIGHT_EYE,
      E_LEFT_EAR,
      E_RIGHT_EAR,
      E_LEFT_SHOULDER,
      E_RIGHT_SHOULDER,
      E_LEFT_ELBOW,
      E_RIGHT_ELBOW,
      E_LEFT_WRIST,
      E_RIGHT_WRIST,
      E_LEFT_HIP,
      E_RIGHT_HIP,
      E_LEFT_KNEE,
      E_RIGHT_KNEE,
      E_LEFT_ANKLE,
      E_RIGHT_ANKLE,
      E_NUM_HUMAN_KEYPTS
  } HumanKeyPoint_e;
  

• Illustration:

  

• Related Data Types and Interfaces

  PoseBox_t

  ALGO_HumanPose_Detect

  ALGO_HumanPose_Cls