Face Pose Recognition Algorithm

---

REVISION HISTORY

Revision No.	Description	Date
101	Initial release	03/22/2024

---

1. Algorithm Description

The face pose recognition algorithm primarily recognizes the pose of the eyes and mouth in the facial area, with pose attributes including:

• Eyes (open/closed)
• Mouth (open/closed)

Model Introduction:

model	Function	Resolution (w*h)	Input Format
headpose_det36s.img	Face detection	640*352	yuvsp420_nv12
headpose_det48s.img	Face detection	800*480	yuvsp420_nv12
headpose_cls11.img	Face pose classification	112*112	yuvsp420_nv12

2. Interface Call Flow

The algorithm interface call flow is ALGO_HeadPose_CreateHandle->ALGO_HeadPose_InitHandle->ALGO_HeadPose_GetInputAttr->ALGO_HeadPose_SetParams->ALGO_HeadPose_Detect->ALGO_HeadPose_Attr->ALGO_HeadPose_DeinitHandle->ALGO_HeadPose_ReleaseHandle.

3. Function Module API

API Name	Function
ALGO_HeadPose_CreateHandle	Create handle
ALGO_HeadPose_InitHandle	Initialize handle
ALGO_HeadPose_GetInputAttr	Get input attributes
ALGO_HeadPose_SetParams	Set parameters
ALGO_HeadPose_Detect	Face pose detection
ALGO_HeadPose_Attr	Face pose attribute recognition
ALGO_HeadPose_DeinitHandle	Handle de-initialization
ALGO_HeadPose_ReleaseHandle	Release handle

3.1. ALGO_HeadPose_CreateHandle

• Function

  Create handle

• Syntax

  MI_S32 ALGO_HeadPose_CreateHandle(void** handle);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.2. ALGO_HeadPose_InitHandle

• Function

  Initialize handle

• Syntax

  MI_S32 ALGO_HeadPose_InitHandle(void *handle, const PoseInit_t *init);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  init	Initialization parameters	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.3. ALGO_HeadPose_GetInputAttr

• Function

  Get input attributes

• Syntax

  MI_S32 ALGO_HeadPose_GetInputAttr(void *handle, PoseInputAttr_t *det_input_attr, PoseInputAttr_t *cls_input_attr);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  det_input_attr	Detection model input attributes (width, height, image format)	Output
  cls_input_attr	Recognition model input attributes (width, height, image format)	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.4. ALGO_HeadPose_SetParams

• Function

  Set parameters

• Syntax

  MI_S32 ALGO_HeadPose_SetParams(void *handle, const PoseParams_t* params);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  params	Parameters	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.5. ALGO_HeadPose_Detect

• Function

  Face pose detection

• Syntax

  MI_S32 ALGO_HeadPose_Detect(void *handle, const PoseInput_t *input, PoseBox_t boxes[MAX_POSE_OBJECT], MI_S32 *num_boxes);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  input	Detection model input data 800*480 yuv420_nv12	Input
  boxes	Detection box results	Output
  num_boxes	Number of detection boxes	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.6. ALGO_HeadPose_Attr

• Function

  Face pose attribute recognition, recognizing what kind of pose attributes the eyes and mouth have.

• Syntax

  MI_S32 ALGO_HeadPose_Attr(void *handle,  const PoseInput_t *input, PoseBox_t* box, PoseCls_t attrs[E_NUM_HEAD_POSE_ATTR], int *num_attrs);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input
  input	Recognition model input data 224*224 yuv420_nv12	Input
  box	Face pose detection box	Input
  attrs	Face pose attribute results	Output
  num_attrs	Number of face pose attributes	Output

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.7. ALGO_HeadPose_DeinitHandle

• Function

  Handle de-initialization.

• Syntax

  MI_S32 ALGO_HeadPose_DeinitHandle(void *handle);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

3.8. ALGO_HeadPose_ReleaseHandle

• Function

  Release handle

• Syntax

  MI_S32 ALGO_HeadPose_ReleaseHandle(void *handle);
  

• Parameters

  Parameter Name	Description	Input/Output
  handle	Handle	Input

• Return Value

  0: Success.

  Non-zero: Failure.

• Dependencies

  Header file: sgs_pose_api.h

  Library file: libsgsalgo_pose.so, libsgsalgo_pose.a

4. Data Types

Data Type	Definition
PoseInit_t	Initialization parameters structure
PoseInput_t	Input image attributes structure
PoseInputAttr_t	Model input data structure
PoseCls_t	Face pose attribute recognition result structure
PoseBox_t	Face pose attribute detection box structure
PoseParams_t	Input parameters structure
HeadPoseAttr_e	Face pose attribute information

4.1. PoseInit_t

• Description

  Initialization parameters structure

• Definition

  typedef struct
  {
      char ipu_firmware_path[MAX_POSE_STRLEN];
      char pose_model[MAX_POSE_STRLEN];
      char cls_model[MAX_POSE_STRLEN];
      MI_BOOL create_device;
      MI_BOOL destroy_device;
  } PoseInit_t;
  

• Members

  Member Name	Description
  ipu_firmware_path	IPU firmware binary path
  pose_model	Detection model path (e.g., ./models/headpose_det48y_20230628.img)
  cls_model	Recognition model path (e.g., ./models/headpose_land_20240321.img)
  create_device	Whether to create device: if false, create device externally; if true, create device internally in the algorithm library
  destroy_device	Whether to destroy device: if false, destroy device externally, if true, destroy device internally in the algorithm library

• Related Interfaces

  ALGO_HeadPose_InitHandle

4.2. PoseInput_t

• Description

  Input image data structure

• Definition

  typedef struct
  {
      void *p_vir_addr;
      MI_PHY phy_addr;
      MI_U32 buf_size;
      MI_U64 pts;
      MI_U16 width;
      MI_U16 height;
  } PoseInput_t;
  

• Members

  Member Name	Description
  p_vir_addr	Virtual address
  phy_addr	Physical address
  buf_size	Data size
  pts	Timestamp (can be left empty)
  width	Image width
  height	Image height

• Related Data Types and Interfaces

  ALGO_HeadPose_Detect

  ALGO_HeadPose_Attr

4.3. PoseInputAttr_t

• Description

  Model input attributes structure

• Definition

  typedef struct
  {
      MI_U32 width;
      MI_U32 height;
      MI_IPU_ELEMENT_FORMAT format;
  } PoseInputAttr_t;
  

• Members

  Member Name	Description
  width	Input image width
  height	Input image height
  format	Input image format

• Related Data Types and Interfaces

  ALGO_HeadPose_GetInputAttr

4.4. PoseCls_t

• Description

  Face pose attribute recognition result structure

• Definition

  typedef struct
  {
      MI_S32 cls;
      MI_FLOAT score;
  } PoseCls_t;
  

• Members

  Member Name	Description
  cls	Face pose attribute (correspondence see HeadPoseAttr_e)
  score	Face pose attribute score

• Related Data Types and Interfaces

  ALGO_HeadPose_Attr

4.5. PoseBox_t

• Description

  Face pose attribute detection box structure

• Definition

  typedef struct
  {
      MI_U32 x;
      MI_U32 y;
      MI_U32 width;
      MI_U32 height;
      MI_S32 cls;
      MI_FLOAT score;
      MI_U64 pts;
      MI_FLOAT keypts[MAX_POSE_NUM_KEYPTS][2];
      MI_U64 track_id;
  } PoseBox_t;
  

• Members

  Member Name	Description
  x	Top-left x-coordinate
  y	Top-left y-coordinate
  width	Width of the box
  height	Height of the box
  cls	Class of the box
  score	Score of the box
  pts	Timestamp (can be left empty)
  keypts	Keypoint sequence
  track_id	Tracking ID

• Related Data Types and Interfaces

  ALGO_HeadPose_Detect

  ALGO_HeadPose_Attr

4.6. PoseParams_t

• Description

  Input parameters structure

• Definition

  typedef struct
  {
      MI_S32 disp_width;
      MI_S32 disp_height;
      MI_FLOAT min_width;
      MI_FLOAT min_height;
      MI_FLOAT det_threshold;
      MI_FLOAT cls_threshold;
      MI_FLOAT attr_threshold[MAX_POSE_NUM_ATTR];
  } PoseParams_t;
  

• Members

  Member Name	Description
  disp_width	Display width
  disp_height	Display height
  min_width	Minimum width for detection
  min_height	Minimum height for detection
  det_threshold	Detection threshold
  cls_threshold	Recognition threshold (not used in face pose recognition)
  attr_threshold	Attribute thresholds: the first is the eye openness threshold, the second is the mouth openness threshold; higher thresholds mean stricter standard for determining eye open/mouth open

• Related Data Types and Interfaces

  ALGO_HeadPose_SetParams

4.7. HeadPoseAttr_e

• Description

  Face pose attribute information

• Definition

  typedef enum
  {
      E_EYE_OPENNESS = 0,
      E_MOUTH_OPENNESS,
      E_NUM_HEAD_POSE_ATTR
  } HeadPoseAttr_e;
  

• Related Data Types and Interfaces

  PoseCls_t

  ALGO_HeadPose_Attr