Keywords Spotting algorithm

REVISION HISTORY

Revision No.
Description
 Date
1.0.0
  • Initial release
    		•  12/23/2024

    1. Overview

    1.1. Algorithm Description

    Keywords Spotting (KWS) is an algorithm that detects whether there is a specified wake-up word in the voice stream.

    1.2. Note

    The wake-up word can be customized. Please contact FAE if necessary.

    1. Generally, 4 characters are the best. If it is too short, it is prone to misrecognition. If it is too long, it is inconvenient for users to call and remember.

    2. The greater the differentiation between the initials and finals of adjacent Chinese characters in the command words, the better;

    3. Comply with the user's language habits, try to use common expressions, and make the content specific and direct;

    4. Avoid using everyday expressions, such as "吃饭啦";

    5. Rare characters and characters with zero initial consonants should be avoided as much as possible. For example, the two characters "语音" in "语音识别" are both characters with zero initial consonants.

    6. It is best not to use modal particles in the words of the command, such as "啊", "呢", etc.

    7. Avoid using reduplication, such as "你好你好";

    1.3. Model and command word list

    Model Name Supported Command Words
    kws_help_call.img "呼叫手机", "救命救命"

    2. API Reference

    2.1. Functional Module API

    API Name Function
    ALGO_KWS_CreateHandle Create kws algorithm handle
    ALGO_KWS_InitHandle Initialize the kws algorithm
    ALGO_KWS_GetInputLength Get the number of sampling points of the kws algorithm input
    ALGO_KWS_SetParams Set kws algorithm parameters
    ALGO_KWS_Run kws algorithm run
    ALGO_KWS_DeInitHandle kws handle deinitialization
    ALGO_KWS_ReleaseHandle Release kws algorithm resources

    2.2. ALGO_KWS_CreateHandle

    • Function

      Create a handle to kws.

    • grammar

      MI_S32 ALGO_KWS_CreateHandle(KWS_HANDLE* handle);

    • Parameters

      Parameter Name Description Input/Output
      handle handle output

    • Return value

      Return value Result
      0 Success
      Other values Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/ libsgsalgo_kws.a

    • Example

      none.

    2.3. ALGO_KWS_InitHandle

    • Function

      Initialize the kws handle.

    • grammar

      MI_S32 ALGO_KWS_InitHandle(KWS_HANDLE handle, const KwsInit_t init_info);

    • Parameters

      Parameter Name Description Input/Output
      handle kws handle input
      init_info Parameters for initialization handle Input

    • Return value

      Return value Result
      0 Success
      Other values Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/ libsgsalgo_kws.a

    • Notice

      • The KWS algorithm only supports 16kHz sampling rate and 16bit sampling width.

    • Example

      none.

    2.4. ALGO_KWS_GetInputLength

    • Function

      Get the number of samples input into the algorithm each time.

    • grammar

      MI_S32 ALGO_KWS_GetInputLength(KWS_HANDLE handle, MI_S32* input_length);

    • Parameters

      Parameter Name Description Input/Output
      handle kws algorithm handle input
      input_length sample length output

    • Return value

      Return value Result
      0 Success
      Other values Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/ libsgsalgo_kws.a

    2.5. ALGO_KWS_SetParams

    • Function

      Set the configuration parameters of kws

    • grammar

      MI_S32 ALGO_KWS_SetParams(KWS_HANDLE handle, const KwsParams_t* params);

    • Parameters

      Parameter Name Description Input/Output
      handle kws algorithm handle input
      params kws algorithm parameter structure input

    • Return value

      Return value Result
      0 Success
      Other values Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/ libsgsalgo_kws.a

    2.6. ALGO_KWS_Run

    • Function

      Run the kws algorithm.

    • grammar

      MI_S32 ALGO_KWS_Run(KWS_HANDLE handle, KwsInput_t* srcData, KwsResult_t* results);

    • Parameters

      Parameter Name Description Input/Output
      handle kws algorithm handle input
      srcData kws algorithm input data input
      result kws algorithm output result output

    • Return value

      Return value Result
      0 Success
      Other values Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/libsgsalgo_kws.a

    2.7. ALGO_KWS_DeInitHandle

    • Function

      Deinitialize the kws handle.

    • grammar

      MI_S32 ALGO_KWS_DeInitHandle(KWS_HANDLE handle);

    • Parameters

      Parameter Name Description Input/Output
      handle kws algorithm handle input

    • Return value

      Return value Result
      0 Success
      Non-zero Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/libsgsalgo_kws.a

    2.8. ALGO_KWS_ReleaseHandle

    • Function

      Release the handle.

    • grammar

      MI_S32 ALGO_KWS_ReleaseHandle(KWS_HANDLE handle);

    • Parameters

      Parameter Name Description Input/Output
      handle algorithm handle input

    • Return value

      Return value Result
      0 Success
      Other values Failed, refer to Error Code

    • Dependency

      • Header file: sgs_kws_api.h

      • Library files: libsgsalgo_kws.so/libsgsalgo_kws.a

    • Example

      See Demo code

    3. KWS data type

    Data Type Definition
    MAX_KWS_STRLEN Maximum length of kws algorithm file name
    MAX_KWS_NUM Maximum number of command words for the kws algorithm
    KwsInit_t kws algorithm initialization data structure
    KwsParams_t kws algorithm parameter setting structure
    KwsInput_t kws algorithm input data structure
    KwsResult_t kws algorithm output structure
    KWS_HANDLE kws handle

    3.2. MAX_KWS_STRLEN

    • illustrate

      Defines the maximum length of the kws algorithm file name.

    • Definition

      #define MAX_KWS_STRLEN 256

    3.3. MAX_KWS_NUM

    • illustrate

      The maximum number of command words for the kws algorithm.

    • Definition

      #define MAX_KWS_NUM 16

    3.4. KwsInit_t

    • illustrate

      kws algorithm initialization parameter structure.

    • Definition

      typedef struct KwsInit_
{
    MI_U8 ipu_firmware_bin[MAX_KWS_STRLEN];
    MI_U8 kws_model_path[MAX_KWS_STRLEN];
    MI_U8 units_path[MAX_KWS_STRLEN];
    MI_U32 beam_size;
    MI_BOOL create_device;
    MI_BOOL destroy_device;
}KwsInit_t;

    • Member

      Member Name Description
      ipu_firmware_bin The path of ipu_firmware_bin. If not specified, no need to set it.
      kws_model_path Path to the kws model
      units_path Modeling units for kws
      beam_size The size of beam_search. The larger the size, the higher the recall and the higher the false positive rate. The recommended value is 10. It must be greater than 0.
      create_device Whether to create ipu device
      destroy_device Whether to destroy ipu device

    • Related data types and interfaces

      ALGO_KWS_InitHandle

    3.5. KwsParams_t

    • illustrate

      Define the kws algorithm parameter structure.

    • Definition

      typedef struct
{
    MI_FLOAT threshold[MAX_KWS_NUM];
    MI_BOOL strict[MAX_KWS_NUM];
    MI_U8 key_words[MAX_KWS_NUM][MAX_KWS_STRLEN];
    MI_U32 words_num;
}KwsParams_t;

    • Member

      Member Name Description
      threshold kws threshold, recommended value is -8, the larger the value, the lower the recall and the lower the false detection.
      strict Whether to use a strict model, the recommended value is true. When false, the recall is high and the false detection is high
      key_words Set the command word. The encoding format must be utf-8. Check whether the code file is uft-8.
      words_num Number of command words

    • Related data types and interfaces

      ALGO_KWS_SetParams

    3.6. KwsInput_t

    • illustrate

      Define the kws algorithm input data structure.

    • Definition

      typedef struct
{
    void* p_vir_addr;
    MI_S32 sample_rate; //only support 16000
    MI_S32 bit_width; //only support 16
    MI_S32 channel; //only support 1
    MI_U16 length;
}KwsInput_t;

    • Member

      Member Name Description
      p_vir_addr Address of input data
      sample_rate The sampling rate of the input voice data, only 16k is supported
      bit_width The bit width of the input voice data, only supports short type 16 bits
      channel The number of voice channels. Only 1 channel is supported.
      length The length of the input data, which is consistent with ALGO_KWS_GetInputLength

    • Related data types and interfaces

      ALGO_KWS_Run

    3.7 KwsResult_t

    • illustrate

      Defines the handle type of the KWS algorithm.

    • Definition

      typedef struct
{
    MI_FLOAT score;
    MI_U8 centence[MAX_KWS_STRLEN];
    MI_U32 id;
}KwsResult_t;

    • Member

      Member Name Description
      score The score of the command word
      centence Recognized command word
      id Belongs to which command word

    • Notes

      none.

    • Related data types and interfaces

      ALGO_KWS_Run

    3.8. KWS_HANDLE

    4. Error code

    Error code Value Description
    E_ALGO_SUCCESS 0 Operation successful
    E_ALGO_HANDLE_NULL 1 The algorithm handle is null
    E_ALGO_INVALID_PARAM 2 Invalid input parameter
    E_ALGO_DEVICE_FAULT 3 Hardware error
    E_ALGO_LOADMODEL_FAIL 4 Failed to load model
    E_ALGO_INIT_FAIL 5 Algorithm initialization failed
    E_ALGO_NOT_INIT 6 Algorithm not initialized yet
    E_ALGO_INPUT_DATA_NULL 7 Algorithm input data is empty
    E_ALGO_INVALID_INPUT_SIZE 8 Invalid algorithm input data dimension
    E_ALGO_INVALID_LICENSE 9 Invalid license
    E_ALGO_MEMORY_OUT 10 Out of memory
    E_ALGO_FILEIO_ERROR 11 File read/write operation error
    E_ALGO_INVALID_OUTPUT_SIZE 12 Invalid algorithm output data dimension