MI IPU CASCADE API

---

REVISION HISTORY

Revision No.	Description	Date
1.0	Initial release.	01/07/2025
1.1	Add new APIs 'MI_IPU_PCIECasWrapper_QueryRemoteInputBufInfo', 'MI_IPU_PCIECasWrapper_SetRemoteInputBuf', 'MI_IPU_PCIECasWrapper_GetRemoteInputBuf', 'MI_IPU_PCIECasWrapper_RunService' and 'MI_IPU_PCIECasWrapper_StopService'. Add new struct 'MI_IPU_PCIECasServiceConfig_t', 'MI_IPU_PCIECasTensorDesc_t', 'MI_IPU_PCIECasInputDesc_t', 'MI_IPU_PCIECasTransferLinklist_t' and 'MI_IPU_PCIECasTransferDesc_t'. Add new error code 'E_MI_IPU_PCIE_CAS_ERR_INVALID_SUBNET_ID', 'E_MI_IPU_PCIE_CAS_ERR_MODEL_CORRUPTED', 'E_MI_IPU_PCIE_CAS_ERR_MAP' and 'E_MI_IPU_PCIE_CAS_ERR_DEVICE_NOT_READY'. Add 'ipu_service' demo code.	02/28/2026

---

1. OVERVIEW

---

1.1. Module Description

The IPU Cascade API is designed for application scenarios where multiple IPUs process the same LLM (Large Language Model) in parallel. Its main function is to accelerate the inference of LLM models on the board. The IPU Cascade API maintains a consistent style with the general MI IPU interface, facilitating the extension of PCIe cascade capabilities in existing projects.

1.2. Basic Structure

• RC (Root Complex) Application: Responsible for reading cascade offline models, creating devices/channels, preparing input/output Tensors, and controlling EP-side service programs to execute model inference.
• EP (Endpoint) Service: Runs on IPU boards connected via the PCIe bus, responsible for responding to RC commands and calling IPU Cascade API to complete model inference.

1.3. Module Functions

The IPU Cascade API supports the following features:

1. Supports collaborative inference of cascade models across local and remote IPU boards

2. Supports controlling remote IPU boards for non-cascade model inference

3. Supports multiple channels

4. Supports specifying priority for IPU inference tasks

5. Supports single-input inference and multi-input inference per batch

1.4. Application Scenarios

• Large Model Cascade: Splits cascade large models into multiple sub-networks and distributes them across multiple IPUs to improve throughput and capacity.
• Remote Inference Acceleration: Models run entirely on EP, while RC is responsible for data scheduling.

1.5. Working Principle

Before calling the IPU Cascade module for model inference, users first need to use the IPU SDK toolchain to convert the original large language model into an offline cascade model file supported by the hardware. Then, by calling the IPU Cascade API, load the offline cascade model locally on the board and control the remote board to accelerate the offline cascade model inference.

1.6. Interface Call Flow

1. MI_IPU_PCIECasWrapper_GetOfflineStaticInfo
2. MI_IPU_PCIECasWrapper_CreateDevice
3. MI_IPU_PCIECasWrapper_CreateCHN
4. MI_IPU_GetInOutTensorDesc
5. MI_IPU_PCIECasWrapper_SubNetInvoke
6. MI_IPU_PCIECasWrapper_DestroyCHN
7. MI_IPU_PCIECasWrapper_DestroyDevice

1.7. Example

The SDK provides examples at sdk/release_feature/source/ipu/ipu_client/ (RC) and sdk/release_feature/source/ipu/ipu_service/ (EP). You can directly refer to how they organize parameters and call public APIs.

1. ipu_client

   #include <fcntl.h>
   #include <sgs_rpmsg.h>
   #include <string.h>
   #include <sys/ioctl.h>
   #include <sys/types.h>
   #include <sys/mman.h>
   #include <unistd.h>
   #include <stdio.h>
   #include <errno.h>
   #include <pthread.h>
   #include <getopt.h>
   #include <stdbool.h>
   #include <stdlib.h>
   
   #include "mi_sys.h"
   #include "mi_ipu.h"
   #include "mi_ipu_pcie_cascade_datatype.h"
   #include "mi_ipu_pcie_cascade.h"
   #include "ipu_client.h"
   
   static void _showUsage(void)
   {
       printf("Usage: ./prog_ipu_ipu_client [-m] [-i][...]\n");
       printf(" -m,       model file\n");
       printf(" -i,       input tensor file\n");
       printf(" -l,       loop count, if not set, only invoke once\n");
       printf(" -n,       nbatch\n");
       printf(" -j,       subnet id\n");
       printf(" -a,       ipu affinity, multi core use\n");
       printf(" -p,       invoke priority, -10~39\n");
   }
   
   static MI_S32 IPU_Client_GetInputTensorData(char *InputPath, void **pInput, int *filelen_in)
   {
       int fd1;
       void *pmem = NULL;
       off_t filelen1;
   
       if (!InputPath){
           return -1;
       }
   
       fd1 = open(InputPath, O_RDONLY, 0644);
       if (fd1 < 0)
           return -1;
       filelen1 = lseek(fd1, 0L, SEEK_END);
       if (filelen1 <= 0){
           return -1;
       }
       lseek(fd1, 0L, SEEK_SET);
       pmem = mmap(NULL, filelen1, PROT_READ, MAP_SHARED, fd1, 0);
       close(fd1);
       if (pmem == MAP_FAILED)
           return -1;
       *pInput = pmem;
       *filelen_in = filelen1;
   
       return MI_SUCCESS;
   }
   
   static MI_S32 IPU_Client_InsertInputNbatch(void *pInput, MI_IPU_SubNet_InputOutputDesc_t *pDesc,
                                                       MI_IPU_BatchInvokeParam_t *pInvokeParam)
   {
       int cnt, batch_cnt;
       int total_cnt = 0;
       char *_pInput = (char *)pInput;
   
       for (batch_cnt = 0; batch_cnt < pInvokeParam->u32BatchN; batch_cnt++) {
           for (cnt = 0; cnt < pDesc->u32InputTensorCount; cnt++) {
               memcpy(pInvokeParam->astArrayTensors[total_cnt].ptTensorData[0], (void *)_pInput, pDesc->astMI_InputTensorDescs[cnt].s32AlignedBufSize);
               MI_SYS_FlushInvCache(pInvokeParam->astArrayTensors[total_cnt].ptTensorData[0], pDesc->astMI_InputTensorDescs[cnt].s32AlignedBufSize);
               _pInput += pDesc->astMI_InputTensorDescs[cnt].s32AlignedBufSize;
               total_cnt++;
           }
       }
       return MI_SUCCESS;
   }
   
   MI_S32 IPU_Client_AllocTensors(MI_IPU_SubNet_InputOutputDesc_t *pDesc,
                                       IpuClientTestParam_t *pstTestParam,
                                       IpuClientRunbufInfo_t *pstIpuRuntimeInfo,
                                       IpuClientTensorAddrInfo_t *pstTensorAddrInfo,
                                       EN_IpuBufType eBufType, MI_U32 u32DeviatonBytes)
   {
       int i, j, s32Ret;
       MI_U32 u32RequireBufSize = 0, u32Nbatch;
       MI_U32 u32InCnt;
   
       MI_PHY *pu64TensorPA;
       void **ppTensorVA;
   
       MI_PHY u64PABase;
       void *pVABase;
   
       u32Nbatch = pstTestParam->u32Nbatch;
       if (!u32Nbatch) {
           u32Nbatch = 1;
       }
   
       switch (eBufType) {
           case EN_INPUT_BUF:
               for (j = 0; j < u32Nbatch; j++) {
                   for (i = 0; i < pDesc->u32InputTensorCount; i++) {
                       u32RequireBufSize += pDesc->astMI_InputTensorDescs[i].s32AlignedBufSize;
                       u32RequireBufSize += ALIGN_UP(u32DeviatonBytes, ALIGN_SIZE);
                       u32RequireBufSize += u32DeviatonBytes;
                   }
               }
   
               pu64TensorPA = &pstIpuRuntimeInfo->u64InTensorPA;
               ppTensorVA = &pstIpuRuntimeInfo->pInTensorVA;
               pstIpuRuntimeInfo->u32InTensorSize = u32RequireBufSize;
               break;
           case EN_OUTPUT_BUF:
               for (j = 0; j < u32Nbatch; j++) {
                   for (i = 0; i < pDesc->u32OutputTensorCount; i++) {
                       u32RequireBufSize += pDesc->astMI_OutputTensorDescs[i].s32AlignedBufSize;
                       u32RequireBufSize += ALIGN_UP(u32DeviatonBytes, ALIGN_SIZE);
                       u32RequireBufSize += u32DeviatonBytes;
                   }
               }
   
               pu64TensorPA = &pstIpuRuntimeInfo->u64OutTensorPA;
               ppTensorVA = &pstIpuRuntimeInfo->pOutTensorVA;
               pstIpuRuntimeInfo->u32OutTensorSize = u32RequireBufSize;
               break;
           default:
               u32RequireBufSize = 0;
               printf("error: don't support tensor type: %d\n", eBufType);
               return -1;
       }
   
       s32Ret = MI_SYS_MMA_Alloc(0, NULL, u32RequireBufSize, pu64TensorPA);
       if (s32Ret != MI_SUCCESS) {
           printf("fail to allocate %s buffer\n", eBufType==EN_INPUT_BUF?"input":"output");
           return -1;
       }
       s32Ret = MI_SYS_Mmap(*pu64TensorPA, u32RequireBufSize, ppTensorVA, TRUE);
       if (s32Ret != MI_SUCCESS) {
           printf("Error: fail to map %s address, error=%d\n", eBufType==EN_INPUT_BUF?"input":"output", s32Ret);
           return -1;
       }
   
       if (u32DeviatonBytes != 0) {
           u64PABase = ALIGN_UP(*pu64TensorPA, ALIGN_SIZE) + u32DeviatonBytes;
       } else {
           u64PABase = *pu64TensorPA;
       }
       pVABase = *ppTensorVA + (u64PABase - *pu64TensorPA);
   
       if (eBufType == EN_INPUT_BUF) {
           pstTensorAddrInfo->u32InCnt = pDesc->u32InputTensorCount * u32Nbatch;
           for (i = 0; i < u32Nbatch; i++) {
               for (j = 0; j < pDesc->u32InputTensorCount; j++) {
                   pstTensorAddrInfo->astArrayTensors[i*pDesc->u32InputTensorCount+j].phyTensorAddr[0] = u64PABase;
                   pstTensorAddrInfo->astArrayTensors[i*pDesc->u32InputTensorCount+j].ptTensorData[0] = pVABase;
   
                   u64PABase += pDesc->astMI_InputTensorDescs[j].s32AlignedBufSize;
                   if (u32DeviatonBytes != 0) {
                       u64PABase = ALIGN_UP(u64PABase, ALIGN_SIZE) + u32DeviatonBytes;
                   }
                   pVABase = *ppTensorVA + (u64PABase - *pu64TensorPA);
               }
           }
       } else {
           u32InCnt = pDesc->u32InputTensorCount * u32Nbatch;
           pstTensorAddrInfo->u32OutCnt = pDesc->u32OutputTensorCount * u32Nbatch;
           for (i = 0; i < u32Nbatch; i++) {
               for (j = 0; j < pDesc->u32OutputTensorCount; j++) {
                   pstTensorAddrInfo->astArrayTensors[u32InCnt+i*pDesc->u32OutputTensorCount+j].phyTensorAddr[0] = u64PABase;
                   pstTensorAddrInfo->astArrayTensors[u32InCnt+i*pDesc->u32OutputTensorCount+j].ptTensorData[0] = pVABase;
                   u64PABase += pDesc->astMI_OutputTensorDescs[j].s32AlignedBufSize;
                   if (u32DeviatonBytes != 0) {
                       u64PABase = ALIGN_UP(u64PABase, ALIGN_SIZE) + u32DeviatonBytes;
                   }
                   pVABase = *ppTensorVA + (u64PABase - *pu64TensorPA);
               }
           }
       }
       return 0;
   }
   
   void IPU_Client_FreeTensors(IpuClientRunbufInfo_t *pstIpuRuntimeInfo, EN_IpuBufType eBufType)
   {
       if (eBufType == EN_INPUT_BUF) {
           MI_SYS_Munmap(pstIpuRuntimeInfo->pInTensorVA, pstIpuRuntimeInfo->u32InTensorSize);
           MI_SYS_MMA_Free(0, pstIpuRuntimeInfo->u64InTensorPA);
       } else {
           MI_SYS_Munmap(pstIpuRuntimeInfo->pOutTensorVA, pstIpuRuntimeInfo->u32OutTensorSize);
           MI_SYS_MMA_Free(0, pstIpuRuntimeInfo->u64OutTensorPA);
       }
   }
   
   int main(int argc, char * argv[])
   {
       int opt;
       int count = 0;
       double linux_time = 0, total_linux_time = 0, total_ipu_time = 0;
       MI_U64 total_bw = 0, total_bw_rd = 0, total_bw_wr = 0;
       int  file_len_in = 0;
       void *pInput = NULL;
       MI_U32 u32ChnId;
       MI_S32 s32Ret;
       int cnt, batch_cnt;
       int total_cnt;
   
       struct timespec invoke_start, invoke_end;
       IpuClientTestParam_t stTestParam;
   
       MI_IPU_PCIECasDevAttr_t stCASDevAttr;
       MI_IPU_PCIECasOfflineModelStaticInfo_t stCASOfflineModelInfo;
       MI_IPU_ModelDesc_t *pstModelDesc;
       MI_IPU_SubNet_InputOutputDesc_t *pstTensorDesc;
   
       MI_IPU_SubNetBatchInvokeParam_t stSubNetInvokeParam;
       MI_IPU_BatchInvokeParam_t stInvokParam;
       MI_IPU_PCIECasSubNetBatchInvokeParam_t stIPUCASInvokeParam;
       MI_IPU_RuntimeInfo_t  stRuntimeInfo;
       IpuClientTensorAddrInfo_t stTensorAddrInfo;
       IpuClientRunbufInfo_t stIpuRunBufInfo;
       MI_IPU_DevAttr_t stIPUDevAttr;
       MI_IPUChnAttr_t stIPUChnAttr;
   
       memset(&stTestParam, 0, sizeof(stTestParam));
       while (true) {
           int option_index = 0;
           static struct option long_options[] =
           {
               {"model",           required_argument, 0,  'm' },
               {"input",           required_argument, 0,  'i' },
               {"loop_count",      required_argument, 0,  'l' },
               {"nbatch",          required_argument, 0,  'n' },
               {"sub_net_id",      required_argument, 0,  'j' },
               {"affinity",        required_argument, 0,  'a' },
               {"priority",        required_argument, 0,  'p' }
           };
           opt = getopt_long(argc, argv, "m:i:o:l:n:j:a:p", long_options, &option_index);
           if (opt == -1) {
               break;
           }
           switch (opt) {
               case 'm':
                   stTestParam.ModelPath = optarg;
                   break;
               case 'i':
                   stTestParam.InputPath = optarg;
                   break;
                   break;
               case 'l':
                   stTestParam.u32LoopCount = strtoul(optarg, NULL, 16);
                   break;
               case 'n':
                   stTestParam.u32Nbatch = strtoul(optarg, NULL, 16);
                   break;
               case 'j':
                   stTestParam.u32SubNetId = strtoul(optarg, NULL, 10);
                   stTestParam.u32SubNetInvoke = 1;
                   break;
               case 'a':
                   stTestParam.u32IpuAffinity = strtoul(optarg, NULL, 10);
                   break;
               case 'p':
                   stTestParam.s32TaskPrio = strtoul(optarg, NULL, 10);
                   break;
               default:
                   _showUsage();
                   return 0;
           }
       }
       if (argc <= 2 || !stTestParam.ModelPath || !stTestParam.InputPath) {
           _showUsage();
           goto SYS_EXIT;
       }
   
       if (stTestParam.u32LoopCount == 0)
           stTestParam.u32LoopCount = 1;
   
       if (stTestParam.u32Nbatch == 0)
           stTestParam.u32Nbatch = 1;
   
       if (stTestParam.u32SubNetInvoke == 0)
           stTestParam.u32SubNetInvoke = 1;
   
       MI_SYS_Init(0);
   
       s32Ret = MI_IPU_PCIECasWrapper_GetOfflineStaticInfo(NULL, stTestParam.ModelPath, &stCASOfflineModelInfo);
       if (s32Ret != MI_SUCCESS)
       {
           printf("fail to get %s static info\n", stTestParam.ModelPath);
           goto SYS_EXIT;
       }
   
       memset(&stIPUDevAttr, 0, sizeof(stIPUDevAttr));
       memset(&stCASDevAttr, 0, sizeof(stCASDevAttr));
       stIPUDevAttr.u32CoreMask = stTestParam.u32IpuAffinity;
       stIPUDevAttr.u32MaxVariableBufSize = stCASOfflineModelInfo.au32VariableBufferSize[E_MI_IPU_PCIE_CAS_LOCAL_VARIABLE_BUFFER_IDX];
   
       memcpy(&stCASDevAttr.stIPUDevAttr, &stIPUDevAttr, sizeof(stIPUDevAttr));
       memcpy(stCASDevAttr.au32MaxVariableBufSize, stCASOfflineModelInfo.au32VariableBufferSize, sizeof(MI_U32)*stCASOfflineModelInfo.u32SocNum);
       stCASDevAttr.u32SocNum = stCASOfflineModelInfo.u32SocNum;
   
       s32Ret = MI_IPU_PCIECasWrapper_CreateDevice(&stCASDevAttr, NULL, NULL, E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
       if (s32Ret != MI_SUCCESS)
       {
           printf("Fail to Create Device, ret=%d\n", s32Ret);
           goto SYS_EXIT;
       }
   
       memset(&stIPUChnAttr, 0, sizeof(stIPUChnAttr));
       stIPUChnAttr.u32InputBufDepth = 0;
       stIPUChnAttr.u32OutputBufDepth = 0;
       stIPUChnAttr.u32BatchMax = stTestParam.u32Nbatch;
       stIPUChnAttr.u32SubNetId = stTestParam.u32SubNetId;
   
       s32Ret = MI_IPU_PCIECasWrapper_CreateCHN(&u32ChnId, &stIPUChnAttr, NULL, stTestParam.ModelPath, E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
       if (s32Ret != MI_SUCCESS)
       {
           printf("Fail to Create CHN, ret=%d\n", s32Ret);
           goto DESTROY_DEV;
       }
   
       pstModelDesc = malloc(sizeof(MI_IPU_ModelDesc_t));
       if (!pstModelDesc)
       {
           printf("Fail to malloc model description\n");
           goto DESTROY_CHN;
       }
       memset(pstModelDesc, 0, sizeof(MI_IPU_ModelDesc_t));
   
       s32Ret = MI_IPU_GetSubNetDesc(u32ChnId, pstModelDesc);
       if (s32Ret != MI_SUCCESS)
       {
           printf("Get model desc failed!\n");
           goto DESTROY_CHN;
       }
       if (stTestParam.u32SubNetId >= pstModelDesc->u32SubNetNum)
       {
           printf("This model(sub net num %u) not support sub net id %u!\n", pstModelDesc->u32SubNetNum, stTestParam.u32SubNetId);
           goto DESTROY_CHN;
       }
       pstTensorDesc = &pstModelDesc->astSubNetDesc[stTestParam.u32SubNetId].stSubNetInputOutputDesc;
   
       // allocate input/output buffers
       memset(&stIpuRunBufInfo, 0, sizeof(stIpuRunBufInfo));
       s32Ret = IPU_Client_AllocTensors(pstTensorDesc, &stTestParam, &stIpuRunBufInfo, &stTensorAddrInfo, EN_INPUT_BUF, 0);
       if (s32Ret) {
           goto DESTROY_CHN;
       }
   
       s32Ret = IPU_Client_AllocTensors(pstTensorDesc, &stTestParam, &stIpuRunBufInfo, &stTensorAddrInfo, EN_OUTPUT_BUF, 0);
       if (s32Ret) {
           IPU_Client_FreeTensors(&stIpuRunBufInfo, EN_INPUT_BUF);
           goto DESTROY_CHN;
       }
   
       s32Ret = IPU_Client_GetInputTensorData(stTestParam.InputPath, &pInput, &file_len_in);
       if (s32Ret != MI_SUCCESS)
       {
           printf("Get input data failed!\n");
           goto FREE_TENSOR;
       }
   
       memset(&stInvokParam, 0, sizeof(stInvokParam));
       memset(&stSubNetInvokeParam, 0, sizeof(MI_IPU_SubNetBatchInvokeParam_t));
       memset(&stIPUCASInvokeParam, 0, sizeof(MI_IPU_PCIECasSubNetBatchInvokeParam_t));
       stInvokParam.u32BatchN = stTestParam.u32Nbatch;
       stInvokParam.s32TaskPrio = stTestParam.s32TaskPrio;
       stInvokParam.u32IpuAffinity = stTestParam.u32IpuAffinity;
   
       for (int i = 0; i < stTensorAddrInfo.u32InCnt + stTensorAddrInfo.u32OutCnt; i++)
       {
           stInvokParam.astArrayTensors[i] = stTensorAddrInfo.astArrayTensors[i];
       }
   
       stSubNetInvokeParam.eBatchMode = E_IPU_BATCH_N_BUF_MODE;
       stSubNetInvokeParam.stInvokeParam = stInvokParam;
       stIPUCASInvokeParam.stSubnNetBatchInvokeParam = stSubNetInvokeParam;
   
       s32Ret = IPU_Client_InsertInputNbatch(pInput, pstTensorDesc, &stInvokParam);
       if (s32Ret != MI_SUCCESS)
       {
           printf("Insert input failed\n");
           goto MUNMAP;
       }
   
       total_cnt = stInvokParam.u32BatchN * pstTensorDesc->u32InputTensorCount;
       // invalid output buffer
       for (batch_cnt = 0; batch_cnt < stInvokParam.u32BatchN; batch_cnt++)
       {
       for (cnt = 0; cnt < pstTensorDesc->u32OutputTensorCount; cnt++)
       {
           memset(stInvokParam.astArrayTensors[total_cnt].ptTensorData[0], 0, pstTensorDesc->astMI_OutputTensorDescs[cnt].s32AlignedBufSize);
           MI_SYS_FlushInvCache(stInvokParam.astArrayTensors[total_cnt].ptTensorData[0], pstTensorDesc->astMI_OutputTensorDescs[cnt].s32AlignedBufSize);
           total_cnt++;
       }
       }
   
       memset(&stRuntimeInfo, 0, sizeof(stRuntimeInfo));
       while (count  < stTestParam.u32LoopCount)
       {
           clock_gettime(CLOCK_MONOTONIC, &invoke_start);
           s32Ret = MI_IPU_PCIECasWrapper_SubNetInvoke(u32ChnId, &stIPUCASInvokeParam, &stRuntimeInfo, stTestParam.u32SubNetId, E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
           if (s32Ret != MI_SUCCESS)
           {
               printf("Invoke failed\n");
               goto FREE_TENSOR;
           }
           clock_gettime(CLOCK_MONOTONIC, &invoke_end);
   
           linux_time = (invoke_end.tv_sec * 1000000 + invoke_end.tv_nsec / 1000)
               - (invoke_start.tv_sec * 1000000 + invoke_start.tv_nsec / 1000);
           printf("Run model %s loop%d success, invoke time: %f us\n",
               stTestParam.ModelPath, count, linux_time);
   
           printf("============ Loop%d ipu_time=%lluus\n", count, stRuntimeInfo.u64IpuTime);
           total_linux_time += linux_time;
           total_ipu_time += stRuntimeInfo.u64IpuTime;
           total_bw += stRuntimeInfo.u64BandWidth;
           total_bw_rd += stRuntimeInfo.u64BandWidthRead;
           total_bw_wr += stRuntimeInfo.u64BandWidthWrite;
           count++;
       }
       printf("cycles=%lfus linux_fps=%lffps ipu_fps=%lffps bandwidth_total=%llubytes bandwidth_rd=%llubytes bandwidth_wr=%llubytes\n",
           total_ipu_time/stTestParam.u32LoopCount,
           1/(total_linux_time/stTestParam.u32Nbatch/stTestParam.u32LoopCount/1000000),
           1/(total_ipu_time/stTestParam.u32LoopCount/1000000),
           total_bw/stTestParam.u32LoopCount, total_bw_rd/stTestParam.u32LoopCount, total_bw_wr/stTestParam.u32LoopCount);
   
   MUNMAP:
       if (stTestParam.InputPath)
           munmap(pInput, file_len_in);
   
   FREE_TENSOR:
       IPU_Client_FreeTensors(&stIpuRunBufInfo, EN_INPUT_BUF);
       IPU_Client_FreeTensors(&stIpuRunBufInfo, EN_OUTPUT_BUF);
   
   DESTROY_CHN:
       MI_IPU_PCIECasWrapper_DestroyCHN(u32ChnId, E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
   
   DESTROY_DEV:
       MI_IPU_PCIECasWrapper_DestroyDevice(E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
   
   SYS_EXIT:
       MI_SYS_Exit(0);
   
       return s32Ret;
   }
   

2. ipu_service

   #include <stdio.h>
   #include <stdlib.h>
   #include <error.h>
   #include <sys/mman.h>
   #include <fcntl.h>
   #include <string.h>
   #include <sys/ioctl.h>
   #include <sys/types.h>
   #include <unistd.h>
   #include <errno.h>
   #include <pthread.h>
   #include <stdbool.h>
   #include <poll.h>
   #include <assert.h>
   #include <signal.h>
   #include <getopt.h>
   
   #include "mi_sys.h"
   #include "ipu_service.h"
   
   static MI_U32 g_u32MaxWorkThread = 0;
   
   static void _IPU_Service_ShowUsage(const char *progName)
   {
       printf("Usage: %s [options]\n", progName);
       printf("Options:\n");
       printf(" -t        show number of work thread\n");
       printf(" -h        show this help message\n");
   }
   
   static MI_S32 _IPU_Service_ParseArgs(int argc, char *argv[])
   {
       int opt;
       int option_index = 0;
   
       static struct option long_options[] =
       {
           {"t",     required_argument, 0,     't'},
           {"help",      no_argument,       0, 'h'},
           {0, 0, 0, 0}
       };
   
       while ((opt = getopt_long(argc, argv, "t:h", long_options, &option_index)) != -1)
       {
           switch (opt)
           {
               case 't':
               {
                   MI_U32 u32ThreadCount = strtoul(optarg, NULL, 10);
                   g_u32MaxWorkThread = u32ThreadCount;
                   break;
               }
               case 'h':
                   _IPU_Service_ShowUsage(argv[0]);
                   return 1; // Signal to exit after showing help
               default:
                   printf("Unknown option: %c\n", opt);
                   _IPU_Service_ShowUsage(argv[0]);
                   return -1;
           }
       }
   
       // Check for remaining non-option arguments
       if (optind < argc)
       {
           printf("Unknown arguments: ");
           for (int i = optind; i < argc; i++)
           {
               printf("%s ", argv[i]);
           }
           printf("\n");
           _IPU_Service_ShowUsage(argv[0]);
           return -1;
       }
   
       return 0;
   }
   
   int main(int argc, char *argv[])
   {
       MI_S32 s32Ret;
       MI_IPU_PCIECasServiceConfig_t stCasServiceCfg;
       // Parse command line arguments
       s32Ret = _IPU_Service_ParseArgs(argc, argv);
       if (s32Ret < 0)
       {
           return -1;
       }
       else if (s32Ret > 0)
       {
           return 0; // Help was shown, exit normally
       }
   
       printf("IPU Service starting with thread number: %u\n", g_u32MaxWorkThread);
   
       MI_SYS_Init(0);
       memset(&stCasServiceCfg, 0, sizeof(stCasServiceCfg));
       stCasServiceCfg.u32MaxWorkThread = g_u32MaxWorkThread;
       if (MI_IPU_PCIECasWrapper_RunService(&stCasServiceCfg) != MI_SUCCESS) {
           printf("Failed to start EP service\n");
           return -1;
       }
   
       MI_IPU_PCIECasWrapper_StopService();
       MI_SYS_Exit(0);
   
       printf("Service exited cleanly\n");
       return 0;
   }
   

---

2. API REFERENCE

---

2.1. Functional Module APIs

API Name	Function
MI_IPU_PCIECasWrapper_GetOfflineStaticInfo	Read static information of cascade offline model
MI_IPU_PCIECasWrapper_CreateDevice	Create cascade device
MI_IPU_PCIECasWrapper_CreateCHN	Create cascade channel
MI_IPU_PCIECasWrapper_SubNetInvoke	Trigger cascade model inference
MI_IPU_PCIECasWrapper_QueryRemoteInputBufInfo	Query remote input buffer info
MI_IPU_PCIECasWrapper_SetRemoteInputBuf	Write remote input buffers
MI_IPU_PCIECasWrapper_GetRemoteInputBuf	Read remote input buffers
MI_IPU_PCIECasWrapper_DestroyCHN	Destroy cascade channel
MI_IPU_PCIECasWrapper_DestroyDevice	Destroy cascade device
MI_IPU_PCIECasWrapper_RunService	Start EP-side service
MI_IPU_PCIECasWrapper_StopService	Stop EP-side service

---

2.2. MI_IPU_PCIECasWrapper_GetOfflineStaticInfo

• Function

  Parse the offline model header information to obtain the number of SoCs, model size, and variable buffer requirements.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_GetOfflineStaticInfo(SerializedReadFunc pReadFunc,
                                                  char *pReadCtx,
                                                  MI_IPU_PCIECasOfflineModelStaticInfo_t *pStaticInfo);
  

• Parameters

  Parameter Name	Description	Input/Output
  pReadFunc	User-defined file reading function (set to NULL to use the default file reading function provided by IPU Cascade API)	Input
  pReadCtx	Model path	Input
  pStaticInfo	Pointer to the offline cascade model static information structure	Output

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasOfflineModelStaticInfo_t stStaticInfo;
  memset(&stStaticInfo, 0, sizeof(stStaticInfo));
  if (MI_IPU_PCIECasWrapper_GetOfflineStaticInfo(NULL,
  
                                               stTestParam.ModelPath,
                                               &stStaticInfo) != MI_SUCCESS)
  {
  
    printf("fail to read cascade model info\n");
    return -1;
  }
  

---

2.3. MI_IPU_PCIECasWrapper_CreateDevice

• Function

  Create a cascade device and initialize the cascade runtime environment.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_CreateDevice(MI_IPU_PCIECasDevAttr_t *pstIPUDevAttr,
  
                                          SerializedReadFunc pReadFunc,
                                          char *pReadCtx,
                                          MI_IPU_PCIECasWorkMode_e eWorkMode);
  

• Parameters

  Parameter Name	Description	Input/Output
  pstIPUDevAttr	Cascade device attributes, including SoC count and variable buffer limits for each SoC	Input
  pReadFunc	Optional firmware reading function	Input
  pReadCtx	Firmware path or context	Input
  eWorkMode	Work mode: E_MI_IPU_PCIE_CAS_WORK_MODE_RC or E_MI_IPU_PCIE_CAS_WORK_MODE_EP	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasDevAttr_t stCasAttr;
  memset(&stCasAttr, 0, sizeof(stCasAttr));
  stCasAttr.u32SocNum = stStaticInfo.u32SocNum;
  memcpy(stCasAttr.au32MaxVariableBufSize,
  
       stStaticInfo.au32VariableBufferSize,
       sizeof(stCasAttr.au32MaxVariableBufSize));
  if (MI_IPU_PCIECasWrapper_CreateDevice(&stCasAttr,
  
                                       NULL,
                                       NULL,
                                       E_MI_IPU_PCIE_CAS_WORK_MODE_RC) != MI_SUCCESS)
  {
  
    printf("create cascade device failed\n");
    return -1;
  }
  

---

2.4. MI_IPU_PCIECasWrapper_CreateCHN

• Function

  Create a cascade channel.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_CreateCHN(MI_IPU_CHN *ptChnId,
  
                                       MI_IPUChnAttr_t *pstChnAttr,
                                       SerializedReadFunc pReadFunc,
                                       char *pReadCtx,
                                       MI_IPU_PCIECasWorkMode_e eWorkMode);
  

• Parameters

  Parameter Name	Description	Input/Output
  ptChnId	Returns the successfully created channel ID	Output
  pstChnAttr	Channel attributes (input/output queue depth, subnet ID, etc.)	Input
  pReadFunc	Model reading function, can be NULL	Input
  pReadCtx	RC: model path; EP: model physical address context	Input
  eWorkMode	Work mode: RC/EP	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_CHN u32ChnId = 0;
  MI_IPUChnAttr_t stChnAttr;
  memset(&stChnAttr, 0, sizeof(stChnAttr));
  stChnAttr.u32InputBufDepth  = 2;
  stChnAttr.u32OutputBufDepth = 2;
  if (MI_IPU_PCIECasWrapper_CreateCHN(&u32ChnId,
  
                                    &stChnAttr,
                                    NULL,
                                    stTestParam.ModelPath,
                                    E_MI_IPU_PCIE_CAS_WORK_MODE_RC) != MI_SUCCESS)
  {
  
    printf("create cascade channel failed\n");
    return -1;
  }
  

---

2.5. MI_IPU_PCIECasWrapper_SubNetInvoke

• Function

  Execute cascade model inference.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_SubNetInvoke(MI_IPU_CHN u32ChnId,
  
                                          MI_IPU_PCIECasSubNetBatchInvokeParam_t *pstInvokeParam,
                                          MI_IPU_RuntimeInfo_t *pstRuntimeInfo,
                                          MI_U32 u32SubNetId,
                                          MI_IPU_PCIECasWorkMode_e eWorkMode);
  

• Parameters

  Parameter Name	Description	Input/Output
  u32ChnId	Channel ID to execute	Input
  pstInvokeParam	Cascade batch parameters, including input/output Tensor information and shared output position	Input
  pstRuntimeInfo	Inference runtime information (time, bandwidth, etc.)	Output
  u32SubNetId	Target subnet ID	Input
  eWorkMode	Work mode: RC/EP	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasSubNetBatchInvokeParam_t stInvokeParam;
  memset(&stInvokeParam, 0, sizeof(stInvokeParam));
  stInvokeParam.stSubnNetBatchInvokeParam.u32BatchN = stTestParam.u32Nbatch;
  /* Fill input/output Tensors according to model */
  if (MI_IPU_PCIECasWrapper_SubNetInvoke(u32ChnId,
                                         &stInvokeParam,
                                         &stRuntimeInfo,
                                         stTestParam.u32SubNetId,
                                         E_MI_IPU_PCIE_CAS_WORK_MODE_RC) != MI_SUCCESS)
  {
      printf("invoke cascade subnet failed\n");
      return -1;
  }
  

---

2.6. MI_IPU_PCIECasWrapper_QueryRemoteInputBufInfo

• Function

  Query the input tensor buffer information (count, size, and physical address) of a specific EP so that the RC side can build transfer linklists based on the real layout.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_QueryRemoteInputBufInfo(MI_IPU_CHN u32ChnId,
                                                       MI_U32 u32EpId,
                                                       MI_IPU_PCIECasInputDesc_t *pstCasInputDesc);
  

• Parameters

  Parameter Name	Description	Input/Output
  u32ChnId	Channel ID to execute	Input
  u32EpId	Target EP board index	Input
  pstCasInputDesc	Returned remote input tensor descriptor pointer, containing u32TensorCount and each tensor buffer size/address	Output

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasInputDesc_t stRemoteInputDesc;
  memset(&stRemoteInputDesc, 0, sizeof(stRemoteInputDesc));
  if (MI_IPU_PCIECasWrapper_QueryRemoteInputBufInfo(u32ChnId,
                                                    0,
                                                    &stRemoteInputDesc) != MI_SUCCESS)
  {
      printf("query remote input buffer failed\n");
      return -1;
  }
  

---

2.7. MI_IPU_PCIECasWrapper_SetRemoteInputBuf

• Function

  Write local tensor data to the remote EP input buffer according to the provided linklist descriptor.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_SetRemoteInputBuf(MI_IPU_CHN u32ChnId,
                                                 MI_U32 u32EpId,
                                                 MI_IPU_PCIECasTransferDesc_t *pstCasTransferDesc);
  

• Parameters

  Parameter Name	Description	Input/Output
  u32ChnId	Channel ID to execute	Input
  u32EpId	Target EP board index	Input
  pstCasTransferDesc	Transfer descriptor, containing u32LinklistCount entries (up to MI_IPU_PCIE_CASCADE_MAX_LINKLIST_CNT)	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasTransferDesc_t stTransferDesc;
  memset(&stTransferDesc, 0, sizeof(stTransferDesc));
  stTransferDesc.u32LinklistCount = 1;
  stTransferDesc.astTransferLinklist[0].phySrcAddr = stLocalTensor.phyTensorAddr[0];
  stTransferDesc.astTransferLinklist[0].phyDstAddr = stRemoteInputDesc.astInputTensroDesc[0].phyTensorAddr;
  stTransferDesc.astTransferLinklist[0].u64TransferSize = stRemoteInputDesc.astInputTensroDesc[0].u64TensorBufSize;
  if (MI_IPU_PCIECasWrapper_SetRemoteInputBuf(u32ChnId,
                                              0,
                                              &stTransferDesc) != MI_SUCCESS)
  {
      printf("set remote input buffer failed\n");
      return -1;
  }
  

---

2.8. MI_IPU_PCIECasWrapper_GetRemoteInputBuf

• Function

  Read data from the remote EP input buffer according to the linklist descriptor.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_GetRemoteInputBuf(MI_IPU_CHN u32ChnId,
                                                 MI_U32 u32EpId,
                                                 MI_IPU_PCIECasTransferDesc_t *pstCasTransferDesc);
  

• Parameters

  Parameter Name	Description	Input/Output
  u32ChnId	Channel ID to execute	Input
  u32EpId	Target EP board index	Input
  pstCasTransferDesc	Transfer descriptor. Each entry specifies remote source address (phySrcAddr), local destination (phyDstAddr), and transfer size	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

• Example

  MI_IPU_PCIECasTransferDesc_t stPullDesc;
  memset(&stPullDesc, 0, sizeof(stPullDesc));
  stPullDesc.u32LinklistCount = 1;
  stPullDesc.astTransferLinklist[0].phySrcAddr = stRemoteInputDesc.astInputTensroDesc[0].phyTensorAddr;
  stPullDesc.astTransferLinklist[0].phyDstAddr = stLocalTensor.phyTensorAddr[0];
  stPullDesc.astTransferLinklist[0].u64TransferSize = stRemoteInputDesc.astInputTensroDesc[0].u64TensorBufSize;
  if (MI_IPU_PCIECasWrapper_GetRemoteInputBuf(u32ChnId,
                                              0,
                                              &stPullDesc) != MI_SUCCESS)
  {
      printf("get remote input buffer failed\n");
  }
  

---

2.9. MI_IPU_PCIECasWrapper_DestroyCHN

• Function

  Destroy the specified cascade channel.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_DestroyCHN(MI_IPU_CHN u32ChnId,
  
                                        MI_IPU_PCIECasWorkMode_e eWorkMode);
  

• Parameters

  Parameter Name	Description	Input/Output
  u32ChnId	Channel ID to destroy	Input
  eWorkMode	Work mode: RC/EP	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasWrapper_DestroyCHN(u32ChnId, E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
  

---

2.10. MI_IPU_PCIECasWrapper_DestroyDevice

• Function

  Destroy the cascade device.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_DestroyDevice(MI_IPU_PCIECasWorkMode_e eWorkMode);
  

• Parameters

  Parameter Name	Description	Input/Output
  eWorkMode	Work mode: RC/EP	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasWrapper_DestroyDevice(E_MI_IPU_PCIE_CAS_WORK_MODE_RC);
  

---

2.11. MI_IPU_PCIECasWrapper_RunService

• Function

  Start the cascade service loop on the EP side, establishing PCIe channels, heartbeat checking, and request handling threads.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_RunService(const MI_IPU_PCIECasServiceConfig_t *pstConfig);
  

• Parameters

  Parameter Name	Description	Input/Output
  pstConfig	EP service configuration, including max worker threads (u32MaxWorkThread), PCIe controller ID (u32PcieId), heartbeat timeout threshold (u32HeartbeatTimeoutCountThreshold), and heartbeat interval (s32HeartbeatTimeoutMs)	Input

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasServiceConfig_t stServiceCfg = {
      .u32MaxWorkThread = 4,
      .u32PcieId = 0,
      .u32HeartbeatTimeoutCountThreshold = 30,
      .s32HeartbeatTimeoutMs = 1000,
  };
  if (MI_IPU_PCIECasWrapper_RunService(&stServiceCfg) != MI_SUCCESS)
  {
      printf("run pcie cascade service failed\n");
      return -1;
  }
  

---

2.12. MI_IPU_PCIECasWrapper_StopService

• Function

  Stop EP-side service threads and release related resources.

• Syntax

  MI_S32 MI_IPU_PCIECasWrapper_StopService(void);
  

• Parameters

  None.

• Return Value

  • MI_SUCCESS indicates success

  • Non-MI_SUCCESS indicates failure. Refer to Error Codes

• Dependencies

  • Header file: mi_ipu_pcie_cascade.h

  • Library file: libmi_ipu.so

• Example

  MI_IPU_PCIECasWrapper_StopService();
  

---

3. DATA TYPES

---

3.1. Data Type Definitions

Data Type	Function
MI_IPU_PCIECasWorkMode_e	Define RC/EP work modes
MI_IPU_PCIECasOfflineModelStaticInfo_t	Define IPU offline cascade model static information structure
MI_IPU_PCIECasDevAttr_t	Define IPU cascade device attribute structure
MI_IPU_PCIECasPosition_t	Define model output position and offset structure
MI_IPU_PCIECasSubNetBatchInvokeParam_t	Define cascade model subnet batch parameter structure
MI_IPU_PCIECasServiceConfig_t	Define EP service runtime configuration
MI_IPU_PCIECasTensorDesc_t	Define remote tensor buffer descriptor
MI_IPU_PCIECasInputDesc_t	Define remote input tensor collection descriptor
MI_IPU_PCIECasTransferLinklist_t	Define transfer linklist node structure
MI_IPU_PCIECasTransferDesc_t	Define transfer linklist descriptor

---

3.2. MI_IPU_PCIECasWorkMode_e

• Description

  Define the work modes supported by IPU Cascade API, distinguishing between RC and EP side interface calls

• Syntax

  typedef enum {
      E_MI_IPU_PCIE_CAS_WORK_MODE_RC = 0,
      E_MI_IPU_PCIE_CAS_WORK_MODE_EP,
  } MI_IPU_PCIECasWorkMode_e;
  

• Members

  Member Name	Description
  E_MI_IPU_PCIE_CAS_WORK_MODE_RC	Call API in RC mode
  E_MI_IPU_PCIE_CAS_WORK_MODE_EP	Call API in EP mode

---

3.3. MI_IPU_PCIECasOfflineModelStaticInfo_t

• Description

  Define the IPU offline cascade model static information structure

• Syntax

  typedef struct MI_IPU_PCIECasOfflineModelStaticInfo_s {
      MI_U32 u32SocNum;
      MI_U64 u64OfflineModelSize;
      MI_U32 au32VariableBufferSize[SGS_MULTI_SOC_NUM];
  } MI_IPU_PCIECasOfflineModelStaticInfo_t;
  

• Members

  Member Name	Description
  u32SocNum	Number of SoCs required to run the offline cascade model
  u64OfflineModelSize	Size of the offline cascade model
  au32VariableBufferSize	Variable buffer size required by the offline model on each SoC

---

3.4. MI_IPU_PCIECasDevAttr_t

• Description

  Define the IPU cascade device attribute structure

• Syntax

  typedef struct MI_IPU_PCIECasDevAttr_s {
      MI_U32 u32SocNum;
      MI_U32 au32MaxVariableBufSize[SGS_MULTI_SOC_NUM];
      MI_IPU_DevAttr_t stIPUDevAttr;
      MI_U32 au32Reserve[8];
  } MI_IPU_PCIECasDevAttr_t;
  

• Members

  Member Name	Description
  u32SocNum	Number of SoCs required to run the offline cascade model
  au32MaxVariableBufSize	Variable buffer size required by the offline model on each SoC
  stIPUDevAttr	MI IPU device attribute
  au32Reserve	Reserved field

---

3.5. MI_IPU_PCIECasPosition_t

• Description

  Define the cascade offline model output offset information structure

• Syntax

  typedef struct MI_IPU_PCIECasPosition_s {
      MI_U32 u32OutIndex;
      MI_U64 u64OffsetByBytes;
  } MI_IPU_PCIECasPosition_t;
  

• Members

  Member Name	Description
  u32OutIndex	Index of the output requiring offset
  u64OffsetByBytes	Byte offset relative to the output start address

---

3.6. MI_IPU_PCIECasSubNetBatchInvokeParam_t

• Description

  Define batch parameter information for inference of cascade offline models

• Syntax

  typedef struct MI_IPU_PCIECasSubNetBatchInvokeParam_s {
      MI_IPU_SubNetBatchInvokeParam_t stSubnNetBatchInvokeParam;
      MI_IPU_PCIECasPosition_t * pstPosition;
      MI_U32 u32PositionNum;
  } MI_IPU_PCIECasSubNetBatchInvokeParam_t;
  

• Members

  Member Name	Description
  stSubnNetBatchInvokeParam	Standard subnet batch parameter, inherited from MI IPU
  pstPosition	Pointer to array of cascade offline model output offset information
  u32PositionNum	Number of outputs requiring offset in the cascade offline model

---

3.7. MI_IPU_PCIECasServiceConfig_t

• Description

  Define the EP-side service runtime configuration.

• Syntax

  typedef struct MI_IPU_PCIECasServiceConfig_s {
      MI_U32  u32MaxWorkThread;
      MI_U32  u32PcieId;
      MI_U32  u32HeartbeatTimeoutCountThreshold;
      MI_S32  s32HeartbeatTimeoutMs;
      MI_U32  au32Reserve[8];
  } MI_IPU_PCIECasServiceConfig_t;
  

• Members

  Member Name	Description
  u32MaxWorkThread	Maximum number of worker threads on EP
  u32PcieId	Selected PCIe controller/link ID
  u32HeartbeatTimeoutCountThreshold	Heartbeat timeout counter threshold indicating RC disconnect
  s32HeartbeatTimeoutMs	Heartbeat interval in milliseconds
  au32Reserve	Reserved

---

3.8. MI_IPU_PCIECasTensorDesc_t

• Description

  Define a remote input tensor buffer descriptor.

• Syntax

  typedef struct MI_IPU_PCIECasTensorDesc_s {
      MI_U64 u64TensorBufSize;
      MI_PHY phyTensorAddr;
  } MI_IPU_PCIECasTensorDesc_t;
  

• Members

  Member Name	Description
  u64TensorBufSize	Tensor buffer size in bytes
  phyTensorAddr	Physical address of the buffer (EP view)

---

3.9. MI_IPU_PCIECasInputDesc_t

• Description

  Define the input tensor collection information on the remote EP.

• Syntax

  typedef struct MI_IPU_PCIECasInputDesc_s {
      MI_U32 u32TensorCount;
      MI_IPU_PCIECasTensorDesc_t astInputTensroDesc[MI_IPU_MAX_TENSOR_CNT];
  } MI_IPU_PCIECasInputDesc_t;
  

• Members

  Member Name	Description
  u32TensorCount	Number of input tensors
  astInputTensroDesc	Array that describes each tensor's physical address and buffer size (up to MI_IPU_MAX_TENSOR_CNT)

---

3.10. MI_IPU_PCIECasTransferLinklist_t

• Description

  Define a linklist node for a transfer operation.

• Syntax

  typedef struct MI_IPU_PCIECasTransferLinkList_s {
      MI_PHY phySrcAddr;
      MI_PHY phyDstAddr;
      MI_U64 u64TransferSize;
  } MI_IPU_PCIECasTransferLinklist_t;
  

• Members

  Member Name	Description
  phySrcAddr	Source physical address (RC or EP)
  phyDstAddr	Destination physical address (RC or EP)
  u64TransferSize	Bytes transferred in this node

---

3.11. MI_IPU_PCIECasTransferDesc_t

• Description

  Define the chain of transfer operations.

• Syntax

  typedef struct MI_IPU_PCIECasTransferDesc_s {
      MI_U32 u32LinklistCount;
      MI_IPU_PCIECasTransferLinklist_t astTransferLinklist[MI_IPU_PCIE_CASCADE_MAX_LINKLIST_CNT];
  } MI_IPU_PCIECasTransferDesc_t;
  

• Members

  Member Name	Description
  u32LinklistCount	Number of transfer nodes, up to MI_IPU_PCIE_CASCADE_MAX_LINKLIST_CNT
  astTransferLinklist	Array describing each transfer's source/destination/size

---

4. ERROR CODES

---

Table 4-1 IPU Cascade Error Codes

Error Code	Macro Definition	Description
0	MI_SUCCESS	Success
1	E_MI_IPU_PCIE_CAS_ERR_INVALID_PARAM	Invalid or missing parameter
2	E_MI_IPU_PCIE_CAS_ERR_NOMEM	Insufficient system memory
3	E_MI_IPU_PCIE_CAS_ERR_NOBUF	Cascade buffer unavailable or not allocated
4	E_MI_IPU_PCIE_CAS_ERR_TIMEOUT	Operation timeout
5	E_MI_IPU_PCIE_CAS_ERR_PCIE_INIT_FAIL	PCIe / communication initialization failure
6	E_MI_IPU_PCIE_CAS_ERR_MISMATCH_MODEL	Offline model mismatch with platform/mode
7	E_MI_IPU_PCIE_CAS_ERR_INVALID_SOC_NUM	Invalid SoC count declared by the model
8	E_MI_IPU_PCIE_CAS_ERR_RC_INVOKE_FAIL	RC-side inference failure
9	E_MI_IPU_PCIE_CAS_ERR_EP_INVOKE_FAIL	EP-side inference failure
10	E_MI_IPU_PCIE_CAS_ERR_FAIL	Unknown or general failure
11	E_MI_IPU_PCIE_CAS_ERR_FILE_OPERATION	File read/write failure
12	E_MI_IPU_PCIE_CAS_ERR_CONNECTION_FAIL	RC–EP connection failure
13	E_MI_IPU_PCIE_CAS_ERR_MISMATCH_MSG	Message type/content mismatch
14	E_MI_IPU_PCIE_CAS_ERR_CONNECTION_TIMEOUT	Heartbeat or connection timeout
15	E_MI_IPU_PCIE_CAS_ERR_RESPONSE_THREAD_FAIL	Response-thread creation/runtime failure
16	E_MI_IPU_PCIE_CAS_ERR_NO_ENOUGH_EP	Not enough active EPs
17	E_MI_IPU_PCIE_CAS_ERR_INVALID_CHNID	Invalid or nonexistent channel ID
18	E_MI_IPU_PCIE_CAS_ERR_INVALID_SUBNET_ID	Invalid subnet ID requested by RC
19	E_MI_IPU_PCIE_CAS_ERR_MODEL_CORRUPTED	Offline model is corrupted
20	E_MI_IPU_PCIE_CAS_ERR_MAP	Address mapping failure (MMU/MAP error)
21	E_MI_IPU_PCIE_CAS_ERR_DEVICE_NOT_READY	Device is not ready for the requested operation

---