Alkaid SDK Architecture

1. SDK Overview

Alkaid SDK is a software development kit based on the Linux kernel, which includes bootloader, kernel, riscv (optional), rootfs and MI multimedia modules. Users can configure and compile a software that can run on the Sigmastar platform through makefile.

The architecture of Alkaid SDK is shown in the figure below:

The entire architecture is divided into application layer, software library layer, driver layer and hardware layer from top to bottom.

  1. Application layer: Linux applications developed by users, running in Linux user mode;

  2. MI user interface layer: The interface of MI SDK in user mode. Applications access MI Driver through this layer to operate hardware.

  3. Driver layer: The specific implementation of MI driver;

  4. Hardware layer: Hardware of Sigmastar platform.

2. MI Function Module

The Sigmastar platform supports multimedia functions such as audio, image processing, and display technology. In order to facilitate users to quickly develop audio and video applications, MI (Middle Interface) serves as the middle layer between user programs and Sigmastar Stream IP. It implements access control to hardware IP downward and provides user programs with a simple and easy-to-use API upward.

The MI function modules supported by the Pcupid platform are:

Module Function
MI_SYS MI system initialization, memory buffer pool management, data flow management between modules
MI_SENSOR Get camera interface information, adjust resolution and frame rate, etc.
MI_ISP Image signal processing, HDW, 3D/2D noise reduction, 3A algorithm, WDR and other functions
MI_VIF Enable video input devices, video input channels, bind video input channels, etc.
MI_SCL Image data for crop/scaling/mirror/flip/pixel conversion and other functions
MI_DISP Video display driver, performs hardware puzzle on front-end input data and finally outputs it to MIPI/HDMI/LVDS/TTL
MI_IPU IPU deduction acceleration of AI models, allowing multiple AI model deductions to be implemented through channels
MI_VDISP Combine multiple images into one full-frame output
MI_RGN Perform Cover, OSD, Frame and other operations on images
MI_PSPI PSPI obtains image data from the sensor, or transmits image data to the panel via PSPI
MI_IVE Provides image algorithm APIs such as filtering, color space conversion, thresholding, morphological processing, etc.
MI_IQSERVER Image quality calibration service, used to complete data communication between the calibration tool and the development board
MI_GFX Provides fast graphics drawing functions for drawing UI
MI_FB Manage UI graphic layers and overlay UI onto the video preview path

3. SDK Directory Structure

  • boot: uboot related boot code, supports multiple system upgrade methods (currently supports sd card upgrade, network upgrade, usb upgrade and other methods) and boot splash screen and other functions;

  • kernel: Linux related environment settings, device tree, network card, touch screen, USB, sdmmc, Gsensor and other peripheral drivers;

  • SDK: Including mi_sdk, sensor driver, demo app and other codes;

  • project: Contains the environment variables, root filesystem and configuration information used in the entire project, the drivers and libraries generated in other directories that need to be packaged and burned to the machine, and the related files needed to be burned to the machine in the end;

  • riscv: Contains the rtos system source code running on the riscv core, as well as the mounriver project files.

3.1. boot

The directory structure is consistent with the original U-Boot. Users should pay attention to the following directories:

.
├── ....
├── cmd
│   └── sstar  //Related cmd files under uboot provided by sstar, including estar.c file for system upgrade
├── drivers
│   └── sstar//Peripheral drivers provided by sstar
├── configs
│   └──pcupid_comake_pi_d1_emmc_arm64_defconfig//Project configuration file, which differs depending on the model used
|                                //See the definition of CONFIG_UBOOT_CONFIG parameter in alkaid defconfig for details.
└── ....

3.2. kernel

The directory is basically the same as the original kernel. Users should focus on the following directories:

.
├── ....
├── arch
│   └── arm
│        ├──boot
│        │    └──dts
│        │        └── pcupid-comake-pi-d1-dual-sensor.dts//Device tree file
│        └── configs
│             └── pcupid_comake_pi_d1_emmc_dual_sensor_defconfig//Project configuration file, which differs depending on the model used
|                            //See the definition of CONFIG_KERNEL_CONFIG parameter in alkaid defconfig for details
│   └── arm64
│        ├──boot
│        │    └──dts
│        │        └── pcupid-comake-pi-d1-dual-sensor.dts//Device tree file
│        └── configs
│             └── pcupid_comake_pi_d1_emmc_dual_sensor_defconfig//Project configuration file, which differs depending on the model used
|                            //See the definition of CONFIG_KERNEL_CONFIG parameter in alkaid defconfig for details.
├── drivers
│   └── sstar //Peripheral drivers provided by sstar
└── ....

3.3. SDK

MI SDK source code implementation and APP demo

sdk
├── linux    //Linux-related code in mi sdk, used to recompile mi sdk after modifying kernel source code
├── verify
│     └── common    //Components required by most sample apps
│     └── prebuild_libs    //The third-party library files required for compiling part of sample_code
│     └── sample_code
│          ├──source    //Various demo examples, users can also add their own demo here
│          │    └──Makefile    //Add the demo name here under DualOS on judgment to compile
│          └──out    //After compiling, you can find the bin file here
│     └── release_feature
│          ├──source
│          │    └──dla    //ipu board demo
│          └──out    //After compiling, you can find the bin file here
│     └── opendla //Provide users with open-source model reference examples for rapid deployment on the SGS platform.
│     └── sstar_ota //Provide users with reference examples for online remote updates on the SGS platform.
|── driver
|     └──SensorDriver//sensor drv source code, RTOS also uses the drv source code here
|     └──PspiDeiver //Source code for the pspi sensor driver

3.4. project

.
├── board
│   ├── pcupid
│   │   ├── boot
│   │   │   └── nor/nand/emmc
│   │   │       ├── ipl //ipl is part of the bootloader
│   │   │       │   ├── IPL.bin
│   │   │       │   └── IPL_CUST.bin
│   │   │       └── partition//Used to store flash list files
│   │   │           ├── flash_list.nri
│   │   │           └── flash.nri
│   │   ├── dla_file//If the chip has an ipu, there will be corresponding ipu fw bin and model bin
│   │   ├── iqfile//IQ file
│   │   ├── json //External parameter json file obtained by MI
│   └── uboot//Public version default uboot package file
├── configs
│   └── verify
│   │   └── defconfigs
│   │       └── dispcam_pcupid.spinand.glibc-10.2.1-squashfs.comake_pi_d1.1024.bga_ddr4_riscv_defconfig//Project configuration file, such as KERNEL/MMA/SENSORIQ configuration, etc. Different models use different configuration files. Here we take SSM001A-S01A-S as an example.
├── image//All configuration files/tools/scripts for making images.
│   ├── build//Some image production tools that have been compiled, such as lfs/fwfs production tools
│   ├── busybox//The compiled busybox
│   ├── configs//Partition-related configuration files, please pay special attention
│   │   └── general
│   │       ├── spinand.squashfs.riscv.partition.config//Nand flash partition script
│   │       └── nor.squashfs.partition.config//Nor flash partition script, all partition information is configured here, including the IPL selection mentioned above.
│   ├── packaging  //Image packaging script
│   │   └── common
│   │       ├── customer.mk//Determine which files to package into the customer partition (UBI) and output execution commands to the script
│   │       ├── misc.mk//Determine which files to pack into the misc partition (FWFS)
│   │       ├── miservice.mk//Determine which files to pack into the miservice partition (UBI), usually MI dynamic library and kernel KO
│   │       ├── ramdisk.mk//Determine which files to pack into ramdisk and which applications to execute in the startup script. Ramdisk is also the first rootfs to be suspended.
│   │       ├── rootfs_fastboot.mk//rootfs fastboot version
│   │       └── rootfs.mk//Same as above, both manage the package information of roofs. This is the second mounted rootfs, which is actually mounted on ramdisk. For details, see ramdisk.mk.
│   ├── image.mk//Partition creation script. According to the partition table information, the corresponding tool will be called here to create the corresponding bin file in the images/ directory.
│   │             //And generate the partition table images/boot/PARTINFO.pni.
│   ├── ota.mk//If you use the ota tool provided by sstar, here is the corresponding script for making ota bin.
│   ├── script.mk//Script for making partition burning, generated in images/scripts. Generally, when uboot is burning, this script will be called to erase/write the flash.
│   ├── etc
│   │   └── init.d//System initialization configuration
│   ├── firmwarefs-fuse//fwfs linux application
│   ├── fuse//lfs or fwfs The libraries that need to be depended on for execution
│   ├── littlefs-fuse//lfs linux application
│   ├── makefiletools//Tools for making rootfs
│   ├── rootfs//rootfs package
│   └── systemtap//kernel debugging tool
├── kbuild
│   └── customize
│       └── $(KERNEL_VERSION)
│           └── pcupid
│               └── dispcam //Mainly focus on the following scripts, which decide which ko will be copied from kernel to project
├── release
│   ├── chip
│   │   └── pcupid//Library file corresponding to SDK
│   │       └── dispcam//MI SDK is mainly in here
│   └── include//SDK header file
├── scripts//Scripts used by project, users do not need to pay attention to them
└── tools//Debugging tools, iqtool, etc.

3.4 riscv (optional)

.
├── kernel
│   └── rtk
│        ├──proj //riscv rtos system source code
├── mounriver //mounriver project related files
└──

4. SDK compilation process

For the specific compilation process, please refer to Development Environment Setup and Burning User Guide. Here is just a brief introduction:

When make image is executed, the makefile will automatically compile the boot, kernel, riscv (optional), and sdk source codes, and copy the compiled output files to project/image. Finally, it will create the partition table, file system, etc., and package them to the project/image/output/images directory.

5. Files generated by SDK

The SDK will generate different burning files and burning scripts according to the configuration selected by the user (nandflash, norflash or emmc), and place the corresponding files in the project\image\output\images directory

Note: Before compiling, make sure the toolchain path is set correctly.

5.1. spi-nor image file

FILE DESCRIPTION
auto_update.txt uboot command during network upgrade, execute the script under scripts
load_addr.txt uboot, kernel loading address range
boot.bin IPL and uboot.bin will be packaged into a unified boot.bin to boot the system
kernel Linux core, including device tree, gpio, Gsensor and other peripheral drivers.
rootfs.sqfs Linux root file system, stores files required for booting and system configuration
misc.fwfs Stores boot logo image configuration and other information, which can be accessed by uboot and kernel
customer.jffs2 Stores libs and apps defined by users according to their own needs, such as wifi drivers and network dependency libraries
miservice.sqfs Stores mi_lib and mi related drivers
partition_layout.txt nor flash partition information table

5.2. spi-nand image file

FILE DESCRIPTION
auto_update.txt uboot command during network upgrade, execute the script under scripts
load_addr.txt uboot, kernel loading address range
OnebinnandBurnImgConfig.cfg ISP_TOOL script for burning SPINand
boot.bin IPL and uboot.bin will be packaged into a unified boot.bin to boot the system
kernel Linux core, including device tree, gpio, Gsensor and other peripheral drivers.
rootfs.sqfs Linux root file system, stores files required for booting and system configuration
misc.fwfs Stores boot logo image configuration and other information, which can be accessed by uboot and kernel
customer.ubifs Stores libs and apps defined by users according to their own needs, such as wifi drivers and network dependency libraries
miservice.ubifs Stores mi_lib and mi related drivers
ubia.bin Customer-defined partition
partition_layout.txt nandflash partition information table
riscvfw risc-v firmware

5.3. emmc image file

FILE DESCRIPTION
auto_update.txt uboot command during network upgrade, execute the script under scripts
load_addr.txt uboot, kernel loading address range
boot.bin IPL and uboot.bin will be packaged into a unified boot.bin to boot the system
kernel Linux core, including device tree, gpio, Gsensor and other peripheral drivers.
rootfs.ext4 Linux root file system, stores files required for booting and system configuration
misc.fwfs Stores boot logo image configuration and other information, which can be accessed by uboot and kernel
customer.ext4 Stores libs and apps defined by users according to their own needs, such as wifi drivers and network dependency libraries
miservice.ext4 Stores mi_lib and mi related drivers

6. root filesystem

Please refer to partition table