Skip to content

OTA Packaging and Upgrading

REVISION HISTORY

Revision No. Description Date
0.1 Initial version 02/27/2020
0.2 Support using mtd-utils to handle bad blocks during upgrade 08/14/2020
0.3 Added --script-add function, which can package any shell script to be processed 05/11/2021
1.2 This version is mainly modified for better support for users to do A/B partition backup upgrade function
1. Support one source file with multiple dst path upgrades
Example:
otapack xxx -s ./images/kernel -d “/dev/mtd8 /dev/mtd9” xxx
The "-d" parameter is followed by a space-separated number of paths. In the above example, it can be upgraded simultaneously mtd8 and mtd9 during the upgrade, or either one, only when the destination path is opened at the same time will it cause the upgrade process to fail. This function can avoid packing the same src twice, and also allow users to choose the destination path to upgrade.
2. Added option "-l --add-log", users can mark each file to be upgraded with a string
Example:
otapack xxx -s ./images/kernel -l kernel xxx
Then in otaunpack's main.c you can get this string in the callback function of fpFileOpen, which can identify the user's packaged file in otaunpack, and thus more easily develop the user's upgrade code.
3. The verification mechanism of the upgrade file is changed from checksum to md5		 06/09/2021
1.3 This version is further optimized on the original version 1.2.
1. Remove ps8UserLog in fpFileOpen
2. Add callback function fpFileAccess in otaunpack
3. This function will be executed first when upgrading any file or partition, and ask whether it can be upgraded, return -1 to skip, return 0 means allow upgrade.
4. fpFileAccess has four formal parameters respectively:
"const char *pOpenData", "unsigned int u32ImageId", " unsigned int u32PathId", "const char *ps8UserLog".
"const char *pOpenData": The path of the target file or partition to be upgraded.
"unsigned int u32ImageId", the unique id corresponding to the file or partition to be upgraded.
"unsigned int u32PathId": When an image corresponds to multiple Dst target partitions or file locations, the sequence number of upgrading according to the order.
"const char *ps8UserLog": The string brought in by -l when packing this upgrade image.		 08/11/2021
1.5 This version supports generating partition table when packing and specifying mask for upgrading when upgrading
1. Generate partition table when packing
Example: otapack --out-layout SStarOtaLayout.txt -r SStarOta.bin
2. Specify partition upgrade by mask
Example: otaunpack -x SStarOta.bin.gz -y SStarOtaLayout.txt -m 0x2		 06/05/2023
1.6 This version is further optimized on the original 1.5 version.
1. Description of OTA return value
2. Support streaming update function
Example: otaunpack - i "External Commands"		 12/25/2024
1.7 Support 'Sha256'. 08/14/2025

1. Overview

This article introduces the process of packaging, unpacking and upgrading in OTA upgrade for Sigmastar's IPC and NVR platform. It supports partition packaging, partition upgrading, individual file upgrading, batch packaging and upgrading by comparing new and old folder files, and file differential upgrading.

For the maintenance of the upgrade package backend server, terminal download and management, please integrate with third-party services.

2. Partition Packaging Process

2.1. Introduction of Packaging Tool

The packaging tool runs on a Linux server with Ubuntu 20.04 as the operating environment. It only generates the required file header according to the current file to be upgraded.

Directory of the packaging and unpacking tools:

sdk/verify/sstar_ota

Compiling the packaging and unpacking tools:

# Enter the sdk/verify/sstar_ota directory for complete packaging compilation
make

After compilation is complete, the packaging tool otapack and the unpacking tool otaunpack are located in ./out/arm/app/

2.1.1. Introduction of otapack tool command parameters

-c --create: Create an empty upgrade package file header. This command can use -b or -e to add a script for starting or ending the upgrade when creating the file header.

-b --begin-script: Specify a shell script file on the server and package it into the upgrade package for the script to be executed on the board before upgrading.

-e --end-scrip: Specify a shell script file on the server and package it into the upgrade package for the script to be executed on the board after upgrading.

-a --append: Append new header information to an already created upgrade package. The new information must inform the relevant information about the upgrade. For specific methods, please refer to the examples or configure according to your needs in the remaining parameter introduction.

-s --src-file: Specify the path of source file on the server that needs to be updated.

-d --dst-file: Specify the path of the destination file or partition node to be updated on the target board.

-t --dst-file-size: The size of the destination file or partition to be updated, in bytes, can be expressed in hexadecimal or decimal.

-m --file-mode: Specify the permission of the updated file, please refer to Linux's mode_t.

-o --diff-old: Specify the old file or folder directory on the server for differential upgrade.

-n --diff-new: Specify the new file or folder directory on the server for differential upgrade.

-l --add-log: Specify a string information so that each block or file can get this string information when upgrading, limited to 512 bytes maximum. This feature is supported from version 1.2 onwards.

--block-update: Set the source file specified on the server as raw partition data.

--ubi-update: Set the source file specified on the server as a volume of UBI file system.

--file-update: Set the source file specified on the server as a single file upgrade.

--file-add: Set the source file specified on the server as a new file added to the target board.

--file-delete: Only update the packaged data header and delete the specified destination file on the board.

--file-update-diff: Package the differential data of the new and old files set on the server.

--dir-update: Scan the contents of the new and old folders and compare the differences, additions and reductions, and then package them in batches.

--dir-update-diff: Scan the contents of the new and old folders and compare the differences, additions and reductions, and then package them in batches. The differential files are packaged with their differential data.

--script-add: Package a shell script specified by --src-file on the server to execute on the target board. The script file name displayed when executing is shown by --dst-file. This feature is supported from version 0.3 onwards.

--out-layout: Output the layout partition table file of ota bin. The parameter '-r' specifies the ota bin file.

--default-mask: Specifies the upgrade mask of the target file or partition node that needs to be updated.

--help: Print help information.

--debug: Enable debugging information.

2.1.2. Introduction of otaunpack tool command parameters

-x: Unpack and upgrade compressed files

-r: Unpack and upgrade uncompressed files

-t: Specify a writable folder path for differential upgrade.

2.1.3. Example of packaging process

  1. Create an empty header:

    otapack -c SstarOta.bin

  2. If necessary, you can use -b/-e command to add scripts (start.sh and end.sh need to be created first)

    otapack -c SstarOta.bin -b start.sh -e end.sh

  3. Update header data according to the files to be packaged:

    otapack -a SstarOta.bin -s ./images/kernel -d /dev/mtd3 -t 0x500000 --block-update -l kernel

  4. Repeat step 3 to package all files.

  5. Compress the upgrade package:

    gzip SstarOta.bin

2.2. Introduction of packaging in ALKAID

The OTA packaging process has been integrated into Makefile.

When the program is compiled and packaged, enter the command under project:

make image-ota

The following interactive interface will appear, specifying the script path to be executed on the board, you can choose to add or not add (you need to create the script in advance, the default path is project/image directory):

Start scripts:

End scripts:

When packaging partitions, list the partition data to be packaged:

Make ipl ?(yes/no)

After making relevant choices, SstarOta.bin.gz will be generated under project/image/output/images/

The partition config file will configure the partition for packaging.

Example:

There is a variable in the file spinand.squashfs.partition.config: OTA_IMAGE_LIST, append the partition name that needs to be packaged after this variable, and add the field xxx$(OTABLK) in the partition configuration to configure the target node path that needs to be upgraded for this partition. Only when OTA_IMAGE_LIST adds a partition name, it will ask whether this partition needs to be upgraded when make image-ota.

All logic of partition packaging script is implemented in image/ota.mk, which can be studied by yourself if interested .

Note:

  1. This packaging process does not support CIS and rootfs partition packaging.
  2. It does not support separate partition packaging in boot partition, such as it does not support separate packaging of ipl/ipl_cust/uboot etc.

2.3. RAW DATA Packaging without File System

The steps of partition upgrade are similar, and it will fill the files that need to be upgraded into the corresponding target partitions, so the packaging process is also roughly the same. The slight difference is that in RAW DATA partition packaging, you need to implement the creation and data filling of upgrade package header by yourself. Here we take spinand's boot partition packaging as an example:

define makeota
@read -p "Make $(1) ?(yes/no)" select;    \
if [ "$${select}" == "yes" -o "$${select}" == "y" ]; then    \
    $(PROJ_ROOT)/image/makefiletools/bin/otapack -s $(2) -d "$(3)" -t $(4) -m $(5) $(6) -l "$(1)" -a $(IMAGEDIR)/SStarOta.bin; \
    if [ -n "$($(1)$(BLKENV))" ]; then    \
        echo -e "$($(1)$(BLKENV))" | sed 's/^/\/etc\/fw_setenv /g' > $(IMAGEDIR)/$(1)_otaenv.sh;    \
        $(PROJ_ROOT)/image/makefiletools/bin/otapack -s $(IMAGEDIR)/$(1)_otaenv.sh -d $(1)_otaenv.sh --script-add -l "$(1)" -a $(IMAGEDIR)/SStarOta.bin;    \
    fi;    \
fi;
endef

boot_$(FLASH_TYPE)_mkota_:
    $(call makeota,$(patsubst%_$(FLASH_TYPE)_mkota_,%,$@),$(IMAGEDIR)/boot.bin,$($(patsubst%_$(FLASH_TYPE)_mkota_,%,$@)$(OTABLK)),$($(patsubst %_$(FLASH_TYPE)_mkota_,%,$@)$(PATSIZE)),0,--block-update)

2.4. ubifs/squashfs/jffs2/lwfs/lfs/ext4 packaging

If you create these partitions and add them to OTA_IMAGE_LIST, you don't need to add special handling in ota.mk, and the partition upgrade files of this type will be handled uniformly.

The partition upgrade method of UBIFS is slightly different from the other two formats, so please use --ubi-update when packaging UBIFS.

2.5. Individual file packaging

The option for individual file packaging is --file-update, and there is no specific packaging for file updates in ALKAID yet, so users need to add it manually.

2.6. Script file packaging

The option --script-add can package a shell script file on the server for execution on the board. The shell script file name packaged on the board is specified by --dst-file, or it can be unspecified.

2.7. Differential data packaging

The option for differential data packaging of individual files is --file-update-diff, which can be used for raw images such as kernel, uboot, etc., as well as for image files of read-only file systems, but not for writable file systems.

Differential upgrade relies on third-party open source bsdiff and bspatch.

The bsdiff tool will be used when packaging, and the bspatch will be used on the board.

The source code of bsdiff and bspatch tools is in project/tools/bsdiff

Compilation method:

tar -vxf bzip2-1.0.6.tar.gz
tar -vxf bsdiff-4.3.tar.gz
cd bzip2-1.0.6
make
cd bsdiff-4.3
make

After compilation, you can see the executable files bsdiff and bspatch in the current directory:

The default compilation is for pc use, please change Makefile to compile bspatch for board use.

The compiled bsdiff should be placed in /usr/bin/ path on the pc, or in the location specified by the environment variable $PATH.

Similarly, bspatch on the board should also be placed in the corresponding location, or in the location specified by the environment variable $PATH.

2.8. Scan new and old folders and batch file packaging

Folder scanning is divided into two types, one is to scan out the different files and force update.

--dir-update

Command example:

otapack -a ./test-diff.bin -o ./old/ -n ./new/ -d /config --dir-update

Another type is to scan out the different files and perform differential upgrade.

--dir-update-diff

Command example:

otapack -a ./test-diff.bin -o ./old/ -n ./new/ -d /config --dir-update-diff

3. Partition Upgrade Process

3.1. Upgrade Process

3.2. Precautions before Partition Upgrade

Please make sure that the partition is umounted successfully before partition upgrade. You can do related umount actions in start scripts. If you need to update files, please make sure that they are in a writable file system and have write permission.

Note:

The Comake PI D2 defaults to eMMC. When performing an OTA upgrade, you need to first enable write permission for boot1 by entering the following command:

echo 0 > /sys/block/mmcblk0boot1/force_ro

3.3. Upgrade of Compressed Upgrade Package

otaunpack -x SstarOta.bin.gz

3.4. Upgrade of Uncompressed Upgrade Package

otaunpack -r SstarOta.bin

3.5. Upgrade UI Display

You can specify a full-screen background image to paste on the framebuffer during the upgrade. The image supports jpg and png formats. At the same time, draw a progress bar in the framebuffer. otaunpack supports progress bar and text UI rotation from version 0.2 onwards. Set -s 0/½/3 to support None/90°/180°/270° rotation.

otaunpack -x SstarOta.bin.gz -p upgrade.jpg -s 1

3.6. Differential Upgrade

If there is differential upgrade data in the compressed package, you need to specify a temporary writable folder. If not specified, a tmp.diff file will be temporarily generated in the current directory. Command:

otaunpack -x SstarOta.bin.gz -t /tmp

3.7. Upgrade status acquisition

After the upgrade is completed, OTA UPGRADE STATUS will be displayed, which can get the upgrade status.

If OTA UPGRADE STATUS is 0, the upgrade is successful. For other return values, please refer to the chapter 5. OTA Return Value.

3.8. SPI-NAND bad block processing

When using the --block-update command to package the upgrade program, if the set is mtdblock or mmcblk block device node, this operation will not handle bad blocks. It is recommended that nor flash and emmc upgrade can use this node. If spinand needs to upgrade and handle bad blocks at the same time, please use --block-update to set it to mtd character device node. Otapack and otaunpack applications can support using mtd-utils to handle bad blocks during the upgrade in version 0.2 and later. The call is nand_write and flash_eraseall in busybox.

3.9. Streaming Update

When the local space does not have enough space to store the OTA package, please use streaming update. User needs to write data to the standard output pipe through an external process, and then use otaunpack to read the data from the standard output pipe for update. The command is as follows:

otaunpack - i "External Commands"

If differential upgrade is used, the command is as follows:

otaunpack -i "External Commands" -t /tmp

4. A/B partition packaging and upgrading

4.1. A/B partition packaging

The upgraded image file may have multiple matches for the target partition or file to be upgraded during the upgrade, and there will generally be two groups of partitions A and B, which are backups of each other.

On the basis of not considering the startup process of A/B partitions as backups of each other, A/B partition packaging needs to consider matching multiple upgrade destination paths or matching multiple backup upgrade files, specifically there are two situations:

  1. A single image source file corresponds to multiple target addresses

    For example, KERNEL partition corresponds to /dev/mtd4 and /dev/mtd5, then when packaging, -d option can have multiple destination paths such as:

    ./otapack -s ./KERNEL -d “/dev/mtd4 /dev/mtd5”

  2. Multiple image source files correspond to a single target address

    There is no difference from the non-A/B partition packaging setting. The same destination address can be filled in when packaging.

4.2. Partition table

Support output partition table after version 1.3

Command:

./otapack --out-layout layout.txt -r pack.bin

--out-layout: Specify the name and path of the output partition table file

-r: Specify OTA upgrade package, must be uncompressed

Output partition table format:

0,boot.bin,/dev/mtd0,0x1
0,boot.bin,/dev/mtd1,0x1
1,kernel,/dev/mtd3,0x1
1,kernel,/dev/mtd4,0x1
2,kernel_otaenv.sh,kernel_otaenv.sh,0x1
3,misc.fwfs,/dev/mtd6,0x1
4,miservice.ubifs,/dev/ubi0_0,0x1
5,customer.ubifs,/dev/ubi0_1,0x1

Each line is used to describe the information of a partition, the format is as follows:

[image id],[dst file name],[destination path],[upgrade mask]

image id: Each partition or file has a unique number.

dst file name: The file name filled in when packaging the partition or file.

destination path: When there are multiple targets there may be different destination path for the same partition represented by multiple lines, such as boot.bin partition in the above table.

upgrade mask: You can manually modify mask to change upgrade strategy. upgrade mask must be hexadecimal, and must have prefix '0x'.

4.3. A/B partition upgrade

After using the otapack tool to output the partition table, you can manually modify the upgrade mask to define the upgrade method.

Please note that the partition table can only be generated by the otapack tool, and you cannot modify any characters except the upgrade mask.

The command to use the partition table for upgrading adds -y to specify the partition table and -m to configure the upgrade mask to be referenced during the upgrade on the basis of the original command using otaunpack.

When upgrading, the parameter -m and the upgrade mask defined in the partition table specified by -y are logically anded (&), if the result is not 0, it means that it can be upgraded, then otaunpack will perform the upgrade operation.

The tool otapack defaults to outputting an upgrade mask of 0x1, and using -m 0x1 when upgrading with otaunpack can default to upgrading all partitions.

Example:

KERNEL partition has 2 destination paths, corresponding to 2 lines of information in the partition table:

1,kernel,/dev/mtd3,0x2
1,kernel,/dev/mtd4,0x4

If the destination path /dev/mtd3 is defined as A partition, /dev/mtd4 is defined as B partition, then the former uses an upgrade mask of '0x2' (bit1), and the latter uses '0x4' (bit2). Then when upgrading A partition, the command used is:

./otaunpack -r pack.bin -y layout.txt -m 0x2

The command used to upgrade B partition is:

./otaunpack -r pack.bin -y layout.txt -m 0x4

If only some partitions in the defined partition table are divided into A/B partitions, as shown in the following table, then A partition is still specified by specifying an upgrade mask of '0x2', B partition's upgrade mask is '0x4', non-A/B partition's upgrade mask is '0x1'

0,boot.bin,/dev/mtd0,0x2
0,boot.bin,/dev/mtd1,0x4
1,kernel,/dev/mtd3,0x2
1,kernel,/dev/mtd4,0x4
2,kernel_otaenv.sh,kernel_otaenv.sh,0x1
3,misc.fwfs,/dev/mtd6,0x1
4,miservice.ubifs,/dev/ubi0_0,0x1
5,customer.ubifs,/dev/ubi0_1,0x1

Then there are multiple choices when upgrading.

The command used to upgrade only A partition is:

./otaunpack -r pack.bin -y layout.txt -m 0x2

The command used to upgrade only B partition is:

./otaunpack -r pack.bin -y layout.txt -m 0x4

The command used to upgrade A partition and normal partition is:

./otaunpack -r pack.bin -y layout.txt -m 0x3

The command used to upgrade B partition and normal partition is:

./otaunpack -r pack.bin -y layout.txt -m 0x5

5. OTA Return Value

  1. The description of the OTAPACK return value is shown in the table.

    Return value Description
    0 Return success
    -1 Return failure

  2. The description of the OTAUNPACK return value is shown in the table.

    Return value Description
    0 Return success
    1 Parameter error or upgrade package verification failed
    2 begin-script script execution failed
    3 An IO exception occurs during the upgrade process

The return value can be obtained through the shell command "echo $?". The return value flow chart is explained as follows: