RISCv Debug Sop
REVISION HISTORY¶
| Revision No. | Description | Date |
|---|---|---|
| 1.0 | Initial release | 12/16/2024 |
1. How to troubleshoot RISCv not starting?¶
If you encounter no log output from RISCv, the first step is to confirm whether the image has been successfully loaded. The steps to confirm whether the image has been successfully loaded are as follows.
1.1. Confirm RISCv Load Address¶
The load address of the image can be found in the compilation configuration file, for example in mak/options_pcupid_riscv_isw.mak:
# Feature_Name = uImage load address # Description = define uImage load address # Option_Selection = ADDRESS CONFIG_RTOS_LOAD_ADDR = 0x26800000
From this, it can be seen that the RISCv image will be loaded from the storage medium (such as Flash, EMMC) to DDR at the address 0x26800000.
1.2. Determine Code Segment Size¶
The RISCv image will contain a code segment (text section) and a data segment (data section). The code segment stores the program's instructions, which will not change during execution, while the data segment stores the program's initialized global variables, which will be modified during execution. Therefore, to confirm whether the image is loaded correctly, you only need to compare the code segment of the image. To determine the size of the code segment, you can use the following command on the compilation server:
readelf -S build/pcupid_riscv_isw/out/pcupid_riscv_isw.elf There are 28 section headers, starting at offset 0x2292a4: Section Headers: [Nr] Name Type Addr Off Size ES Flg Lk Inf Al [ 0] NULL 00000000 000000 000000 00 0 0 0 [ 1] XRAM PROGBITS 10000000 061fcc 000000 00 W 0 0 1 [ 2] .text PROGBITS 10000000 001000 041148 00 AX 0 0 256 [ 3] .rodata PROGBITS 10041148 042148 01e780 00 A 0 0 8 [ 4] PREMAIN_INITCALL PROGBITS 1005f8c8 061fcc 000000 00 W 0 0 1 [ 5] NORM_INITCALL PROGBITS 1005f8c8 0608c8 000028 00 WA 0 0 4 [ 6] APPLICATION_INITC PROGBITS 1005f8f0 0608f0 00001c 00 WA 0 0 4 [ 7] XRAM0 PROGBITS 1005f90c 06090c 000004 00 WA 0 0 4 [ 8] .cli_cmd_list PROGBITS 1005f910 060910 0002a0 00 WA 0 0 4 [ 9] .cam_dev_list PROGBITS 1005fbb0 061fcc 000000 00 W 0 0 1 [10] .data PROGBITS 1005fbb0 060bb0 001414 00 WA 0 0 8 [11] RW_STATICBOOT PROGBITS 10060fc4 061fc4 000008 00 WA 0 0 1 [12] DEBUG_AREA PROGBITS 10060fcc 061fcc 000000 00 W 0 0 1 [13] .bss NOBITS 10061000 061fcc 00f470 00 WA 0 0 64 [14] .sys_stack NOBITS 10070470 061fcc 000500 00 WA 0 0 1 [15] .data2 NOBITS 101b8000 062000 008000 00 WA 0 0 1 [16] .debug_info PROGBITS 00000000 061fcc 0cc52a 00 0 0 1 [17] .debug_abbrev PROGBITS 00000000 12e4f6 019483 00 0 0 1 [18] .debug_loc PROGBITS 00000000 147979 0475fc 00 0 0 1 [19] .debug_aranges PROGBITS 00000000 18ef78 004590 00 0 0 8 [20] .debug_line PROGBITS 00000000 193508 055e97 00 0 0 1 [21] .debug_str PROGBITS 00000000 1e939f 01d66b 01 MS 0 0 1 [22] .comment PROGBITS 00000000 206a0a 000011 01 MS 0 0 1 [23] .debug_frame PROGBITS 00000000 206a1c 00f1a0 00 0 0 4 [24] .debug_ranges PROGBITS 00000000 215bbc 000ae0 00 0 0 1 [25] .symtab SYMTAB 00000000 21669c 009070 10 26 926 4 [26] .strtab STRTAB 00000000 21f70c 009a6f 00 0 0 1 [27] .shstrtab STRTAB 00000000 22917b 000129 00 0 0 1 Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), p (processor specific)
It can be seen that the size of .text is 0x41148.
1.3. Confirm whether the RISCv image is loaded correctly¶
The RISCv image will be loaded during the IPL_CUST phase, so you can use the u-boot command line to confirm whether the RISCv image is loaded correctly. After entering the u-boot command line, configure the correct network environment, and then use the tftp command to dump the data content on DDR:
SigmaStar # tftpput 0x26800000 0x41148 riscvfw_dump.bin
Using sstar_emac device
TFTP to server 10.21.2.10; our IP address is 10.24.16.137; sending through gateway 10.24.16.254
Filename 'riscvfw_dump.bin'.
Save address: 0x26800000
Save size: 0x41148
Saving: T ##################
43 KiB/s
done
Bytes transferred = 266568 (41148 hex)
SigmaStar #
Use a comparison tool to compare the burned image and the dumped file to see if they are consistent.
2. How to determine if RISCv is in a running state¶
You can use a register read/write tool (e.g., system tool, riu_r) to obtain the values of the following registers:
| Bank | Offset | Bits | Description |
|---|---|---|---|
| 0x1E | 0x35 | [0] | 0: disable 1: enable |
3. How to determine if RISCv is in TCM Mode¶
You can use a register read/write tool (e.g., system tool, riu_r) to obtain the values of the following registers:
| Bank | Offset | Bits | Description |
|---|---|---|---|
| 0x802 | 0x18 | [2:0] | 3b'000: icache mode |
| 3b'001: reserved | |||
| 3b'010: tcm mode |
4. How to check thread priority and CPU loading status in RISCv¶
Use the CLI command taskstat, this command will count the CPU loading status of all threads within 1 second after input.
| ID | PRIO | STAT | CPU | STACK USAGE | NAME | HANDLER |
|---|---|---|---|---|---|---|
| 1 | 111 | B | 0.0 | 776/3072 | SYS_CUST | 0x1002a250 |
| 2 | 2 | B | 0.0 | 496/2048 | CONSOLE | 0x1002c6b8 |
| 3 | 2 | X | 0.0 | 1320/4096 | MENU | 0x1002a268 |
| 4 | 0 | R | 99.9 | 144/2044 | IDLE | 0x00000000 |
| 5 | 127 | B | 0.0 | 192/2040 | Tmr Svc | 0x1003919c |
| 6 | 99 | B | 0.0 | 648/1528 | NonSecureWorld | 0x1002e900 |
| 7 | 64 | B | 0.0 | 776/2040 | rpmsg_dualos | 0x1001b194 |
5. How to check the status of all interrupts in RISCv¶
Use the CLI command intrstat to get the number of interrupt triggers, interrupt duration, and interrupt registration names:
| INT | Count | MaxTimeUs | AvgTimeUs | Handler | DevId | Affinity | Name |
|---|---|---|---|---|---|---|---|
| 162 | 1 | 7 | 14 | 0x10016000 | 0x10072740 | 0x00000001 | pwm_group3 |
| 365 | 13 | 22 | 22 | 0x1001B0B0 | 0x00000000 | 0x00000001 | RPMSG_L2R |
| 105 | 0 | 0 | 4294967297 | 0x1000664C | 0x1006230C | 0x00000001 | bdma7 |
| 105 | 0 | 0 | 4294967297 | 0x1000664C | 0x1006228C | 0x00000001 | bdma6 |
| 105 | 0 | 0 | 4294967297 | 0x1000664C | 0x1006220C | 0x00000001 | bdma5 |
| 105 | 0 | 0 | 4294967297 | 0x1000664C | 0x1006218C | 0x00000001 | bdma4 |
6. How to debug when Riscv serial port is stuck and cannot input¶
6.1. Reasons¶
The input of the serial port relies on the CONSOLE and MENU tasks, which have a default priority of 2. When the Riscv CPU is completely preempted by other higher-priority tasks, the serial port may become unresponsive.
6.2. Common Issues¶
1) Customer code logic is abnormal, calling time-consuming API in the rtos_application_initcall function, where the initcall priority belongs to SYS_CUST (111).
2) Threads created by the customer using CamOsThreadCreate are using while(1){……CamOsMsDelay()} for polling execution tasks. Delay does not release CPU resources; it should be changed to CamOsMsSleep.
3) The system's IRQ has taken away all Riscv loading; such issues are generally bugs in the public version, or the customer has time-consuming operations in the interrupt handler while the number of interrupts is high, leading to processing overload.
6.3. Troubleshooting Method¶
Check what the current Riscv is doing by reading the PC pointer continuously about 10 times, then use addr2line to resolve the PC pointer to the corresponding code location.
6.3.1. Method to Obtain the PC Pointer¶
1) If the Arm serial port is working normally, use the following command:
/customer # ./riux32_r 0x803 0x1 BANK:0x0803 16bit-offset 0x01 0x1002D630
2) If the Arm serial port is also stuck, use the Debug serial port with SStarSystemTool to read by selecting the X32 register and filling in 0x803 to view the value at offset:0x1.
6.3.2. Method to Resolve the PC Pointer¶
1) First, find the corresponding RISCv compiled ELF file for the problematic environment:
MOUNRIVER compilation file path: rtk\proj\obj\PCUPID.elf, Linux compilation path: rtk\proj\build\pcupid_riscv_isw\out\pcupid_riscv_isw.elf.
2) Use the addr2line command in the Linux command line to resolve the PC pointer; multiple addresses can be resolved simultaneously:
riscv64-unknown-elf-addr2line -e xxx.elf 0x1002D630 0x1002D630
or
addr2line -e xxx.elf 0x1002D630 0x1002D630
7. How to troubleshoot Riscv Exception error reasons¶
The reasons for exceptions can be mainly divided into the following 4 types:
- DATA ABORT: Accessing an illegal memory address; it is recommended to check whether there is an error in the process of obtaining the memory address.
- UNDEFINED INSTRUCTION: The CPU executes an unrecognizable instruction; it is recommended to check whether the function pointer is incorrect or if the memory where the function resides has been corrupted.
- PREFETCH ABORT: The CPU reads instructions from an illegal memory address; it is recommended to check whether the function pointer is incorrect.
- SYSTEM ASSERT: The code flow actively triggers an exception.
Exception information is mainly divided into 3 parts:
- Exception register info: Prints some main status register information of the RISCV CPU.
- Exception type: Prints the reason for the exception.
- Panic message: Prints the backtrace recorded during the exception and the specific location that triggered the assert.
Exception Example:
Exception without dump info
Exception type: SYSTEM ASSERT (240), Param: 0x100425c4
Panic at 0x10004d90 (unknown symbol)
Panic message: Test Assert
Call Stack Backtrace Begin:
#0 0x10004d90 (unknown symbol)
#1 0x1000550a (unknown symbol)
#2 0x10025038 (unknown symbol)
#3 0x1002313c (unknown symbol)
#4 0x1002c5e6 (unknown symbol)
#5 0x1002316a (unknown symbol)
#6 0x1002c5e6 (unknown symbol)
#7 0x1003885a (unknown symbol)
#8 0x10038854 (unknown symbol)
#9 0x1002acd8 (unknown symbol)
#10 0x1002b34e (unknown symbol)
#11 0x1002be12 (unknown symbol)
#12 0x1000311c (unknown symbol)
Based on the Panic backtrace, refer to the methods mentioned in previous sections to resolve the PC pointer.
aarch64-linux-gnu-addr2line -e pcupid_riscv_isw.elf 0x10004d90 0x1000550a 0x10025038 0x1002313c 0x1002c5e6 0x1002316a 0x1002c5e6 0x1003885a 0x10038854 0x1002acd8 0x1002b34e 0x1002be12 0x1000311c
Based on the backtrace, check the code and find that the call to CamOsPanic("Test Assert"); is at line 63 of riscv_gpio.c.
/home/beck.zhang/5_2.3.0_p3p/riscv/kernel/rtk/proj/build/pcupid_riscv_isw/out/riscv_gpio.c:63 ………………………… /home/beck.zhang/5_2.3.0_p3p/riscv/kernel/rtk/proj/build/pcupid_riscv_isw/out/core_state.c:160