FreeRTOS with STM32 Cube

Cube Sucks. But, if you’ve gotta use it, at least it has demos and things right? And you can use FreeRTOS, that’s integrated, right?

Not really. At the time of writing, working with STM32WB, the out of the box FreeRTOS demos will have problems with printf. Or, well, anything that uses a raw malloc() call.

Out of the box, the cube demo code uses FreeRTOS’s heap_4.c for memory management. This is fine. It means that you select a heap size in your FreeRTOSConfig.h file, and the port provides pvMalloc and pvFree that do the right thing. But, if any code, (anywhere!) uses malloc() itself, directly, then…. where does that come from? Well, it ends up in _sbrk() and the out of box implementation in the cube demos is completely busted. They provide you with the same implementation as for the non FreeRTOS demo.

So, looking at a “classic” memory arrangement, with heap starting at the end of BSS/DATA, and stack growing downwards, you have something like this. (_estack and _end are in the default out of box linker scripts from ST) FreeRTOS’s heap_4.c declares “ucHEAP” (you can statically provide it to, if you like) based on your user provided configTOTAL_HEAP_SIZE which then becomes another (somewhat large) chunk of “DATA” space.

The _sbrk() implementation basically checks that malloc() is asking for space that’s above _end and below the current stack pointer. This is fine when you’re not using an RTOS, but! The RTOS tasks have their own stacks, allocated from the RTOS heap! This is good and right, but it means that the out of box sbrk implementation is now absolutely garbage. It will never succeed, as every single tasks’ stack pointer will be below _end.

You have options here. https://nadler.com/embedded/newlibAndFreeRTOS.html has written extensively, though I think they make it all sound really complicated.

Option 1 – Just don’t call malloc (or… call it early…)

This is a garbage option. You either need to make sure you never call malloc, or make sure that you only ever call malloc before you start the FreeRTOS task scheduler. This is safe, as the memory you allocate here will be in safely in the gap above DATA on the diagram, but you must make all malloc() calls before starting the task scheduler. This requires no changes to the _sbrk() system call implementation

Option 2 – Use newlib completely

This is Nadler’s approach. Switch heap_4 out for heap_3, rewrite your system calls (including _sbrk() to make all of FreeRTOS use standard newlib malloc/free natively. This isn’t a bad option, but it’s fairly messy, isn’t really something the FreeRTOS people like much, and puts a lot of reliance on how well newlib behaves. The upside here is that you only have one memory management mechanism for the entire system here. (I failed to get this working at the time of writing)

Option 3 – Fix _sbrk() only

You can also just fix _sbrk. The upside here is the change is trivial. The downside is that you now have two heaps. One that you statically give to FreeRTOS via the config options for heap_4.c (ucHEAP in the DATA section), and a new one, growing upwards on demand, classic style, above DATA, show in red on this diagram. Remember, that in the FreeRTOS context, all of the space marked in purple and red is untouched, or “wasted”. You want to size your ucHEAP that you leave “enough” for untracked raw malloc users, but still big enough for your polite, well behaved FreeRTOS consumers.

The change itself is straightforward, just stop looking at the stack pointer, and look at the linker script provided pointer to the end of ram.

--- /out-of-box/application/sysmem.c	2021-03-26 10:02:54.045633733 +0000
+++ /fixed/application/sysmem.c	2021-04-07 13:58:27.610330156 +0000
@@ -37,18 +37,19 @@
 **/
 caddr_t _sbrk(int incr)
 {
 	extern char end asm("end");
 	static char *heap_end;
 	char *prev_heap_end;
+	extern char _estack;
 
 	if (heap_end == 0)
 		heap_end = &end;
 
 	prev_heap_end = heap_end;
-	if (heap_end + incr > stack_ptr)
+	if (heap_end + incr > &_estack)
 	{
 		errno = ENOMEM;
 		return (caddr_t) -1;
 	}
 
 	heap_end += incr;

And that’s it. The _estack symbol is already provided in the out of box linker scripts. Risks here are that you are only checking against end of ram. Conceivably, if you ran lots of deeply nested code with lots of malloc() calls before you start the scheduler, you might end up allocating heap space up over your running stack. You’d have to be very tight on memory usage for that to happen though, and if that’s the case, you probably want to be re-evaluating other choices.

Option 4 – Make newlib use FreeRTOS memory

There is another option, which is to make newlib internally use FreeRTOS pvMalloc/pvFree() instead of their own malloc/free. This is the logical inverse of Option 2, and has the same sorts of tradeoffs. It’s probably complicated to get right, but you end up with only one memory management system and one pool.

Autofailing autotools are autohell, as always

“Needed” a new version of something, so build from source. Configure not available, because… autohell, so run “autogen.sh” instead.

karlp@nanopiduo2:~/src/libgpiod$ ./autogen.sh 
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
aclocal: warning: couldn't open directory 'm4': No such file or directory
autoreconf: configure.ac: tracing
autoreconf: configure.ac: creating directory autostuff
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
configure.ac:92: error: possibly undefined macro: AC_CHECK_HEADERS
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:172: error: possibly undefined macro: AC_LIBTOOL_CXX
configure.ac:174: error: Unexpanded AX_ macro found. Please install GNU autoconf-archive.
configure.ac:179: error: possibly undefined macro: AC_LANG_PUSH
configure.ac:181: error: possibly undefined macro: AC_LANG_POP
configure.ac:188: error: Unexpanded AX_ macro found. Please install GNU autoconf-archive.
autoreconf: /usr/bin/autoconf failed with exit status: 1
karlp@nanopiduo2:~/src/libgpiod$

Whatever, this is “traditional” that you can’t actually use autohell without adding more autohell from somewhere else, but ok, bend the knee, install autoconf-archive and continue….

karlp@nanopiduo2:~/src/libgpiod$ ./autogen.sh 
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
aclocal: warning: couldn't open directory 'm4': No such file or directory
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
configure.ac:92: error: possibly undefined macro: AC_CHECK_HEADERS
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:172: error: possibly undefined macro: AC_LIBTOOL_CXX
configure.ac:179: error: possibly undefined macro: AC_LANG_PUSH
configure.ac:181: error: possibly undefined macro: AC_LANG_POP
autoreconf: /usr/bin/autoconf failed with exit status: 1
karlp@nanopiduo2:~/src/libgpiod$

wat? really? that’s it? Thanks for nothing. Google and the accumulated wisdom of years of hate suggest that, as this is a “new” machine, I might not have installed pkg-config. (Given that the whole thing with autohell is finding things, you might think it could give you a better error?) ok, Install that, try again

karlp@nanopiduo2:~/src/libgpiod$ ./autogen.sh 
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
aclocal: warning: couldn't open directory 'm4': No such file or directory
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
configure.ac:172: error: possibly undefined macro: AC_LIBTOOL_CXX
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1
karlp@nanopiduo2:~/src/libgpiod$

Progress, but… still not there. Still absolutely atrociously useless error messages. Now, I’ve read the README, it says to install autoconf-archive, and that’s about it. I have python3-dev and libstdc++-9-dev I’m not sure what else now…

Oh right. libtool, another gross piece of historic nonsense, that mostly just gets in the way, and leaves pretend files around the place.

Now autogen will ~finish, and run configure, happily checking whether I’m running on a 30 year old unpatched sparc solaris, so it applies the right workarounds, but don’t worry…

configure: error: "libgpiod needs linux headers version >= v5.10.0"

I mean, I’m super glad we included all that “portable” autohell bullshit when we’re explicitly tied to linux anyway. And not just any linux. At the time of writing, 5.10.0 was three days old.

At this point, there are now _other_ problems that autotools, but this is just another example of why you should NEVER be using autotools in 2020.

Jenkins agent systemd unit file

So, you try to setup an agent, and instead of getting a nice package or anything, you get told, “run this long java command line app” …. wat. Here’s a (very) basic system unit file to run your agent. You can copy the command lines from what jenkins provides itself.

$ cat /etc/systemd/system/jenkins-agent.service 
[Unit]
Description=My radical Jenkins agent
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/java -jar /var/opt/jenkins-agent/agent.jar -jnlpUrl https://radical.example.org/computer/blah/slave-agent.jnlp -secret @/var/opt/jenkins-agent/my.jenkins.secret-file -workDir /var/opt/jenkins-agent
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

(And remember, JDK8 for your agent, until https://issues.jenkins-ci.org/browse/JENKINS-63313 is fixed)

Then, just systemctl enable jenkins-agent and systemctl start jenkins-agent.

Vodafone Huawei R216 teardown

For work reasons, we had a couple of these LTE mobile hotspots deployed permanently. After about two years, they’ve failed, both with puffed up batteries. For no good reason I can think of, the devices don’t work with the battery removed, they just blink constantly. Still, they worked well right up to that point, so, what’s in one? I’ve not tried powering one up again properly, I’m not excited about ordering a spare battery for one, and it’s not (currently) interesting enough to try and provide power to the right place to fake the battery.

You can find the FCC internal photos under QISR216 but it’s blurry. We know it’s from Huawei, but that’s about it. You can also find a GPL compliance tarball that implies it runs android, and has some more details if you’re good at digging into that sort of thing (I’m not really) R216-open resource code.tar.gz

Here’s some photos of it’s guts with a bit of decoding. There’s a few spaces on the PCB for what I suspect is a SD card, and a 5GHz wifi radio, but it’s kinda ugly trawling through old product announcements to see if any of it was ever used. The row of circular pads on the PCB edge are actually available without disassembly, (there’s a row of holes inside the battery compartment that provides access to them), so that’s presumably a “useful” port for something. There’s another set of test pads by the wifi chip.

bcm43241 wifi chipset

The silver circle in the white field is actually a super slim pushbutton!

And now the “real” side

Green is HiSilicon HI6559, a PMIC from reading elsewhere. Red is some Micron flash, and a HiSilicon Hi6921M. You can read a bit more about the part here: https://github.com/Huawei-LTE-routers-mods/README/blob/master/balong_series.md

Blue is HiSilicon Hi6361gfc, presumably the LTE modem? Purple is the Qorvo multiband power amplifier

White box? No idea, not sure what would ever go on that footprint. The blank on the left is presumably a microsd socket.

There’s another one of these superslim membrane switches in the top right, between the micro usb and the optional external antenna connectors.

sunxi-fel: nanopi duo2 – latest kernel + openwrt rootfs

So yeah, back on getting all of that friendlyarm nanopi duo2 supported again. Want a newer kernel to get the bluetooth stuff in, but not interested in trying to port openwrt forwards (yet?) Instead, build your own kernel as per: http://linux-sunxi.org/Mainline_Kernel_Howto and build openwrt as normal. (well… normal, but add this patch to skip a manual step: https://github.com/karlp/openwrt/commit/6438d22e98df353afe2635667b7dc29c97bd1196 )

Now, make sure you _BUILT IN_ any kernel modules you want, because we aint doin no module installation and loading bullshit.

And then just…. fel load it, as per http://linux-sunxi.org/FEL/USBBoot#Booting_the_whole_system_over_USB_.28U-Boot_.2B_kernel_.2B_initramfs.29

$ sunxi-fel uboot u-boot-sunxi-with-spl.bin write-with-progress 0x42000000 /home/karlp/src/linux-git/arch/arm/boot/zImage  write-with-progress 0x43300000 /home/karlp/src/openwrt-git/bin/targets/sunxi/cortexa7/openwrt-sunxi-cortexa7-sun8i-h3-nanopi-duo2-iotbox-rootfs.uboot write 0x43000000 ~/src/linux-git/arch/arm/boot/dts/sun8i-h3-nanopi-duo2-iotbox.dtb write 0x43100000 my-fel-bootz.scr
100% [================================================]  5229 kB,  842.6 kB/s 
100% [================================================] 13228 kB,  845.3 kB/s 
$ cat my-fel-bootz.cmd 
setenv bootargs console=ttyS0,115200 console=ttyGS0,115200 earlyprintk root=/dev/ram0
bootz $kernel_addr_r $ramdisk_addr_r $fdt_addr_r
$

And presto. ramdisk from openwrt rootfs as root, upstream kernel of whatever you like.

Shitty blog post, but saving this for next time so I don’t lose it again.

Actually using FEL to boot openwrt on friendlyarm nanopi duo2

Following on from where we left off, now let’s actually use FEL for the whole process, not just to get uboot loaded. In the last piece, once uboot was loaded, we used uboot+tftp to boot our openwrt ramdisk, but FEL can do it all in one hit, over usb, according to http://linux-sunxi.org/FEL/USBBoot#Booting_the_whole_system_over_USB_.28U-Boot_.2B_kernel_.2B_initramfs.29 and that’s way more useful on things that might not have an ethernet port onboard.

We basically just take our steps from the last one, and add them as options to the sunxi-fel tool.

First, we trim down our “my.env” file, as now it just needs to boot.


$ cat my-fel.cmd
setenv bootargs console=ttyS0,115200 earlyprintk
bootm $kernel_addr_r - $fdt_addr_r

Now, we convert that to the “scr” file that uboot will run magically.

$ tools/mkimage -C none -A arm -T script -d my-fel.cmd my-fel.scr

Now, use sunxi-fel with all the wonderful paths into the build guts of openwrt.

$ sunxi-fel uboot u-boot-sunxi-with-spl.bin write-with-progress 0x42000000 /home/karlp/src/openwrt-git/bin/targets/sunxi/cortexa7/openwrt-sunxi-cortexa7-sun8i-h3-nanopi-neo-initramfs-kernel.bin write 0x43000000 ~/src/openwrt-git/build_dir/target-arm_cortex-a7+neon-vfpv4_musl_eabi/linux-sunxi_cortexa7/linux-4.14.82/arch/arm/boot/dts/sun8i-h3-nanopi-neo.dtb write 0x43100000 my-fel.scr
100% [================================================] 4013 kB, 770.4 kB/s

Because openwrt’s buildroot builds us a nice kernel+ramdisk image out of the box, (well, if you select “ramdisk” in the image settings) we don’t need to write the uImage and rootfs separately as indicated on the http://linux-sunxi.org/FEL/USBBoot page.

For reference, the openwrt diffconfig to build this is just

$ ./scripts/diffconfig.sh
CONFIG_TARGET_sunxi=y
CONFIG_TARGET_sunxi_cortexa7=y
CONFIG_TARGET_sunxi_cortexa7_DEVICE_sun8i-h3-nanopi-neo=y
CONFIG_DEVEL=y
CONFIG_TARGET_INITRAMFS_COMPRESSION_NONE=y
CONFIG_TARGET_ROOTFS_INITRAMFS=y

Booting openwrt on nanopi duo2 (or a nanopi neo) via FEL

Edit: See the next post to use just FEL via USB, (no cheating and using tftp afterwards like we do here)

(Very rough unformatted, just so I don’t lose it)

FriendlyARM NanoPi Duo2 is ~similar enough to a FriendlyARM NanoPi Neo to get started. Later, we’ll work on… later stuff.

I don’t want to go hacking around with sd cards and crap while I’m developing stuff, and FEL mode is great.

So… get a uboot built (2018.11 is what I tested) using “make nanopi_neo_defconfig” (You will almost definitely want to enable “Networking Support->Random ethaddr if unset”

Now, patch sunxi-fel: https://github.com/linux-sunxi/sunxi-tools/pull/117

Now, make an environment file with some helpers…. (you can do this by hand if you like)

#=uEnv
myvar=world
bootcmd=echo "Hello $myvar."
bootargs=console=ttyS0,115200 earlyprintk
serverip=192.168.3.6
resetserver=setenv serverip 192.168.3.6
loadkernel=tftp $kernel_addr_r build_dir/target-arm_cortex-a7+neon-vfpv4_musl_eabi/linux-sunxi_cortexa7//sun8i-h3-nanopi-neo-kernel.bin
loaddtb=tftp $fdt_addr_r build_dir/target-arm_cortex-a7+neon-vfpv4_musl_eabi/u-boot-nanopi_neo/u-boot-2018.05/arch/arm/dts/sun8i-h3-nanopi-neo.dtb
loaddisk=tftp $ramdisk_addr_r
# Running dhcp will reset serverip....
winning=run resetserver && run loadkernel && run loaddtb && bootm $kernel_addr_r - $fdt_addr_r

(the magic variables are reffered to at http://linux-sunxi.org/FEL/USBBoot#Booting_the_whole_system_over_USB_.28U-Boot_.2B_kernel_.2B_initramfs.29)
The magic trick of bootm $kernel_addr_r – $fdt_addr_r I got from extracting the openwrt sdcard image file, and reading the “boot.scr” file in the first partition.

Now, go and build openwrt master

Now, setup a dhcp/tftp server as appropriate for your network.

Now, use (patched) sunxi-fel (fixing udev perms if necessary):

sunxi-fel uboot u-boot-sunxi-with-spl.bin write 0x43100000 my.env

Now, on serial console, use uboot to dhcp an address and run stuff…


dhcp
(ctrl-c to abort the dumb attempts at autofinding an image)
run resetserver
run winning

Now, enjoy the fruits of a whole lot of people’s labour…..


=> bootm $kernel_addr_r - $fdt_addr_r
## Booting kernel from Legacy Image at 42000000 ...
Image Name: ARM OpenWrt Linux-4.14.82
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 4013368 Bytes = 3.8 MiB
Load Address: 40008000
Entry Point: 40008000
Verifying Checksum ... OK
## Flattened Device Tree blob at 43000000
Booting using the fdt blob at 0x43000000
EHCI failed to shut down host controller.
Loading Kernel Image ... OK
Loading Device Tree to 49ff9000, end 49fff4b7 ... OK

Starting kernel ...

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.82 (karlp@beros) (gcc version 7.3.0 (OpenWrt GCC 7.3.0 r8575-a734450d6f)) #0 SMP PREEMPT Mon Nov 26 20:27:17 2018
[ 0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=30c5387d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: FriendlyARM NanoPi NEO

boring bootlog snipped, you know what they look like

From here, we need to move to a real separate device tree for the duo2, prepare to get the AP6212 module up and running, and handle how overlays/custom versions should be handled….

tracker-miner-fs stuck at 100% alternative solution

There’s plenty of posts about problems with tracker-miner-fs being a CPU hog, even if it’s theoretically “idle”. “tracker status” will say that indexing is complete, and all miners are idle, but that’s clearly a lie, and you process stuck running from bootup.

Most “solutions” on the internet revolve around just turning off tracker completely, or at least the filesystem miner. I don’t want that, I actually like having search work. What I wanted was it to, you know, work :)

So, while experimenting with resetting it, I noticed that it was always getting stuck trying to recurse a particular directory tree. It’s 6GB, but that was unlikely to be problem. However, one directory there contained a lot of files. Lots like, 600,000 files or so, in one directory. This is enough to make ls hang, so, maybe it was enough to make tracker-miner-fs hang too? (Side note, if you want to count files in a directory like this, find . | wc -l will work, where ls doesn’t.)

I didn’t actually need those files anymore, so I just deleted the suspect directories entirely, and then restarted the trackers with tracker daemon --stop; tracker daemon --kill; tracker daemon --start (The kill was to kill the errant tracker-miner-fs process, the others all stopped gracefully)

And presto, tracker status starts reporting sane and plausible results, the tracker-miner-fs runs at 100% for a short period, reporting status correctly, then the tracker extractor takes over. And we have search again!

Nitsuka DX2E office phone teardown.

Full model on back of phone: DX2E-12BTXH. Manufactured October 1999. Internal PCB is clearly multi model, and also has the model MH 6330(1) and DX7C-12ANU5-A1
I have a few of these, they’re quite common, you can buy them on ebay for ~$35 or so, or similar models. I had the (very silly, but there was going to be some learning involved) idea of turning one of them into …. a phone. The plan is/was to hookup my own hardware to the handset, buttons and lcd screen, a little linux SoM or similar, and have it run some SIP softphone, running off 18650 batteries. Basically, make a phone…. into a phone. Not all terribly complicated, really.

So, time to open it up. Immediate difficulties are that the entire phone body is a giant PCB with button press pads. I’d loosely been planning on making a little USB HID device to capture all the buttons. (As we’re making a softphone, we don’t need DTMF or anything silly) Gonna have to rethink that now :) Anyway, what’s inside? Super easy to open, plain phillips head screws, no glue, no latches.

Three major ICs, the LCD connector, and the wonderful world of PSTN power supplies.

Let’s start at the top.

X2 is labelled Mitsubishi, M38003M6, and is by all accounts, a mask rom H8/300L series. Part numbers match up for Renesas (the current owner at least) I suspect this is handles all the programmable buttons and the LCD? X1 is a bit of a mystery, it’s labelled NEC D65825GF043. No idea what this is. Custom asic for phones? *shrugs*
Down to the bottom…

X3, Toshiba TC35324F. I found TC35300 and TC35310, both DTMF decoders. I suspect that this listens in on it’s own keypad? Seems a bit excessive though?

X7 (what happened to X4/X5/X6) Is a plain old MC34119. An audio amplifier. The fun things with old tech like this, it talks about being “low current” with only 2.7mA Iq, suitable for batteries :) The datasheet I found even excitedly mentions that it contains 45 active transistors! Whee.

Finally, back side. Just keypads and LEDs really.

Anyway. There it is. Not really sure what next. It wouldn’t be completely unreasonable to just make a whole new pcb that fits under all the button pads, but that’s a lot more than I was planning on. JLPCB and elecrow will make pcbs that big for ~$10 each, but it’s a tedious button pad design. Could do a smaller PCB, only do some of the buttons, just for less hassle, but, meh. Could try and probe up some buttons, lay a big rats nest of wire down. Possibly the easiest way would be to desolder one of the ICs and put something in it’s place, a bunch of selective trace cutting, see if all the buttons go there? Or maybe just look for a different phone altogether…

Cypress CY8CKIT-049-42XX – PSoC 4200 kit – first impressions

(This was a draft I write in 2015, thought I might as well publish it where I’d left it)
Cool packaging, just like in the PSoC5LP kit. Despite looking similar, with a breakoff section that plugs into USB, the “programmer” side actually isn’t. It’s Cypress’s USB Serial bridge chip, the CY7C65211, and you program by using the UART bootloader on the PSoC 4200 target. Totally functional, totally valid. Not nearly as useful as a full debugger for application development of course, but in effect, it’s a full eval board for the usb serial chip. You can get a config tool that turns it into USB-SPI, USB-I2C, and USB-GPIO dongles. Neat, but still, I didn’t get this to evaluate usb serial bridge chips. On the bright side, it does at least detect properly out of the box as a serial UART in linux. (Unlike the KitProg on the PSoC5LP kit)