linux-rockchip v3.17-next on minix neo x5 mini

This is a braindump post, probably of very little interest to anyone but myself. It outlines further adventures towards getting a plain linux system booted on a minix neo x5 mini.

rk3066 support is expanding in the bleeding edge linux trees, and seeing as I don’t care about HDMI (yet) for my purposes, I thought it would be a good way to go forward. Device tree support, recent, near vanilla code, a snowflakes chance of getting a patch to work, and a single clear path to upstream. Of course, I thought for practice, I should start with one of the existing “gpl release” trees, all built on 3.0.x. That was an adventure. I tried the “omegamoon” repository, which _seemed_ to be the most uptodate, with backports of “important” things. https://github.com/omegamoon/Rockchip-GPL-Kernel

We ignore all the build script crap, at least until we’ve looked at them all, because we’re trying to work out what they’re doing, not just monkeys on keyboards.

$ export ARCH=arm
# This line your toolchain, mine is from "yum install gcc-arm-linux-gnu" on fedora 20
# Note, that you can actually skip this with the right entries in your .config
$ export CROSS_COMPILE=arm-linux-gnu-

In any tree, have a look in arch/arm/configs for an appropriate defconfig for your device…. (This is where it gets fun!)
So, let’s start with something basic

make rk3066b_sdk_defconfig
# now we would need to go into menuconfig and make sure we've turned on drivers we might need....
# but let's just build this one first and see what we get....
make -j8

And… it fails here. Awesome. Turns out all the 3066 configs here don’t even compile, presumably because of rk3188 support getting patched in, without any testing. so, toss this repo out and try another…

We’re going back to a “clean” rk3066 tree, following the directions here: http://hwswbits.blogspot.com/2013/11/linux-on-ug008-tv-box-rk3066.html This site has been generally helpful, and the ug008 is a very similar device to the minix neo x5 mini. I originally intended to go all the way back to basics, just follow any build guide first, make sure the code worked as is, before I tried anything more specific to the x5mini.

git clone https://github.com/Galland/rk3x_kernel_3.0.36 galland-rk
cd galland-rk
cp galland.config .config
make menuconfig
General Setup->Cross Compiler tool prefix -> Update the prefix so you don't have to export CROSS_COMPILE...
make

watch it fail…

/home/karlp/projects/galland-rk3x-3.0.36/scripts/gen_initramfs_list.sh: Cannot open '../initramfs/initramfs.cpio'

Ok, that was clearly skipping steps. Why do I need a damn initramfs, I just want to build in the modules I need in one go! But, ok, we’re going to follow the instructions given.

clone galland initramfs image, choose to use the one with rk mtd nand support, as we are hoping eventually to get linux, not android. rebuild
watch it fail, no uudecode. wtf?! and yum search uudecode only turns up a perl module?! Turns out you need the sharutils package. Ok, my bad, this was listed at the top of galland’s page of requirements. No idea why I haven’t had this installed already before, but so be it, install and make again…

Realise it doesn’t compile either. Bugs in the drivers/mtk_wcn_combo/drv_wlan/wlan/common/wlan_lib.c file. Awesome. I did consider resetting this repository to the date the blog post was written, and also looked at the rest of the forks, but ultimately decided, I wasn’t interested in trying this any further, they were all deadends. On to “near” mainline.

from #linux-rockchip on freenode, on earlier discussions I’d come across a more recent repo, on kernel.org no less, from one of the people who’s involved in mainlining efforts for rockchip support.

So, this is now…. _very recent_
Git repo: git://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip.git
Branch: v3.17-next/rk3xxx-dts

I’ve hacked a .config together, but I don’t know if it’s worth sharing at this point. So many options!

make -j8
# That spat out arch/arm/boot/zImage
make scripts # This makes the dts/dtb compiler
make rk3066a-bqcurie2.dtb # The only rk3066 in the tree, this is where I'll eventually have to make one for the minix neox5 mini
# that spat out arch/arm/boot/rk3066a-bqcurie2.dtb
cat arch/arm/boot/zImage arch/arm/boot/rk3066a-bqcurie2.dtb > zImage_with_dtb
[rktools]/rkcrc -k zImage_with_dtb > kernel.img
rkflashtool w kernel kernel.img (Or flashkit, or whatever)

Yay, now we actually get a booting kernel that we built ourselves….

425482 Starting kernel...@0x60408000

Uncompressing Linux... done, booting the kernel.
Booting Linux on physical CPU 0x0
Linux version 3.16.0-rc3 (karlp@teros) (gcc version 4.8.2 20140120 (Red Hat 4.8.2-2) (GCC) ) #1 SMP Wed Jul 30 21:08:48 GMT 2014
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=10c53c7d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine model: bq Curie 2
..... snip....
Please append a correct "root=" boot option; here are the available partitions:
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.16.0-rc3 #1
..... snip (same on cpu2) ....

whee, that’s actually doing the right thing. It actually gets all the way to failing to mount a rootfs, which is reasonable, as I hadn’t configured one, mounted one, or set anything up on an sd card. So, I guess next, we’re looking at making at least a stub rootfs, and looking at maybe a more specific device tree file for this hardware.

Note: If you don’t concat the DTS to the end of zImage, you’ll get a boot something like this…

 424337 Starting kernel...@0x60408000

Uncompressing Linux... done, booting the kernel.

Error: unrecognized/unsupported machine ID (r1 = 0x00000bfa).

Available machine support:

ID (hex)	NAME
ffffffff	Generic DT based system
ffffffff	Rockchip Cortex-A9 (Device Tree)

Please check your kernel config and/or bootloader.

A good write up of this is at http://billauer.co.il/blog/2014/02/uboot-linux-dtb-fdt-device-tree/

minix neo x5 mini – backup partitions and investigate

Urgh, I hate the massive pile of rom noise around anything to do with android devices.

I’m using https://github.com/neo-technologies/rkflashtool because it appears to be actually maintained. https://github.com/linuxerwang/rkflashkit is also somewhat convenient, though it’s gui only. If it was a nice python tool with a gui front end that would be even better. (no idea why you need waf and install either, simple “python run.py” is perfectly suitable.) libusb+python is perfectly acceptable for this sort of thing. No idea why people went straight to C code. https://github.com/neo-technologies/rockchip-mkbootimg looks good too, but I’ve not poked it yet. Anything that’s being maintained basically.

A commonly referred tool, https://github.com/naobsd/rkutils while often pointed to, is effectively a dead end with zero follow up commits. I’ve no interest in following that nest of forks.

Anyway, here’s the bits that actually worked to open up the boot partition image extracted with either rkflashkit or rkflashtools…

  • ./rkflashtool r boot > boot.img
  • dd if=boot.img of=bootimg.gz skip=8 bs=1 count=20000000
  • mkdir hohoho && cd hohoho
  • gunzip < ../bootimg.gz | cpio -i --make-directories

According to this review of boot.img formats, this means the x5mini uses format 4.

karlp@teros:~/projects/rkflashtool/hoho (master)$ ls
charger       init.goldfish.rc       init.usb.rc               sbin                 ueventd.rk30board.rc
data          init.rc                proc                      sys
default.prop  init.rk30board.rc      res                       system
dev           init.rk30board.usb.rc  rk30xxnand_ko.ko.3.0.36+  ueventd.goldfish.rc
init          init.trace.rc          rk30xxnand_ko.ko.3.0.8+   ueventd.rc
karlp@teros:~/projects/rkflashtool/hoho (master)$ 

Still learning the pieces, this is as much diary as blog.

Minix NEO X5 mini – teardown and serial console

Well, I bought a “Minix NEO X5 mini” for various experiments, eventually planning just a plain linux server, with ethernet and wifi, and a pile more ram and flash than the regular OpenWRT router platforms. I chose this based mostly on the smallest, cheapest that had a physical ethernet port. The other contender was the UG008. (And UG008B) Those are marginally smaller, and have one less fullsize USB, also, I _thought_ I’d read good instructions on getting linux running on the x5mini, but it turned out the instructions were for the x5 (full size) which is very similar, but not the same. I’ve since found instructions for the ug008, maybe I should have bought that. It also has a power button and from the pcb pics, looks like the pads for test points are bigger. No heatsink though, and I’d heard it’s wifi wasn’t as good. (Rumous and hearsay, I’ve no idea really) The second fullsize USB port however, is kinda of interesting. The x5mini came with a rather unusual fullsize USB A male-male cable. And there’s no microusb port for the OTG like normal. Plugging this in to a regular pc host would be disastrous, so the supplied android has a special menu option to enable “connecting to pc” which resets on every boot. If you do have it turned on, and you push the pinhole on the bottom while plugging in the power, you’ll get a usb device that ADB can find. dodgy, a regular OTG port would have been preferable.

So yeah, it works pretty well out of the box, but there’s enough reviews of that. We’re here to pull it apart. The little screws are glued in, so you need a good screwdriver, I almost stripped them getting them out. Then, whee, brown PCB! And whee, they put a weight in it to feel pro ;)

The wifi (rtl8188es) antenna is soldered on, then soldered to an adhesive antenna on the lid, a little awkward, but perfectly reasonable of course. Immediately on the board you can see two missing ICs, one, labelled “WIFI2″ is almost certainly Bluetooth, which is not on the x5mini, but is on the x5. More curious though is missing QFP48 IC by the recovery button, and it’s 12 pin “STM_DEBUG” connector. It certainly reminds me of an STM32 footprint, though the debug for cortex-m is normally 10 or 20 pins, not 12? No idea. Two test points on this side, T23 and T24, if it’s indeed a bluetooth module, this is possibly a uart between them? *shrugs*

Back to the “top” side, ethernet is an SMSC LAN8720A, power is a TI T659102, which seems to be pretty common on these boards. There’s two filled jumper blocks with interesting text, “USB1 Device / USB1 Host” and “Auto power on enable/disable” but I’ll leave them for a later time. There’s a rather unexpected battery connector too. Minix doesn’t have any battery backed devices in their catalog, so someone’s design company has been working on this. So much for thinking that minix was an OEM.

T306 is RX, T307 is TX and T308 is GND, for a serial console at 115200, 8N1

minix-neo-x5-mini-pcb-top-notes-30
minix-neo-x5-mini-pcb-bottom-notes-30

Finally, here’s most of the bootup

Using SWO/SWV streaming data with STLink under linux – Part 2

In Part 1, we set up some (very) basic code that writes out data via Stimulus Channel 0 of the ITM to be streamed otu over SWO/SWV, but we used the existing ST provided windows tool, “STLink” to be view the stream. Now let’s do it in linux.

OpenOCD has some very draft support for collecting this data, but it’s very rough around the edges. [1]

I wrote a tool based on my own decoding of USB traffic to be a little more flexible. You connect to the STLink hardware, and can start/stop logging, change trace files, and change which stimulus ports are enabled. It is quite rough, but functional. It should not be underestimated how important being able to start/stop tracing is. In the ARM debug docs, turning on or reconfiguring trace is undefined as far as having the output bitstream be properly synced. (Section D4.4 of “Flush of trace data at the end of operation” in the Coresight Architecture spec, and most importantly, “C1.10.4 Asynchronous Clock Prescaler Register, TPIU_ACPR” in the ARMv7M architecture reference manual)

Don’t get me wrong, although my tool works substantially better than OpenOCD does, it’s still very rough around the edges. Just for starters, you don’t have debug or flash at the same time! Having it integrated well into OpenOCD (or pyOCD?) is definitely the desired end goal here.

Oh yeah, and if your cpu clock isn’t 24MHz, like the example code from Part 1, then you must edit DEFAULT_CPU_HZ in the top of hack.py!

So, how do you use it?

First, get the source from github: https://github.com/karlp/swopy. You need pyusb 1.x, then run it, and type connect

karlp@tera:~/src/swopy (master)$ python hack.py 
:( lame py required :(
(Cmd) connect
STLINK v2 JTAG v14 API v2 SWIM v0, VID 0x483 PID 0x3748
DEBUG:root:Get mode returned: 1
DEBUG:root:CUrrent saved mode is 1
DEBUG:root:Ignoring mode we don't know how to leave/or need to leave
(1682, 2053)
('Voltage: ', 2.9293697978596906)
DEBUG:root:enter debug state returned: array('B', [128, 0])
('status returned', array('B', [128, 0]))
('status is: ', 'RUNNING')
(Cmd) 

Yes, there’s lots of debug. This is not for small children. You have been warned, but there is some help!

(Cmd) help

Documented commands (type help ):
========================================
connect     raw_read_mem32   run       swo_read_raw
magic_sync  raw_write_mem32  swo_file  swo_start   

Undocumented commands:
======================
EOF          exit  leave_state  raw_read_debug_reg   swo_stop
enter_debug  help  mode         raw_write_debug_reg  version 

(Cmd) 

The commands of interest are swo_file, swo_start and swo_stop. So, enter a file name, and start it up…

(Cmd) swo_file blog.bin
(Cmd) swo_start 0xff
INFO:root:Enabling trace for stimbits 0xff (0b11111111)
DEBUG:root:READ DEBUG: 0xe000edf0 ==> 16842752 (0x1010000) status=0x80, unknown=0x0
DEBUG:root:WRITE DEBUG 0xe000edfc ==> 16777216 (0x1000000) (res=array('B', [128, 0]))
DEBUG:root:READMEM32 0xe0042004/4 returned: ['0x0']
DEBUG:root:WRITEMEM32 0xe0042004/4 ==> ['0x27']
DEBUG:root:WRITEMEM32 0xe0040004/4 ==> ['0x1']
DEBUG:root:WRITEMEM32 0xe0040010/4 ==> ['0xb']
DEBUG:root:STOP TRACE
DEBUG:root:START TRACE (buffer= 4096, hz= 2000000)
DEBUG:root:WRITEMEM32 0xe00400f0/4 ==> ['0x2']
DEBUG:root:WRITEMEM32 0xe0040304/4 ==> ['0x0']
DEBUG:root:WRITEMEM32 0xe0000fb0/4 ==> ['0xc5acce55']
DEBUG:root:WRITEMEM32 0xe0000e80/4 ==> ['0x10005']
DEBUG:root:WRITEMEM32 0xe0000e00/4 ==> ['0xff']
DEBUG:root:WRITEMEM32 0xe0000e40/4 ==> ['0xff']
DEBUG:root:READMEM32 0xe0001000/4 returned: ['0x40000000']
DEBUG:root:WRITEMEM32 0xe0001000/4 ==> ['0x40000400']
DEBUG:root:READMEM32 0xe000edf0/4 returned: ['0x1010000']
DCB_DHCSR == 0x1010000
(Cmd) rDEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
unDEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 16 bytes of trace buffer
DEBUG:root:Wrote 16 trace bytes to file: blog.bin
DEBUG:root:reading 18 bytes of trace buffer
DEBUG:root:Wrote 18 trace bytes to file: blog.bin

Ok, great. but… where’d it go? Well. It’s in the native binary ARM CoreSight trace format, like so…

karlp@tera:~/src/swopy (master *)$ tail -f blog.bin | hexdump -C 
00000000  01 54 01 49 01 43 01 4b  01 20 01 37 01 31 01 38  |.T.I.C.K. .7.1.8|
00000010  01 0d 01 0a 01 54 01 49  01 43 01 4b 01 20 01 37  |.....T.I.C.K. .7|
00000020  01 31 01 39 01 0d 01 0a  01 54 01 49 01 43 01 4b  |.1.9.....T.I.C.K|
00000030  01 20 01 37 01 32 01 30  01 0d 01 0a 01 54 01 49  |. .7.2.0.....T.I|
00000040  01 43 01 4b 01 20 01 37  01 32 01 31 01 0d 01 0a  |.C.K. .7.2.1....|
00000050  01 54 01 49 01 43 01 4b  01 20 01 37 01 32 01 32  |.T.I.C.K. .7.2.2|
00000060  01 0d 01 0a 01 54 01 49  01 43 01 4b 01 20 01 37  |.....T.I.C.K. .7|
00000070  01 32 01 33 01 0d 01 0a  01 54 01 49 01 43 01 4b  |.2.3.....T.I.C.K|
00000080  01 20 01 37 01 32 01 34  01 0d 01 0a 01 54 01 49  |. .7.2.4.....T.I|
00000090  01 43 01 4b 01 20 01 37  01 32 01 35 01 0d 01 0a  |.C.K. .7.2.5....|
000000a0  01 54 01 49 01 43 01 4b  01 20 01 37 01 32 01 36  |.T.I.C.K. .7.2.6|
000000b0  01 0d 01 0a 01 54 01 49  01 43 01 4b 01 20 01 37  |.....T.I.C.K. .7|
000000c0  01 32 01 37 01 0d 01 0a  01 54 01 49 01 43 01 4b  |.2.7.....T.I.C.K|
000000d0  01 20 01 37 01 32 01 38  01 0d 01 0a 01 50 01 75  |. .7.2.8.....P.u|
000000e0  01 73 01 68 01 65 01 64  01 20 01 64 01 6f 01 77  |.s.h.e.d. .d.o.w|
000000f0  01 6e 01 21 01 0d 01 0a  01 54 01 49 01 43 01 4b  |.n.!.....T.I.C.K|
00000100  01 20 01 37 01 32 01 39  01 0d 01 0a 01 54 01 49  |. .7.2.9.....T.I|
00000110  01 43 01 4b 01 20 01 37  01 33 01 30 01 0d 01 0a  |.C.K. .7.3.0....|
00000120  01 68 01 65 01 6c 01 64  01 3a 01 20 01 32 01 34  |.h.e.l.d.:. .2.4|
00000130  01 35 01 32 01 20 01 6d  01 73 01 0d 01 0a 01 54  |.5.2. .m.s.....T|
00000140  01 49 01 43 01 4b 01 20  01 37 01 33 01 31 01 0d  |.I.C.K. .7.3.1..|
00000150  01 0a 01 54 01 49 01 43  01 4b 01 20 01 37 01 33  |...T.I.C.K. .7.3|

Which is ugly, but you get the idea.

This is where swodecoder.py comes in. The author of the original SWO support in OpenOCD has some code to do this too, it’s more forgiving of decoding, but more likely to make mistakes. Mine is somewhat strict on the decoding, but probably still has some bugs.

Usage is pretty simple

$ python swodecoder.py blog.bin -f
Jumping to the near the end
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 73
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 67
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 75
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 32
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 50
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 49
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 53
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 51
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 13
Not in sync: invalid byte for sync frame: 1
Not in sync: invalid byte for sync frame: 10
TICK 2154
TICK 2155
TICK 2156
TICK 2157
---snip---
TICK 2194
TICK 2195
TICK 2196
Pushed down!
held: 301 ms
Pushed down!
TICK 2197
held: 340 ms
TICK 2198
TICK 2199
Pushed down!
TICK 2200
TICK 2201
---etc---

You (hopefully) get the idea. When the writes to the stimulus ports are 8bit, swodecoder.py simply prints it to the screen. So here we have a linux implementation of the SWV viewer from the windows STLink tool. It’s got a lot of debug, and a few steps, but the pieces are all here for you to go further.

In part three, we’ll go a bit further with this, and demonstrate how SWO lets you interleave multiple streams of data, and demux it on the host side. That’s where it starts getting fun. (Hint, look at the other arguments of swodecoder.py and make 16/32bit writes to the stimulus registers)

To stop SWO capture, type “swo_stop” and press enter, or just ctrl-d, to stop trace and exit the tool.

[1] Most importantly, you can not stop/start the collection, you can only set a single file at config time, which isn’t very helpful for running a long demon. Perhaps even worse, OpenOCD is hardcoded to only enable stimulus port 0, which is a bit restrictive when you can have 32 of them, and being able to turn them on and off is one of the nice things.

Using SWO/SWV streaming data with STLink under linux – Part 1

This is part 1 in a short series about using the SWO/SWV features of ARM Cortex-M3 devices [1]
This post will not explain what SWO/SWV is, (but trust me, it’s cool, and you might work it out by the end of this post anyway) but will focus on how to use it.

First, so you have a little idea of where we’re going, let’s start at the end…

enum { STIMULUS_PRINTF }; // We'll have more one day
 
static void trace_send_blocking8(int stimulus_port, char c) {
        while (!(ITM_STIM8(stimulus_port) & ITM_STIM_FIFOREADY))
                ;
        ITM_STIM8(stimulus_port) = c;
}
 
int _write(int file, char *ptr, int len)
{
        int i;
        if (file == STDOUT_FILENO || file == STDERR_FILENO) {
                for (i = 0; i < len; i++) {
                        if (ptr[i] == '\n') {
                                trace_send_blocking8(STIMULUS_PRINTF, '\r');
                        }
                        trace_send_blocking8(STIMULUS_PRINTF, ptr[i]);
                }
                return i;
        }
        errno = EIO;
        return -1;
}

You can get this code from either:

  • My github repository
  • The swo-1-printf directory in swo-stlink-linux-1

    That’s all[2] you need to have printf redirected to an ITM stimulus port. It’s virtually free, doing nothing if you don’t have debugger connected. [3]

    Groovy. If you have the Windows STLink Utility, you can use this right now. Enter the correct clock speed of your main app, and choose stimulus 0 (or all) and watch your lovely console output.

    stlink-windows-swv

    Ok, that’s cool, but weren’t we going to do this in linux? We were, and we will, but let’s stop here with a good working base, so we can focus on just the extra stuff later.

    [1] Cortex M4 too, but not M0, that’s another day altogether. Specifically, STM32L1 parts, but the concepts and code are the same
    [2] Expects you have your general makefiles all set up to do “the right thing” for newlib stubs and so on.
    [3] Except for generating the formatted string of course, that’s not free. And it does take a _little_ bit of time to write the characters out without overflowing, but that’s a story for another day.

Who pays for shipping? FedEx, TI and overseas agents

Update: I lost this in my drafts folder for over a year!

I live in Iceland. It’s a nice place to live, but it’s small, so I often have to buy things from overseas, particularly in for hobby electronics gear, but anything small and cheap has this problem.

Now, let’s get this straight, I’m not complaining about customs or VAT. (Not today anyway) VAT here in Iceland is 25.5%, and is applied at the end, on top of shipping charges. Customs rate isn’t always as easy to work out.

Now, in this example, I bought a device from Texas Instruments, for $US4.99, or about 630 ISK. This was advertised as shipping included, worldwide. Add 25.5% VAT, 790 ISK. Not bad. So it arrived yesterday.

Now, what does that even mean? They’ve charged me ~473 ISK for “customs handling” It doesn’t seem to be anything like the VAT rate, or the VAT rate plus the customs rate. (FIXME – finish getting customs rate)

Customs officially don’t collect anything like this, they simply say that agents are allowed to charge for filing paperwork.

So, this seems to be some arbitrary amount they just felt like charging. Next we have 750 ISK for being “an unregistered customer.” Well, that sounds awesome. I didn’t choose this company, I ordered a product from Texas Instruments with shipping included. Who knows if it’s even possible to become a registered customer.

Then there’s VAT on those charges. So, a whole 120% charge (of original purchase) on being the wrong sort of customer and another 75% (of original purchase) for “handling”. I can’t actually see where they charge the VAT/Customs on the product itself, but the end result is that I’ve paid 170% of the original purchase ON TOP of the basic vat rate + the product price. (1345 ISK + 790 ISK)

How is this remotely reasonable for “shipping included” ? And who’s responsible? Texas Instruments clearly believes they paid FedEx, (or presumably FedEx wouldn’t accept the shipment?) I can only assume that FedEx delivers the package to Iceland, believing that they’ve been paid, but I have zero insight into what the deal is between the local agent (Hraðflutningar ehf)

I believe the party responsible is IceTransport. As they said earlier, local customs agents, defined here, are allowed to charge basically whatever they like. So they do. I presume they treat customers shipping outbound through them quite a lot better, but what are my options at the end of the day? I refuse to pay their arbitrary charges, they hold the parcel, and they’ve already been paid by FedEx/Texas Instruments? So they bin the package, and are already in front? Or do I just suck it up, and pay through the nose? They’re both distasteful.

Customs official answer on handling charges:

http://www.tollur.is/displayer.asp?cat_id=2816&module_id=210&element_id=10177

Decoding Vendor Specific USB protocols with Wireshark lua plugins

Earlier this week I was doing some reverse engineering and confirmation of behaviour for a USB tool. I got wireshark to sniff the traffic, (Not going into that here, it’s relatively straightforward and documented enough on the web to get by) but as it’s a vendor specific protocol, it was just lots of bytes.

I was decoding them by hand, and then copying and pasting into a python script (I had pretty good sources of what all the bytes meant, I just had no good way of visualising the stream, which is where wireshark and this post comes in)

I have written a custom wireshark dissector before, but I wasn’t super happy with the mechanism. I have been doing some work with Lua in my dayjob, and had read that wireshark supported dissector plugins could be written in Lua now. Seemed like a better/easier/more flexible approach.

I got to work, following these (somewhat) helpful resources:

These were all generally very helpful, but there two things I wrestled with, that I didn’t feel were at all well described. From here on out, I’m going to assume that you know what’s going on, and just need help with things that are not covered in any of the earlier links.

Little Endian

You declare ProtoField’s as just uint8, uint16, uint32. This is fine and sane. But there’s a few ways of working with it when you add it to the tree.

f.f_myfield = ProtoField.uint32("myproto.myfield", "My Field", base.HEX)
-- snip --
mytree:add_le(f.f_myfield, buffer(offset, 4)

This way works very well, selecting “myfield” in the packet view correctly highlights the relevant bytes. But say you want to get the value, to add to the info column for instance, you might do this.. (if you read the api guide well enough)

local val = buffer(offset, 4):le_uint()
-- normal add, we've done the LE conversion
-- if you don't do :le_uint() above, and do a :add_le() below,
-- the info column will show the backwards endian value!
mytree:add(f.f_myfield, val)
pinfo.cols["info"]:append(string.format(" magicfield=%d", val))

At first glance, this works too. The Info Column shows the right value, the tree view shows the right value. BUT clicking on the tree view doesn’t highlight the bytes.

Here’s how to do it properly:

local val = buffer(offset, 4)
mytree:add_le()(f.f_myfield, val)
pinfo.cols["info"]:append(string.format(" magicfield=%d", val:le_uint()))

Ok, a little fiddly, but you would probably get there in the end.

Reading existing USB properties

The docs talk about doing something like:

local f_something = Field.new("tcp.port")

Except, I didn’t find anywhere that described what magic strings were available. I tried using the values available in the display filter box, like, “bEndpointAddress” but never got anywhere. One of the samples led me to this tidbit:

       local fields = { all_field_infos() }
       for ix, finfo in ipairs(fields) do
            print(string.format("ix=%d, finfo.name = %s, finfo.value=%s", ix, finfo.name, getstring(finfo)))
        end

When you click on a packet, this will dump lots to the console, and you can hopefully work out the magic values you need!

Synchronising packets

TCP streams are easy, you have sequence ids to correlate things. USB isn’t quite the same. You can see the “URB” has two frames, from host to device and device to host, which are in sync, but for the very common case of writing to an OUT endpoint, and getting a response on an IN endpoint, you don’t get any magical help.

I found a way of doing this, but it’s not ideal, and tends to mess up the display in wireshark if you click on packets in reverse order. This is because I just set a state variable in the dissector when I parsed the OUT packet, and check it when I parse the IN packet. It works, but it was less that ideal. Sometimes you need to click forwards through packets again. Sometimes the tree view would show the right values too, but the info column would be busted. Probably doing something wrong somewhere, but hard to know what.

Actually hooking it up

Finally, and the most frustrating, was how to actually hook it up! The demos, all being TCP related, just do:

the_table = DissectorTable.get("udp.port")
the_table:add(9999, my_protocol)

Ok, well and good, but how on earth do I register a USB dissector? You need to register it by the class which is sort of ok, but good luck having dissectors for multiple vendor specific classes. I didn’t see a way of adding a dissector based on the VID:PID, though I think that would be very useful.

usb_table = DissectorTable.get("usb.bulk")
-- this is the vendor specific class, which is how the usb.bulk table is arranged.
usb_table:add(0xff, my_proto)
-- this is the unknown class, which is what shows up with some usb tools?!
usb_table:add(0xffff, my_proto)

Update 2013-Jan-16One of the sigrok developers got the vid/pid matching working, in another example of a lua plugin.

This skims over a lot, but it should help.
Final working code:

I’ve totally glossed over exactly _what_ or _why_ I was decoding vendor specific usb classes. That’s a topic for another day :)

Here’s some pictures though

How it looks before you write a plugin.

How it looks before you write a plugin.

With the plugin, note that the info column isn't always showing the proper values.  No idea why.  wireshark's weird

With the plugin, note that the info column isn’t always showing the proper values. No idea why. wireshark’s weird

PS: How about that title! linkbait to the max! keywords to the rescue! (or at least, the ones I had tried searching for)

Installing Eagle 6.5 on Fedora 18

I’ve spoken about this before, but here’s how to install current Eagle 6.5.0 on Fedora 18 64bit.

As before, this may be incomplete for a fresh install, ie, I’ve already installed all the 32bit compat libs. But Eagle expects to find OpenSSL 1.0.0. No other version at all.

You need openssl-libs.i686 to be installed, but that will be 1.0.1e, and none of the existing symlinks are suitable.

$ sh ./eagle-lin-6.5.0.run
/tmp/eagle-setup.11607/eagle-6.5.0/bin/eagle: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory

However, you can just symlink it a bit more…

/usr/lib $ sudo ln -s libssl.so.1.0.1e libssl.so.1.0.0
/usr/lib $ sudo ln -s libcrypto.so.1.0.1e libcrypto.so.1.0.0

STM32 Unique ID Register on L1 parts

Last week I mentioned I was seeing duplicate “Unique” ID registers on some STM32L151 parts [1], and included some memory dumps of the three unique ID registers as proof.

However, I had foolishly assumed that on the L1 the Unique ID register was three contiguous 32 bit registers, as it is on the F0, F1, F2, F3 and F4. (The base address changes, but that’s normal and expected)

On the L1, the registers are actually at offset 0, offset 4, and offset 0×14. Thanks for nothing ST! :(
(Oh, and L1 Medium+ and High Density devices use a different base address too, good job)

Here’s some more complete memory dumps for the same three parts I was looking at last week.

UID31:0 UID63:32 UID95:64
Part A 0x0e473233 0×30343433 0×00290031
Part B 0x0e473233 0×30343433 0×00320032
Part C 0x0e473233 0×30343433 0×00380030

Reading other reference manuals, and seeing that the UID registers often have 8 bits of unsigned “Wafer number”, 7 bytes of ASCII Lot number, and 4 bytes of X/Y wafer coordinates in BCD, I would interpret my part “A” above as

Wafer Number Lot Number X/Y coords
Hex 0x0e 0×47323330343433 0×00290031
Natural 0x0e G230443 X=0029, Y=0031

For reference, here are some full dumps of that section of memory. “0x7b747800″ is what I had been looking at as UID bits 95:64, note that there are other bits in this section with fixed values, no idea what they mean :)

Part A

(gdb) x /20x 0x1FF80050
0x1ff80050: 0x0e473233  0x30343433  0x7b747800  0x50505091
0x1ff80060: 0x00000011  0x00290031  0x11000000  0x11000011
0x1ff80070: 0x00000000  0x00000000  0x029f067e  0x035a0000
0x1ff80080: 0x035a0000  0x035a0000  0x035a0000  0x035a0000
0x1ff80090: 0x035a0000  0x035a0000  0x035a0000  0x035a0000
(gdb) 

Part B

(gdb) x /20x 0x1FF80050
0x1ff80050: 0x0e473233  0x30343433  0x7b747800  0x50505091
0x1ff80060: 0x00000011  0x00320032  0x11000000  0x11000011
0x1ff80070: 0x00000000  0x00000000  0x02a50685  0x035e0000
0x1ff80080: 0x035e0000  0x035e0000  0x035e0000  0x035e0000
0x1ff80090: 0x035e0000  0x035e0000  0x035e0000  0x035e0000
(gdb)

Part C

(gdb) x /20x 0x1FF80050
0x1ff80050: 0x0e473233  0x30343433  0x7b747800  0x50505091
0x1ff80060: 0x00000011  0x00380030  0x11000000  0x11000011
0x1ff80070: 0x00000000  0x00000000  0x02a50689  0x035e0000
0x1ff80080: 0x035e0000  0x035e0000  0x035e0000  0x035e0000
0x1ff80090: 0x035e0000  0x035e0000  0x035e0000  0x035e0000
(gdb)

[1] Again, these are STM32L151C6T6 parts, revision V, package markings “GH254 VG” and “CHN309″

STM32 Unique ID register not so unique (Or, how to read docs properly)

UPDATE: This post is WRONG! See updated information here

The findings below were based on expecting the UID register to be contiguous as it is on all other STM32 parts. This is not true on the L1 family, and I hadn’t taken enough care with reading the reference manual.

Original post below


Following up from when I wrote about it earlier, it turns out that the “unique” id isn’t as unique as it is meant to be.

On my desk I have three different STM32L151C6T6 revision “V” parts, with exactly the same 96bit unique id. The parts all have package labels “GH254 VG” and CHN309

UID[32:0] (0x1FF80050) UID[63:32] (0x1FF80054) UID[96:64] (0x1FF80058)
Hex 0x0e473233 0×30343433 0x7b747800
Decimal 239546931 808727603 2071230464

According to reports on the irc channel ##stm32, this has also been seen (at least) on stm32f407vet6 parts.

Not fun :(