Category Archives: software

tracker-miner-fs stuck at 100% alternative solution

There’s plenty of posts about problems with tracker-miner-fs being a CPU hog, even if it’s theoretically “idle”. “tracker status” will say that indexing is complete, and all miners are idle, but that’s clearly a lie, and you process stuck running from bootup.

Most “solutions” on the internet revolve around just turning off tracker completely, or at least the filesystem miner. I don’t want that, I actually like having search work. What I wanted was it to, you know, work :)

So, while experimenting with resetting it, I noticed that it was always getting stuck trying to recurse a particular directory tree. It’s 6GB, but that was unlikely to be problem. However, one directory there contained a lot of files. Lots like, 600,000 files or so, in one directory. This is enough to make ls hang, so, maybe it was enough to make tracker-miner-fs hang too? (Side note, if you want to count files in a directory like this, find . | wc -l will work, where ls doesn’t.)

I didn’t actually need those files anymore, so I just deleted the suspect directories entirely, and then restarted the trackers with tracker daemon --stop; tracker daemon --kill; tracker daemon --start (The kill was to kill the errant tracker-miner-fs process, the others all stopped gracefully)

And presto, tracker status starts reporting sane and plausible results, the tracker-miner-fs runs at 100% for a short period, reporting status correctly, then the tracker extractor takes over. And we have search again!

GVFS mount points, or, how to completely ignore usability while building a usability feature.

Gnome happily mounts my phone’s MTP shares for me. I can even browse for them in “files” and drag and drop and everything. It’s pretty cool. And, magically, behind the scenes, it even exists in the filesystem so I can use rsync and cp and things.

But the mount point…. Oh boy. What on earth were they thinking. /run/user/1000/gvfs/mtp\:host\=%5Busb%3A002%2C010%5D/
Are you fucking kidding me? That’s an url encoded way of saying [usb:002,010] Why did it need to be url encoded? No idea. What about the content? Oh, that’s the USB bus, and device address. Spectacular. That means it depends on not only what physical port it’s plugged into, but what order it was plugged in. (And that unplugging and replugging will make a new path) Let’s not use things like iManufacturer, iProduct, iSerial, vid, pid, anything actually device specific no, not here.

I’m sure this was done to try and ensure that the mount point would be unique. Great. Mission accomplished. However, it could be made unique, and usable at the same time. And then this thing, that exists out of a usability feature, would actually be more usable.

Netbeans 8.x “Could not get children iterator” or “NULL VALUE RETURNED FOR CHILDREN”

If you’re getting messages like this when debugging and trying to look at variables, chances are you might be debugging some glib2 code. If you are, welcome to https://bugzilla.gnome.org/show_bug.cgi?id=749092

It’s easy enough to manually patch the file, and presto, working debug again. It’s been fixed in GDB, I was seeing this in Fedora 22, which is not _entirely_ up to date these days, but you tend to expect that plain old host debugging should be pretty stable. blog post because the netbeans error string doesn’t help you find that gdb bug. You need to run gdb on the command line to find the real error: Python Exception iter() returned non-iterator of type '_iterator':

More arm android box conversion adventures

TL;DR: Still failing. Gave up again.

Back when I bought a RK3066 based device, I had seen rockchip developers start contributing directly upstream.  However, the RK3066 just missed the boat, and while I poked and prodded a bit, it was all just a bit too difficult and a bit too complicated a bit too much not much fun.  Still, I followed what was going on.  The plan was to use the swell of consumer electronics to get a cheap android tv box that I could run linux on, and use as a small home server.  “Just use a raspi” you say.  Well, no.  Raspi3 is at least decently powerful, but running everything off a sdcard, and not having a case is really not what I was looking for.  Little tiny cute black boxes, with a remote control and a power supply and ethernet and usb shipped for $30?  That’s what I want to reuse.  Rockchip continued to contribute upstream, so I bought a MXQ 4K (RK3229).  New hotness I thought, but the cheap end.  I still believed that rockchip was doing the best job of upstream support.  Note: My copy of this device has a much smaller heatsink, Spectek flash, not Sandisk, and some generic no name ram chips. (See pics)

Top side, smaller heatsink, no name ram.

Top side, smaller heatsink, no name ram.

Maybe they are.  And I’ve not worked on amlogic or sunxi chips, but goddamn, the work environment is still just as sucky.  A garbage world of “custom roms” and complicated documentation detailing multiple ways of doing things with rarely any explanation of why.  I’ve still no idea, but some things seem a little clearer than last time… maybe?

If you just want to run linux, in _theory_ you can just use the existing android kernel shipped stock.  It has support for all your hardware, you just need to replace the root file system (RFS) and amend the kernel command line to say where your rootfs is.  Except, because some of the drivers are actually separate binary proprietary modules, you end up needing a fucking initrd/initramfs timewaste so it can have such core details like a functional fucking nand driver.  Why the fuck rockchip is keeping _this_ bit proprietary is beyond me.  So that gets non-trivial quickly.  You can in theory extract the modules from your android system image.  If you have a stock firmware, (“update.img”) you can use rkunpack from the rkflashtool repository.  you’ll probably have to unpack more than once.  Documentation on what’s what is less than forthcoming.  The “system.img” wil be a raw ext4 image (in theory) and you can mount it locally to explore with “sudo mount -o loop blah.img somemountpoint”

anyway, while some of this is just for my own notes, not really for anyone else, one thing has stuck out.  First, the default serial console is at 1500000 baud.  Thanks for nothing rockchip.  On a cp2102 serial dongle, you’ll need to use the silabs baud rate aliasing software to get to this baud rate.  Thanks for nothing silabs. and yeah, again, thanks for nothing rockchip.  The serial console is in the picture below.

Bottom side, spectek flash, serial console details

Bottom side, spectek flash, serial console details

Oh yeah, the “thing that has stuck out”  Despite being “unbrickable” because they have a maskrom bootloader, and normally just going into recovery mode,  I’ve had great difficulty.  After some bad flashing,  (bad why? no idea, rkflashtool or upgrade_tool both successfully report the flashing complete, then I reboot, and get…. bad things) the serial console will hang on a line like…

In
300MHz
DDR3
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
mach:2
OUT
Boot1 Release Time: 2016-03-15, version: 2.31
ChipType = c 275
No.1 FLASH ID:2c 64 64 56 a5 4
ECC:60
ReadRetry pageadd=15d800  ecc=3c err=ffffffff
Read pageadd=15d800  ecc=3c err=ffffffff
spare: 0x0:b8679151 b8679151 b8679151 b8679151 
ReadRetry pageadd=15d801  ecc=3c err=ffffffff
Read pageadd=15d801  ecc=3c err=ffffffff
spare: 0x0:527ded0c 527ded0c 527ded0c 527ded0c 
SdmmcInit=0 20
StorageInit ok = 134644
SecureMode : SBOOT_MODE_NS
powerOn 812121
Usb re Boot. 6812118

At this point, rkflashtool v will “fail” like so… (notice mangled text)

$ rkflashtool v
rkflashtool: info: rkflashtool v5.2
rkflashtool: info: Detected RK322X...
rkflashtool: info: interface claimed
rkflashtool: info: chip version: 322A-��..��-��

upgrade_tool will still say the device is connected, in “Loader” mode, and TestDevice, ReadFlashID, ReadFlashInfo and ReadChipInfo all still work. But attempting to download any image will fail saying parameter is invalid. rkflashtool p agrees.

 rkflashtool p
rkflashtool: info: rkflashtool v5.2
rkflashtool: info: Detected RK322X...
rkflashtool: info: interface claimed
rkflashtool: info: reading parameters at offset 0x00000000
rkflashtool: info: size:  0x0000ffff
rkflashtool: fatal: Bad parameter length!

One time this happened, I got out of it by having the USB-A-A cable disconnected (oh yeah, A-A for OTG? thanks for fucking nothing rockchip, hooray port overloads and reverse powering disasters) holding the recovery button down (with a toothpick down the SPDIF hole. (button down the AV hole _should_ be different but seems to behave the same for me)) and then plugging in the main power supply. Perhaps it had been a problem with powering the device purely via the A-A cable as I’d normally been doing?

Who knows. It’s currently in this state again, the unbrickable “bricked” Even shorting pins 7-8 on the NAND chip itself and powering up isn’t helping. (The “standard” rockchip method of getting back to maskrom mode) I get a different log, but it’s still non-responsive, even after inserting the A-A cable. (Still in loader, but not really working)

DDR Version V1.04 20160504
In
300MHz
DDR3
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
mach:2
OUT
Boot1 Release Time: 2016-03-15, version: 2.31
ChipType = c 276
No.1 FLASH ID:2c 64 64 56 a5 4
FlashLoadPhyInfo fail 2bc!!
sync para 32
sync para 32
ECC:60
ReadRetry pageadd=15d800  ecc=3c err=ffffffff
Read pageadd=15d800  ecc=3c err=ffffffff
spare: 0x0:b8679151 b8679151 b8679151 b8679151 
ReadRetry pageadd=15d801  ecc=3c err=ffffffff
Read pageadd=15d801  ecc=3c err=ffffffff
spare: 0x0:527ded0c 527ded0c 527ded0c 527ded0c 
SdmmcInit=0 20
StorageInit ok = 93093
SecureMode : SBOOT_MODE_NS
powerOn 770567
Usb re Boot. 6770565

So. No results, no future, but at least a weird log for the internet to archive if it helps anyone sometime.

oh udev. oh yes

I try and do the “right thing” when it comes to the future and how linux distros see their roles and the way things should be.  I think systemd has some generally good ideas, though I think upstart has far far better documentation.  Anyway, given that linux distros still “protect” users from plugging in usb devices, except for all those ones that they _don’t_ protect you from, if you’re working with embedded electronics, sooner or later you need to “fix” permissions for some device you have.  No matter, the canonical advice for a “long time” has been something like adding a file in /etc/udev/rules.d/doesnt_really_matter.rules with some content like…


ATTRS{idVendor}=="aaaa", ATTRS{idProduct}=="bbbb", GROUP="plugdev"

And you added yourself to the plugdev group, and everything was pretty much ok. Now, it’s the “future” and we can’t use anything so silly and outdated as groups to assign permissions. Heaven help us, what would become of us all if people just used groups?! Now we need to use “tags” and a magical “uaccess” incantation. Except for all those devices that we do use groups for, like usb modems, usb serial ports, usb printers, wireshark capture permissions, virtualization, colour correction devices, basically everything except “unknown” devices.

But ok, good and well, we’ll learn the new magic incantation for udev….

ATTRS{idVendor}=="aaaa", ATTRS{idProduct}=="bbbb", TAGS+="uaccess"

And…. it doesn’t work. Why? Good fucking luck finding out. If you’re lucky, someone might suggest that the name of your file isn’t right. Why should that matter? Welllll. because it does. See https://lists.freedesktop.org/archives/systemd-devel/2015-August/033946.html and https://bugzilla.redhat.com/show_bug.cgi?id=1074122#c0 for some hints and clues.

TL;DR: pretty much must be named 70-doesnt-matter.rules. At least until system decides that it should be a different number without documenting it anywhere.

Disable Alt-F1 shortcut on gnome

Current versions of gnome’s “Keyboard shortcuts” control panel applet don’t have an entry for “System->Show the activities overview“, so you can’t edit it.  Instead, it’s hard coded to the winkey, and  Alt-F1.  No. Bad gnome.  Don’t do that. I want to use that, (in my case, in IntelliJ)

What to do?

gsettings to the rescue.

$ gsettings get org.gnome.desktop.wm.keybindings panel-main-menu
['<Super>s', '<Alt>F1']
$ gsettings set org.gnome.desktop.wm.keybindings panel-main-menu "['<Super>s']"

Presto.

Using NetBeans for STM32 development with OpenOCD

This post supersedes http://false.ekta.is/2012/05/using-netbeans-for-stm32-development-with-stlink-texane/ it has been updated for 2016 and current best software tools.  Again, this is only focussed on a linux desktop environment.

Required pieces haven’t changed much, you still need:

  1. A tool chain.
  2. GDB Server middleware (OR just a tool flashing if you want to live in the dark ages)
  3. A “sexy” IDE (If you disagree on wanting an IDE, you’re reading the wrong post)

Getting a toolchain

New good advice

Advice here hasn’t changed.  The best toolchain is still gcc-arm-embedded  They steadily roll out updates, it’s the blessed upstream of all ARM Cortex GCC work, and it has proper functional multilib support and proper functional documentation and bug reporting.  It even has proper multi platform binaries for windows and macs.  Some distros are packaging “arm-none-eabi-XXXXX” packages, but they’re often old, repackages of old, poorly packaged or otherwise broken.  As of November 2015 for instance, ArchLinux was packaging a gcc 5.2 binary explicitly for arm-cortex, that did not support the -mmcu=cortex-m7 option added in gcc 5.x series.  Just say no.

I like untarring the binaries to ~/tools and then symlinking to ~/.local/bin, it avoids having to relogin or start new terminals like editing .bashrc and .profile does.
~/.local/bin$ ln -s ~/tools/gcc-arm-none-eabi-5_2-2015q4/bin/arm-none-eabi-* .

Old bad advice

The internet is (now) full of old articles recommending things like “summon-arm-toolchain” (Deprecated by the author even) “code sourcery (lite)” (CodeSourcery was bought by Mentor, and this has been slowly killed off.  Years ago, this was a good choice, but all the work they did has long since been usptreamed)  You can even find advice saying you need to compile your own.  Pay no attention to any of this.  It’s well out of date.

GDB Server middleware

New good advice

Get OpenOCD. Make sure it’s version 0.9 or better. 0.8 and 0.7 will work, but 0.9 is a _good_ release for Cortex-M and STM32 parts. If your distro provides this packaged, just use it. Fedora 22 has OpenOCD 0.8, Fedora 23 has OpenOCD 0.9. Otherwise, build it from source

Old bad advice

Don’t use texane/stlink.  Just don’t.  It’s poorly maintained, regularly breaks things when new targets are introduced and not nearly as flexible as OpenOCD.  It did move a lot faster than OpenOCD in the early days, and if you want a simpler code base to go and hack to pieces for this, go knock yourself out, but don’t ask for help when it breaks.

Netbeans

No major changes here, just some updates and dropping out old warnings.  You should still setup your toolchain in netbeans first, it makes the autodetection for code completion much more reliable.  I’ve updated and created new screenshots for Netbeans 8.1 the latest current release.

First, go to Tools->Options->C/C++->Build Tools and Add a new Toolchain…

Adding a new toolchain to netbeans

Adding a new toolchain to netbeans

Put in the “Base directory” of where you extracted the toolchain.  In theory netbeans uses the base directory and the “family” to autodetect here, but it doesn’t seem to understand cross tools very well.

Base path for new toolchain and name

Base path for new toolchain and name

Which means you’ll have to fill in the names of the tools yourself, as shown below.  Click on “Versions” afterwards to make sure it all works.

Adding explicit paths to netbeans toolchains

Adding explicit paths to netbeans toolchains

“Versions” should show you the right thing already, as shown

Versions from our new toolchain (and an error from gdb we must fix)

Versions from our new toolchain (and an error from gdb we must fix)

If you’re getting the error about ncurses from gdb, this because newer gdb builds include the curses “tui” interface to gdb in the standard build.  (Yay! this is a good thing!)  However, as the g-a-e toolchains are all provided as 32bit, you may be missing the 32bit ncurses lib on your system.  On Fedora, this is provided in the ncurses-libs.i686 package.

Ok, now time to build something.  This is your problem, I’m going to use one of the libopencm3 test programs right now, specifically, https://github.com/libopencm3/libopencm3/tree/master/tests/gadget-zero (the stm32l1 version)

Programming your device

Netbeans doesn’t really have any great way of doing this that I know of.  You can use build configurations to have “run” run something else, which works, but it’s a little fiddly.  I should spend more time on that though.  (See later for a way of doing it iteratively via the debugger console in netbeans)

In the meantime, if you just want to straight out program your device:

$ openocd -f board/stm32ldiscovery.cfg -c "program usb-gadget0-stm32l1-generic.elf verify reset exit"

No need for bin files or anything, there’s a time and a place for those, and if you don’t know and can explain why you need bin files, then elf files are just better in every way.

Debugging your part

GDB is always going to be a big part of this, but, assuming you’ve got it flashed, either by programming as above, then you can debug in netbeans directly.  First, make sure OpenOCD is running again, and just leave it running.


$ openocd -f board/stm32ldiscovery.cfg

First, install the gdbserver plugin, then choose Debug->Attach Debugger from the menu.

Attach to a gdbserver

Attach to a gdbserver

  • Make sure that you have “gdbserver” as the debugger type.  (Plugin must be installed)
  • Make sure that you have “ext :3333” for the target.  By default it will show “remote host:port” but we want (need) to use “extended-remote” to be able to restart the process. (See the GDB manual for more details)
  • Make sure that you have the right project selected.  There’s a bug in the gdbserver plugin that always forgets this.

At this point, “nothing” will happen.  If you look at the console where OpenOCD is running, you’ll see that a connection was received, but that’s it.
Press the “Pause” button, and you will stop execution wherever the device happened to be, and netbeans will jump to the line of code. In this example, it’s blocked trying to turn on HSE, as there’s no HSE fitted on this board:

Source debugging in netbeans via OpenOCD

Source debugging in netbeans via OpenOCD

If you now set a breakpoint on “main” or anywhere early and press the “Restart” icon in the top, OpenOCD will restart your process from the top and stop at the first breakpoint. Yay!  If restart fails, make sure you used “extended-remote” for the target!

Bonus

If you click the “Debugger console” window, you can actually flash your code here too.  Leave the debugger running!  (No need to stop the debugger to rebuild) Make a change to your code, rebuild it, and then, on the “Debugger console” just enter “load” and press enter.  You’ll see the OpenOCD output as it reflashes your device.  As soon as that’s done, just hit the “Restart” icon in netbeans and start debugging your new code!

Facebook share scraping image selector open graph woes

So, the share buttons on my website have been giving blank images, or the wrong images for a while. You’re meant to go and add a whole bunch of OpenGraph meta tags, but W.T.F the tag for the image to use must be absolute. wat

No, really, WAT. For double fun wat, it must include the protocol too! You have to use a second tag if you want to use https!

So, using the “modern” og:image tags is just right out then. (for realz, who the fuck though absolute urls were a good idea. I generate galleries, the galleries just run on disk, you don’t need to know where in the site they’re mounted or anything) Web browsers themselves are perfectly capable of loading a relative url image. Facebook’s Object Debugger on the other hand, not so much. It will take an img with src=../blah.jpg" and turn it into http://domain/blah.jpg, completely ignoring the path sections.

However… it can deal with relative urls that don’t have .. in them. “src=img/blah.jpg” is perfectly ok. Dumb, dumb and dumber.

<img src="../blah.jpg"> FAIL
<img src="blah.jpg"> GOOD
<img src="relative/blah.jpg"> GOOD

WAT

version control of tools – part 2

Oh look! again! Remember kids, it’s not just compilers that you should keep around, but libc/binutils, and all libraries you’re ever going to use.

Today it’s avr-libc, and this gem:

In file included from ../../common/FreqCounter.h:17:0,
                 from ../../common/FreqCounter.c:35:
/usr/avr/include/util/delay.h: In function '__vector_21':
/usr/avr/include/util/delay.h:246:2: error: __builtin_avr_delay_cycles expects a compile time integer constant
  __builtin_avr_delay_cycles(__ticks_dc);
  ^

Apparently it was always actually a requirement in the notes, but everyone did it anyway. Now, you need to write a wrapper function like this sort of shit:

void do_what_I_say_delay_us(int us) {
     while (us-- > 0) {
         _delay_us(1);
     }
}

Hooray, isn’t that better for everyone! See http://nongnu.org/avr-libc/user-manual/group__util__delay.html for more information, if you can read carefully enough :)

version control of tools

Ahh, it’s always the right thing to do, but normally it’s so heavy to check in things like GCC/Binutils versions into every project. It’s the sort of thing you do if you’re a big serious company, working on big serious things, with big serious support. Besides, I’m not relying on anything weird or esoteric right?

Wrong.


../../common/pjrc/usb_debug_only.c:96:45: error: variable 'device_descriptor' must be const in order to be put into read-only section by means of '__attribute__((progmem))'
static uint8_t PROGMEM device_descriptor[] = {

Thanks for nothing avr-gcc updates. I understand what you’re getting at, but still, this used to compile, and put things in PROGMEM. Now it fails the compile. GCC bug in question is https://gcc.gnu.org/bugzilla/show_bug.cgi?id=44643, or at least, that’s where it came from. A nice case of, “everybody’s using this code, but it was never actually meant to work, so let’s make it fail for them all, like it was meant to anyway.”

Compilers are awesome.