tracker-miner-fs stuck at 100% alternative solution

There’s plenty of posts about problems with tracker-miner-fs being a CPU hog, even if it’s theoretically “idle”. “tracker status” will say that indexing is complete, and all miners are idle, but that’s clearly a lie, and you process stuck running from bootup.

Most “solutions” on the internet revolve around just turning off tracker completely, or at least the filesystem miner. I don’t want that, I actually like having search work. What I wanted was it to, you know, work :)

So, while experimenting with resetting it, I noticed that it was always getting stuck trying to recurse a particular directory tree. It’s 6GB, but that was unlikely to be problem. However, one directory there contained a lot of files. Lots like, 600,000 files or so, in one directory. This is enough to make ls hang, so, maybe it was enough to make tracker-miner-fs hang too? (Side note, if you want to count files in a directory like this, find . | wc -l will work, where ls doesn’t.)

I didn’t actually need those files anymore, so I just deleted the suspect directories entirely, and then restarted the trackers with tracker daemon --stop; tracker daemon --kill; tracker daemon --start (The kill was to kill the errant tracker-miner-fs process, the others all stopped gracefully)

And presto, tracker status starts reporting sane and plausible results, the tracker-miner-fs runs at 100% for a short period, reporting status correctly, then the tracker extractor takes over. And we have search again!

Nitsuka DX2E office phone teardown.

Full model on back of phone: DX2E-12BTXH. Manufactured October 1999. Internal PCB is clearly multi model, and also has the model MH 6330(1) and DX7C-12ANU5-A1
I have a few of these, they’re quite common, you can buy them on ebay for ~$35 or so, or similar models. I had the (very silly, but there was going to be some learning involved) idea of turning one of them into …. a phone. The plan is/was to hookup my own hardware to the handset, buttons and lcd screen, a little linux SoM or similar, and have it run some SIP softphone, running off 18650 batteries. Basically, make a phone…. into a phone. Not all terribly complicated, really.

So, time to open it up. Immediate difficulties are that the entire phone body is a giant PCB with button press pads. I’d loosely been planning on making a little USB HID device to capture all the buttons. (As we’re making a softphone, we don’t need DTMF or anything silly) Gonna have to rethink that now :) Anyway, what’s inside? Super easy to open, plain phillips head screws, no glue, no latches.

Three major ICs, the LCD connector, and the wonderful world of PSTN power supplies.

Let’s start at the top.

X2 is labelled Mitsubishi, M38003M6, and is by all accounts, a mask rom H8/300L series. Part numbers match up for Renesas (the current owner at least) I suspect this is handles all the programmable buttons and the LCD? X1 is a bit of a mystery, it’s labelled NEC D65825GF043. No idea what this is. Custom asic for phones? *shrugs*
Down to the bottom…

X3, Toshiba TC35324F. I found TC35300 and TC35310, both DTMF decoders. I suspect that this listens in on it’s own keypad? Seems a bit excessive though?

X7 (what happened to X4/X5/X6) Is a plain old MC34119. An audio amplifier. The fun things with old tech like this, it talks about being “low current” with only 2.7mA Iq, suitable for batteries :) The datasheet I found even excitedly mentions that it contains 45 active transistors! Whee.

Finally, back side. Just keypads and LEDs really.

Anyway. There it is. Not really sure what next. It wouldn’t be completely unreasonable to just make a whole new pcb that fits under all the button pads, but that’s a lot more than I was planning on. JLPCB and elecrow will make pcbs that big for ~$10 each, but it’s a tedious button pad design. Could do a smaller PCB, only do some of the buttons, just for less hassle, but, meh. Could try and probe up some buttons, lay a big rats nest of wire down. Possibly the easiest way would be to desolder one of the ICs and put something in it’s place, a bunch of selective trace cutting, see if all the buttons go there? Or maybe just look for a different phone altogether…

Cypress CY8CKIT-049-42XX – PSoC 4200 kit – first impressions

(This was a draft I write in 2015, thought I might as well publish it where I’d left it)
Cool packaging, just like in the PSoC5LP kit. Despite looking similar, with a breakoff section that plugs into USB, the “programmer” side actually isn’t. It’s Cypress’s USB Serial bridge chip, the CY7C65211, and you program by using the UART bootloader on the PSoC 4200 target. Totally functional, totally valid. Not nearly as useful as a full debugger for application development of course, but in effect, it’s a full eval board for the usb serial chip. You can get a config tool that turns it into USB-SPI, USB-I2C, and USB-GPIO dongles. Neat, but still, I didn’t get this to evaluate usb serial bridge chips. On the bright side, it does at least detect properly out of the box as a serial UART in linux. (Unlike the KitProg on the PSoC5LP kit)

GVFS mount points, or, how to completely ignore usability while building a usability feature.

Gnome happily mounts my phone’s MTP shares for me. I can even browse for them in “files” and drag and drop and everything. It’s pretty cool. And, magically, behind the scenes, it even exists in the filesystem so I can use rsync and cp and things.

But the mount point…. Oh boy. What on earth were they thinking. /run/user/1000/gvfs/mtp\:host\=%5Busb%3A002%2C010%5D/
Are you fucking kidding me? That’s an url encoded way of saying [usb:002,010] Why did it need to be url encoded? No idea. What about the content? Oh, that’s the USB bus, and device address. Spectacular. That means it depends on not only what physical port it’s plugged into, but what order it was plugged in. (And that unplugging and replugging will make a new path) Let’s not use things like iManufacturer, iProduct, iSerial, vid, pid, anything actually device specific no, not here.

I’m sure this was done to try and ensure that the mount point would be unique. Great. Mission accomplished. However, it could be made unique, and usable at the same time. And then this thing, that exists out of a usability feature, would actually be more usable.

Netbeans 8.x “Could not get children iterator” or “NULL VALUE RETURNED FOR CHILDREN”

If you’re getting messages like this when debugging and trying to look at variables, chances are you might be debugging some glib2 code. If you are, welcome to https://bugzilla.gnome.org/show_bug.cgi?id=749092

It’s easy enough to manually patch the file, and presto, working debug again. It’s been fixed in GDB, I was seeing this in Fedora 22, which is not _entirely_ up to date these days, but you tend to expect that plain old host debugging should be pretty stable. blog post because the netbeans error string doesn’t help you find that gdb bug. You need to run gdb on the command line to find the real error: Python Exception iter() returned non-iterator of type '_iterator':

More arm android box conversion adventures

TL;DR: Still failing. Gave up again.

Back when I bought a RK3066 based device, I had seen rockchip developers start contributing directly upstream.  However, the RK3066 just missed the boat, and while I poked and prodded a bit, it was all just a bit too difficult and a bit too complicated a bit too much not much fun.  Still, I followed what was going on.  The plan was to use the swell of consumer electronics to get a cheap android tv box that I could run linux on, and use as a small home server.  “Just use a raspi” you say.  Well, no.  Raspi3 is at least decently powerful, but running everything off a sdcard, and not having a case is really not what I was looking for.  Little tiny cute black boxes, with a remote control and a power supply and ethernet and usb shipped for $30?  That’s what I want to reuse.  Rockchip continued to contribute upstream, so I bought a MXQ 4K (RK3229).  New hotness I thought, but the cheap end.  I still believed that rockchip was doing the best job of upstream support.  Note: My copy of this device has a much smaller heatsink, Spectek flash, not Sandisk, and some generic no name ram chips. (See pics)

Top side, smaller heatsink, no name ram.

Top side, smaller heatsink, no name ram.

Maybe they are.  And I’ve not worked on amlogic or sunxi chips, but goddamn, the work environment is still just as sucky.  A garbage world of “custom roms” and complicated documentation detailing multiple ways of doing things with rarely any explanation of why.  I’ve still no idea, but some things seem a little clearer than last time… maybe?

If you just want to run linux, in _theory_ you can just use the existing android kernel shipped stock.  It has support for all your hardware, you just need to replace the root file system (RFS) and amend the kernel command line to say where your rootfs is.  Except, because some of the drivers are actually separate binary proprietary modules, you end up needing a fucking initrd/initramfs timewaste so it can have such core details like a functional fucking nand driver.  Why the fuck rockchip is keeping _this_ bit proprietary is beyond me.  So that gets non-trivial quickly.  You can in theory extract the modules from your android system image.  If you have a stock firmware, (“update.img”) you can use rkunpack from the rkflashtool repository.  you’ll probably have to unpack more than once.  Documentation on what’s what is less than forthcoming.  The “system.img” wil be a raw ext4 image (in theory) and you can mount it locally to explore with “sudo mount -o loop blah.img somemountpoint”

anyway, while some of this is just for my own notes, not really for anyone else, one thing has stuck out.  First, the default serial console is at 1500000 baud.  Thanks for nothing rockchip.  On a cp2102 serial dongle, you’ll need to use the silabs baud rate aliasing software to get to this baud rate.  Thanks for nothing silabs. and yeah, again, thanks for nothing rockchip.  The serial console is in the picture below.

Bottom side, spectek flash, serial console details

Bottom side, spectek flash, serial console details

Oh yeah, the “thing that has stuck out”  Despite being “unbrickable” because they have a maskrom bootloader, and normally just going into recovery mode,  I’ve had great difficulty.  After some bad flashing,  (bad why? no idea, rkflashtool or upgrade_tool both successfully report the flashing complete, then I reboot, and get…. bad things) the serial console will hang on a line like…

In
300MHz
DDR3
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
mach:2
OUT
Boot1 Release Time: 2016-03-15, version: 2.31
ChipType = c 275
No.1 FLASH ID:2c 64 64 56 a5 4
ECC:60
ReadRetry pageadd=15d800  ecc=3c err=ffffffff
Read pageadd=15d800  ecc=3c err=ffffffff
spare: 0x0:b8679151 b8679151 b8679151 b8679151 
ReadRetry pageadd=15d801  ecc=3c err=ffffffff
Read pageadd=15d801  ecc=3c err=ffffffff
spare: 0x0:527ded0c 527ded0c 527ded0c 527ded0c 
SdmmcInit=0 20
StorageInit ok = 134644
SecureMode : SBOOT_MODE_NS
powerOn 812121
Usb re Boot. 6812118

At this point, rkflashtool v will “fail” like so… (notice mangled text)

$ rkflashtool v
rkflashtool: info: rkflashtool v5.2
rkflashtool: info: Detected RK322X...
rkflashtool: info: interface claimed
rkflashtool: info: chip version: 322A-��..��-��

upgrade_tool will still say the device is connected, in “Loader” mode, and TestDevice, ReadFlashID, ReadFlashInfo and ReadChipInfo all still work. But attempting to download any image will fail saying parameter is invalid. rkflashtool p agrees.

 rkflashtool p
rkflashtool: info: rkflashtool v5.2
rkflashtool: info: Detected RK322X...
rkflashtool: info: interface claimed
rkflashtool: info: reading parameters at offset 0x00000000
rkflashtool: info: size:  0x0000ffff
rkflashtool: fatal: Bad parameter length!

One time this happened, I got out of it by having the USB-A-A cable disconnected (oh yeah, A-A for OTG? thanks for fucking nothing rockchip, hooray port overloads and reverse powering disasters) holding the recovery button down (with a toothpick down the SPDIF hole. (button down the AV hole _should_ be different but seems to behave the same for me)) and then plugging in the main power supply. Perhaps it had been a problem with powering the device purely via the A-A cable as I’d normally been doing?

Who knows. It’s currently in this state again, the unbrickable “bricked” Even shorting pins 7-8 on the NAND chip itself and powering up isn’t helping. (The “standard” rockchip method of getting back to maskrom mode) I get a different log, but it’s still non-responsive, even after inserting the A-A cable. (Still in loader, but not really working)

DDR Version V1.04 20160504
In
300MHz
DDR3
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
mach:2
OUT
Boot1 Release Time: 2016-03-15, version: 2.31
ChipType = c 276
No.1 FLASH ID:2c 64 64 56 a5 4
FlashLoadPhyInfo fail 2bc!!
sync para 32
sync para 32
ECC:60
ReadRetry pageadd=15d800  ecc=3c err=ffffffff
Read pageadd=15d800  ecc=3c err=ffffffff
spare: 0x0:b8679151 b8679151 b8679151 b8679151 
ReadRetry pageadd=15d801  ecc=3c err=ffffffff
Read pageadd=15d801  ecc=3c err=ffffffff
spare: 0x0:527ded0c 527ded0c 527ded0c 527ded0c 
SdmmcInit=0 20
StorageInit ok = 93093
SecureMode : SBOOT_MODE_NS
powerOn 770567
Usb re Boot. 6770565

So. No results, no future, but at least a weird log for the internet to archive if it helps anyone sometime.

More excessive customs and handling fees

Bought a somewhat fancy keyboard. (more on that later) Bought some custom made icelandic keys to go on it. Paid regular VAT on both. Bought some plastic inserts to help mount the new custom keys on the new keyboard. Got charged 7.5% customs. What? What is this duty for? It’s certainly not protecting any local industry. And it’s not to cover recycling of the parts or their packaging, there’s a separate charge for that. Still, only 7.5% still “not much” in the scheme of things. And then of course, the ever delightful hammer of icelandic online shopping, the flat rate 550kr “handling charge” of Iceland post. Effective net percentage on top for “fees”? 72.4% That’s unreasonable. I’m not complaining about VAT, (even though, because too many companies in europe don’t know how to take VAT off to send to Iceland, outside the EU, I’d already paid VAT in another country for this order) I’m complaining about the excessive handling charges with no alternative path. Good job Icelandic Customs, they legally allow “any” charge by the handler. Of course, I can’t control what shipping partner my parts are eventually delivered by, so I have no way of avoiding this. Iceland Post offers no way of preprocessing or providing more information to reduce or eliminate the charge either. It’s simply an impediment on trade, with no gains for anyone other than the postal service.20160630-customs-keyboard-parts-real

oh udev. oh yes

I try and do the “right thing” when it comes to the future and how linux distros see their roles and the way things should be.  I think systemd has some generally good ideas, though I think upstart has far far better documentation.  Anyway, given that linux distros still “protect” users from plugging in usb devices, except for all those ones that they _don’t_ protect you from, if you’re working with embedded electronics, sooner or later you need to “fix” permissions for some device you have.  No matter, the canonical advice for a “long time” has been something like adding a file in /etc/udev/rules.d/doesnt_really_matter.rules with some content like…


ATTRS{idVendor}=="aaaa", ATTRS{idProduct}=="bbbb", GROUP="plugdev"

And you added yourself to the plugdev group, and everything was pretty much ok. Now, it’s the “future” and we can’t use anything so silly and outdated as groups to assign permissions. Heaven help us, what would become of us all if people just used groups?! Now we need to use “tags” and a magical “uaccess” incantation. Except for all those devices that we do use groups for, like usb modems, usb serial ports, usb printers, wireshark capture permissions, virtualization, colour correction devices, basically everything except “unknown” devices.

But ok, good and well, we’ll learn the new magic incantation for udev….

ATTRS{idVendor}=="aaaa", ATTRS{idProduct}=="bbbb", TAGS+="uaccess"

And…. it doesn’t work. Why? Good fucking luck finding out. If you’re lucky, someone might suggest that the name of your file isn’t right. Why should that matter? Welllll. because it does. See https://lists.freedesktop.org/archives/systemd-devel/2015-August/033946.html and https://bugzilla.redhat.com/show_bug.cgi?id=1074122#c0 for some hints and clues.

TL;DR: pretty much must be named 70-doesnt-matter.rules. At least until system decides that it should be a different number without documenting it anywhere.

Disable Alt-F1 shortcut on gnome

Current versions of gnome’s “Keyboard shortcuts” control panel applet don’t have an entry for “System->Show the activities overview“, so you can’t edit it.  Instead, it’s hard coded to the winkey, and  Alt-F1.  No. Bad gnome.  Don’t do that. I want to use that, (in my case, in IntelliJ)

What to do?

gsettings to the rescue.

$ gsettings get org.gnome.desktop.wm.keybindings panel-main-menu
['<Super>s', '<Alt>F1']
$ gsettings set org.gnome.desktop.wm.keybindings panel-main-menu "['<Super>s']"

Presto.

Using NetBeans for STM32 development with OpenOCD

This post supersedes http://false.ekta.is/2012/05/using-netbeans-for-stm32-development-with-stlink-texane/ it has been updated for 2016 and current best software tools.  Again, this is only focussed on a linux desktop environment.

Required pieces haven’t changed much, you still need:

  1. A tool chain.
  2. GDB Server middleware (OR just a tool flashing if you want to live in the dark ages)
  3. A “sexy” IDE (If you disagree on wanting an IDE, you’re reading the wrong post)

Getting a toolchain

New good advice

Advice here hasn’t changed.  The best toolchain is still gcc-arm-embedded  They steadily roll out updates, it’s the blessed upstream of all ARM Cortex GCC work, and it has proper functional multilib support and proper functional documentation and bug reporting.  It even has proper multi platform binaries for windows and macs.  Some distros are packaging “arm-none-eabi-XXXXX” packages, but they’re often old, repackages of old, poorly packaged or otherwise broken.  As of November 2015 for instance, ArchLinux was packaging a gcc 5.2 binary explicitly for arm-cortex, that did not support the -mmcu=cortex-m7 option added in gcc 5.x series.  Just say no.

I like untarring the binaries to ~/tools and then symlinking to ~/.local/bin, it avoids having to relogin or start new terminals like editing .bashrc and .profile does.
~/.local/bin$ ln -s ~/tools/gcc-arm-none-eabi-5_2-2015q4/bin/arm-none-eabi-* .

Old bad advice

The internet is (now) full of old articles recommending things like “summon-arm-toolchain” (Deprecated by the author even) “code sourcery (lite)” (CodeSourcery was bought by Mentor, and this has been slowly killed off.  Years ago, this was a good choice, but all the work they did has long since been usptreamed)  You can even find advice saying you need to compile your own.  Pay no attention to any of this.  It’s well out of date.

GDB Server middleware

New good advice

Get OpenOCD. Make sure it’s version 0.9 or better. 0.8 and 0.7 will work, but 0.9 is a _good_ release for Cortex-M and STM32 parts. If your distro provides this packaged, just use it. Fedora 22 has OpenOCD 0.8, Fedora 23 has OpenOCD 0.9. Otherwise, build it from source

Old bad advice

Don’t use texane/stlink.  Just don’t.  It’s poorly maintained, regularly breaks things when new targets are introduced and not nearly as flexible as OpenOCD.  It did move a lot faster than OpenOCD in the early days, and if you want a simpler code base to go and hack to pieces for this, go knock yourself out, but don’t ask for help when it breaks.

Netbeans

No major changes here, just some updates and dropping out old warnings.  You should still setup your toolchain in netbeans first, it makes the autodetection for code completion much more reliable.  I’ve updated and created new screenshots for Netbeans 8.1 the latest current release.

First, go to Tools->Options->C/C++->Build Tools and Add a new Toolchain…

Adding a new toolchain to netbeans

Adding a new toolchain to netbeans

Put in the “Base directory” of where you extracted the toolchain.  In theory netbeans uses the base directory and the “family” to autodetect here, but it doesn’t seem to understand cross tools very well.

Base path for new toolchain and name

Base path for new toolchain and name

Which means you’ll have to fill in the names of the tools yourself, as shown below.  Click on “Versions” afterwards to make sure it all works.

Adding explicit paths to netbeans toolchains

Adding explicit paths to netbeans toolchains

“Versions” should show you the right thing already, as shown

Versions from our new toolchain (and an error from gdb we must fix)

Versions from our new toolchain (and an error from gdb we must fix)

If you’re getting the error about ncurses from gdb, this because newer gdb builds include the curses “tui” interface to gdb in the standard build.  (Yay! this is a good thing!)  However, as the g-a-e toolchains are all provided as 32bit, you may be missing the 32bit ncurses lib on your system.  On Fedora, this is provided in the ncurses-libs.i686 package.

Ok, now time to build something.  This is your problem, I’m going to use one of the libopencm3 test programs right now, specifically, https://github.com/libopencm3/libopencm3/tree/master/tests/gadget-zero (the stm32l1 version)

Programming your device

Netbeans doesn’t really have any great way of doing this that I know of.  You can use build configurations to have “run” run something else, which works, but it’s a little fiddly.  I should spend more time on that though.  (See later for a way of doing it iteratively via the debugger console in netbeans)

In the meantime, if you just want to straight out program your device:

$ openocd -f board/stm32ldiscovery.cfg -c "program usb-gadget0-stm32l1-generic.elf verify reset exit"

No need for bin files or anything, there’s a time and a place for those, and if you don’t know and can explain why you need bin files, then elf files are just better in every way.

Debugging your part

GDB is always going to be a big part of this, but, assuming you’ve got it flashed, either by programming as above, then you can debug in netbeans directly.  First, make sure OpenOCD is running again, and just leave it running.


$ openocd -f board/stm32ldiscovery.cfg

First, install the gdbserver plugin, then choose Debug->Attach Debugger from the menu.

Attach to a gdbserver

Attach to a gdbserver

  • Make sure that you have “gdbserver” as the debugger type.  (Plugin must be installed)
  • Make sure that you have “ext :3333” for the target.  By default it will show “remote host:port” but we want (need) to use “extended-remote” to be able to restart the process. (See the GDB manual for more details)
  • Make sure that you have the right project selected.  There’s a bug in the gdbserver plugin that always forgets this.

At this point, “nothing” will happen.  If you look at the console where OpenOCD is running, you’ll see that a connection was received, but that’s it.
Press the “Pause” button, and you will stop execution wherever the device happened to be, and netbeans will jump to the line of code. In this example, it’s blocked trying to turn on HSE, as there’s no HSE fitted on this board:

Source debugging in netbeans via OpenOCD

Source debugging in netbeans via OpenOCD

If you now set a breakpoint on “main” or anywhere early and press the “Restart” icon in the top, OpenOCD will restart your process from the top and stop at the first breakpoint. Yay!  If restart fails, make sure you used “extended-remote” for the target!

Bonus

If you click the “Debugger console” window, you can actually flash your code here too.  Leave the debugger running!  (No need to stop the debugger to rebuild) Make a change to your code, rebuild it, and then, on the “Debugger console” just enter “load” and press enter.  You’ll see the OpenOCD output as it reflashes your device.  As soon as that’s done, just hit the “Restart” icon in netbeans and start debugging your new code!