iPad Thumbscanner

Well, my iPad 2 arrived yesterday. Loving it.

But that’s not what this post is about.

At work, one of our products is a thumbscanner system. I was discussing with our CEO the other day how the best way to have this in a unit was, and ideally an iPad or other tablet would be best.

The issue with the iPad is that you cannot just connect any hardware to it.

But then I remembered how some people had connected all types of USB hardware, including in one case a USB-ADB adaptor and an old, old keyboard. All you need is one of the iPad Camera Connection Kit thingos.

Guess I’ll have to get one of those, and see what I can manage to get it to do…

…The trigger for thinking about this: Oscium’s iMSO-104 turns your iPad into a mixed signal oscilloscope

Sonoff Basic unable to connect to Wifi on AC power

I have a few of the Sonoff Basic devices. I’d flashed one of them with a custom firmware before, but then was using the stock firmware, and having them operate in LAN mode.

However, that’s less than awesome: for one, it means I need to have (and maintain) a custom service running somewhere on my network, and because of the way you need to provision them, it’s hard to move this to a different machine.

With ESPhome, I’ve started re-flashing my old ESP8266 devices, so I had a go at some of the Sonoff. Neither of the ones I had installed (with the stock firwmare) had headers soldered onto the UART pins, but it turned out that one of my USB-UART devices had the pins in the correct order that I was able to (after disconnecting it from the mains, of course), hold the button down, press the USB-UART device into position, and then connect the USB port. After a couple of seconds, I released the button, and, eventually (after a couple of tries), have the device in “flash” mode.

Of course, I only discovered this after totally disconnecting one, soldering on some header pins and then connecting the device using hookup wires.

Then I uploaded the custom firmware. At which point I was able to re-upload firmware using the OTA mode, which means I no longer had to juggle the serial connector, USB and GPIO0 button.

So, I then flashed another one, this time without having to solder, and had both of them configured as devices on my IoT network.

But, there was a problem. They would only connect to the Wifi when they were connected to the serial connector. When I disconnected the serial port, and reconnected them to the mains supply, they would operate correctly, but would not connect to Wifi. Even though one of them was literally less than a metre from the router.

I did a bit of research, and there apparently are a batch that are like this. Tasmota firmware has issues that mention this on GitHub, but then in those issues they are marked as “resolved”. Unfortunately there was not really a good resolution - more than likely it was just some type of stale-bot just closing issues that had not been updated in a certain time frame. At least one of these suggested that a newer firmware worked, but that was no good for me.

A couple of these issues identified that there was a missing capacitor, but one of them mentioned that they had soldered together the 3v3 supply, and the RX and TX pins.

So, I ran some experiments. Connecting the 3v3 pin to either of those other pins had no effect, but connecting the RX and TX together suddenly allowed my device with the header pins to connect!

Luckily, these pins are adjacent, so I found a jumper (I remember when these were on just about every hard drive, but I couldn’t find one today with a single pin), and hooked it up.

All good!

Unfortunately, when putting it back together I didn’t align the top cover with the button correctly, so I managed to break that. Which just means I can’t control the switch directly, only remotely. That’s pretty annoying (it’s one of the reasons I went with Sonoff over some other solutions), but at least it works on HomeKit now.

Hacking the DETA rewireable plug

There are a bunch of inexpensive IoT devices coming onto the market. One of these ranges are the GridConnect devices available from Bunnings. I’ve been eyeing these off, because they include some fairly reasonable looking switches and plugs, and, most importantly, are legal to have installed in your home in Australia.

I’m not okay with sticking a bunch of devices into my home network that communicate with the outside world. I like to use Apple’s HomeKit, which means the communication is funnelled through their systems, and everything I have configured as a HomeKit accessory either has no cloud component, and is where possible isolated onto a seperate wireless network.

In fact, most of my devices are home built, and have limited accesibility: they are permitted to connect to an MQTT server and nothing else.

Most of the commercially available items in Australia fail this test: they all have their own little cloud service. I dislike this for a couple of reasons: the first is security and privacy. I don’t trust that these providers will protect my data.

The second is a little more subtle: if an IoT provider goes out of business (or decides to end-of-life some products), then you are no longer able to access them in the way you might have liked.

So, I’ve been creating my own devices, or re-purposing commercial devices where possible. I grabbed a couple of the Mirabella globes, and was able to flash them with an ESPHome firmware, that I believe hardens the device somewhat. Having controllable light globes is neat, but realistically, they are of limited use. Having to use an App to turn on and off your lights is unacceptable, but also so is having an always listening microphone. And, again, what happens when you want to turn your lights on and your internet is down?

So, I’m more interested in smart switches.

I was not able to find much information about the Deta line of products that support GridConnect, so I went and bought the cheapest one, that I thought I was going to be realistically able to reverse engineer.

Opening up the “user accessible” region, we see the three screws for attaching the cable, and some lovely 2.0mm triangular screws.

IMG 2119

After spending more money on a set of screwdriver bits that had this size and shape, we are able to see the top of the PCB.

IMG 2120

Note that there are two relays: this is safer than things like the Sonoff Basic, that only switch the active wire, rather than both.

The underneath shows clearly the separation between the different AC lines.

IMG 2121

Finally, we can see the brains behind this switch:

IMG 2122

Note the code on the microcontroller: TYWE2S. Plugging that into a search engine yielded some interesting results.

It turns out that not only are these using an ESP8285 (which is just an ESP8266 with onboard flash, and in a smaller package), but they are running the Tuya firmware. So, it wasn’t even necessary to have opened up the casing.

Since I had a Raspberry Pi Zero W configured to run tuya-convert, I built up a simple firmware that would enable me to run OTA updates after flashing, and settled down to working out which GPIO pins are attached to which parts: the button, LED and relay.

(Oh, and this was interesting: both the active and neutral lines are toggled by relays, as opposed to something like the Sonoff, that only toggles the active line).

At some point, possibly because I was tired, I flashed a firmware that accessed GPIO pins 6 and 7. This locks the device up, and prevents booting. So, I then soldered some wires onto the device (after opening it back up again: so it turned out I did need that fancy screwdriver bit) to get access to the UART.

Interestingly, the ESPHome firmware detects that it’s been unable to boot, and boots into a safe mode. So, again, in hindsight, I still didn’t need to open it up.

Through trial and error, I was able to determine which GPIO was the button. That’s the easiest (and was helpful for me to be able to determine the LED and relay).

Surprisingly, it was GPIO1. Those familiar with the ESP8266 may know this is normally the TX pin for the UART. Which explains why, when I had the serial port connected, I was seeing a bunch of weird things when I pushed the button.

Once I had that, it was a matter of trying each GPIO in turn to see which was the relay and which was the LED. I did them in pairs, and happened to choose GPIO13 and GPIO14 in the same test, so for a while I thought maybe the relay and the LED were hardwired together.

Anyway, I now have a working firmware for this device:

substitutions:
  device_name: deta_plug

wifi:
  # Hah!

esphome:
  name: $device_name
  platform: ESP8266
  board: esp01_1m

binary_sensor:
  - platform: status
    name: "Status"

  - platform: gpio
    pin:
      number: GPIO1
      inverted: true
      mode: INPUT_PULLUP
    name: GPIO1
    on_press:
      - switch.toggle: relay

switch:
  - platform: gpio
    id: led
    pin:
      number: GPIO13
      inverted: true
  - platform: gpio
    id: relay
    pin: GPIO14
    on_turn_on:
      - switch.turn_on: led
      - mqtt.publish:
          topic: HomeKit/${device_name}/Switch/On
          retain: ON
          payload: 1
    on_turn_off:
      - switch.turn_off: led
      - mqtt.publish:
         topic: HomeKit/${device_name}/Switch/On
         retain: ON
         payload: 0


sensor:
  - platform: wifi_signal
    name: "WiFi signal sensor"
    update_interval: 60s

ota:

logger:

mqtt:
  broker: "mqtt.lan"
  discovery: false
  topic_prefix: esphome/${device_name}
  on_message:
    - topic: HomeKit/${device_name}/Switch/On
      payload: "1"
      then:
        - switch.turn_on:
            id: relay
    - topic: HomeKit/${device_name}/Switch/On
      payload: "0"
      then:
        - switch.turn_off:
            id: relay

I’m using my MQTT ⟺ HomeKit bridge, since that works great, but you could easily change the MQTT topics, or do whatever else you want to do.

Update: turns out, I just needed to search for the code: this page has everything I would have needed.

Review: miglia TVmini2/equinox The Tube

I lashed out yesterday, and bought something that I’ve been wanting for a while.

A USB TV Tuner stick.

I went for the miglia TVmini2, as it was (I thought) fairly inexpensive. I’m not sure I made the right decision right now.

For starters, it doesn’t actually come with the software. Sure, it’s got a CD with some demo software of other stuff, but the actual software (which is actually written by another company, called equinox), The Tube, needs to be downloaded from the internet before installation.

Which wouldn’t be too bad, but it’s a 104Mb download. Which meant I couldn’t (a) start playing with it as soon as I bought it, since I was in town, and not near a free wireless point, and (b) start playing with it as soon as I got home, since it took about 30 minutes to grab.

It gets worse, though. Even though the software really requires a dongle (the USB tuner is really just a dongle, after all), the manufacturers insist that you register the software, using the included key string. And it only works on one machine without de-registration, apparently. Which kind of sucks, because I’d like to have it moved between more than one machine.

The hardware itself seems okay. I haven’t been able to make it work with any other software yet. I’ve tried a couple of downloads, but they wouldn’t recognise it.

System Profiler reports the device as looking like:

TVMini2:

Version: 1.00

Bus Power (mA): 500

Speed: Up to 480 Mb/sec

Manufacturer: Miglia

Product ID: 0x0069

Serial Number: 0000000001

Vendor ID: 0x18f3

The software, on the other hand, seems to have some good features, but quite a few flaws.

The manufacturers have decided to make the application scriptable. This is a very good thing, as it will allow me to, for instance, set up Salling Clicker to be able to control it. Which will be very good. I’d also be able to make a rudimentary controller for network or local remote control. Which I may do: more on that later.

Another good feature is the simplicity of the software. The basic window is shown below.

You basically have a video section, with a controller on the right. This appears to be fixed. Although you can hide the main video area, they resize together. Which is a bad thing, if you like to resize the video to a small size while you write a review of something, for instance. Like I’d like to do now. It means that the controller becomes fairly unusable fairly quickly:

This is as small as the remote area gets. Any further resizing just shrinks the video display in that window. You can hide the Details area (this is shown above), but you can’t hide the record area.

You can hide the remote, too. This makes shrinking the video a bit more usable:

I anticipate being able to build another controller that floats, and can be used in conjunction with the view shown above. More below on some issues that might appear with this, when I discuss the AppleScript interface in a bit of detail.

The controller itself is mostly okay. The main/default view has a list of channels. This is customisable. You can scan for channels, and then delete the ones you don’t watch, and rename the ones you do. Scanning takes a long time, and is probably worth re-doing every now and then, just in case a new station has been added.

The bottom area of the controller is the recording pane. You can pause live TV, record the current stream, and if you have been watching a channel for a little while, skip back through the already viewed stuff. I did a quick test where I watched for a while, then went back to try to record a section from the past. The program developed a spinning beach ball, and I had to Force Quit. I’ll continue to test it again later, as this is a nice feature.

There’s also a button link to their other software, MediaCentral, which totes itself as a total replacement for FrontRow. And, I’ve had a bit of a look at it. It looks pretty good, actually. May be worth investing in, and using this on the Mac Mini when I finally get it, and set it up as my Media Centre. (Typing Central reminds me to just let all of the yanks out there know the correct way to spell centre. Yank is a slang term Australians use to refer to Americans).

If you record live TV, it stores it in the recordings pane.

You can then export from here, in order to be able to view on another device, or in another program. It exports raw DV fairly quickly (almost real time, I think, perhaps even faster), but as expected exporting to m4v takes a bit longer. It appears that these are the only formats it will export: iPod, iPhone and AppleTV are all m4v, and iMovie is DV. I’m still going to record some stuff and see what the quality is like. Gotta wait for some decent content first.

Which brings me nicely to the next topic. Electronic Program Guide and Scheduled Recording. The interface is, again, fairly simple. All programs appear in the one list, there doesn’t appear to be a method of filtering so sort by channels. You can search or filter by title and description. To schedule a recording, simply click the black dot next to the program time.

Clicking on a title will show the details about that program. This is obtained from the digital TV data stream, although it is possible to add in an external TV guide, using the open XMLTV standard. Unfortunately, there doesn’t seem to be a way to subscribe to an XML feed. EPG via RSS, that would be awesome.

If you click on the “Show Recordings” button, it will filter to only display the programs you have scheduled to record. You can then de-select them by re-clicking the red dot. Fairly easy to understand. It also seems to have some smarts about overlapping programs, and will display a different coloured dot when they clash.

Okay, feature list is pretty much done. What doesn’t it do, or what does it do badly?

AppleScript support.

You can do quite a few things with AppleScript, such as pause, play, start recording and so on. It appears that you cannot get decent data from the EPG, for instance. So I can’t build a fully replacement EPG in another application, unless I access the data directly from the file. This file is stored as an SQLite database. So I should be able to grep data out of this using the relevant tools. This seems to be an oversight, since you can schedule recordings, and access the EPG data, but I can’t figure out how to get meaningful data from the EPG. This would be a necessity for a proper Salling Clicker interface, that allowed me to just run the program in full-screen mode and only use the phone for the remote, doing all of the programming with that instead.

You can also export from AppleScript. Not sure if you can choose to export DV, which I would probably want to do. So I can post-process the data and remove advertisements, for instance. But I cannot find a way to choose a channel by channel name. Which would be useful for a remote program, either on phone or computer.

Possibly the biggest annoyance: in every instance I have seen, this application is called The Tube. But to access it in AppleScript, you need to use TheTube. Why do this? You can have spaces in application names in AppleScript and it works fine.

Data file formats.

The channel data is stored as an XML/plist file. Which is a good thing. You can hand-tweak this file, which is much faster than, and less error-prone than deleting them inside the software. Especially since right-clicking to delete doesn’t actually select the channel, so the previously selected channel gets deleted instead.

Scheduled programs are also plist files. You could create these files with another piece of software, and just rely on The Tube to record them. I haven’t tried creating a file and seeing if it automatically records, or whether you need to start/re-start The Tube to get it to notice. Still on the TODO list.

The video data is stored in a package. There are a couple of TIFFs, for preview purposes, but these aren’t always the correct aspect ration. In fact, since most of my Digital TV seems to be broadcast in 16:9, with black bars on the sides, this is always the wrong aspect ratio as these TIFFs are all 4:3.

Bonus points if you can name the song and artist this preview is from!

Inside the package is a plist file with data about the recording. But the actual data is stored in two files, Media.tvi and Media.tvm. I’m still trying to find out what file format these are - VLC doesn’t seem to recognise them, although I suspect they are a standard file format, just hidden. I’d like to be able to get this data without necessarily having to use The Tube. According to one website, this data is just the raw MPEG stream. Changing the file extension to .mpg or .mpeg doesn’t allow for playback Might need more research.

New MacBook Pro

As I sit here, home from work with a back issue (funny story about that: remember how teachers always told students off about pulling out a chair from behind someone as they are about to sit down is dangerous? I can confirm that indeed it is. One of my students did it to me on Monday, and my back, including bits which are nowhere near the impact points, is totally screwed. And I’ve got State selections this weekend… but I digress), at least I have a light shining at the end of the tunnel. MacBook Pro, 15“; 2.16GHz/2GB/200GB On order now. My first laptop computer. Not a moment too soon. OS X on a Dell is better than Windows, but there are still some issues (random application crashes, inability to run VMWare Fusion and CrossOver Office properly, Missing Sync doesn’t run at all).

The Key The Secret (Glamourously Developed Edit)Urban Cookie CollectiveThe Key : The Secret ★★

DoubleCommand

The first hassle I had with the MacBook Pro is to do with the keyboard. I spend a lot of time in the Terminal, and use Control sequences (^C to break, ^A to jump to start of line, etc). However, the fn key is where I expect the ctrl key to be. Luckily, DoubleCommand can change this. You can also map something useful, like Forward Delete to the extra enter key (which I hadn’t even noticed existed!). Speaking of forward delete, you can also map Shift-Delete to this keystroke too.

Installing Fedora 7

With the purchase of my MacBook Pro, now called arne, I’ve decided to move all of my file serving from the NSLU2 to the Dell. Rather than run Windows on this box, I decided to install Fedora. OS X just wasn’t that stable on the old thing, unsurprisingly, since it was a severely hacked version. So, I bought a new hard drive (so I can keep the OS X and Windows installations on their original disks in case I need to go back), and installed it. I made my first mistake in installing it as the Secondary Slave, since that was the only available channel. I installed Fedora, and it wouldn’t boot. So I tried again. By this time, I had figured out that it was something to do with the GRUB bootloader not being installed, so I moved the hard drives around, but still no joy. I spent ages trying to get it to work, including using a (very) old Debian CD I had lying around (which, by the way, I’d never actually installed on anything!). No joy here, as this only recognised the first 8 GB of the 320GB hard disk! Finally, I almost gave up. I started to install Windows, just to get the disk reformatted (I’d accidentally hit the power at some stage, and nothing else was recognising it). So, I let it get through the formatting stage, and then reinstalled Fedora. And finally, it worked. Now, I just need to get everything on there I need: Samba, AppleShare Filing Protocol, mDNS (Bonjour), and Print/Scan/Fax sharing. It already comes with some cool stuff, like virtualisation (I’m going to compare this to Parallels and VMWare, and choose the best), and a torent server/client. Oh, and the name of that machine is now poul. I’ve gone down the road of four letter machine names, which are also the first names of designers. Finally, this is my first offline blog post, on the train on the way home. It’s not like I could do any real work on the short trip…

xkcd Success

Ah, I’ve been there. Mine was dual-booting OS X onto an unsupported Dell PC. I had a second Hard Disk that I was putting it onto, yet somehow I managed to screw up the Windows partition as well. Luckily I didn’t destroy the laptop. And I never got near the sharks…

And, yet another plug for xkcd.com - if you haven’t subscribed to his RSS feed, then you really should.

Transfer Errors? Check the cable!

Just spent about twenty minutes trying to transfer a file from a computer to my iPod. Errors, input/output from the Finder. Lockups when transferring via the Terminal. Occasional disconnections of the device. Thought it might be to do with the fact it was a 2.7G disk image. Tried splitting it into smaller chunks. Unplugged it, and plugged it in without the extension USB cable. Transfer of file was successful first time. Either the cable, or that USB port was damaged!

MacBook Pro Kernel Panic WPA Enterprise

Well, I love my MacBook Pro. Except for one little thing. If I connect to a WPA Enterprise network, and I am running on battery power, then I get a Kernel Panic. Every time, within a couple of minutes. Plugging into power means no panic, using WPA personal (at home) also means no panic. Although, apparently the lack of panic at home may be more to do with the fact I’m running an Airport Express router. I’ll switch over to my other router (b-only, still WPA) and see if it panics then. Apparently, Apple is aware of the problem.