Wlkie-tlkie

Wlkie-tlkie

A mobile (iOS) frontend for Rdio Scanner.

https://gitlab.com/bettse/wlkie-tlkie

Background

A few years back I picked up an RTL SDR (software defined radio) after seeing a number of cool talks using it at DefCon. I went through the usual projects, dumping pager data (POCSAG), flight data (ADS-B), and listening to traditional FM. Then, as is often the case, it went into a box.

More recently, a friend who is also interested in RTL SDR showed me his setup, using trunk-recorder, to record his local law enforcement. I was intrigued, but didn’t find law enforcement very compelling. Since I live downtown, near a number of businesses, I started looking into what business radio was around me. Skipping over the time I spent researching at the radio reference wiki and experimenting with gqrx, I found the frequencies used by the Target store near me, and more interesting, its Loss Prevention team.

I set up Rdio Scanner, a web based frontend, for listening to the captured audio. However, the next time I went shopping at Target, I became curious what might be happening as I shopped that I wasn’t aware of. I tried using Rdio Scanner on my iPhone’s browser, and although the design was responsive, it was problematic to leave the screen on with the browser in the foreground so “live” mode would work. I would frequently pull my phone out to find out it had gone to sleep, or had navigated away from the page, or switched apps.

This inspired me to try writing an app of my own, using Rdio Scanner’s backend, that would play the clips through an AirPod as I walked around the store.

It’s worth noting that although I hadn’t written any Swift recently, or any Swift 5.1, I have written toy iOS apps in the past, so I was already familiar with the language and some of the design patterns.

Phase 1: Graphql Subscriptions and audio

I’d already seen, from my work adding a download button to Rdio Scanner, that it used graphql between its backend and frontend and used subscriptions for live mode. I used this as my jumping off point and started by using adding Apollo’s iOS client library to my project and setting up the requisite code generation.

My work was unexpectedly slowed by Apollo’s iOS examples using ViewControllers, while the project I had created was using SwiftUI. I spent an evening reading blog posts on SwiftUI, and how it handled data flow between UI components and ‘backend’ pieces of code. I eventually managed a solution, although its a bit hacky, and CallsStore class is likely doing far more than it should.

Once I had events arriving, I began to work on playing the audio. I was very fortunate that the audio was m4a formatted, which is easily supported by AVAudioPlayer, so I only needed to massage it from a byte array into a Data object. I was able to successfully play the audio clips as soon as they arrived.

It was at this point that I moved from testing using the simulator to using a real device. I discovered very quickly that real device app backgrounding resulted in the websocket connection being dropped, which hadn’t happened in the simulator. Nothing I read online suggested there was any way around this with current APIs (there are some deprecated ones for maintaining a TCP connection).

Phase 2: Push notifications and voip

Credit for the idea to use push notification belongs to my friend, mentioned earlier, who also shared an interest in RTL SDR. I had to make changes both to the app code and Rdio Scanner in parallel, but for readability I’m going to cover each part independently.

Backend (Rdio Scanner)

To start with, I knew I needed to make modifications to Rdio Scanner to store push specific data (device token, what talkgroups were enabled), and to have it send to Apple. I forked the project to GitLab and created a branch with my work.

In order to send push events to Apple, the developer account needs to enable it for the app in the Apple Developer Portal, and create various keys and certificates. The steps necessary are well documented (with screenshots!) on various blog posts.

I added @parse/node-apn to Rdio Scanner to handle the communication to Apple, although since it supports both current and legacy APNS APIs (and the certificates/keys/ids required differ between the two methods), I did run into a little friction using it at first. On the other hand, Rdio Scanner’s use of Apollo Server’s PubSub made subscribing (internally) to the new call events very straight forward.

For device registration, there are two parts: a migration to create an apnsDevices table, and a new graphql mutation, apns-device, that accepted deviceToken and selection(JSON string describing the systems and talkgroups of interest). The mutation would create records, which would be referenced during a new call event to determine what events to dispatch to APNS. If a talkgroup in the selection was set to false, the corresponding record would be deleted, effectively deregistering the device (for that talkgroup).

App

Adding background audio playing from a push notification was not without its fair share of challenges. Primarily because app backgrounding, pausing, and termination is left up to the OS, which makes debugging particularly difficult. Although the app is fairly reliable now, there are still times when a new audio clip can’t be played for difficult to quantify reasons.

For performance, the push event only contains the call id, and a request for the rest of the call data is made. The call would then be added to a queue, which would show up in the UI, and be played if the app state was set to do so.

Since the app operates in the background, and I didn’t want to be disturbed late night or early morning, I added a large play and pause button to control if audio was played immediately, or queued for later. For cases when audio was boring, annoying, or had some technical issue, I also added a fast-forward that would jump to the next call in the on-screen queue.

I discovered that playing a silent audio clip when the app first loads in the foreground is necessary to initialize the audio session that will be used later. I also chose the “duck other” option so that the apps audio wouldn’t impact anything else playing.

For polish, I also added a settings page in the app. The user can define the Rdio Scanner graphql base URL, and it will populate a list of talkgroups with toggles, grouped by system.