Zeo Sleep Monitor - Android Data Export

Hey, thanks for the Tips!

I did update my Wireshark and finally got the previous version of your LUA dissector to run… but your warnings are right, while miraculous… it was a bear to get to work right.

I will be most happy to switch to the new version.

I did go through your script… but even with a folding editor it was dense for me… both familar and not so familar… LUA looks a lot like C.

I’ll have to find this Legacy mode your speaking of… nice tip.

Good and bad to hear about the Mobile vs Bedside… Good since I can’t verify it either way… Bad since we don’t know yet how to run those test modes. Not so bad for people who just want the post processed information.

I found the legacy mode… but it made [Decode As…] really unusable for me… then it crashed… so I just went back to using the new LUA dissector with the new Wireshark interface… its harder to read… but at least I know how to make it [Decode As…] and there is still much I don’t know about that feature… for instance I have to set the BT RFCOMM value to 4 or it won’t process the packets into Headband protocol messages… the default is “0” which never works.

The RFCOMM DCLI=4 thing was based upon my traffic … the 4 means DLCI channel 4, and obviously on your captures that channel is 0. The Wireshark LUA interface requires that I set some value; I didn’t know if that value changed or not (and now I know it does).

I find I can set multiple values for that within the dissector definition. So I’ve updated the LUA. Give it a try.
zeohbbt.lua (15.4 KB)

That is really much improved Mike!

I didn’t have to [Decode As…] anymore at all… it just parsed the packets right away.

I think I’ll spend some more time learning Wireshark, I saw someone with a lot more experience parse a bunch of packets into “streams” and “follow” conversations.

I seem to recall a version of Wireshark that could even capture and replay packets back… if that could be made to work with an Rfcomm session… it would be nice.

Quick note… [I think] discovered how to activate the [debug logging] mode for the Zeo… if it wasn’t already doing that already. It could have been I just didn’t know how to download the logs.

Regardless…

Its been logging now and the entries have ‘tags’ of [V…D…I…W…E] ect… with timestamps.

Could be useful or useless… I don’t know, but it happened while trying to do something entirely different.

Posted the details to the blog

Combining the debug data with the protocol observations might help understand some things.

You didn’t mention in your blog post HOW you activated the Zeo App’s Verbose-level logging mode … mine appears to only be in Info-level mode. You can also view these logs in real-time from Eclipse or Android Studio rather than emailing them to yourself…

FYI you can generate those logs right from the Developer Options menu … its right at the top. Using the power button is convenient for a “popup” in the middle of an application when you don’t want to send the app you are working on into the background.

My logs for some reason are so large, the stupid Email client won’t send the attachments.


Updated:
Oh … looking at the Zeo App’s code and your blog post, I can guess you re-signed the Zeo App after likely altering some of its permissions. The Zeo App auto-turns off its “verbose” log messages if its signature matches a hardcoded numeric string. Altering the signature will have the side effect of putting the Zeo App into “verbose” log message mode.

Yes… I deleted the META-INF in the original APK without extracting or touching the other compiled components. The signature is by its nature exposed. Then I made a new keystore and used that with the java jarsigner to re-sign it. I didn’t even use eclipse.

Interesting what you said about Info vs Verbose modes… I didn’t know it would do that… I was just bumbling around in the dark.

I did get a slight clue though since in the back and forth on that Android developer forum in 2012 someone mentioned developer builds versus release builds and toggling logging.

I just did not know what the trigger was. When I got the first Bug report log dumps… I knew something was happening that I didn’t expect.

I see the Top of the Developer menu [now] that you mention it… I didn’t think it was click-able I thought it was just a menu Title bar… but your right it does work.

I Really like the idea of viewing them ‘real time’ as opposed to emailing them… I was hoping there would be a way to do that. I assume it has something to do with the ADB… which I know very little about.

Long term figuring out the Bluetooth protocol will probably be the best thing… since it opens up so many possibilities. Accessing the database was just a [thought experiment] that has gone a bit too far… the debug logging was a really nice side benefit (though now I worry about shutting it off ) :slightly_smiling:

I think I will have to take a break tomorrow… its a work day… and there is a problem with Outlook and autodiscover misbehaving… someone set all of the users primary smtp addresses away from the auth domain… and they look untethered from autodiscover… oy… I got hopes I can use a .prf file to reconnect them… or some wicked scripting to adjust the profile configs in their registry… I already have a headache just thinking about “that”

Although the Zeo App’s debug messages have some interest, everything from the App will still be limited to the post-processed data that the Headband sends out.

I got my ZeoCompanion app for Android up and running in an “alpha” release mode. I’m able to directly interact with the headband, receiving data and sending commands. I had to incorporate the same packet re-assembly logic as was needed in the Wireshark dissector.

The next step I’m going to take is to see if I can discover how to put the Headband into Test mode, and therefore gain access to the pre-processed data.

Hi Mike,

Sorry about the slow reply.

While, I’m interested, I’m not sure that I have the chops or time to contribute meaningfully right now.

This is why:

  1. I’m technical, but I’m a systems guy. I’m fairly familiar with Git, so no issues there. I’m strong with scripting languages, and expect that normally I’d be able to pick Java up fairly quickly but…
  2. I have a new baby at home (1 month) and am back to work next week, so I expect I’ll be overloaded trying to figure out how life works.

All that said, I’m more than happy to help test, an AM interested in learning some Java / mobile app development and would be interesting in tinkering once I figure out how life works. It would just be a while before I was more of a help than a hindrance.

-Alex

That was really fast… congratulations!

I wish you all the luck with the test modes. I haven’t seen anything about them anywhere, but I will be keenly interested to hear if you can activate them.

The two quotes I read about the bedside versus the mobile model seemed to indicate the beside model ran all the time broadcasting its raw measurements and ate up a lot of power.

Nothing concrete to offer… but I would [guess] they would need raw sampling data to fine tune the software code computing the FFT.

I have seen on a different project board an FFT function approximated by a neural net using some sort of fine tuned mathematical matrix transform equation in code. That is they used a standard maths library for the matrix calculations… but they had to provide the coefficients using Matlab or something… if they had the raw data, and could keep working it back and forth they could increase the accuracy… of filtering to filter out unimportant things to deduce the sleep states. I don’t think that helps much.

But it would be a good reason for the raw data to be made available in a test mode.

The big win for transmitting only processed results would be savings in processing power… and not having to write reduction code for every platform.

Just for reference: Zeo IP was bought by Resmed, and I’ve pinged my one contact there (again) - but my last requests were over a year ago and never got a response.

I’m not sure if I’m going to write a full app or not … depends on whether or not there is more to pull from the Zeo than just its present dataset. But if I do, of course I’ll be looking for some beta testers at least.

@Agaricus I hope you do get a reply at some point. We really do not want their intellectual property (which would be the FFT and their analytics). We would just like to be able to access the Test mode of the headband (and know the formats of the Test messages) to allow ourselves to be able to perform our own analytics if we so wished.

Unfortunately, I found a headband command over Bluetooth that bricked my headband. Fortunately I bought two. But I’m not going to risk bricking my second headband, so I’m not going to probe the headband’s test modes any further. I may be able to revive the bricked headband by opening it up and disconnecting/reconnecting the battery to make it do a true power-on reset.

Attached is a final copy of the ZeoMobile Headband Bluetooth protocol. I was able to successfully write and use a new Android App to fully interact with the headband just like the Zeo App. The dangerous message type (#173 aka 0xAC) is documented as “DANGER - this will brick the headband” in the protocol document. Likely its a message/command used to flash the headband or wipe the headband’s flash.

Zeo Headband Bluetooth Raw Data Format.txt (22.6 KB)

I am writing a ZeoCompanion Android App that I’ll put up on GitHub. It will feature a much more comprehensive automatic exporter, plus a sleep journal that works side-by-side to augment the Zeo’s hypnogram. I’ll have the headband bluetooth classes in there as well.

I’m really sorry to hear that Mike.

In the back of my mind I was worried something like that might happen.

So that the rest of us don’t fall to that same fate… could you relate any of the details?

I really appreciate the work that you have done.

Sorry you lost your Zeo. I am however, quite glad to hear that you’re creating the companion app. Having the sleep journal to be able to populate the various fields in the Zeo CSV will be quite nice. I appreciate all you’ve done. Thank you!

The ZeoCompanion App is nearly done. Its turned into a bigger project than I thought. I have been stuck in my house while remodeling is done, so thats given me alot of time to work on it.

The ZeoCompanion app will provide:

  1. Automated daily email export of Zeo CSV data or snapshot images
  2. Manual email or other sharing of Zeo CSV data exports or snapshot images
  3. More basic analytics of daily sleep history and showing the 30-sec hypnogram
  4. A dashboard with trend-over-time basic analytics
  5. A sleep journal that extends and is more customizable than what was formerly on myZeo.com, including tracking of going-to-sleep problems and nighttime-wakeups; its integrated into the CSV export. Gibson’s Healthy Sleep Formula is also integrated into the journal.
  6. Optional (not for the feint-of-heart) tool that provides direct access to the headband (and could be used eventually to complete eliminate the need for the existing Zeo App).

I’ll likely start a new thread here when it’s first release is ready some time next week. It will be open-source and posted to GitHub then as well.

2 Likes

Really Impressive. I bet Steve and everyone using the Zeo for Sleep Monitoring will be pleased to adopt this as the best tool for their research. I will be very interested in trying it too. Can’t wait to see it!

Version 0.9.5a of the App is now available. Please see thread

Anything for IOS devices?

Sorry no. iOS and Android are written in two separate languages, so I cannot just “compile” the ZeoCompanion App for use on iOS. Also, the ZeoCompanion relies on the Zeo Mobile Android App to access the headband, and since the Zeo Mobile iOS App is gone, there would be no data for an iOS-based ZeoCompanion App to access.