This is a general topic for discussing ideas about personal dashboards for your self-tracking data. I started it with some posts from earlier topics that were happening on the forum, sparked by @LNP.
I apologize for some slight timeline issues on the posts and making small edits to add coherence, but I think it will be worth it. #virtualbreakout
Would you be interested in:
the ULTIMATE tracking aggregator with analytics and custom dashboards?
(SIMILAR TO EXIST.IO BUT WITH MORE CUSTOM FUNCTIONALITY)
I’m thinking about building an online platform that will be different from Exist.io in the following ways:
It will also support the upload of .CSV files from unsupported trackers and define custom scripts to extract attributes from those files.
The custom tracking will also allow for numbered variables, and can be sorted into different groups!
You will be able to create your own pages and subpages with custom dashboards! (Visualize any combination of variables with a scatterplot, timeline, etc.)
Comment “I’m interested” if you would like to know if I build this platform ! 
Also… Is there anything else that you would include in a platform like this?
Hi Lars,
I’ll offer some counsel based on a decade of observing projects similar to these. The goal of having a dashboard like product to track everything has attracted dozens (perhaps even hundreds) of programmers. Sometimes these were commercial projects, sometimes they were hobbyist or community or more generally open projects. A few have survived. Zenobase, exist.io, and Gyroscope are the most well known examples in the QS Community. (Q: Who am I leaving out?) In general this has proven to be much more difficult than most people realize when they begin, due to heterogeneity on both sides of the system: users have more divergent needs than is obvious from the beginning; and, data flows from services and apps are more idiosyncratic and less stable than is obvious. You have to be prepared for a long haul. Maybe you have source of income that allows you to tinker freely for years; that’s great. Or perhaps you have a startup intensity that allows you to raise money to support the development and customer acquisition phase. That can also work. Or maybe you have a specific use case you envision, a problem to solve for people who have no other solution and will pay you for your early versions and support you as you go forward incrementally. Also can work! But I encourage you to ask the question: How does this survive for, say, three years in a way that allows me to devote significant development and support time to it.
An alternative approach is to contribute to the development of an open collection of Jupyter notebooks. See this section of Open Humans for some inspiration: https://exploratory.openhumans.org/.
Open Humans can manage the user authorization workflow. Then you can build your own
dashboards and share them in a variety of ways, as well as run them at a URL and allow authorized users to operate the dashboard with their own data.
This doesn’t work for supporting a commercial path of the conventional sort, probably, but it gives you access to some good technical and platform capacity, as well as to a small but very knowledgeable group of potential collaborators.
Here are some of the notebooks currently available, if you want to browse examples: https://exploratory.openhumans.org/notebooks/
Here’s the open source dashboard I built 3 years ago and am still using today - https://github.com/Andreilys/personal_dashboard
I ended up building this because I couldn’t find any dashboards that were real-time. This creates an annoying user experience where you need to actively refresh the dashboard/app to get your latest data. I wanted to be able to, at a glance, see how my day has been going and make corrections accordingly which is what I was able to accomplish with a combo of javascript long polling and flask.
I’m actually planning on rewriting the whole project and creating a micro-service for the API which will act as an interface for a bunch of my other projects that rely on this data. So if you want to collaborate feel free to email me - andreilyskov [at] gmail [dot] com
I’m a Software developer who can’t find an aggregator/Dashboard tool that I’m satisfied with! So I want to hear from the experience of others who have tried to create their own thing, to not make the same mistakes!
I would love to chat with others who are also passionate about Quantified Self and Software/IT, who can guide me in the right direction 
Hello Lars,
I’m in the same process as you are building a QS dashboard using python. It’s something I have been thinking of for a while but haven’t started until a couple of weeks ago. Here are some of my thoughts on my experience so far as well as some of the questions you asked:
It’s ugly (no CSS effort), its incomplete. But it is still something I DO watch everyday when I boot my PC up. 2 reasons to focus on an MVP: 1/ it’s something tangible so it motivates you to continue, 2/ you’ll already have to learn and do a lot to have even a basic thing working (chose a library for building the interface, connect data to the API, learn how to embed a graph, …)
For the interface I’m using dash. Advantages: easy integration of interactive plotly graphs (it’s done for that). Drawbacks: less flexibility than other solutions like React.js (which I don’t know yet, so learning dash is easier)
For the database: you can start with a very basic SQLite database, which is a SQL database but stored in a file (no need to install any additional DB software), then upgrade to an MySQL or an equivalent. But at the very beginning (for the mvp) just stick to csv files, then make it more robust.
for importing your data, you could add an “upload” block direclty in your interface (drag and drop your csv) so you don’t need to go through an intermediary email. (just an idea)
I personally am not interested in correlations for now, as I find through my experience that its a lot of noise. I’m focusing on building something that gives me a good and flexible view of the metrics I care about so that I can myself spot correlations just by watching the trends. Automated correlations are definitely not on my priority list.
Start small, then progressively move towards your ‘100 variables through hopefully more than 50 years’ vision.
I was also considering using dash to build the dashboard. Can you expand more on why it isn’t flexible? What can’t you do when using dash?
https://plotly.com/dash/ ? it just draws things it does not aggregate, clean or analyze. Its the last step in your Python journey.
@LNP it is flexible, works fine. It’s just less flexible than more complete frameworks like React.js. I’ve run into a problem for example where I can’t assign multiple callbacks to a same output, there are some workarounds but it is a bit clumsy. Also it does not play well with adding your own custom javascript code. But for an MVP it does the job very well.
This may be a foolish suggestion as I’m over my head technically, but would there be a benefit in “joining forces” using the Open Humans platform to collaborate on dashboard development in a notebook anybody can run?
I’d be interested in a virtual conference! Happy to volunteer some time as well to make it happen.
Good discussion in this post. I am searching no. of sites for tracking data information. I never got this data. Finally i get here the complete useful information regarding personal dashboards for self tracking data. thank you for sharing.
If you’re interested in correlations, a hot startup is Bioloop Sleep for improving your performance by focusing on sleep patterns. For Fitbit, Oura and Apple Watch. Worth a look!
The goal was originally the same as yours, but focus is paramount in QS dashboards.
Hi Kastan, I agree that focus is important if the goal is to create a product that has a clearly understandable and standard use case to support success in a crowded field of other products. But many personal dashboards aren’t products, but resources for a personal workflow. In that case, they may benefit from being more general purpose aggregators that allow their creators/users to shift focus over time. (What if you wake up tomorrow aren aren’t so interested in sleep anymore?)
Here are some of the self-tracking dashboards I have been using since 2015. Most of the data is observed by me and recorded manually - some tracking dimensions contain data for all members of my family. Dashboards are built using Power BI connecting to data hosted in a cloud-based SQL server database.
Love the dashboards @Sergio! Are they available publicly anywhere?
I recently started playing around with various ways to visualize my data as well, inspired by many of the posts here. It’s unexpectedly difficult to present the data in a clear yet informative fashion.
You can play around with it here: https://alexey.io
I’m interested
How can we flag the post? I want to keep it
I undeleted my post, no need to flag it. I just hadn’t seen anyone else posting “I’m interested” in this thread, so I thought I had responded incorrectly. Would love to see something along the lines of what you’re proposing to build!
Hey LarsNyborgPedersen,
Answering some of your questions:
Would it be best to put it in a .CSV
It seems like CSV is the most universal format now (at least for such kind of that), so I’ll bet on it.
Would there be any benefits to making this a local desktop app instead of a webapp?
Of course. If you’ll decide one day to switch over from Macs to Windows (or otherwise) - web application has you covered.
Are there more automated ways to get data from my 5-8 different trackers to my webapp? Not all of them have public API’s.
That’s true - some of them are not public, some of them are public but you have to pay for the REST API data access (and I think it’s fair taking into account that storage and processing can cost plenty). I’d stick to one platform that allows you to backup/export data to CSV anytime you want.
Hope that helps
