Has anyone used PYTHON to create the PERFECT tracking aggregator/Dashboard webapp for themselves?

Quantified Self Data Formats, Export and APIs
1

Has anyone used PYTHON to create the PERFECT tracking aggregator/Dashboard webapp for themselves?

I’m a Software developer who can’t find an aggregator/Dashboard tool that I’m satisfied with! So I want to hear from the experience of others who have tried to create their own thing, to not make the same mistakes!

This is why I’m not satisfied with Excel/Spreadsheet:

  • I’m using 5-8 different trackers, and their date formats are not consistent, and also the data has to be cleaned up, and I’m not very good at working in VBA/Excel.
    I don’t know how to setup my data, in a way that will allow me to see correlations between variables from different trackers…
    I’m better at using Python, and I will probably also use Python more than VBA/Excel in my Software Engineer career I think.

This is why I’m not satisfied with Exist.io/Gyroscope etc.

  • They don’t allow me to import my .CSV tracking data from Chronometer and Notion.

This is my perfect solution:

  • Each month I download my .CSV files and send them to my E-mail. This should trigger IFTTT to send those .CSV files to my Webapp by using my webapp’s public API (Or will this be overkill? Don’t know how hard it is to make a public API…)
  • After the .CSV files are sent to my webapp, then my webapp should extract the information about all variables and put it in a consistent format that can be worked with.
    • Do you think it would be best to put it in a consistent format in a database or a new .CSV file?
  • After I have the data in a consistent format that can be worked with, I should be able to do the following:
    • See change through time on a graph for each variable, and also put more than one variable on the same graph.
    • Making scatterplots with 2 variables
    • Find correlation between any 2 variables:
      • The 2 variables both can be numbers or both consist of categories or one be a number variable while the other consists of categories.
      • I should be able to pick a variable, and then the webapp calculates the correlation between that variable and ALL other variables and present a sorted list.
    • Maybe also other useful ways of working with the data? I’m new to this whole field, so I don’t know of other useful ways of working with the data…

My questions to you:

  • Which mistakes did you do when trying to create your own aggregator/dashboard webapp?
  • Are there any of the things I mentioned where I could have used existing solutions, instead of coding them myself?
  • Are there more automated ways to get data from my 5-8 different trackers to my webapp? Not all of them have public API’s. Is the only solution then to download the .CSV files?
  • Would there be any benefits to making this a local desktop app instead of a webapp?
  • I want to store information on more than 100 variables through hopefully more than 50 years. Would it be best to put it in a .CSV file or actual database? And which kind of database should I use? I’m pretty new to working with databases…

I would love to chat with others who are also passionate about Quantified Self and Software/IT, who can guide me in the right direction :slight_smile:

1 Like
2

Hi Lars, I hope you find the help you need, but it may be difficult to get into this level of support on a forum thread. This would be the perfect topic for a conference breakout session: “Creating a QS Dashboard Using Python.” Unfortunately, not the time to have an in person conference! If there’s interest, though, we could arrange some kind of virtual workshop. Forum users: message me if you would want to participate in something like this, or reply here.

1 Like
3

I would definitely be interested! :slight_smile:

4

or reply here.

I’m curious to lurk something like that, though I doubt I’d understand most of it.

5

7 posts were merged into an existing topic: Personal Dashboards for Self-Tracking Data

11