The diagram below illustrates some of the tech i use in my QS projects, including examples of how it is being used .
I would preface by saying that I face a very similar problem to that mentioned by @Krishnan and @justintimmer - multiple years worth of tracking data (and growing!) and the need for an efficient way to store and manage it, as well as innovative ways to tap into the value of these growing datasets.
Those with backgrounds in traditional Business Intelligence functions will recognize the data platform depicted here as a very classic distributed data pipeline feeding a data warehouse. Data comes in from various sources on the left hand side, is cleansed and transformed before being loaded into a traditional data warehouse. On the right hand side, you see the visualization and analytics reporting options.
Here are a few examples of how I utilize my data:
Visualizing Growth - We have three kids so I can easily generate time-series of their physical growth directly from my iPhone using Microsoft's PowerBI app. These are useful during pediatric visits and help build on the conversation between parent and pediatrician. Most of the measures i'm tracking are recorded manually except for weight which is loaded automatically from a Withings WIFI scale.
Reporting on Vaccinations - I track all medicines and their dosages taken by my family, including vaccines. From my phone I can also quickly pull up immunization records for the whole family compared to the vaccination schedule we are following, or any allergic reaction we've had to a medication over time.
Anticipating Symptoms - I also track all the health symptoms we experience over the course of the year. Symptoms are normalized to the WHO's ICD 10 classification. It was only after collecting a few years of symptoms data that I could begin to detect patterns in how our body responds to changes in weather, season, and a host of other factors. This gives us clarity in planning for vacations, for example. In general it gives us a pulse to understand how factors may be impacting any symptoms we feel.
Tracking Debits/Credits - Here is an example where a Machine Learning algorithm is being used successfully. I track daily credit/debit transactions. Before importing them to the data warehouse you see in the diagram, they are categorized according to a list I defined (i.e. Food, Groceries, Shopping). I wanted a simple way to look at the transaction's description and assign a category. I also wanted a way to learn from previously categorized transactions, so that the process become more accurate over time. For this I designed a Text Classification algorithm using Microsoft's ML Studio.
My background is IT, so i apologize if this comes across as very tech-oriented perspective. The reality is that I am much more interested in making self-tracking an effective tool for the whole family, regardless of the technology behind it.
Having said that, today more than ever, the technology options in front of us are really amazing. The cloud services alone, including data and machine learning services, such as the one I mentioned earlier, are helping us all "cash in" on the data we keep. In the diagram you see, I pay a total of about USD 18 per month to Microsoft for the following:
- database in the cloud (Azure SQL Server)
- machine learning in the cloud (Azure ML Studio)
- data factory in the cloud (Azure Data Factory)
- analytics and visualization in the cloud (Microsoft PowerBI)
The items in dashed lines don't exist yet, but I would like to add them. These items include the sci-fi sounding chatbots or natural voice processing tasks. There is a practical reason for them however and it relates to Google, Amazon and Microsoft beginning to introduce "home" products like Alexa. These products listen and talk to you about your music, the weather and your groceries. How great would it be if these products could also answer in a context that is based on your self-tracking data!