Unlike most people who find this site, I have absolutely no idea what a start point is, or would be. I worked on some data mining projects during my MBA and find myself fascinated with data analysis and visualization. The down side is that I have no idea where to start. I do have some medical issues and need a hands-on project to grasp the “big data” concept, and quantified self seemed like a great first step.
Here is where I need help:
Where do I start, as in exactly (how do I record basic data, such as my GPS history via iPhone, access it and then visualize/manipulate it).
I’ve realized that I will have to learn some programming language(s), but I’m not sure which ones to go with to focus strictly on data mining and analysis, and then visualization. I plan to use my eventual skills in the business world by helping interpret data, visualize it, and hopefully give me insight that my peer’s and/or competitors would not have.
I spent the previous 10 years in the military, so I have the education and knowledge of a pre-K student. For example, I read in a couple of books and websites to pursue Python, but after trying to “begin” I can’t figure out how to make the text I’m writing (coding) ‘work’ (compiling???).
Finally, I understand that a TON of data is available and there is this mysterious “API” that somehow queries for the data, but I still don’t know where that fits in. So, do I create a website or use a MySQL server (I don’t know how to do that either), and how do I create applications or API’s (whatever it would be called) to query information and store it on my server, and then, how do I physically analyze the data and visually represent it?
From my attempt at research, I believe that I would like a website that is my project gateway, and I need a server, and subsequently, some method of turning raw data into visualized products so that I can understand it and then learn what I did wrong so that I can alter my techniques in the future.
If someone can PLEASE tell me where I should actually begin, I would appreciate it. I apologize for the lengthy post. Thank you.
Thanks for your post and your trust in us!
First of all: if I had a ton of funding, I’d hire an A-team to help you find out exactly what you need to know and see what I can learn…
You asked where you should begin when beginning a QS project. It’s a very good question, indeed! Not to be trifled with.
my short answer: set clear (SMART) goals and find out how to best measure your progress.
Long answer: you can do the ‘big data’ approach (in which case I won’t be of much help, there are others here who are more versed in this topic), and just try to get as much data as you can. In this case it’s important that you build whatever data format you end up with centred around a timeline. Personally, I use whatever data format I can read into Matlab and store those in my dropbox folder. Some files are a file per day or per session, others are continuous Excel files. Surely there are tons of more elegant and cohesive ways, but we all got only a limited amount of time at our hands your needs will often change rather quickly. Also, using big data to come to sensible conclusions using blind algorithms can be very tricky, because what comes out is often garbage…
you can also use a hypothesis driven approach: do a needs assessment, see what variables you want to control, which aspects do you want to manipulate. Write down for yourself exactly what you want to achieve and ask yourself what possible next steps you can take to achieve it. (Then maybe take a long hike or do a home improvement project to take your mind off of it.) Then decide what measurable thing can be your ‘surrogate marker’ for progress or success. And then start measuring, get a decent baseline. (A sufficiently long period of time during which you don’t try to change, just collect data.) Once you have that, start changing one thing at a time. If you think you found something, experiment…
