Hello! Seeking advice on the best way to analyse a collection of text files

Hi there. Long time forum lurker (signed up in 2012, but this is my first post). I currently track activity with an UP24, and I’ve been tracking my schedule to varying degrees of precision using iCal/Calendar. I also use Moves, and I really want to build a Reporter.app habit, but I fell off the wagon a few weeks in.

I’m a writer, primarily, and I tend to track my thoughts in logs and journals— plain text files, often prefixed with specific keywords (journal, workout log, task_archive etc). I’m wondering if there’s some way of extracting some value from these files. Each file is prefixed with a timestamp, so it shouldn’t be too easy to fit them into a chronology. I’m wondering how I might go about tracking instances of specific keywords (or even significantly occurring words within the bodies of text over time?) and visualising that in some way?

Does anyone have any leads for text analysis tools or apps that might be useful for me here?

Thanks in advance for any suggestions!

Thinking further on this. As I explain in the first post, having experimented with a range of other tracking tools, I’ve found it easiest to log the things I want to observe/report on in text files. My workflow:

  • fire up NVAlt (OSX) or 1Writer (iOS). 1Writer automatically names new files with a timestamp. In NVAlt I use a text snippet to achieve the same.
  • first line: hashtag to categorise the log (e.g. #journal, #ate, #draft, #workout)
  • enter appropriate content, and done.

I’m trying to figure out whether I can extract the following from my current logging practise:

  • productivity (1): I’m a writer, among other things. Productivity in my case could be tracked by counting the number of files created within a certain folder against each day/date. If I could break that down to report against specific hashtags (contained on the first line of every document), that would be a bonus.

  • productivity (2): Number of @done items in an archive file. I use TaskPaper-style task lists via FoldingText (OSX) and Editorial (iOS). I’d be interested in reporting on the number of “done” items each day. Is there a relationship between the number of tasks done and the amount of writing I’ve done over a specific time period? Can I effectively split my days between making and doing, or do I need to block longer periods of time for specific modes of work?

  • food log: I don’t need to track calories or portions— for now, I just want to log what I eat, when, and how that affects my mood/energy/cognition.

  • workouts: I track home and gym workouts with a list of the exercises I’ve done that session, how many reps or how much time spent (for timed exercises like planks). While this makes for the easiest data capture I’ve found for all of the workout trackers I’ve played with, I don’t yet know how to extract totals to determine whether I’m making any progress here. It could be argued that all I need to know is whether I’ve worked out or not? I’m mindful of not wasting time trying to track things simply for the sake of tracking…

  • mood/energy level/cognition: haven’t thought this through yet. The goal would be to report on how mood/energy/level/cognition relate to food consumed, productivity and workouts. However, the data is currently wrapped up in the text of journal entries. I’d love to be able to create a system that would report on the occurrence or frequency of words in journal entries that could help me analyse/determine how I’m thinking/feeling, but I’m not sure how easy that would be to a) extract b) correlate.

Ultimately, I’d like to pull my own reporting system together, maybe with some kind of “at a glance” dashboard that allows me to quickly get a sense of where I’m at— to inspire me to a) maintain my efforts and b) continue to improve wherever possible. Again— I’m not expecting any one system to do this all for me, but if anyone has any suggestions for ways of working towards this, ways of extracting or reporting on the kind of data I’m talking about capturing, or even challenges for my thinking, I’d appreciate it.