Hello! I’m new here, not a researcher or academic by any means, just a programmer who falls in this niche (which is relevant to this post actually).
This idea was inspired by google reCatchpa; which seeks to detect if a user is a human/bot not through a catchpa string but by tracking all of the ways a user interacts with the page. For instance if the catchpa is clicked 0.2 seconds after page load, or the mouse moves in a perfectly straight line with 0 jitters to click the box, it is probably a bot doing it.
This got me thinking about how the digital metrics when interacting with a computer (mouse position / movement, opening/closing tabs, switching windows/applications) are different when I am focused on a task properly versus when I’m getting distracted, procrastinating, or engaging in compulsive behavior’s like tab hoarding or opening social media.
When I started paying attention the qualitative/quantitative differences between (focused) mindful and mindless computer usage are quite stark; such that I think you could take all this data in the form of logs mostly and use it to make a digital ‘mindfulness assistant’ app. (And I don’t think it’d be that hard actually).
A simple example for how this could work is say you select in the app that you are doing ‘a writing task
about abc topic in xyz program’
Right off the bat the app now knows, and knows to watch for:
- keystrokes / typing
- when xyz program is open or closed, and how it is interacted with (like if typing is being done with xyz selected or not)
- that you are engaging with topic abc, using AI the app can check whether a tab, resource, website etc is relevant to topic abc when opened, and if not restrict it or remind the user
Then using audio/visual indicators; the app can call the user back to focus on the task at hand if they get sidetracked into something else.
What’s cool to me about this (and makes me think it could have legs) is that it’s a pure software / pure vanilla PC solution that doesn’t need so much as a webcam (but a mic and webcam would certainly help). Combined WITH eye tracking it would be even more potent.
With user feedback to let the app know when they are/were focused or distracted etc you could even fine tune some simple machine learning models on that users personal data, which should get you to ‘seemingly reading your mind’ levels of effectiveness.
I’d love to get some input on this and hear your thoughts, and if anyone is interested I’d be happy to talk about collaborating!
security and privacy are the big thing. It would pretty much be a high tech keylogger on steroids. Probably by Spring or Summer of next year though I see current ML/AI/LLM models being miniaturized enough to support fully local (self hosted would also be a good option). Analyzing content for semantics will require an LLM which for now is mostly an offsite thing, everything else should be able to stay local. I wonder if some kind of VM / private silo instance could work.