Track mood and energy level by photo

Hello, QS community

I want to have a tool which could grade energy level and mood by a photo. So, I could take my photo twice a day and track it.
I have enough experience to build an NN model on my own. However, I don’t have a proper dataset on this.

I would be glad to have some help, specifically:

  1. If you have some information about this topic (articles, prototypes, applications)
  2. If you know where to find a relevant dataset
  3. If you are willing to help me to collect such data

There are quite a few facial expression databases; maybe you could even find a pre-trained model and use transfer learning to make the model work better for you. But I don’t know how feasible grading “energy levels” or detecting subtle moods is; manually entering your mood into an app like Daylio might be less tedious than having to take pictures of yourself smiling forcibly :slight_smile:

…unless you’re planning to set up a webcam that automatically logs your mood every time you leave the house, or step in front of a mirror…

The other approach to mood detection is voice analysis.

I made some research beforehand. All datasets I found are indeed facial expression datasets. To classify a facial expression is very specific task and not the thing I need.
I’ve was thinking of using speech data, but this looks harder than with photos.

I want to track more subtle things than facial expression. I want to know if the person is tired / having a bad mood. I believe that if you are not technically showing “sad” face (even if you are forcing a smile), it’s still possible to understand that you have bad mood by looking at person. But this is only a hypothesis.

It is an interesting idea, for some people it’s a easier to manually grade your state than to take a picture (or record a voice).

Interesting! Some things that came to mind that might at least generate ideas:

this project:

this app company for sures has exactly the data you would probably want, but no idea how sharable it is (but could be used for data collection):

If you were interested in collecting your own dataset, I’d be interesting in chatting more about that if you think that would be useful to you. I’m an experimentalist in psychology/animal behavior, and work a lot with computer science folks!