No need to wear your wearable to bed: If you carry a step counter right up to bed (and don’t read in bed), the last and first steps of the day are a good approximation of your bedtime (and activity levels; see e.g. in this experiment).
I also use the “Flip to Shhh” feature on my Android phone while sleeping, so that would be another, convenient way to capture rest times (if Google let me have the data)…