Drowning in data

The longer I’ve run, and certainly the longer I’ve coached, the more I find that the answer to any question is ‘it depends’! Maybe I should stop writing, and just create one mega-article with the words ‘it depends’ over and over again. What does ‘it’ depend on? The answer is helpfully vague: context. Which could mean anything and everything, but that’s why ‘it depends’.

Anyway, it’s certainly the answer to today’s question, which is about whether the ever-increasing amount of data we get from our watches and other wearable tech is reliable, useful, or both (reminder: it depends!). My ‘headline’ (possibly unpopular) opinion on this is that in the main, these data sets are created by tech manufacturers to persuade us that their device is more valuable to us than that of another manufacturer. The algorithms behind the stats are kept secret, and although the manufacturers claim that the devices have been rigorously tested, it’s usually them doing the testing! On the other hand, the manufacturers are also keenly engaged in trying to improve their tech, and bring it to the standard of lab-based measurement - should I be giving them more credit? I thought I’d have a look at the most recent research data I could find. Here’s an overview of three wearable tech data sets (sleep, heart rate, HRV); in each case I offer my personal experience (I wear a Garmin Fenix 6 24/7) alongside a (recent-ish) research study.

Sleep

My personal experience is that Garmin sleep tracking is poor. In particular, the watch is not able to distinguish between lying very still in bed, and actual sleep. I often take over an hour to fall asleep, and wake up during the night, so this frequently leads to Garmin congratulating me on my excellent sleep when actually it’s been a disrupted, short night. On other occasions I can wake up feeling rested and refreshed, but Garmin claims I’ve had a restless night and am going to have a horrible day. None of this concerns me on a personal level (I’d rather listen to my body), but I worry that anyone making decisions about, for example, ‘do I feel rested enough to do a great job of my interval session today?’ might be relying on dodgy data if they outsource this decision to their device sleep score.

In terms of the research, this study from 2024 tested multiple devices, and found research-grade accuracy for measuring total sleep time (with the exception of the cheapest Garmin device tested), but multiple issues with the detection of sleep stages (light, deep and REM sleep). Conclusion: if you’re interested in knowing how long you slept for, your device can do this (my experience to the contrary!). But you should not rely on your ‘sleep score’ as a measure of how restful and restorative your sleep was - these scores are based in part on the proportions of light, deep and REM sleep you had, and the devices are not yet measuring these well (in particular, they tended to overestimate REM sleep).

Heart rate

I find wrist-based heart rate monitors to be pretty variable (and I see a lot of data!). The two most common problems I notice are either watches which never show a heart rate change (e.g. the athlete has done an intense interval session, and their watch has recorded 130bpm throughout, without variation), or those which at some point during a session shoot up to 175bpm+ and stay there, even on easy runs. Even watches which seem to generate a bigger range of data can show surprising jumps and drops through a session. My theory on this is that since these sensors are making an optical measurement, if the fit of the watch to your wrist is not good enough, your data will be inaccurate. I’m a big fan of both the Garmin chest strap HRM and the Coros armband HRM - although of course I can’t confirm via a lab-based test, these both seem to generate steady data which corresponds to what I’d expect to be happening in the session.

What does the research say? There are a lot of studies on this topic (one example here) and my best interpretation (as a non-statistician) is that the outcomes are mixed. Some studies seem to be suggesting that the devices tested agree well with lab testing, other note quite a lot of anomalies at different intensities of exercise. I’d argue it’s not very helpful if your device has been shown to be accurate in e.g. moderate intensity exercise, but loses this accuracy at high intensities. In many cases, the researchers used commercially available chest strap heart rate monitors as their reference measurement - which is a useful confirmation of their belief in the validity of those devices!

In conclusion here, if you want to make any training decisions based on heart rate (and we haven’t even got into the thorny issue of ‘zone’ training, and how your device has decided what these zones are!), you really need a heart rate monitor which is not worn on your wrist.

HRV (heart rate variability)

HRV is one of the newer metrics available, and is still misunderstood in some quarters. To save space here, if you aren’t familiar with HRV, here’s a good summary.

I have generally been impressed with Garmin’s HRV monitoring. For the most part, when my HRV drops it corresponds to a period when I’m one or more of exhausted, overtraining or sick. High HRV periods correspond to times when I feel relaxed and well rested. What I haven’t found, though, is HRV data telling me anything I didn’t know already! I shouldn’t need my watch to tell me to modify training because I’m knackered. I know some runners who say that they think their HRV drops a couple of days before they notice the first symptoms of illness - which could perhaps be useful, but could also lead to modifying training frequently when it wasn’t actually necessary.

This study (2025) looked at resting heart rate and HRV across a range of devices, and while they had positive things to say about Oura and WHOOP, they weren’t a fan of Garmin. It was also interesting to read the (gently expressed) frustration of the researchers at not having access to proprietary algorithms, as this limited their ability to understand what each device was doing in terms of processing the data it received.

VO2 max and lactate threshold

I’m going to save these for another time! It is worth a deeper dive that explains what each of these metrics is, as well as how each is measured both by the watch and in a lab, before looking into the validity. Watch this space if you’re interested.

In sum (as you can probably tell!) I’m somewhat suspicious of ever-increasing encouragement to outsource our perceptions of our health, wellness and fitness to a device, and to an algorithm we’re not allowed to understand. And certainly the answer to whether the data is useful or reliable remains 'it depends'! By all means use the data as one piece of information among many to help you make decisions and track progress. But don’t allow the tech to overrule your own perception and understanding of your body.