There are poorly designed scientific studies, and then there’s poor reporting on scientific studies by journalists who should know better.

A prime example of the latter is the response to a research letter published last week in the Journal of the American Medical Association called “Accuracy of Smartphone Applications and Wearable Devices for Tracking Physical Activity Data.”

The gist of the research: Counting steps has been shown to be good for your health; plenty of smartphone apps as well as wearables count your steps now; might they be equally good at that purpose?

Here are the results of the study, which had 18 participants use four apps (two Android, two iOS) and five wearable devices on a treadmill. One test had them walk 500 steps, the other 1500 steps. The horizontal bars are the range of steps reported (or misreported); the dot is the average.

What might you reasonably conclude from those results?

That the Nike Fuelband is wildly underreporting and a waste of money? Probably.

That if you really want a super-accurate step count, there’s no substitute for the Fitbit One or Fitbit Zip wearables (which are cheaper than the less accurate Fitbits)? Certainly.

That smartphone apps are pretty good at tracking your steps, and can substitute for a wearable if you don’t want to spend the money? Sure, why not.

You might be surprised to learn, then, that Mother Jones, an otherwise thoughtful and reputable publication, went with the headline. “Science Says Fitbit is a Joke.” Jezebel, picking up the story from Mother Jones, took its conclusion a step further: “Your Fitbit is Bullshit, Says Science.”

Of course, science says no such thing. In the case of two Fitbit trackers, this science says the exact opposite.

So what gives? Well, first of all there’s the layer of interpretation added to those results by the study’s authors, who clearly want to nudge us towards the third conclusion. The research letter opens with:

Despite the potential of pedometers to increase physical activity and improve health, there is little evidence of broad adoption by the general population. In contrast, nearly two thirds of adults in the United States own a smartphone and technology advancements have enabled these devices to track health behaviors such as physical activity and provide convenient feedback. New wearable devices that may have more consumer appeal have also been developed.

and ends with:

Our study is limited by being conducted with young, healthy volunteers in a controlled setting with a convenience sample of a small number of applications and devices. Results should be confirmed in other settings and with other devices. Increased physical activity facilitated by these devices could lead to clinical benefits not realized by low adoption of pedometers. Our findings may help reinforce individuals’ trust in using smartphone applications and wearable devices to track health behaviors, which could have important implications for strategies to improve population health.

Got that? It’s a very preliminary study, the result needs checking, but basically apps are good enough for most people to see positive results if they use them for exercise. No mention of the frankly superb performance from two of the Fitbit trackers, which suggests a little confirmation bias is at work. But hey, basically a legitimate summary of the data.

The game of telephone began with the title of the University of Pennsylvania’s press release: “Smartphone apps just as accurate as wearable devices for tracking physical activity.” From that, a week later, Mother Jones took the conclusion that wearables are a waste of money. That despite getting this quote from the study’s senior author, Mitesh Patel:

“Smartphones may be harder to carry with more vigorous activity such as running or biking, and that might be one reason an individual chooses to use a wearable device.”

That just begins to scratch the surface of the problem. Nobody carries their smartphone on their person the whole time. The larger they get, the harder it is to fit them into pockets rather than purses, where they’re less likely to count steps. (More studies on that, please, science.) Then there are all the steps you miss when the device is plugged in — an agonizing loss, as all competitive Fitbit fanatics know.

Bottom line? Be skeptical of any report of a scientific study, especially if it anthropomorphizes “science.” Go to the source data whenever you can. Draw your own conclusions. Understand that a popular brand such as Fitbit is a likely target for sniping.

And definitely don’t buy a Nike Fuelband.

Have something to add to this story? Share it in the comments.