GPS – Less is better

In my previous posting on the issue of GPS track log accuracy, I made a case that the 500-600 points produced by recording every 6 meters (20 feet) or so should be plenty to give an accurate picture of the average 4 mile hike and that the additional 9000 or so points proffered by Garmin’s preferred (to put it mildly!) “1-second distance accumulation” method should not, assuming that the points are accurate, change the track length by any significant amount.

But we have the simple fact that my friends and I have gotten a substantial discrepancy on every single mountain hike we’ve taken with my new Garmin 62stc device, errors ranging from 10-40% – which I would call significant! So which one is right? Or, I should say, which is more accurate? Because nothing is perfect, after all. This is all about the accuracy.

Accuracy

I want to make it clear that I am not meaning to “dis” the accuracy of the GPS unit itself. The reason I didn’t take the thing back to REI after its first hike was that I could tell that it was handling the challenging conditions of hiking in the North Cascades quite well. It acquires a signal very quickly and holds it even under deep forest cover. It does a good job of tracking the switchbacks, reacting to a sudden direction change much faster than my other models. Each individual reading from the device is quite good, given the conditions.

But there will always be errors. Garmin only claims accuracy down to 2-3 meters (7-10 feet) in the very best conditions. If you are a few feet away from where you thought you were it is no big deal. The problems arise when you combine a whole bunch of slightly inaccurate readings:

• Just how big are these errors?
• Will they tend to cancel out?
• Or will they tend to accumulate?

Looking at the last point first: if the errors were even as small as 3-4 inches, 9000 of them would accumulate to about a half mile! So this is not a trivial question.

Error analysis is difficult, by its very nature, especially when you have no independent means of verification. Nice straight lines just don’t exist in the forests on the sides of mountains. The trails tend to be hidden under the trees and thus not visible on aerial or satellite photos, and I don’t have a ball of twine long enough to stretch along the trail. So there will have to be a lot of guesswork and some hand-waving, but I will try to make the estimates very conservative and give the benefit of every doubt.

Assumptions

• It is very, very conservative to say that you might expect any recorded point along the track to be off by a foot or so from the true path.
• Because of the steepness and roughness of the trails, we are often traveling at only 1 or 2 feet per second; we hardly ever average better than 3 fps (2 mph).

So the  “1-second distance accumulation” method will be recording points only a couple of feet apart, about one for each step we take. (I ask you to meditate for a moment on the wisdom of recording points a foot apart when we only know the position of each point to within ten feet… Now back to the conservative assumptions.)

• So it safe to assume that the errors in these points could easily be on the same order of magnitude as the distances being measured

Here come the numbers

Let me compare this to the situation you have when riding in a vehicle at, say, 20 mph (or 30 fps). In one second you travel about 30 feet and record a point that is off by, say 1 foot.

Sometimes the error in that point may be in the direction of travel, so the estimated distance might be as little as 29 feet or as much as 31 feet; but that sort of error will tend to cancel itself out – if this reading is a foot short, the next will likely be a foot long. This is not a problem since we are just interested in the total.

The sort of error that will NOT cancel is when the estimated point is off the true path to one side or the other. If this point is a foot off to the left, then the next might be off to the right, compounding the error rather than canceling. If the next point is off to the left also, then it might not add to the error, but there is no reason to expect that it would cancel it out. And when you go off the track you don’t have to go off to the other side on the next reading, but you do have to get back to the true path eventually if the track is to be worth anything.

What sort of error are we talking about here? The worst case would be where the estimated point is a foot to one side of the true path – I’m picturing a a right triangle with a 30 foot leg (the true distance) and a 1 foot leg (the error). The GPS will be measuring the straight line distance from the starting point to the estimated point, i.e., the hypotenuse of that triangle. Mr. Euclid assures us that the length of that hypotenuse is the square root of the sum of the squares of 30 and 1, that is, the square root of 901 or 30.0166. So the error is that 0.0166 foot or about 0.2 inch.

Which ain’t much. In terms of percentages an error of 0.2 inch in a measurement of 30 feet works out to about 0.057%. This is a very tiny error that explains why Garmin’s strategy works quite well when you use the device in a car, or even a bike.

But this all depends strongly on the fact that the thing being measured is thirty times larger than the error, more than an order of magnitude. If that ratio came down to 10 (one order of magnitude) the percentage error would still be small – about 0.5% – not enough to bother with.

Things go nuts, though, when the error is on the same order as the thing being measured. If the ratio comes down to 2, we get

And it gets quite silly when the ratio is 1:

This is enough to bother a hiker – if you were training for a trip down into the Grand Canyon by making “10 mile” hikes that were in fact only 6 miles long, you could get a nasty surprise when you get to Arizona!

Now I’m not saying that you will always get 40% error on a real life hike. There are lots of reasons why you might do better than this. But back when I was using the default settings on the Garmin I did in fact record some errors of that size. And while the errors are not always that large, they are ALWAYS ON THE HIGH SIDE, just as this analysis would predict. (Again, if an error takes you off the track, you have to make another error to get back on it.)

(By the way, the same discussion applies to the odometer. Assuming that they are using the same method to stoke the odometer, it would explain why I long ago gave up on its consistently, and often laughably, high readings. On one hike the odometer gained a quarter mile while we were sitting down having lunch.)

Conclusion

I know that this does not prove anything. The thing I hope you take away is this: the small errors that are unavoidable in each point of a track will tend to accumulate, which means that sometimes less is better. And given our slow rate of travel and difficult sky conditions, it would be very reasonable to expect the “1-second distance accumulation” method to generate errors on the scale that I’ve observed on the trail.

Ironic Epilogue

The cruel irony is that Garmin’s decision to paste the one-per-second track length into the track log makes almost no difference in most situations, such as in a vehicle. Or when one is power-walking along an urban trail with a clear view of the sky in all directions. So it may not do harm in those situations, it never does any good! This is the easiest kind of cost-benefit analysis:

• Substantial Cost.
• No Benefit. Ever.

In my last note to Garmin, I also made an appeal to their sense of fairness:

If a user believes that the “1 second distance accumulation” is a good idea and gives her good results, then she can choose that option in the track log setup. But if she has a valid reason NOT to sample that many points and wants to make a track with the points spaced further out FOR WHATEVER REASON (but her reason, not yours), then there is currently no way for her to measure the length of that track in the field. You may not like her reason, you may snicker behind your hand that she would want to do something you think is foolish, but it is (pardon the expression) none of your freakin’ business!

 

GPS – Just because you can doesn’t mean you should

After going back and forth since last June with the email support staff at Garmin Ltd I finally received a reply with some substance:

When the user uses anything but the one second track interval the track log is “missing” data.  We are putting a value for total distance in the GPX file in the track stats tag that gives BaseCamp the 1 second distance accumulation ( even if they are not setup for 1 second distance accumulation ).  So when the user removes the first point BaseCamp can no longer use our 1 second distance accumulation value and has to fall back on the old method of summing the delta between all of the points.  The problem is that all of the points are not there, because they are setup to record at a less than 1 second interval.  So when this happens they think they are just removing 1 point, while in reality they are removing possibly thousands of points along the entire track which is why the distance changes.

The substance is that they have finally acknowledged what I have been saying for the last six months and what I suggested in the last paragraph of my first posting on this issue back on June 10, 2014. That’s the good news; the bad news is that they clearly seem to think that substituting in  the “1 second distance accumulation” is a good idea! If it took six months to get them to even admit what they are doing, how long will it take to make them see that what they are doing is just plain wrong?

The Lily and Lizard Lake Trail, Skagit County, WA – 687 points provide a lot of detail.

A Quick Review

After a lot of experimentation, I have my Garmin GPSmap 62stc set up to record a track log point roughly every 20 feet (6 meters). Since we usually average only one or two feet per second when hiking uphill, Garmin’s “1 second distance accumulation” would mean recording a point every foot or so – ten or twenty times as much data. For example, on the first half of last Thursday’s hike I recorded 687 points in roughly 4 miles, at an average distance of about 30 feet, as shown in the picture; the elapsed time was about 2:37 so the 1 second method would have totaled up around 9,400 intervals of about 2 feet each. The total track length (by the 1 second method) was listed as 4.7 miles. When I deleted a single point and forced a recalculation (“losing” almost 9000 points) the length changed to 4.2 miles.

Why my points were 30 feet apart rather than the 20 I asked for is something only Garmin could answer, but it is not a big deal: 687 points is more than enough to make a very good picture of the trail we were on. The track agrees very closely both with my previous tracks (made with different Garmin devices) and with published information and maps of this very popular local trail. We had good weather, we were moving at a good, steady rate with few stops and the device showed an accuracy between 10 and 20 feet (about as good as it gets) every time I checked (which was often).

The Big Question

Pay attention here, as I am pretty sure this is the point where Garmin is going to counterattack (assuming I can get them to respond before I die.)

Even on these winding mountain trails it’s not like we are juking rapidly from side to side as we go along – with the exception of the occasional 179° switchback, you are usually traveling more or less in a straight line. I’ve been paying attention to that this summer – most of the time you can see straight ahead along the trail for 20 or 50 or even 100 feet, then there is a subtle change of direction and you go straight for another stretch. A lot of the trails, including large parts of this one, follow old logging roads and railroads (which are not prone to rapid direction changes) and have only a handful of sharp turns. I think if you joined us on a hike you would agree that if you could accurately record a point every 20 or 30 feet or so you would get a pretty darn good track, and it should contribute almost nothing (except for a whole lot of extra data) to the measured length of the trail to accurately record a few thousand intermediate points along the way.

So this is the first core of the issue – it is all about accuracy. How do we explain that half-mile discrepancy between the “1 second” method and the “20 feet” method? When I have done similar comparisons in town on a nice straight city street with a full, clear view of the sky (and where you can get a good independent verification of the distance) there was no such difference. Garmin is on record with its explanation: when I edit the track I am “removing possibly thousands of points along the entire track which is why the distance changes.”

And I agree completely! Where we differ is that Garmin seems to think that more is necessarily better, that those thousands of additional points must be an improvement. That is a common fallacy – more chocolate is better; more habanero pepper maybe not so much.

I think I made a pretty good argument that those thousands of extra points cannot increase the accuracy by more than few feet in a mere handful of sharp turns, nothing like a half a mile. My argument is that the 1 second method was adding in an additional 9000 tiny errors – that would be a positional error averaging only 3.5 inches per reading, and that on a device that is only claiming accuracy of 10-20 feet.

My Assignment

So my assignment (which I very much do choose to accept!) is to try convince a bunch of engineers who work and probably live in a place with very few hills, very few clouds and very few trees that there are situations where a lot of their precious data is more like pepper than chocolate and is ruining the result rather than flavoring it.

And more poignantly, that this particular kind of pepper never improves the dish – most of the time is does no harm but it can never help.

I intended to dig into the gritty geometric and arithmetic details that support this argument but this is more than enough for one day.

GPS – the adventure continues

Since my last post I have been locked in a death grip with Garmin’s “Help” desk about my new GPSmap 62stc handheld GPS unit. I write in, they respond with a condescending, patronizing reply that picks out one phrase to criticize and ignores the general thrust of my message. I then reply in detail (as I am sure you can believe) and it is picked up by a different “helper” who chooses a different irrelevancy to pick on, and offers a completely different “explanation” or “solution”.

This process started on June 16 and has gone through six or eight iterations. I was going to wait for a some sort of resolution before reporting back to this blog but it is becoming clear that they are so adept at ducking this issue that resolution is very unlikely. But in the meantime, I have accumulated enough evidence to be quite confident that I understand what is going on.

Are you ready?

I my previous post I included this sentence:

 The best I can come up with would be an unlikely and deeply conspiratorial attempt to rig the results to agree with the odometer. But they probably wouldn’t do that.

Since that “4.2 mile” hike toward Excelsior Pass on June 5, we have gone to the Church Mountain Meadows on 6/12, the Goat Mountain Overlook on 6/19, a Middle Fork hike on 6/26 and back to Church Mountain yesterday, July 3. I have tried out all of the suggested settings proffered by Garmin and several of my own and gotten the same sort of results as the Excelsior hike every time. And at least twice, we have been joined by Carl sporting his Garmin 62st device, which seems to differ from mine only in that mine has a camera. He has gotten very similar results and has made a major contribution to my sanity (or what remains of it.)

The responses from Garmin have ranged from the merely patronizing to the positively ridiculous. In response to the fact that the device was generating spurious points when we stopped along the trail one respondent made this suggestion:

… try to stop as little as possible.

Another was quite insistent that accuracy could only be achieved by setting the device to record a track point every second – 3600 points every hour. Never mind that a track can only hold 10,000 points, limiting the device to a trip of about 2 hours and 45 minutes. Never mind that a track of that length causes their own BaseCamp software to bog down to the point of being unusable. And never mind that the “every second” track I made coming down on Goat Mountain is every bit as inaccurate as the one I made going up.

Most of my time with them has been spent dithering over these inaccuracies, where there is no outside authority to settle any dispute. There are hiking maps and websites, which are notorious for their casual inaccuracies –  it was my word against theirs. So I attempted to focus all the attention onto one verifiable, repeatable, incontrovertible inconsistency:

Why does their own device and their own computer software report different lengths for the same track under different circumstances?

It took a couple of exchanges to wrench their attention away from the accuracy question but last week I finally got someone to admit that there was something questionable going on when the removal of one redundant point from a log of over 500 points caused its length to changed from 4.2 miles to 3.6 miles and that he would have to look into it.

Then … silence.

The intervals between responses had been one or two days, mostly, but a full week elapsed before I saw a new email at about midnight before Thursday’s hike. I chose not to look at it before the hike since most of their responses had been simultaneously irrelevant and infuriating (or, in homage to Stephen Colbert, infurelevant) and I didn’t want to be in a bad mood in the morning.

On the hike to Church Mountain Meadows there were even more interesting results than before. I set the device to record a reasonable numbers of “bread crumb” points and kept a frequent watch on it along the way. I could see that it was doing a good job of following the shape of trail, catching even minor changes of direction and only rounding the switchbacks a tiny bit; I was confident that I had a very good track. I also was carrying my old Garmin 60CSx.

We stopped for lunch at a spot where all my previous experiences on Church told me the distance should be about 3 miles. I took the following readings from the devices:

• 62stc Odometer: 3.55 mi
• 62stc track: 3.4 mi
• 60CSx Odometer: 2.9 mi
• 60CSx track: 3.05 mi

I turned off the track logging on the new 62stc at this point, but didn’t save the track, yet. While I was sitting in the snow for the next half hour or so, I checked the 60stc now and then, watching the odometer go from 3.55 to 3.64 to 3.74 and by the time we got up to start hiking back, to 3.82. (That’s why I had turned the track logging off – so all that spurious junk wouldn’t be added to the track log.) All this time the device was sitting stock still on the top of my pack at 4950′ in an open meadow, showing excellent accuracy of about 2 meters (7 feet). Gaining an imaginary quarter mile in half an hour seems like pretty bad behavior, but long experience has driven my expectations for GPS odometers to be very, very low. But that’s another story.

As we prepared to start down the trail, I turned the tracking back on, saved the track (which still showed the 3.4 miles) and a minute later decided to use the “tracback” feature to show the distance back to the trailhead along that new track. Understand that I still thought it was an accurate track, just that its reported length was off  by almost half a mile.

And Lo! and Behold! The device now showed the distance to the trailhead as 3.0 miles!

When I got home and uploaded the stuff to BaseCamp, I got the usual nonsense: the track length showed as 3.4 miles until I deleted one point, whereupon it also changed to 3.1 miles. I ran the track through my spreadsheet and it gave 3.06.

The Bottom Line

So the old 60CSx track was right on the money at 3.05, and the 3.0 on the “tracback” and the 3.1 in BaseCamp can be chalked up to roundoff error.

All the other numbers reported by the 60stc are sheer bullfeathers.

Back to the Garmin “Help” Desk

So then I was ready to read the response from Garmin that I had avoided. There was some more patronizing stuff, but the gist was contained in this paragraph:

I hope this helps to explain this isolated incident that you were experiencing. This is an extremely rare problem. Unfortunately, the only way to really explain this error is to contribute it to corrupt GPX data on your device.

So, ignoring the fact that this was a brand new device when this stuff started happening and that another person’s device was getting the same results, Kris had come up with an “reason” that conveniently would invalidate all the previous data that Carl and I had accumulated, and attribute it to my not knowing how to disconnect the device from my computer!

[ short pause while the lingering rage gradually subsides…]

At the risk of my declining sanity…

Kris had given instructions on how to purge the dread corruption from the device, but since it works quite well around town, I would have to wait to test the inanity of his/her directions. But I couldn’t wait so on Sunday morning I performed the purge and headed over to Stewart Mountain (a big foothill, actually) to check on the progress of the logging operation that is likely to obliterate some of the trails over there. The conditions there are not nearly so challenging for the GPS as our real mountain hikes, but it might be enough to generate some good (i.e., bad) data.

I stopped at a little knoll in an old clearcut when the new device’s odometer just happened to read exactly 2.00 miles; the track log said 2.0; the old GPS’s track was 1.9. I was expecting the “tracback” function to shave a little off that 2.0 mile track length and was disappointed when it didn’t. I headed home.

Back at the computer, however, the old bug reappeared: deleting one redundant point changed the length to 1.9 miles and my spreadsheet came up with 1.895.

I sent this information off to Garmin and finally got the first response that was neither patronizing nor inane:

This problem has been reported. We are still researching this to find out what is causing this and how to correct this. I have added your information/file to the case. We will contact you when we have more information.

Hmm. We shall see.