GPS Bakeoff #2

Not a hike, but I figured that some of you might be interested anyhow...
(Well, maybe it was a bunch of short hikes. )

The first GPS bakeoff
(http://www.vftt.org/forums/showthread.php?t=14406) piqued
my curiosity, so I have performed some more comparisons. This
set of tests compared a Garmin GPSMAP 60CS with a GPSMAP 60CSx
under several conditions.

One of the problems in the first test was that one of the GPSes recorded
trackpoints at much longer intervals than the others, so both GPSes
in this test were set to record at timed intervals--1 sec for
biking, 5 sec for walking, and 30 sec for stationary. This
also eliminates any biases due to position affecting the choice
of which points would be saved by the GPS.

It was not practical to estimate the absolute accuracy, so
repeatability between the directions of an out-and-back route was used
for moving tracks and scatter (2 dimensional standard deviation) was
used for a stationary GPS. Hopefully, these measures correlate with
accuracy.

The data analysis for moving GPS tracks was the same as in the first
bakeoff: The tracks were produced by walking or biking an out-and-back
route. I then computed the area outlined by the track and divided it by
half the track length. This gives an average cross-track width which is
a measure of the repeatability of the tracks generated by the GPS.
In each individual experiment, both tracks were recorded simultaneously
to normalize out external factors as much as possible.

Comparisons between different runs are weaker than comparisons within
a run because the satellite clock errors, satellite ephemeris (orbital
parameters) errors, satellite positions, and the ionosphere may be different.

WAAS was turned off for all of these tests.

Walking tests

The first set of tracks were recorded while walking out to and back
from a nearby store. Tree coverage varies from complete
coverage overhead to a short open area. There were still leaves on
the trees.

In the first run (10), the GPSes were carried vertically in
the pocket of a small daypack with a 3-4in separator between them.
The GPSes were about 1in from my back at about kidney height, so
the signals from some directions were blocked by my back. The 60csx
appears to be more accurate than the 60cs under this condition.

In the second run (22), the external antennas were carried in my hat
and therefore signals were not blocked by my body. Both appear
to give about the same accuracy. The GPSes appear to be more accurate
with the external antenna on the user's head than with the partially
blocked internal antennas.

Biking tests

The biking tests were similar to the walking tests. The bike path
was mostly treed and the GPSes in the same daypack were less blocked
because I was leaning forward. (The bike is a touring bike with
drop handle bars.)
In all of the biking tests, the 60csx had more repeatable tracks
than did the 60cs. The external antenna on top of the head also gave
a more repeatable track than did the internal antenna in the pack.
Test 23 is a repeat of 19 and gave consistent results.

Stationary Tests

In these tests, the GPSes (or external antennas) were located less
than 15 cm apart just below the ceiling of my living room (one story
wood frame house). Only the ceiling and roof are above the mounting
location with no obvious metal above. There is a tree (with leaves) to the
S and SE of this location. Signals are fairly strong at this location.

The figure-of-merit used in these tests is the 2-dimensional standard
deviation (2Dsd) of the points. (Smaller is better.)

All of these tests were performed from about midnight to the following
noon. The tests were also performed in pairs: one 12hr run plus another
12 hr run with the GPS/antenna positions reversed. The averages of the
two runs are presented in the tables.
In these tests, the 60cs outperformed the 60csx and both the internal
and external antennas gave similar results. It would appear that the
60cs is more repeatable when stationary with good signals.

I added a stationary test with poorer signals by placing the two GPSes
at a location inside the room where the signals would be blocked by
metal from some directions and where multipath would be more likely.
The 2Dsd is larger than when the GPSes were ceiling mounted, confirming
that this is a poorer location. And the 60csx now outperformed the
60cs.

Conclusions
These results suggest that the 60cs is more accurate when the GPSes
are stationary with good signals, but when moving or in a poor
signal location, the 60csx is more accurate. The results also
suggest that the external antennas are better than the internal
antennas only if they are mounted in a better receiving location.

Doug

Great info Doug.Have you heard anything about the next generation Garmin. I've heard some rumors of a 3D screen.I have a 60cs and I'm waiting a bit for something new. I don't think it worth it for me to get a 60csx right now.

skiguy said:

Have you heard anything about the next generation Garmin. I've heard some rumors of a 3D screen.I have a 60cs and I'm waiting a bit for something new. I don't think it worth it for me to get a 60csx right now.

Click to expand...

I've heard a rumor of some display enhancement, but have no specifics.

I had a 60cs and was tempted, but had not sprung for the 60csx. I had read reports about the 60csx's improved ability to receive weak and degraded signals, but still had some doubts as to the accuracy of positions derived from these degraded signals. The first bakeoff confirmed the improved reception of poorer signals and showed the accuracy of the 60csx to generally be as good as or better than that of the 60cs on a trail hike in a narrow valley with tree cover. Add a 20% off sale and a US$50 rebate and I broke down and got the fancier toy at a halfway reasonable price (~US$350). The tests reported in this thread have further confirmed the improved performance in a hiking-like environment.

The primary advantages of the 60csx over the 60cs are improved reception of degraded signals and a removable memory card. (I installed a 2GB card and have lots of maps loaded.) The 60cs still works as well as it ever did--generally well enough for most hiking use.

At some point, the performance of newer GPSRs will reach a limiting point and the manufacturers can only add more features. (Don't know if we have reached this point yet. The SIRF Star-III chipset has 200K correlators, there is another chipset with 1M correlators--some of the specs are slightly better, but I haven't seen any good field comparisons between it and a SIRF Star-III based GPS.)

There are also some improvments in the works for the GPS satellites (eg higher power) and a second civilian frequency. There may be soon be two additional systems available (Russian Glonass and European Gallileo). Adding the second frequency and the other satellite systems are all likely to improve performance. But it will be a few years...

Notes:
* Glonass exists right now, but has an inadequate satellite constellation.
* Glonass receivers are significantly more expensive than GPS or Gallileo receivers (the signal structure is different)
* Gallileo will use a similar signal structure to GPS, so dual system receivers should be possilble without excessive difficulty and cost.

BTW:
* Latest firmware for the 60cs: 4.10
* Latest firmware for the 60csx: 3.0.0
* Latest GPSchipset firmware for 60csx: 2.90
* Latest Mapsource: 6.11.6 (you might prefer 6.11.5--see the 6.11.6 release notes)
Available from the http://www.garmin.com or http://www.gpsinformation.org/perry/

Doug

DougPaul said:

I've heard a rumor of some display enhancement, but have no specifics.

I had a 60cs and was tempted, but had not sprung for the 60csx. I had read reports about the 60csx's improved ability to receive weak and degraded signals, but still had some doubts as to the accuracy of positions derived from these degraded signals. The first bakeoff confirmed the improved reception of poorer signals and showed the accuracy of the 60csx to generally be as good as or better than that of the 60cs on a trail hike in a narrow valley with tree cover. Add a 20% off sale and a US$50 rebate and I broke down and got the fancier toy at a halfway reasonable price (~US$350). The tests reported in this thread have further confirmed the improved performance in a hiking-like environment.

The primary advantages of the 60csx over the 60cs are improved reception of degraded signals and a removable memory card. (I installed a 2GB card and have lots of maps loaded.) The 60cs still works as well as it ever did--generally well enough for most hiking use.

At some point, the performance of newer GPSRs will reach a limiting point and the manufacturers can only add more features. (Don't know if we have reached this point yet. The SIRF Star-III chipset has 200K correlators, there is another chipset with 1M correlators--some of the specs are slightly better, but I haven't seen any good field comparisons between it and a SIRF Star-III based GPS.)

There are also some improvments in the works for the GPS satellites (eg higher power) and a second civilian frequency. There may be soon be two additional systems available (Russian Glonass and European Gallileo). Adding the second frequency and the other satellite systems are all likely to improve performance. But it will be a few years...

Notes:
* Glonass exists right now, but has an inadequate satellite constellation.
* Glonass receivers are significantly more expensive than GPS or Gallileo receivers (the signal structure is different)
* Gallileo will use a similar signal structure to GPS, so dual system receivers should be possilble without excessive difficulty and cost.

BTW:
* Latest firmware for the 60cs: 4.10
* Latest firmware for the 60csx: 3.0.0
* Latest GPSchipset firmware for 60csx: 2.90
* Latest Mapsource: 6.11.6 (you might prefer 6.11.5--see the 6.11.6 release notes)
Available from the http://www.garmin.com or http://www.gpsinformation.org/perry/

Doug

Click to expand...

Thanks again as usual Mr. Information Man! Let's us know if you do another Bakeoff on the Trail sometime.

Here's a test that shows the CSX is better than the CS. Also, it would seem that the Magellan Unit also tested in this report has superior performance.

What is the explanation of drifting for a mile (60csx)? It must be "Walking in circles" rather then out of position by a mile. Would not be much good if you were a mile off position.

Tim

Well the tester is analyzing serial data output from the NMEA interfaces on the all the units. The one mile figure is the accumulation of distance travelled while the units were perfectly still. That is not position.

So the units are accumulating a reading much like an odometer. Thats not important, imho, for hiking.

Remix said:

Well the tester is analyzing serial data output from the NMEA interfaces on the all the units. The one mile figure is the accumulation of distance travelled while the units were perfectly still. That is not position.

So the units are accumulating a reading much like an odometer. Thats not important, imho, for hiking.

Click to expand...

I am ignorant, but I am considering a GPS purchase, and thus I'd like to understand this, especially about the Garmin 60CSX, which has come recommended to me by others on this forum.

My odometer does not advance while my car is sitting in the garage, or at the trailhead. Why would a GPS accumulate distance traveled when in fact it is 'perfectly still'?

My only personal experience with GPS is on a boat owned by a friend of mine. When drifting slowly the heading sometimes swings around, but while tied up a the dock, the track log doesn't lengthen any -- it stays put right under the center of the display. While trolling a pattern, we often use the track log to zig zag over a target area. This all makes sense (we're moving.)

What am I missing (still)?

Tim

Tim, yes your GPS receiver can be stationary - but the "transmitters" that are part of it's system aren't - remember we look at a GPS as being "real smart" - but its just the final piece of a system, based on satellites circling the earth at thousands of miles per hour, and intricate geometrical calculations based on precise clock signals.

We look at out GPS's and don't see movement; but its there in the system, and small errors in this system are interpreted as movement. So generally without extra help, our consumer-grade GPS's lack ability for precision with height - and for extreme stationary precision.

Most of our use of GPS's we're either stopped & acquiring a "fix" to determine where we are, or where we may want to return to - or actively travelling -but not sitting stationary with the odometer running. In any event I've never seen much "distance covered" as my gps sat stationary, and you wouldn't need to have concern when buying your first unbit that this is a problem

I understand how the satellites moving can introduce errors, and that the government occasionally changes the signaling (or used to) to throw off the precision (until differential GPS came along.)

BikeHikeSkiFish said:

It must be "Walking in circles"

Click to expand...

Is my original statement correct? I.e., 1 mile of "accumulated error" around its stationary position?

I'm sure I've read in the GPS threads about shutting the unit off or at least stopping the track log when stopping for lunch. This must be why.... ?

Thanks,
Tim

Remix said:

Here's a test that shows the CSX is better than the CS. Also, it would seem that the Magellan Unit also tested in this report has superior performance.

Click to expand...

The author notes that the Magellan unit appears to average the postion more than the CSx does. This is better for some purposes and worse for others.

It is better if you take just one reading if the GPS has been stationary for a while.

It is worse if you are recording a track. Imagine taking the GPS along a path with a sharp corner or a succession of sharp turns. If you average the position (over the recent past) for each trackpoint, the recorded track will not represent the corners or succession of turns accurately.

All GPSes do some position averaging (typically using Kalman Filters). How much to use seems to be affected by company philosophy--no single amount is optimum for all applications and situations.

Doug

bikehikeskifish said:

What is the explanation of drifting for a mile (60csx)? It must be "Walking in circles" rather then out of position by a mile. Would not be much good if you were a mile off position.

Click to expand...

The output location of all GPSes will wander aroud a bit.

For instance, draw a 10m radius circle. Wander around randomly inside it for a while and mark your position once per second. Now add up the distances beween successive points--if you have recorded 1000 points, this sum can be up to 20000m, yet no point is farther than 10m from the center point. This total "travel" distance has no bearing on the accuracy of the measurements--it is only a measure of how much the reported position wanders between successive output position.

Another viewpoint is that the total wandering distance is related to how fast the indicated position wanders, but not how far the indicated position is from the correct position.

Doug

OK, This all makes sense to me now. Thanks.

Tim

bikehikeskifish said:

My odometer does not advance while my car is sitting in the garage, or at the trailhead. Why would a GPS accumulate distance traveled when in fact it is 'perfectly still'?

Click to expand...

A consumer GPS has no way of knowing that it is "perfectly still".* (Humans define "perfectly still" as not moving with respect to a spot on the earth.) But the GPS is moving relative to the satellites due to the rotation of the earth and the orbital motions of the satellites.
V_earth_rotation = 2075 * cos (latitude) mph (east)
V_satellite = 8647 mph (along the orbtal path)
The GPS location is computed from measured distances between the satellites and the GPS and the satellite locations. A number of factors contribute errors to the measured distances which result in the GPS computed location wandering around in a quasi-random fashion.

My only personal experience with GPS is on a boat owned by a friend of mine. When drifting slowly the heading sometimes swings around, but while tied up a the dock, the track log doesn't lengthen any -- it stays put right under the center of the display. While trolling a pattern, we often use the track log to zig zag over a target area. This all makes sense (we're moving.)

Click to expand...

When the GPS is drifting slowly, the random velocity errors can be larger than the actual velocity. Under these conditions, the GPS can report a reversal of direction.

Some GPSes "guess" that the GPS is stationary when the velocity has been below some threshold for some period of time. The GPS may output no velocity when it thinks the GPS is stationary. Note that the guess can be wrong...

What am I missing (still)?

Click to expand...

Hope this helps...

GPSes are amazing, but complicated little beasts. You may have to understand the details of how they operate to explain some of the quirks in their behavior.

* Survey grade GPSes can be told that they are at a fixed location and to compute the location based upon the total measurement period which may last for many hours. Typically the raw data is recorded and then post processed in a computer. This also allows measured satellite positions to be used rather than the estimated postions which must be used in real-time output devices.

Doug

bikehikeskifish said:

I understand how the satellites moving can introduce errors, and that the government occasionally changes the signaling (or used to) to throw off the precision (until differential GPS came along.)

Click to expand...

The government no longer introduces these intentional errors, by order of Pres Clinton. It is very, very, very unlikely that they will ever be reintroduced.

Is my original statement correct? I.e., 1 mile of "accumulated error" around its stationary position?

Click to expand...

The accumulated error (actually distance between successive points) is meaningless with respect to accuracy. What matters is the average error and the distribution of errors. The standard 10m accuracy means that 95% of the time, the error will be less than 10m.

I'm sure I've read in the GPS threads about shutting the unit off or at least stopping the track log when stopping for lunch. This must be why.... ?

Click to expand...

Some people don't like a little "knot" in the track caused by the wandering when stationary. (A track offset by some distance looks prettier, but is no more accurate.) Turning the GPS off, waiting a while, and turning it back on is likely to introduce a jump in position--again prettier than the knot. Another reason for turning the GPS off during extended stops is to save the batteries.[/QUOTE]

Doug