GTS format / Warpscript : Best practices to provide LAT and LON values?

Question

Some questions regarding the GTS format described here :

TS/LAT:LON/ELEV NAME{LABELS} VALUE

• Does it make sense to provide a LAT and LON values like 48.441583584062755 ? or is it more convenient to provide only 6 digits precision like 48.441583 ? What are the pros/cons ?

• If the stored value is 48.441583584062755, is there any existing warpscript function allowing to convert the LAT value to get it with a 6 digits precision ?

Thanks for your help

Answer 1

Internally, locations in Warp 10 are stored as HHCodes in a Long (64 bits). So latitude and longitude are each stored on 32 bits. This means HHcodes in Warp 10 have a maximum error of:

• 180 / 2^32 = 4.190951585769653E-8 degrees of latitude
• 360 / 2^32 = 8.381903171539307E-8 degrees of longitude

To answer your question, 8 decimal digits should be the maximum. As far as I know, there is no real advantage of further reducing the number of decimal digits: it will very slightly reduce the needed bandwidth and the time to parse the data but at the cost of precision.

I'm not aware of any function to restrict the precision of a Double, so you have to do that yourself:

48.441583584062755 10 6 ** * ROUND 10.0 6 ** /

If you use that often, consider creating a macro:

<%
  'precision' STORE
  10 $precision ** * ROUND 10.0 $precision ** /
%>
'reduce_precision' STORE

48.441583584062755 6 @reduce_precision