Stored Procedure float input param truncating value

6.6k Views Asked by At

I have a stored procedure that has input params:

  @Latitude float
, @Longitude float

but when I pass these values:

 @Latitude=-28.640328248358344
,@Longitude=153.61249352645871

the values being stored in the table are:

Lat: -28.6403
Lng: 153.612

If I do a standard insert or update on the table the values are correct.

I tried changing the params to float(54) but it made no difference... The reason I am using a float is because the Lat/Lng are actually computed cols from a geography column (and it states in the designer that they are floats) - I have double checked that it's not related to the computed cols by inserting the lat/lng values from the stored procedure into a couple of varchar columns.

I am using EF and can confirm that the values are correct in the Entities and I have checked the SQL Profiler to confirm that the full values are included in the exec call.

EDIT: I insert these values into a column call GeoLocation of type geography:

GeoLocation = geography::STPointFromText('POINT(' + CONVERT(varchar, @Longitude) + ' ' + CONVERT(varchar, @Latitude) + ')',4326)
2

There are 2 best solutions below

0
On BEST ANSWER

When you cast float to varchar you lose accuracy. Use STR to convert to varchar

A comment says "CONVERT(varchar, @Longitude)". This isn't "truncating input" but normal behaviour

0
On

Try this:

DECLARE @Longitude FLOAT = 41.1234567

SELECT CAST(@Longitude AS VARCHAR(24))

What is your result?? In my case, it seems that SQL Server will always truncate to four digits after the decimal point.....

COuld that be the cause of your truncation?? It's really only the conversion to VARCHAR that causes truncation?? Try storing the values in your sproc as FLOAT and see if those values are truncated, too - I'm pretty sure they won't be!