Sunday, February 19, 2012

Calculation problems

Hi,
Im having a strange problem. Im trying to calculate a percent and store it in the database. The column type is float. Here is my code:

Insert into QuoteLineData
(GM) values ((::price::-::opcost::)/::price::))

Where the price and opcost are parameters in a WEB page.
No mater what I do, it will calculate to 0 or 1, rounding the number up or down.

How can I keep the precision to get a fraction in the float field?

ThanksYour sample code is missing a parenthesis after price-cost, it's probably a typo, but could generate some strange results.

Try inserting 12.34 into the table to make sure the column will accept decimal points.

If that works, try displaying the results of the calculation to make sure decimals are being generated.

If both of those work, try calculating into an intermediate variable and then inserting the intermediate variable into the table.

Good Luck,|||OK,
This is very weird.
The values are coming from MS Front-page.
If I enter an integer for the values the result will be 0 or 1 it will round up or down.
However, if I enter a . After any one of the integers, the calculation is correct and it will give the value with a 6 decimal point precision.

Any ideas what is happening and how I can bypass this problem?

Thanks|||SQL Server has a way of converting results to the lowest common denominator, in this case an int. by adding a "." you have converted the values to non-integer.

Can you cast your value from your web page to a real?|||That worked! Thanks - had to multiplay by 1.0
Simple as that

Thanks

No comments:

Post a Comment