[futurebasic] Re: [FB] Significant digits question.

Message: < previous - next > : Reply : Subscribe : Cleanse
Home   : November 2004 : Group Archive : Group : All Groups

From: tedd <tedd@...>
Date: Sat, 27 Nov 2004 20:51:55 -0500
Darron:

It works, but now the real question -- why doesn't single precision work?

I'm not sure if I want to double the memory requirements of my 
program because of an error that's found in a digit four places to 
the right of the decimal. That's mighty small to cause such a big 
difference.

Isn't there a way to truncate a single precision number? I've tried 
multiplying it by 10,000, taking the integer portion and then 
dividing by 10,000 only to end up where I was. It don't make any 
sense to me. It's probably one of those bit thingies.

Thanks.

tedd


>Ha!  Finally answered one. ;)
>
>Try this:
>
>'--- main ---
>
>window 1,"Test",(0,0)-(300,300)
>dim value#
>dim a$
>
>
>a$ = "123.2222"
>print "In = ";a$
>value# = val(a$)
>print
>print "Out = ";value#
>print
>print "But, what I want is 123.2222"
>
>do
>HandleEvents
>until gFBQuit
>
>
>Darron
>
>
>tedd wrote:
>
>>Hi gang:
>>
>>This is a simple problem, but the solution escapes me. Please 
>>review the following code.
>>
>>I don't want to use "using" -- what I want is a variable that has 
>>the value of 123.2222 derived from a string.
>>
>>I've tried using "dim value as fixed", but I can't get that to work 
>>-- it acts like an integer.
>>
>>Thanks to all those who consider.
>>
>>tedd
>>
>>'--- main ---
>>
>>window 1,"Test",(0,0)-(300,300)
>>dim value as single
>>dim a$
>>
>>a$ = "123.2222"
>>print "In = ";a$
>>value = val(a$)
>>print
>>print "Out = ";value
>>print
>>print "But, what I want is 123.2222"
>>
>>do
>>HandleEvents
>>until gFBQuit
>>
>>
>
>--


-- 
--------------------------------------------------------------------------------
http://sperling.com/