[futurebasic] Re: [FB] Files larger than 2^31 bytes

Message: < previous - next > : Reply : Subscribe : Cleanse
Home   : October 2009 : Group Archive : Group : All Groups

From: Bill Zielenbach <bill@...>
Date: Tue, 13 Oct 2009 08:40:33 -0400
Brian,

Thanks for the suggestion to read a large chunk of the data into memory and
pick it apart record by record from there. This avoids the rather slow line
input # approach I was using for each record.

The records in the file were fixed length, 80 bytes of data followed by a
line feed.

What I tried to do was:

begin record memoryarray
dim  81 records$[4999]   // to give 5000 records in memory (405K bytes,
could be increased)
end record
dim gmemoryarray as memoryarray
dim as SInt64 gmemoryarraypointer,numberofbytes
 
gmemoryarraypointer=varptr(gmemoryarray)
numberofbytes=81*5000

record # fileID,recnum // some starting recnum, incremented by 5000 for each
read
read file # fileid,gmemoryarraypointer,numberofbytes

For I=1 to 5000
A$=gmemoryarray.records$[I-1]
// process A$
Next I

This did not work as expected because I think I gotten bitten by a Pascal
String.
The data in A$ was missing characters that were at the start of the record
in the file.

Instead, I modified the loop section as follows:

dim 81 A$
dim as SInt64 frommemoryarrayloc,secondbyteofA

A$=space$(81) // creates a Pascal String with length byte of 81
secondbyteofA=varptr(A$)+1 // gets the address of the "data" portion of A$
frommemoryarrayloc=gmemoryarraypointer-81

For I=1 to 5000
frommemoryarrayloc=frommemoryarrayloc+81
block move frommemoryarrayloc,secondbyteofA,81 // copies 81 bytes from
memory into the data potion of A$
// process A$
Next I

This works just fine, and the process really flies!

Is there a better way to do this?

Bill