I have a framework on a lot of RAM machines that have matte files with a very large and specially designated matrix Produces This matrix is calculated only once and takes a lot of time. Finally it is stored in a mate file on disk.
During the usage phase, this MAT file should be loaded. The problem is that I do not need all the data - only a few selections of columns from that matrix
For example, I have a matrix 'mark' in a file [crtf.mat] size [500x250000] and type double to load only vectors using 'id' from that matrix You may be interested in:
Sign (:, ID)
Is there any way to do this? I searched the web and nobody has expressed the need for such functionality. I'm thinking of writing a MEX function select_mat ():
sign_sub = select_mat (mat_file, var_name, id);
If you really have a large matrix that you want to load only a few parts then I do not save it as a .mat file. Writing the matrix in your binary file will be more efficient then you can use functions such as skip the different indexed points in the file and only read what you need. For example, save a small sample matrix in a binary file using the first function:
& gt; & Gt; M = magic (5)% # is a sample matrix M = 17 24 1 8 15 23 5 7 14 16 4 6 13 20 22 10 12 1 9 21 3 11 18 25 2 9 & gt; & Gt; Fid = fopen ('bigmatrix.dat', 'w'); Open file to write% # & gt; & Gt; Fwrite (fid, size (m), 'uint8', 'l'); % # Matrix size (required later)% 2 as unsigned 8-bit (1-byte) integers & gt; & Gt; Fwrite (fid, m, 'uint8', 'l'); % # Matrix data is unsigned 8-bit% # (1-byte) integer & gt; & Gt; Fclose (fid); File% / shutdown Now, we can just use the functions to read the third column and:
& gt; & Gt; Coal index = 3; & Gt; & Gt; Fid = fopen ('bigmatrix.dat', 'r'); Open file to read% #% & Gt; Size M = fread (feed, 2, 'UIT 8', 'L'); To get the #% size of the matrix in the #% file, first read two bytes & gt; & Gt; Fseek (fid, sizeM (1) * (colIndex -1), 0); Two% of the value of% # bytes # next to the amount of the column & gt; & Gt; ColData = fread (fid, sizeM (1), 'uint8', 'l'); % # Read column 3 data & gt; & Gt; Fclose (fid); % # Close the file & gt; & Gt; Disp (colData)% # Confirm that the correct column 1 7 13 19 25 was read This is just a simple example. Perhaps you would like to write other information in the file (i.e.) such as matrix In the byte size or the data type of each value. It may seem like more work than just dumping things into the MAT file, and it is, but if the efficiency of file IO operations is a big concern, then in this case your own file format to handle your data Better in making
Comments
Post a Comment