Editing large text (*.CSV) files

[ShotSimon]

I inadvertently created a 3.5Gb file with data using the Write Characters to File.vi and giving it a *.csv extension.  I have seen some info on the forums dealing with files this large but noticed that they point to Open G for the most part.  I was wondering if someone knew how to quickly divide this file up possibly using the DOS "TYPE" command with echo turned off?  This seems to open the file but echoing to the screen takes a really long time.  I tried downloading other text editors with not much luck, EditPad Pro 6 (2GB), and UltraEdit which I forced closed after 10mins.
 
The wild thing is LabVIEW let me create this large file but an't edit it<img height="16 src= Loading Image..." width="16" border="0">
&nbsp;
The data format is as follows:
&nbsp;
1, 21, 2, 9, 16.8652802, 21, 2, 10, 16.834560.
.
.
&nbsp;
Regards,
&nbsp;
-SS

[GerdW]

Hi ShotSimon,I would make a small loop:- read a part of the big file of (let's say) 50MB- write to a uniquely named file using iteration count and format string like "part_%05d.txt"- continue until end of file reached...<img src="../../attachments/ni/170/348411/3/split_file2.png">using "read lines" option to read in approx 20MB (=1 million lines) of the big file. I wonder about the file size, I thought LabView will only create up to 2GB files using standard file functions... This could be a problem when reading that file, too.Message Edited by GerdW on 08-11-2008 09:26 PM


split_file2.png:
Loading Image...

[ShotSimon]

GerdW,
&nbsp;
That solution is working and processing my file correctly.&nbsp; It turns out that I was appending my array incorrectly.
Such as:
&nbsp;
Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink 1, 10, 1, 0, 14.089472Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink 1, 10, 1, 0, 14.0894722, 10, 1, 1, 14.106368Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink 1, 10, 1, 0, 14.0894722, 10, 1, 1, 14.1063683, 10, 1, 2, 14.078720
&nbsp;
It was a 10 day test so I didn't want to lose my data.
&nbsp;
Thanks,
&nbsp;
-SS

[smercurio_fc]

You can also try the free <a href="http://www.filesplitter.org/" target="_blank">FileSplitter</a>.

[ShotSimon]

I'll give the free splitter a try too.&nbsp; Here is the code I created from GerdW *.png.&nbsp; -SS


File Splitter.vi:
http://forums.ni.com/attachments/ni/170/348433/1/File Splitter.vi