We're converting some old master files in to new format files. These master files have around 58 million records each. We wrote COBOL programs to convert the format and write to the new files. In order to speed up the process we are splitting the each master file into 20 parts(i.e., 20 files and spreading the records in to fileslike 1 to 20000 to FILE1, 20001 to 40000 to FILE2, etc.) and running the same program to populate the records in the new format in the NEWFILE, in multiple jobs, after the necessary overrides. It is taking about 5 hours to complete all these jobs, even after using 85% of the CPU.
I would like to further speed up the process. Can someone help/guide me how I can improve the performance? What is the best approach for dealing with such conversions? We need to convert 4 such files. Any help is appreciated.