Is it possible for you to generate a separate text file from Crystal Reports directly? This way you can avoid the middle men. Because this file is purely for data processing, you can format it the way for easy importing to the AS400.
The only reasonable suggestion is to stop trying to convert a PDF back to simple text. Unless you have a converter that is explicitly designed to reverse-engineer in the same way that the PDF procedure generated the PDF, it’s not going to be reliable. There are many “PDF creators” available and they don’t all do it the same way. Some are pretty bad at making the internals of the PDF file look good.