Is this on an AS/400? If so which version of the OS are you using? We have V5R4 and that seems to allow 2.1 billion records (2,147,483,646).
For V5R4 — “Number of records contained in a file member – 4 294 967 294 records”. See Database file sizes. But a PF may have 32767 members and total number of bytes in a file is approx 1.8 TB.
If any system is reaching a limit at 999,999,999 records, it may be from other limits. Is the size limited by the file description? Is the file indexed? If so, what are the key characteristics? What is the record size? Is the owner maxed out on space?
Most importantly, what message ID is sent when the limit is passed?