My first guess would be a corruption of something. Did anything happen when the problem first started (after an update or a new rpm install or something)?
Have you attempted to monitor any performance statistics while the problem is happening? Using the console start monitoring and then try to SSH in from a remote box?
If it was networking, in my experience anyway, it is probably speed/duplex mismatch or something causing a lot of errors (bad network cable, switch port partial failures, etc). I would verify the speed and duplex and then if possible check the error counter on the switch port. You may also try a brand new network cable into a different switch. If there is a physical problem outside this box, this solution would eliminate those instantly. You can then troubleshoot in order to find the specific problem.
If the problem persists despite a new network cable and a totally different switch, then try a detail hardware diagnostic. Most brand name servers and workstations have some kind of hardware testing utility you boot up to and test everything.
If all that fails, I would go back to my original guess of a software corruption of some sort. You can try repair utilities, a restore to a known good time, or reinstalls over the existing but in my experience, the most reliable answer is a fresh build somewhere else and then a transfer of essential files only.
Hope this helps.