Check performance monitor statistics. If that does not show you any issues, put a sniffer (e.g. Wireshark) on the network and see what it shows.
You need to identify is the problem is a network problem, or if the slowdown is within the SQL Server. Based on the fact that you have 6 hard drives and only 4 gigs of RAM I would assume that your disks are being pushed about as hard as they can.
Can you provide more information about your hard disk setup, and your buffer cache hit ratio and buffer cache estimated life time? Both those numbers can be gotten from performance monitor.
Do you have AWE enabled within SQL? Do you have the /3GB and /PAE switches in the boot.ini?