According to Microsoft, XP Pro Workgroups become impractical in environments with more than 10 computers. You need to upgrade to a domain.
Apply Microsoft updates. It is possible that there is malicious software running on your network creating these problems. As the previous poster suggests, peer-to-peer XP workgroups for this large of an environment is not recommended. It would be best to implement an Active Directory environment for a network this size. You can also use a sniffer like Microsoft’s Network Monitor or Wireshark to see what the traffic is actually doing on the LAN. Take a look at the client machine’s event logs to see if they can shed any light on this issue. This is likely not a network (switch) problem but a desktop or server configuration issue. Scan the computers using a good, up-to-date antivirus/spyware application and resolve any issues found.