What is the OS that the single machine is running? Have you turned on the QOS service on that machine? Does that machine support QOS? Mis-configurations can cause the type of problems you are experiencing. Like if you have the wrong IP in there or that one machine is sucking up all your upstream bandwidth.
What is your switch as well? Does it support QOS? Is any of your equipment old?
Get back to me with some more information!