Priority Queueing is only recommended for VoIP because of the way the switch handles the queue. Any frames residing in the queue are given priority over all other traffic in the switch. While the switch is busy processing the Priority Queue all other queues wait.
Imagine now if you tagged several high-utilization applications as Priority, as well. While the switch is busy handling all of this traffic the “non” priority queues are filling up, latency increases, and the potential for dropped packets comes in to play.
You want your switch spending as little time in the Priority Queue as possible, therefore the recommendation that you use it only for VoIP.