There can be many reasons for SMTP time outs. If this is your email server you can check the SMTPSVC logs and see if the FROM: command is getting an acknowledgment. Is the TO: command being generated?
If this is not your server and you are using a free service like Gmail or Yahoo then chances are the problem is not on your side.
I have also seem SMTP time outs when a specific address is black listed and it hangs the machine so it will not try to send another message to it immediately.
Did these problems just develop or are they going on since inception? Are all the people you can not get to on the same domain? Anything in common? Does the list fluctuate?