Let me see if I understand your question. You have a 2003 Small Business Server, is this correct? Do you have a static public IP set for your exchange to use? Did you set up the server using the configuration wizards? What do you mean when you state “I removed any restrictions and started loggind the incoming mail” what exactly did you do? When you refer to “They all come in with the pop connector”, do you mean that you are use a pop connector to get your email from another mail server?
With out additional information, I can only guess at solutions.
If this server is for a business – seek assistance from a local Microsoft certified Small Business Specialist with experience in SBS 2003. As a minimum you should get a static IP; register a domain name, have your ISP setup an MX and A record pointing to your ‘mail.mycompany.com’
Reply with additional information and we can attempt to assist you further.