Would you please be a bit more specific when you say “gets the access”?
Do you mean that the user name / password combination is refused?
Do you mean that some of the users are never prompted?
Also – POP3 is a plaintext protocol – try sniffing some test cases (successful and unsuccessful) and see if you can see any difference.
Try also looking in the security logs.
Hard to help without more specific information.