It depends on what brought the auditors in and why they're there. Is this simply a contracted service whereby you want to learn weaknesses that some auditor might find? Or is this a required audit due to regulatory or other compliance factors?
If this is simply a choice that's intended purely to improve your business practices, then you give access to whatever you feel giving access to. But if this is a regulatory or similar audit, give access to everything your lawyer says to give access to.
Should we be worried about auditing the auditors?
If you didn't verify their professional qualifications first, then yes.
I don’t know that I would necessarily be TOO worried. Who brought them in? If they were brought in by IT mgmt., then it is most likely to identify & address weaknesses in current procedures & how things can be done more efficiently. However, if they’re being brought in by upper mgmt., it COULD be (& this could just be me being paranoid) that they’re looking to make changes in IT World… possibly looking outside. I only say this because I’ve seen it happen before.
I would say give them access only to what you have to. Don’t give them more than they need. It’s better that they have to ask you for something than that you’ve given them more than they have to have (i.e. passwords, etc.).