I work at a major public academic health sciences center and have faced a similar challenge. We set up separate environments for development, test, quality assurance, production, and in some cases, reporting. It may seem like a lot of environments, but each one serves a critical purpose. Development is used to develop new applications and as an initial location to install a new commercial application and become familiar with the product and the technology infrastructure. Test is used by the development team to perform integration/system level testing/debugging. Quality Assurance is used by business analysts and end-users for user acceptance testing. Production is production. And, Reporting is used for ad-hoc reporting that would otherwise slow down the Production database. In some cases we host the application and database for an environment on the same machine when resources are tight, but otherwise we have a separate application and database server per environment. We have co-hosted two environments, like development and test, on the same servers when resources are tight too. The key is to provide separate environments to isolate development/testing activities and to allow separation of duties so that only developers can access development and test, business analysts and end-users can access quality assurance, production, & reporting, and change management personnel can access all environments to implement code and data changes as needed. I hope that this answers your questions.
I would recommend that you create a ‘standard’ test system for each environment (this will probably create some overlap), but you can use this test system for future changes. Also, you will not need to retain ‘experts’ for this system, since your test cases and resullts should be the same each time. You can always develop new test cases when new functionality is added.
When you are ready to test changes; load your current test system, compare the results with the previous test system (they should be the same, since no changes were introduced); implement your current set of changes; run the test system (adding additional test cases for new functionality) and compare with previous test system output. This is a relatively simple process to ensure only those functions that were modified have been changed.
I prefer this technique, rather than copying every db 10th record and turning this into our ‘test’ system. Using the above example, I have a high confidence level that the changes implement will be adequately tested, and we will not have screwed up something else.