I would highly suggest separating the two upgrades. First upgrading your Windows Domino server to 8. Then Installing the new server (are you looking to use the same hardware?) Then when your data is on version 8 the migration should be a lot easier.
Never ever do two components of an upgrade at the same time. You will have a really hard time finding your point of failure.
For more detailed information on the upgrades check out the IBM Redbook site here.