If by performance we are talking simply about the speed of execution then gauging the performance can be a simple matter of recording the time taken to execute tests. This can easily be included in the results output from a test execution. You will have an idea of how long it has taken to run the equivalent test in a different automation tool. So its a case of calculating the difference.

You will also need to take into account the initial outlay to design, develop and maintain a stable automation suite. A tool may perform slightly better than another during execution but if its difficult to set up the suite or maintain scripts then it might not be the one to go for.