The only way to really remove patch panels is to locate the edge switches in the cabnet with the servers. If you have 40 servers, then a 48 port switch with 10g upllinks will work well. So the connections would be server -(1000bT)> switch in rack -(10000 Fiber)> core. Make sure that clusters are scattered between cabnets to make sure to limit the shared points of failure and you are set.
Doing this will limit the “underfloor” time, since the only underfloor work will be when a rack is installed.