If the file is large enough to fill the pipe, or there is enough other trafffic passing across the link, then the latency will go up because the ping has to fight for bandwidth on the line. I’ve seen this on leased lines from the US to China as well.
Another thing to check is to see how many “hops” the packets go across between the two networks with a tracert (or traceroute depending on your OS) from one side to the other side. (I know you said leased line, but sometimes even leased lines have hops as do Frame Relay circuits)
There is a chance that it’s due to a bad port on either router. We had the same kind of problem at our office network and it turned out to be a bad port. The problem is that nothing on the router showed that the port was bad. We had to find the problem do to process of elimination.