Is this a broadband or DSL connection ? If broadband, and the drop times seem to be at the same time, it may be network traffic. I have been told there may be peak usage hours, say 7:00pm when most people may jump on their PC’s that you may see random drops or slow speeds.
Another thing to do is run a virus scan and see if you system has been hijacked and is being used as a bot. It may be using bandwidth you are not aware of or it may be getting used for other reasons.