Router Spikes? Normal?

I am still trying to figure out our network issue, I have been working with our router trying to see if maybe the problem is in our hardware, i have been watching a normal response from the router, I am getting 1ms from a hardwired, and about 15ms wireless, sometimes this jumps up into the 20s as high as 43 on wired for a few seconds, I cannot figure why its jumping, I tried to add bandwidth to network and that does not seem to be the trigger, any ideas?