You need to differentiate between bit latency and packet latency.
Bit latency is impacted by distance traveled, the media used, and to a small degree the communication devices. Distance is most apparent with satellite - adding 40,000 miles to the distance adds almost a quarter second of bit delay, half second on round trips. Media used is most apparent with fiber versus copper segments. Here we are looking at speed of light versus speed of electrical pulse over copper.
The devices in the path will add at least 4 clock times at whatever the internal clock of the device is. It is negligible when looking at total performance.
Bit latency is constant for a given path between two point.
Packet latency is reflected in the processing time of communications, congestion, and bandwidth. The processing time involves the protocol used, the routing methods used, and the efficiency of the device (clock, OS, memory). This is all constant for a given path. Most routers will add between 2 and 10 msec of latency each. Switches add no more than 5 msec, and a cut through switch will add less. Bridges add about 1 msec. Added functions like stateful inspections at a firewall and packet compression add to the latency.
The impact of bandwidth on latency to follow.