Networking Issue

I have VLAN’s 1 and 2 that are routable. Both VLAN’s are getting IP addresses via DHCP.

DHCP Scope 1 is servicing Building 1 on VLAN 1 – 192.168.1.1-254
DHCP Scope 2 is servicing Building 2 on VLAN 2 – 192.168.2.1-254

Sitting in building 1, on VLAN 1, I can ping addresses on VLAN 2 in both buildings
Sitting in building 1, on VLAN 2, I can ping address on VLAN 2 in building 1.
However, sitting in building 1, on VLAN 2, it take 1-2 minutes to ping addresses on VLAN 2 in building 2.

I am confused at why it would take so long to ping systems on the same VLAN in the other building.