Question

  • Creator
    Topic
  • #2171991

    What can I do to speed up the copying of large files?

    Locked

    by dennis.rhine ·

    We are in the process of virtualizing many of the servers we use at the office and have encountered and issue that I didn’t expect and cannot see a reason for. When backing up, or copying large files (over 1GB) the process goes so slow that it practically cannot be done. For example, I copied a couple of smaller files – about 2MB each – in a minute or so each. However, when I tried to copy a 3GB file it took several hours and then was only partially done. The impact on the network is that a backup that should take a few hours has exploded to over 60 hours.

    Has anyone else experenced this? If so, what is the best way to fix it?

All Answers

  • Author
    Replies
    • #2897799

      Clarifications

      by dennis.rhine ·

      In reply to What can I do to speed up the copying of large files?

      Clarifications

    • #2897785

      Are you copying volume-to-volume or to another server?

      by robo_dev ·

      In reply to What can I do to speed up the copying of large files?

      I would guess you’re talking about VMware ESX 4?

      I believe that the free ESXi has some limits on throughput…around 6MBPS.

      What are your approximate measured transfer rates?

      There are some great products from Veeam, FastSCP. (free versions too)

      Note that using plain old unsecure FTP is typically ten times faster than SCP.

      (Actually, I am pretty sure that FastSCP is actually FTP by another name)

      Of course, to/from another server puts great demands on the network, and VMware can be tricky to configure

    • #2897778

      try changing the nic card settings

      by markp24 ·

      In reply to What can I do to speed up the copying of large files?

      Hi

      I have seen this on non vm desktops coping larges data files(ie pst files) to a network server, there nics were set to auto/auto so was the cisco switch,

      once i changed both the nic and the swithc port to 100/full duplex , everything copied much faster! You may want to try this on the host EXI (set it to 100 or gbit/ full duplex and the switch port, then see if that resovles it.

    • #2897733

      Too Narrow network bandwidth

      by chdchan ·

      In reply to What can I do to speed up the copying of large files?

      I think there might be issues with network traffic or bandwidth that bogged down your speed. Even the commonest broadband router should deliver shorter time for file copy than yours. Perhaps, simply use USB drives/Dropbox (data transit) or shared folders (avoid duplication) to work around.

Viewing 3 reply threads