Google

Steering around the potholes with Google Drive

Read one user's difficulties with Google's cloud storage/file synchronization service and how these problems were addressed.

I have about 30GB of files which I recently decided to synchronize across the four computers I use (all running 32-bit Windows 7). Because I've gone almost completely paperless, I scan all the documents I can (better yet, I receive as many as possible in digital format, such as magazines and bank statements) and store them electronically. This consists of the following top-level folders on my D: drive.

  • Books - E-books, technical PDFs, white papers, and other recreational/career documents
  • Data - Financial data, receipts, and memorabilia. Most importantly, my "Work" folder which contains all my writing including the 580 page crime novel I spent almost a year on and which is now undergoing final revision.
  • Installs - Install files such as Office, Adobe and VPN products
  • My Pictures - My digital camera photos for the current calendar year (I archive the prior year's photos to a separate disk)

All told, this consists of almost 11,000 files in over 700 subfolders. It's a lot to manage, but it allows me to manage my job and my life from anywhere, since I'm not dependent on file cabinets or printed manuals. I'd probably need a moving van to haul around the hard copies of all this information!

I had used the free versions of Dropbox (3GB of space) and Google Drive (5GB of space) for some time, but I decided to pay for enough storage with one of these providers so I could keep all my files automatically updated on my systems. Previously I just kept my "Work" folder (about 1GB) in Dropbox and part of my "Data" folder in Google Drive, but I elected to integrate "Work" underneath this "Data" folder and go from there. My goal was one storage repository for everything; no hassles, manual updates or having to go here or there for what I needed.

Choices

It was a tough choice, since the one thing that always impressed me about Dropbox has been its rock-solid reliability. However, the factors that led me to choose Drive over Dropbox involved the following:

  1. Cost ($5 per month for 100GB of Drive space versus $10 per month for the same amount of Dropbox space)
  2. Integration with Google services, as I discussed a few months back
  3. The excellent support I've seen from Google in the past

As the saying goes, little did I know how soon I would be calling on that support.

Setting up a paid Google Drive storage plan was easy; I accessed my Drive account at drive.google.com then clicked "Upgrade storage" in the lower left of my Chrome window. (Figure A)

Figure A

I clicked "Choose this plan" under the 100GB option and after entering my payment information I was the proud owner of some hefty new Drive space. I got this accomplished before breakfast, so I dragged and dropped the remainder of my 30GB of files into the Google Drive folder on my PC, which is located on my D: drive.

Then I observed the Google Drive icon in my system tray. (Figure B)

Figure B

This icon shimmers when Drive is synchronizing files and shows as solid when the synchronization is complete. As I left for work the Drive icon was shimmering, so I figured it would upload my data to Drive and then copy it down to my work PC, which also has Drive installed on it.

The first sign of trouble

I got to work to find that only about 250MB of my data had synchronized from my home computer to my work system, within about forty minutes. That meant it would take over 2.5 hours per GB - more than three days to copy 30GB of data - to complete if the throughput remained the same.

Since I didn't want to use too much bandwidth at the office I came up with a clever solution (so I thought) - I would copy the files from my USB drive into my Google Drive folder on my work PC. Since most of these files were the exact same versions as the ones I was trying to copy down from the Internet, I figured Drive would recognize there was no need to sync these. This would, theoretically speaking, shave days off the synchronization process. From there on out it would be smooth sailing since only a small amount of my data is "dynamic" meaning that it changes frequently.

In a word: wrong!

As I said before, my four main folders are called Books, Data, Installs, and My Pictures. Not long after I copied all my files into my work PC's Google Drive folder, I checked it and found Figure C.

Figure C

Uh, why is there an extra folder called "Installs (1)"?

You can probably guess what happened - duplicate folders were starting to show up. Worse, duplicate files began streaming onto the hard drive - these appeared with (1), (2) or even (3) after the name.

Everything became a game of whack-a-mole after that. I spent a lot of time comparing my online Drive folders with those on my local systems and removing the duplicate objects, but they respawned as soon as I deleted them. I restarted the Drive application a few times to try to get sync working properly. Then I elected to start over; I moved all the data out of my Google Drive folders, deduplicated it using "Free Duplicate File Finder," and deleted it from my online Drive space.

I checked Google Drive help page but it didn't really seem to apply to my situation, so I contacted Google support. I've always had prompt service and good results from Google support, and so in short order I discussed the issue with one of their engineers. He advised me to move my Google Drive folder on my home PC from my D: drive to the same volume where the program was installed (in this case the C: drive). Explaining that sync problems can occur if the Drive repository is on a different volume, he also recommended slowly re-adding my files to my Google Drive folder so as to not choke it with 30GB of data at once.

I gave this a try, spending the weekend carefully feeding my data into my Google Drive folder. Slowly the folder and files began showing up in my online Drive account, and when all seemed to be uploaded I checked my work PC and things looked good. I did have to restart the Drive app a few times since the sync process appeared to hang (it would say it had completed 39/2054 files for instance, but then go no further), but finally it reported "Sync Complete" on my home and work PC. Those red X's next to the folders in the screenshot above (which indicate not all files have synchronized) went away. Situation normal, I presumed.

The return of the villain

You know those horror movies where the bad guy is supposed to be dead and the clueless kids poke him with their foot to see if he's still breathing? Yeah, something like that happened here.

I got up the next morning and tried to open a shortcut to a crucial file in my "Work" folder, which as I mentioned is a subfolder of "Data." The shortcut didn't open.

"How weird," I thought, and opened Windows explorer to examine the contents of "Data." My Work folder had completely disappeared. Cue the scary bass music.

I have a nightly scheduled task which copies all my files onto an external drive for backup purposes. This is different from synchronizing the files, since it never removes anything from the target location. I do this so that if something is accidentally deleted from my hard drive the unwanted removal won't replicate to my external drive, because that would defeat my backup scheme. I can also get stuff I erased then changed my mind about deleting, such as drivers that I might want to reinstall.

I was able to recover my files from backup, and I observed that my online Drive account did show the "Work" subfolder, so not all was lost. I copied the "Work" folder back under "Data." After a while the Google Drive app reported "Sync complete." The "Work" folder never appeared on my office PC, despite the fact Google Drive also reported "Sync complete" on that computer.

Since the Work folder is so critical to me I moved it to my Dropbox account until I could sort things out. What makes me nervous here is not losing stuff I use every day (since I'll notice the absence right away), but missing files which I might not realize are gone for several days or even weeks.

I left the other folders as is. Then I recorded the size and number of items in my top-level folders to compare these between my two systems and track how Google Drive synchronizes items. I've been slowly adding and removing test files to my Google Drive folders and checking to see if these changes are properly replicated. So far all is working as expected, though I've probably changed 25 or less items.

The jury is still out

I haven't given up on using Google Drive just yet, though I have to admit that I'm not willing to take any risks with the data I rely on. I'm awaiting further advice from support on how to properly troubleshoot this problem. There is a sync log for Google Drive, which on Windows 7 systems is located under %LOCALAPPDATA%\Google\Drive\sync_log.log. My log file is 88MB and thus far I've found no explanation in it for the removal of the "Work" folder. There's too much data in there to wander through it without a guide. A Google product forum discusses the issue which shows I'm not alone.

There are a few good takeaways from this experience

Adding 30GB of data to my Google Drive folder all at once was an obvious mistake; lesson learned. It was like trying to drive a school bus down the sidewalk. I'd previously used 4GB of data with Google Drive and had no discernible problems.

I have a ton of data. It's not just the 30GB involved, but the 11,000 files as well. I'm something of a digital packrat, I admit. I have utilities dating back to Windows 95. I think part of the problem here involved the sheer amount of objects. Zipping up many of the older items I don't often access would probably help ensure smoother synchronization, since there would be less material to comb through when analyzing changes.

This problem may not be native to Google Drive. I'm currently experimenting with 50GB of free space from Tresorit and the results have been sluggish as well with such a high data volume, though I've not yet found any discrepancies. It would be interesting to see if similar experiences also occur with Dropbox.

The Google Drive app seems to have performance issues when it's busy for too long (at least in my experience) - having to restart it periodically obviously didn't help with the synchronization process. Again, this is possibly tied into too much data flowing at once.

It would have been a good idea NOT to have the Google Drive program running on another system until all my files were successfully synchronized from my home PC to Google servers. I'm sure this added to the confusion since the work PC was also trying to pull down the changes.

Always have multiple copies of your data, and DON'T synchronize everything to another location; use a standard copy method so you can get back any files erroneously deleted. Schedule the backups so they don't kick off as soon as changes are made, but rather run periodically so you have time to recover.

I'll update you on this problem in the Comments section of this article once I find out more - and if you've seen similar issues please let me know.

Also read:

About

Scott Matteson is a senior systems administrator and freelance technical writer who also performs consulting work for small organizations. He resides in the Greater Boston area with his wife and three children.

25 comments
john
john

So far all the comments seem to be from Windows users - are any Mac users having problems? I have been editing files in Chrome for years and never seen any issues or discrepancies with the sync'ed Google Drive folder  on my Mac.

sdjcs
sdjcs

Figured as a long-time Google Drive user and having previously posted here, that I should post a follow-up. After two consecutive days of Google Drive crashing minutes after starting it  - it basically won't stay running on my system, I'm moving off to SkyDrive. This is not the first time this has happened to me with Google Drive. The fix is to disconnect my storage, rename the Google Drive folder, then reconnect which creates an empty Google Drive folder, and finally copy all my files back. I don't know if SkyDrive is any better, but Google Drive's reliability problems and lack of support make it inappropriate for my business and that of my customers to whom I might otherwise recommend it.  I would urge anyone interested in cloud storage to look at other solutions before Google Drive.

terpua
terpua

Insync fixes the issues you have encountered, except for the "usb" issue but that too is coming. https://insynchq.com I'm one of the co-founders.

CMM Concepts
CMM Concepts

My team is all virtual, so we're looking for a solution that syncs our creative files across a half dozen or so local machines. We tried DropBox and Box for this, but neither worked out for different reasons. (DropBox created even more "conflicting" files than Google and Box is too expensive for a small business owner.) We all installed Drive, thinking that whatever we did on our local machines would then be mirrored to the cloud and to everyone else's files. What a mess. The only time Drive seems to work correctly is if you add an entirely new file. Then, yes, it does show up almost instantly on everyone else's HD. If, however, you move a file, rename a file, delete a file... none of that carries over. The next time someone syncs, the same file with the original name reappears, so you now have two identical files with only different names. When I move a file, I wind up with two copies... one in the new location and one synced back to the original location, I would imagine from someone else's local files. We also wound up with as many as five of the same (large) folders with (1) (2)... (5) appended to each. As soon as someone comes up with an affordable small business solution that syncs everything based on the very last time stamp action - even if it's a move or rename - I'll be all over it. Until then, I have no idea what we're going to do...

Slayer_
Slayer_

I don't use sync since that would be massive for the tablet. But the only real issue I have is the google drive app is friggin stupid. It always tries to open everything as if it was a word document or something. So you have to drill down to the file, tell it to open with your browser, then click it twice, choose where to download the file with your browser (Opera classic only, Chrome gives you no choice where the file goes). Now if it goes to your downloads folder, you need a file management app. I am using ES File manager, to move it from the downloads folder to the folder you want it in. Why can't I just click off the files/folders I want in the drive app, and choose to download them to the folder I want? I am using the drive to store PS1 disc images so I can swap what games I have on the go.

lens20v
lens20v

I use both of these, both freebie versions. For the light load that I give each, it works fine. I upload only those files that I know that I might need to access while away from the desk, not everything. Too many of the desk pc files are rarely accessed anyway, so why bother? Friends like dropbox because of the way they reward you with more space every time a new user joins. In my opinion both work well for light weight users like me. Also I do not upload anything that is sensitive security-wise, I just upload my shareable stuff..

mholzworth
mholzworth

I have been having these same types of issues trying to sync 100GB to my bitcasa unlimited storage size account so it seems to be something a few of these vendors have issues with. I tried the whole glob at once with no luck and have had to break it down further and further and double checking fiule numbers when adding in new files ot make sure nothing is lost in the initial upload which bitcasa seems to like to do...

flyboyanderson
flyboyanderson

We attempted to load 150,000 documents to the Enterprise Edition of Google Drive. It worked but was unstable. We are now using SyncDocs and are results appear much better. But, we are slowly methodically adding the docs and measuring performance along the way. In the interest of full disclosure we were sharing these docs with 300 users which is a whole other can of worms

pkass
pkass

I had also some weird experiences ... Since then I use Google Drive only as temporary storage, always by COPYing data (e.g. private Videos i want to share with only few Person, not the whole world) to Google drive. Or when travelling, I copy all important documents and Infos there to have it accessible on my Smartphone during a trip.

sm
sm

Using Google Drive for half a year I have often seen filename(1) and disappearing files. Never on Dropbox which fully confirms the picture of GD as half-baked. One reason for (1)-files is taking down a computer not noting that GD is not fully synchronized. This happens often when leaving office with your laptop. GD has no buttom to force a synchronization (although you may pause and restart as a work-around) and GD has no mechanism to prepare itself for a part-synchronization. Dropbox never caused similar hassles.

brewerwt
brewerwt

Hi Scott, I went through a very similar experience with Google Drive, except that I was synching 220GB of data across three machines. The biggest thing that saved me was the existence of a fantastic tool called "BeyondCompare." BeyondCompare is a file comparison/synching tool that made the impossible possible. The trick is to use BeyondCompare with a reference backup stored on an outboard passport drive. With that arrangement I can check the integrity of the data stored on all three of my machines within minutes. If Google Drive hiccups and creates a duplicate file, I can find/fix it within seconds. I've even done bit-level compares on all 220GB (500,000 files) just to make sure Google doesn't throw in/out some bits here and there. So far, I have not found single corrupted file after two years of use. Without BeyondCompare I probably would have tossed Google Drive due to lack of trust. With it, I am confident Drive is working and I can prove it. And while I'm at it, here are two other useful tips. 1. NEVER DISCONNECT YOUR GOOGLE DRIVE FROM THE REMOTE SERVER EXCEPT AS A LAST RESORT. I did that on a whim and had to re-synch 220GB of data to restore what I began with. Once Google Drive is disconnected from its server, the folder becomes "radioactive" and the server will have nothing to do with it. It demands a fresh, clean folder. 2. When Google Drive creates a duplicate, delete the original file and remove the parenthetical increment on the duplicate. If you delete the duplicate, Google Drive will just create it again. The only way to stop that is to substitute the duplicate for the original as described. Best Regards, billb historeo.com

dss2
dss2

While I have been a heavy user of Google applications (gmail, calendar, groups, blogger, etc.) since Google evolved from Search to Apps, and I can appreciate many of their services, I have recently become disenchanted due to a similar challenge with Drive (Docs), and Google as a service company. It seems my account was frozen by Google, without ANY notice. As a teacher, using most of Google services heavily for communication and collaboration with my students, this was a major hassle!. I attempted every resolution posted, which was a continuous loop. No phone numbers were available. The communication was zero. It took the assistance of the Better Business Bureau to contact Google, and after more than a month, Google provided me access again, without ANY reasoning. My Drive still is all out of whack (the duplicate file finding is a big job in and of itself). I do not trust Google anymore, and have now looked for alternatives to anything I use Google. This experience has made me extremely angry, due to the lack of communication, arrogance in not providing any reasoning or purpose in their actions, and leaving me with work to do in managing the content I put into Google and it's services over many years. Google SUCKS!

sdjcs
sdjcs

I have used a free 5GB Google Drive account for about a year now to store roughly 2.5GB of files. It has mostly been a good solution for me, and I like the fact that it runs on a server OS. I know DropBox does but as the author points out you get less free storage with DB (I think its 2GB), and I think it is less flexible in where you place the DropBox folder. However, I have had glitches, probably a dozen or more, where files will sometimes fail to sync and the only way to correct the problem has been to pull the offending files out of Google Drive, wait for the sync to complete across several computers, then add the files back in. In two cases I had completely empty the Google Drive folder, reinstall the app, then add the data back. As IT consultant, customers ask me about "cloud" file sync and because of my Google Drive experiences, I suggest DropBox to them because I have found that DropBox seems to be more reliable and easier to fix when a sync has failed (for example, I've never had to empty my DropBox folder before). I'm able to troubleshoot and fix my Drive issues and changing means I have to touch about 20 machines. But I need reliability so I'm thinking I may need to make a switch to DropBox, even if I have to bite the bullet and buy storage.

jon
jon

I have been in a similar paperless scenario for several years - storing files on my local drive and on 2 separate NAS boxes for backup. I used to have Google Desktop Search on my PC and there was a plugin for GDS that would OCR any scanned images. This worked great because all I had to do was scan the file in and GDS would OCR and index it in the background. I now have to make all my PDF's that I scan in Searchable by OCRing them before I store them. Ideally the files would not be stored on my PC on the NAS but windows 7 is not capable of indexing using Desktop Search a NAS drive. Ideally for similar reasons to you I would like to move them to the Cloud However I found programs that index the filenames on Dropbox but not the content. Google Drive does a great job of the indexing and I was thinking about using Google Drive instead of Dropbox but having read your article I'm rather dubious now. Anyone got any good suggestions for indexing content on Dropbox - Thanks

gak
gak

... using it to sync between 2 PCs and 2 MACs. This happens because I know it is a toy and use it accordingly. It originated in the World Of Toys - smart phones and tablets. How it could be different? In addition to what the author have experienced, I also noticed that sometimes the file is not synchronized when its contents has been changed (smart phones to weak to compute hashes?), there is no way to make it to reconnect other than close and restart (smart phones provide more notifications than desktops?), and it may offer to logout and login as the the only solution, which requires all files to be downloaded from the cloud anew (assumed that everybody understands it is a toy?). What troubles me is that the author considers the inability to handle any amount of files, including just 30 GB, to be an OK feature. You know, programs are not buses or cars. The (THE!!!) fundamental property of a computer is that it does not care about volume. It makes 1 trillion of iterations or 10 iterations equally easily. It processes a stream of bytes one million petabytes long as easily as a stream 10 bytes long. Big things just take longer. This is why computers have been invented - if you are able to predict waves in a bowl, you can predict them in an ocean, if you know how to service one customer, you can service the mankind. Lazy, silly or mismanaged programmers may intentionally add limitations, though. Consider the 30 GB failure to be OK, and you will be fed with more junk.

Jimmy5208
Jimmy5208

Thank you for pointing out an interesting service from Tresorit. The link you posted had me excited for a minute, however, the deal for 50 GB expired on May 20. Still downloaded the program. Another 5 free GB of storage never hurt anybody.

YehudahGriffin
YehudahGriffin

...I won't use or trust Google with stuff like that. Good luck.

smmatteson
smmatteson

As I mentioned in my article, I'm providing an update on the current status of this issue in order to educate the readers. To date I have not found a resolution to the synchronization issues described above, nor has Google support been able to assist further. I recently moved all my data out of my Google Drive folder, then added three small Word documents to the top-level location. Last night I created a subfolder in Drive and moved these documents into it using my browser. This morning I checked my PC, which has the Google Drive app installed and is supposedly synchronizing data with my Google account. The new folder was present, but it only contained one of the three files. The other two were still in the top-level folder and had a (1) after their names. In short, synchronization of even three files and their respective locations failed to complete. This disturbing trend has led me to conclude that Google Drive is too unreliable for me to entrust with ANY files of significance. Subsequently, I canceled my paid subscription (I must commend Google in the ease with which I was able to complete that process; I did not have to call up and endure a harangue from a customer service rep) and am now relying largely on Dropbox now for my file synchronization needs. The simple fact is that I absolutely believe any files I put in Dropbox will faithfully appear across all my Dropbox-linked PCs and devices. I have no such faith in Drive. Google has their work cut out for them, improving this product and gaining (or regaining) the trust of their users. While I find many of their other services to be flawless in reliability, Drive is not one of them. Dropbox is twice as expensive as Drive for a reason. I hope this is something they can address and resolve properly, as the concept is promising, but the execution has failed, at least in terms of my own experience.

Michael Kassner
Michael Kassner

I see you haven't tried to edit a file in Drive using a mobile device. Google will only allow you to edit files that are Google App formated. If you try to edit a word.docx it will not allow you to save it as such. I'd love to learn about a work around if there is one.

shaneholloman
shaneholloman

@john  Terrible performance here... os x mavericks 10.9.2. I'm for 1TB of storage. I simply can get a fraction of directories to sync. Google Drive is pretty  sucking for me here as well. FYI, I'm a web developer and need to sync 100,000+ files. I have better luck using Rsync to a VPS rather than using GD. Yeah I'm bummed out because I love the functionality if it actually would sync!

shaneholloman
shaneholloman

@john  yes! Terrible performance here... os x mavericks 10.9.2. I'm signed up for 1TB of storage. I simply can't get even a fraction of directories to sync. Google Drive is pretty much  sucking for me here as well. FYI, I'm a web developer and need to sync 100,000+ files. I have better luck using Rsync to a VPS rather than using GD. Yeah I'm bummed out because I love the functionality if it actually would sync!

arielmon
arielmon

If you want to edit a word, excel or powerpoint document in the web use Office Web Apps (Office 365) Google Drive is for drive documents only. You can store the Office documents in drive and see them but cannot edit them I guess for some copyright restrictions of the Office Documents (.docx, .xlsx, etc)

andybrucenet
andybrucenet

@arielmon  Copyright restrictions on editing a link from your desktop? That is simply not accurate. What is being described (only store Apps documents in GD) sounds like intentional hobbling to promote the vendor's other platforms. Just imagine if Windows Explorer did the same thing: "You can *store* non-Office documents in the Windows file system, but we won't allow you to edit them." That would be ridiculous, and it is just as ridiculous if that is what GD does.

robstewart.nz
robstewart.nz

@andybrucenet @arielmon Hilariously, that's exactly what Windows does do. If you were to download a PDF form onto a Windows computer, but you didn't have a PDF reader on your machine, Windows would quite happily let you store the file without being able to edit it.


The limitation here is not what Google/Microsoft does and doesn't 'allow' you to do, its that there's no compatibility between the software that uses the files. If you install QuickOffice onto Android (in the same way you'd install a PDF reader onto Windows) then you can edit Office files from Drive on Android just fine. The reason this is a problem is because Microsoft hasn't released their software onto all systems, not a limitation of Drive.

Editor's Picks