Welcome back to my discussion of an email migration to Google Apps which I conducted for a client organization called Terrapin Valley. In Part 1, I outlined the small business environment that exists at this five-person company, the challenges we faced in supporting the outdated Windows 2003 Small Business Server which handled email processing, and the requirements for a new email platform better suited to this day and age.
The owner of the company (Ted) and I decided to pursue a plan to migrate the company email to Google Apps, so I hammered out a migration guide with help from Google's documentation and links. Part Two is devoted to the migration process
Enrolling in Google Apps
I began by using our public DNS administration console to adjust the TTL on the DNS MX record for terrapinvalley.org. I set this to five minutes; that would ensure that new email to the organization would start arriving on Google servers within a few minutes once we were ready to proceed with the cutover.I then initialized the Google Apps for Business trial account and the setup wizard kicked off, see Figure A.
Set up wizard
The above screenshot also shows the toolbar (Dashboard, Organization & Users) which is used in the Google Apps Control Panel, so it is handy to see how the administrative options work.
The first step was to verify domain ownership for terrapinvalley.org. There are several ways to do this:
- You can create a customized public DNS record (TXT file) for your domain.
- You can set up a special HTML file / add a <meta> tag on your public web server.
- You can use Google Analytics with a tracking code on your public web server.
All methods involve a security token provided by Google which they will then detect in your DNS settings or on your public website so they can confirm you own (or have administrative rights to) the domain.I opted to go with the special HTML file, which I made available at www.terrapinvalley.org. Once I did this I was able to verify domain ownership with Google and proceed with the setup, as shown in Figure B.
Verify domain ownership
Creating accounts for users within the Google Apps environment
The next step was to provision Terrapin Valley users in Google Apps, which would include the creation of Gmail accounts (with the same existing email addresses at terrapinvalley.org; their email addresses would be their account names) allowing 25GB of space. Since we hadn't gone live yet, these email accounts would merely be used for migration of existing data.
The options allow you to create the accounts manually or import a CSV file to automate the process. You can also use a tool from Google called "Google Apps Directory Sync" to import users via LDAP. For simplicity sake, I opted to create the accounts manually since only five were involved. I also had to set up the appropriate Apps for the users; I chose Gmail and Calendar only, since we could add others later as needed.
Configuring settings to permit copying of existing data to Google serversIn order to migrate data up to Gmail I had to enable a setting called "Allow users to upload mail using the Email Migration API." This is currently available in the Google Apps Control Panel (which I will explore further in a follow-up article to outline the administrative capabilities provided by Google) under Settings and then Email, as shown in Figure C.
Google Apps Control Panel
While I was navigating through this section I also checked off the option to "Enable Offline Gmail for my users." This requires Google Gears, a plugin for the browser which will work in Internet Explorer, Firefox, and Google Chrome. We wouldn't set this up just yet, but would plan to use it down the road.
The next step involved configuring authentication between our existing email environment and Google Apps. Google Apps uses OAuth to handle the process (OAuth stands for open authentication which allows data to be shared without having to provide the source credentials involved).This can be accomplished under the Advanced tools section in the Control Panel, as shown in Figure D.
Advanced tools sectionI clicked on "Manage OAuth domain key" to see what details were involved (Figure E).
Manage OAuth domain keyThe consumer key is the domain name (I have blacked out the actual domain name in the screenshot above). Note the OAuth consumer secret string; this is used by Google for authentication. I did not have to make any changes here, so I went back to Advanced Tools and clicked "Manage third party OAuth Client access," as shown in Figure F.
Manage third party OAuth Client access
I entered the domain name in the "Client Name" field. I then entered "https://apps-apis.google.com/a/feeds/migration/" in the "One or More API Scopes field" and clicked Authorize (other URLs are recommended for different feeds, but at the moment I only needed the migration option).
Now we were ready to migrate data.
Copying all email, contacts and calendar items for up to Google serversGoogle offers a Google Apps Migration tool to facilitate the data upload process. I installed it on a workstation and started it up, as you see in Figure G.
Google Apps Migration tool
I entered the administrator account credentials and clicked Continue:
The correct Outlook profile (MS Exchange) was highlighted for me already. Since this was the first time I'd run the tool, I clicked "Migrate all data". The tool also allows you to migrate only new data which is helpful if you're conducting a slow transition to Gmail which will involve multiple data copying efforts (such as the first pass to get all existing data copied and then subsequent passes to catch the new data users have sent/received since the initial run of this tool).Once I clicked Next I was taken to the screen shown in Figure H.
MigrationI chose to go with the highlighted options (these were checked off by default) and clicked Migrate (Figure I).
Starting the migration process
The tool then ran and gave me progress bars to indicate the completion status of the migration of different categories of data.
When the data was done copying, I checked the migration log file, which on the XP system was located at C:\Documents and Settings\"username"\Local Settings\Application Data\Google\Google Apps Migration\Tracing\ExchangeMigration. (On a Vista or Windows 7 machine the log directory would be at C:\Users\"username"\AppData\Local\Google\Google Apps Migration\Tracing\ExchangeMigration).
No errors were found, so I then verified that the data now existed up in Gmail (and had not been removed from the source mailbox). Even though Gmail isn't really folder-oriented, I was pleased to note that the existing folder structure had been copied up from Outlook and was intact with the correct contents.Then I installed the Google Apps Sync for Microsoft Outlook plugin on the Outlook client and started it up. The plugin provides you with a similar startup screen to the Migration tool, as you can see in Figure J.
Google Apps SyncI signed in and got the following setup box (Figure K).
Set up Google Apps Sync
The Google Apps Sync tool allows you to import data to Gmail, meaning it can perform the same function as the Migration tool. In my case I had opted for the migration tool to ensure I followed the official process, but it's nice that this utility can do something similar.I left all boxes unchecked and clicked "Create profile." The plugin then set up a local PST file with the user name and "Google Apps" in the title. Outlook opened automatically and a new icon appeared in the far left of the system tray (Figure L).
That circular icon represents the Google Apps sync process. It runs automatically when Outlook opens - in fact, you can't manually close it. This poses an issue if you don't want to synchronize data at a particular time, such as when trying to clean up items in Outlook that don't need to get copied up to Gmail. I found a workaround by clicking File, then Work Offline; this stops the Google Apps sync.Right-clicking the Google Apps Sync icon brought up the box in Figure M.
Google Apps Sync options"Set mailbox size limit" allows you to configure how large your local PST file should be permitted to grow; this plays a factor in the synchronization performance (Figure N).
Set limitsChoosing "View sync status" or simply double-clicking the Google Apps Sync icon brings up a synchronization status window (Figure O).
Synchronization status window
(Clicking "show more" will reveal progress bars for Google Tasks, Notes and Google Address Book, if those options are valid).There are some other useful options, available from the Start Menu (Figure P).
Other useful options
Next I ran the migration for all of the users after exporting their existing Outlook rules and signatures, since those can't be copied up to Gmail. Some users had small mailboxes (<10MB) and so the migration process was quite fast and efficient, generally completing within minutes. Some other users had PST files with older data which they wanted to keep, and I was able to conduct a migration of the contents of these objects directly from the PST up to Gmail, thanks to the options in the migration tool.
Jill had a rather large mailbox with dozens of folders, and this took some time to migrate. I made sure to copy her email items first, then calendar, and then finally I set up her contact groups in Gmail and migrated each set of contacts separately so I could then assign them to the appropriate groups in Gmail. This involved running the migration tool several times, but I completed it all after two or three hours.
Jill's account showed many instances of duplicate contacts since apparently there had been extra versions scattered throughout each category. Gmail offers a helpful tool in the Contacts screen called "Find and Merge duplicates" which can be found via the "More" button when viewing the Contacts page. This came in handy to get the duplicates removed.
Once I'd completed the migration of all user data to Gmail (fortunately there were no errors or problems with any of them), I set up the Google Apps Sync tool on the applicable Outlook 2003 clients.
Since all of her data now existed up at Google, my next task was to set up Outlook 2011 for Jill on her Mac. As I previously mentioned, this involved the following steps:
- Set up IMAP account in Outlook 2011 to communicate with Gmail
- Set up local Address Book application to synchronize with Gmail, then set up Outlook 2011 to sync with the Address Book
- Set up local iCal calendar application to synchronize with Gmail
This meant Jill could use Outlook 2011 for email and contacts while utilizing iCal for her calendar items. In all three instances the data would be copied back and forth between Gmail and her Mac. Unfortunately, there is a limitation in Outlook 2011 whereby the calendar cannot synchronize with Gmail, something I hope will be remedied at some point (many users are eagerly awaiting this!)
I then trained the users on how to work with the new setup. We put Ted and Jill's iPhone and iPad configuration on the back burner so we could ensure everyone was satisfied with their desktop/laptop email clients. The users continued accessing their live Terrapin Valley email data via Outlook Web Access; this meant we wouldn't have to deal with multiple Outlook profiles and switching back and forth between old and new data (not to mention the confusion and mistakes than can occur with such an arrangement).
A day or so later everyone felt ready to proceed with the cutover. This was the easiest part. I simply edited the DNS MX record for terrapinvalley.org to list the following Google servers:
I confirmed new email for users was arriving in their Gmail accounts and in their Outlook clients a few minutes later. I tested outbound email as well. I made sure to check the Exchange accounts on the now-defunct Terrapin Valley server to ensure no new mail was arriving in any of them. Since we no longer needed port 25 to be open in the firewall for the mail server to receive inbound messages, I disabled that rule. Finally, I reran the migration tool for all users to get the latest data up to Gmail, and helped the users import their Outlook rules and signatures.
Our migration was complete and we were live on Google Apps.
In my next few articles, I'll discuss what happened next; how we got the mobile devices configured, how we started delving into document sharing, and how the Google Apps control panel works.
Scott Matteson is a senior systems administrator and freelance technical writer who also performs consulting work for small organizations. He resides in the Greater Boston area with his wife and three children.