Migrating emails using IMAP (imapsync) to/from (Gmail, Yahoo, etc)

Hi all!

Last update April 5 2015!

I’ll try use the new Yahoo! Mail for a while but I need my old emails messages. I am a Gmail user and want to copy everything to Yahoo! Mail. After search a lot at Google I found imapsync. It is a Linux program that runs on the command line and can connect to a IMAP server and copy to another one.

Linux Machine

I am a MAC user, so I don’t have Linux machines except servers I don’t want to install things I’ll use just once. So I created a Ubuntu 12.10 64 bits droplet at Digital Ocean and install dependencies and start to sync my emails messages. First we must install dependencies and after install the imapsync. Take a look:

apt-get update
apt-get install libdigest-hmac-perl libdigest-hmac-perl libterm-readkey-perl libterm-readkey-perl libdate-manip-perl libdate-manip-perl libmail-imapclient-perl

Now lets install imapsync downloading the .deb file (version 1.315):

wget http://old-releases.ubuntu.com/ubuntu/pool/universe/i/imapsync/imapsync_1.315+dfsg-1_all.deb
dpkg -i imapsync_1.315+dfsg-1_all.deb

As I want to backup Gmail messages into Yahoo! Mail I must do the follow:

imapsync --host1 imap.gmail.com --user1 username@gmail.com --password1 ******** --host2 imap.mail.yahoo.com --user2 username@yahoo.com --password2 ******* --syncinternaldates --ssl1 -ssl2 --noauthmd5 --split1 100 --split2 100 --port1 993 --port2 993 --exclude "All Mail|Spam|Trash" --allowsizemismatch

Gmail to Google Apps

If you want to migrate a Gmail account to a Google Apps account this will solve your problem. It worked like a charm for me:

imapsync --host1 imap.gmail.com --port1 993 --user1 you@gmail.com --password1 ****** --ssl1 --host2 imap.gmail.com --port2 993 --user2 you@domain.com --password2 ****** --ssl2 --syncinternaldates --split1 100 --split2 100 --authmech1 LOGIN --authmech2 LOGIN --allowsizemismatch --useheader Message-ID

Comments about some parameters:

  • split1 100 and split2 100: Number of messages requests. The default value is 1000 but 100 is a good choice.
  • allowsizemismatch: This options ignore different messages size. Is not present a lot os messages will be ignored because different size between both mail servers.
  • useheader Message-ID: This option is necessary when transferring large folders. If not present the process will be killed and won’t be completed. Now you must wait. If you want you can copy the entire command to another one, like

run_imapsync and give it execute privilegies.

If you also want to let the process running, just run the following and be happy!

./run_imapsync >> /dev/null &

The & will run the process in background mode.

UPDATE Jan 20, 2013

Previously on this post I was using the version 1.315 of imapsync. But there is newer versions that are better than 1.315, that crashes sometimes without reason. Today (Jan 20, 2013), I found the version 1.518 that has some improves. You’ll find the imapsync in this site: http://imapsync.lamiral.info/, but you must to pay to download the newer version. BUT imapsync is for free in GitHub. Lets install the newer version (1.518). First we have to install another dependencies in Ubuntu (my Linux server):

apt-get install makepasswd rcs perl-doc libmail-imapclient-perl make

After you must clone the GIT repository from Github. The URL is https://github.com/imapsync/imapsync. First you have to install git (if not already installed):

apt-get install git-core git-doc git-svn git-gui gitk

Now we can clone the GIT repository:

git clone git://github.com/imapsync/imapsync.git

After that, enter inside the imapsync dir created and run make install:

cd imapsync
make install

Ok! You’ll se something like this:

root@server:~/imapsync# make install
perl -c imapsync
imapsync syntax OK
pod2man imapsync > imapsync.1
mkdir -p /usr/bin
install imapsync /usr/bin/imapsync
chmod 755 /usr/bin/imapsync
mkdir -p /usr/share/man/man1
install imapsync.1 /usr/share/man/man1/imapsync.1
chmod 644 /usr/share/man/man1/imapsync.1

Now you have the newer version installed (1.518):

imapsync -v

UPDATE Apr 25, 2014

Today I started a migration from Google Apps to Gmail and had some problems with my own post πŸ™‚

I created a Digital Ocean droplet with Ubuntu 12.04 64 bits and I installed everything using apt-get (check all needed packages on this hole post). After that I’ve cloned the GitHub repo and run make install. That’s the problem. I have a problem like:

Can't locate File/Copy/Recursive.pm in @INC (@INC contains: ./W/Mail-IMAPClient-3.35/lib /etc/perl /usr/local/lib/perl/5.14.2 /usr/local/share/perl/5.14.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.14 /usr/share/perl/5.14 /usr/local/lib/site_perl .) at ./imapsync line 568.
BEGIN failed--compilation aborted at ./imapsync line 568.

So, we need here the ‘file-copy-recursive’ from Perl. It’s easy to install it:

apt-get install libfile-copy-recursive-perl

Just a tip! Let’s check if your Perl is with all packages Imapsync need:

perl -c imapsync

If you see the follow result everything is ok:

imapsync syntax OK

Now you can go inside the imapsync folder (cloned from GitHub), and run:

make install

Thanks for reading!

Update Apr 5 2015

Thanks for @pgf67 for the contribution. Using the last imapsync version (1.607) you must install some packages:

sudo apt-get install libio-tee-perl libunicode-string-perl

Published by

Junior Grossi

Senior PHP Engineer with TDD focus. Husband & Dad. Creator of Corcel PHP. Organizer of PHPMG. Software Engineer at InterNACHI. Loves coffee and music.

50 thoughts on “Migrating emails using IMAP (imapsync) to/from (Gmail, Yahoo, etc)”

  1. Hi Junior,

    i have this problem with imapsync and imap.gmail.com

    This is the command that i use:

    imapsync –host1 imap.gmail.com –port1 587 –user1 alessandro.feller@aurorascs.com –password1 ***** –tls1 –host2 zimbra.geatrentinaservizi.com –port2 993 –user2 alessandro.feller@aurorascs.com –password2 ****** –ssl2 –syncinternaldates –split1 100 –split2 100 –authmech1 LOGIN –authmech2 LOGIN –allowsizemismatch –useheader Message-ID

    Gut i recived this error

    Transfer started at Tue Nov 24 15:08:54 2015
    PID is 5137
    Log file is LOG_imapsync/2015_11_24_15_08_54_alessandro.feller@aurorascs.com.txt ( to change it, use –logfile filepath ; or use –nolog to turn off logging )
    $RCSfile: imapsync,v $ $Revision: 1.644 $ $Date: 2015/07/17 01:22:52 $
    Here is a [linux] system (Linux zimbra 2.6.32-4-pve #1 SMP Wed Dec 15 14:04:31 CET 2010 x86_64)
    With perl 5.10.1 Mail::IMAPClient 3.34
    Command line used:
    /usr/bin/imapsync –nosyncacls –subscribe –syncinternaldates –fast –dry –host1 imap.gmail.com –port1 993 –regextrans2 s/ /_/f –user1 alessandro.feller@aurorascs.com –password1 MASKED –tls1 –host2 zimbra.geatrentinaservizi.com –user2 alessandro.feller@aurorascs.com –password2 MASKED
    Temp directory is /tmp ( to change it use –tmpdir dirpath )
    PID file is /tmp/imapsync.pid ( to change it use –pidfile filepath ; to avoid it use –pidfile “” )
    Modules version list:
    Mail::IMAPClient 3.34
    IO::Socket 1.31
    IO::Socket::IP ?
    IO::Socket::INET 1.31
    IO::Socket::SSL 1.31
    Net::SSLeay 1.35
    Compress::Zlib 2.021
    Digest::MD5 2.39
    Digest::HMAC_MD5 1.01
    Digest::HMAC_SHA1 1.01
    Term::ReadKey 2.30
    File::Spec 3.3
    Time::HiRes 1.9721
    Unicode::String 2.09
    IO::Tee 0.64
    File::Copy::Recursive 0.38
    Authen::NTLM 1.09
    URI::Escape 3.29
    Data::Uniqid 0.12
    JSON::WebToken ?
    ( use –no-modules_version to turn off printing this Perl modules list )
    Info: turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
    Info: will try to use LOGIN authentication on host1
    Info: will try to use LOGIN authentication on host2
    Info: imap connexions timeout is 120 seconds
    Host1: IMAP server [imap.gmail.com] port [993] user [alessandro.feller@aurorascs.com]
    Host2: IMAP server [zimbra.geatrentinaservizi.com] port [143] user [alessandro.feller@aurorascs.com]
    Failure: can not open imap connection on host1 [imap.gmail.com] with user [alessandro.feller@aurorascs.com]: socket closed while reading data from server

    Can you help me?
    thanks

    1. Hi @Alessandro Feller! Thanks for the comment. According your log file you have the following:

      Failure: can not open imap connection on host1 [imap.gmail.com] with user [alessandro.feller@aurorascs.com]: socket closed while reading data from server

      Imapsync can’t connect with your IMAP server on host1 (Gmail). Maybe there is some port problem. Check the config you’re using and try again. I think this will solve your problem.

      Any question just ask. Thanks πŸ˜‰

  2. Thank you. Imapsync is amazing!
    I had a little problem:


    auth [LOGIN]: 2 BAD parse error: excess characters at end of command

    But after search on google about it I discovery it was a bash escape problem, so:




    gilberto.andrade@BEM-INF007:~/tmp/imapsync> export password1="iio/09&*"
    gilberto.andrade@BEM-INF007:~/tmp/imapsync> echo $password1
    iio/09&*



    1. Hi Gilberto! Thanks for the comment. Your password has a special character / so you have to scape it because linux command patterns. Thanks πŸ˜‰

  3. Thanks @XZtian! I hope you could migrate your emails successfully. If you have any questions would be a pleasure to answer. Thanks for the comment.

    1. Thanks! BTW, in the last imapsync version (1.607) a needed to add a couple of packages:

      sudo apt-get install libio-tee-perl libunicode-string-perl

      Thanks again!

  4. Hello, and thank you for this website.

    I am trying to get away from gmail. If I use this script, is there any way google/gmail will know the destination address? Call me paranoid, but I don’t want them to know where I’m moving all my gmail emails.

    1. Hey Hondaman!

      I really cannot answer your question because I don’t know. But imapsync uses the IMAP protocol and it needs a destination server to do that, so I “think” Gmail knows where your messages are going, but I don’t guess this is a problem.

      Thanks for the message! Cheers!

  5. Hi Grossi,

    It seems like the problem still persists. Any suggestions. I followed the command as you suggested. Adding just –addheaders as it kept saying no headers found.

    https://drive.google.com/fi

    Preferably, I’d like to have the emails from the inbox going to the inbox, sent mail to the sent mail and any subfolders to the relevant subfolders. I can live with it going to IMAP/Sent Items, but then its rather confusing when emails that should be in the sent items appear in the inbox πŸ˜€

    If there are any suggestions for the contacts that get migrated across, that would be fantastic, else I can live with migrating the contacts, manually.

    Any help is most appreciated. Thanking you in advance.

    Best wishes,
    Sathesh.

  6. btw, this is the system I am on:

    $RCSfile: imapsync,v $ $Revision: 1.592 $ $Date: 2014/05/22 10:03:17 $
    Here is a [linux] system (Linux genesis-desktop 3.13.0-27-generic #50-Ubuntu SMP Thu May 15 18:08:16 UTC 2014 i686)
    With perl 5.18.2 Mail::IMAPClient 3.35

  7. Hi Grossi,

    I can’t seem to find the v1.518. Hence have to use the latest version – 1.592

    I seem to be having a problem with my migration. I am tryingt o migrate from Zimbra to Gmail. Its really a trial before I do a large number of accounts as we move to google apps.

    Anyway, my issue is that upon migration, all emails go into the Inbox and has a label for sent items too. So, emails that should be in the sent items are also there. Not sure what could be the possible solution to this. Please advise, if you have any suggestions as to how I can make the migration neat. Also, its importing my contacts..but not correctly πŸ™‚ Any way I can fix this too?

    Thanking you muchly.

    Best wishes,
    Sathesh.

  8. Hi,
    I have used the below command to migrate yahoo account to zimbra

    /usr/bin/imapsync –host1 “hostname1” –port1 993 –user1 “user1@mydomain.com” –password1 “passwd” –ssl1 –host2 “hostname2” –port2 993 –user2 “user2@mydomain.com” –password2 “passwd” –ssl2 –syncinternaldates –nosyncacls –authmech1 LOGIN –authmech2 LOGIN –allowsizemismatch –nofoldersizes –skipsize –fast –subscribe –buffersize 8192000 –sep 1 “/” –prefix1 “”

    Everything is synced without “Inbox”. It is showing the following error while migrating Inbox

    Time: 0 s
    Host1 Folder [Inbox]
    Host2 Folder [Inbox]
    Host2 folder Inbox does not exist
    Creating folder [Inbox]
    Couldn’t create [Inbox]: 15 NO CREATE failed: mailbox already exists

    Any help appreciated ….

      1. Hi Grossi,

        Thanks for your reply. i have tried that also, but again i am getting the same error. will imapsync 1.584 help me to resolve my issue? because right now i m using imapsync 1.315. while syncing i am getting the folder structure of host1(yahoo) and folder structure of host2(zimbra).

        Imapsync output: folder structure list of host1 and host2 while syncing are mentioned below for your reference.

        Host1 folder structure showing "inbox" in that list and Host2 folder structure showing "INBOX". can i able to migrate from "inbox" to "INBOX" using "regextrans" parameter. if it is possible, please tell me how or this is the error beacuse of case sensitive.

        host1: state Authenticated
        host2: state Authenticated
        Using [/] given by –sep1
        Using [] given by –prefix1
        Host1 separator and prefix: [/][]
        Host2 separator and prefix: [/][]
        ++++ Listing folders ++++
        Host1 folders list:
        [Bulk Mail]
        [Draft]
        [Inbox]
        [Sent]
        [Trash]

        Host2 folders list:
        [Chats]
        [Contacts]
        [Drafts]
        [Emailed Contacts]
        [INBOX]
        [Junk]
        [Sent]
        [Trash]

        ++++ Looping on each folder ++++
        Host1 Folder [Bulk Mail]
        Host2 Folder [Bulk Mail]
        Host2 folder Bulk Mail does not exist
        Creating folder [Bulk Mail]
        Subscribing to folder Bulk Mail on destination server
        ++++ Host1 [Bulk Mail] parsing headers ++++
        ++++ Host2 [Bulk Mail] parsing headers ++++
        ++++ Verifying [Bulk Mail] -> [Bulk Mail] ++++
        Time: 1 s
        Host1 Folder [Draft]
        Host2 Folder [Draft]
        Host2 folder Draft does not exist
        Creating folder [Draft]
        Subscribing to folder Draft on destination server
        ++++ Host1 [Draft] parsing headers ++++
        ++++ Host2 [Draft] parsing headers ++++
        ++++ Verifying [Draft] -> [Draft] ++++
        Time: 0 s
        Host1 Folder [Inbox]
        Host2 Folder [Inbox]
        Host2 folder Inbox does not exist
        Creating folder [Inbox]
        Couldn’t create [Inbox]: 15 NO CREATE failed: mailbox already exists

        1. Hi!

          I have to use imapsync some times along this year but I’ve found a lot of problems with newer versions. The only version I have no problem is the 1.518, where you can find on Github and install like I’m saying on the post.

          Any changes in versions brought a lot of problems for me so I strongly recommend you the 1.518.

          Thanks again!

          πŸ˜‰

          1. Thanks a lot for your immediate and valued responses, appreciate your heads up with respect the same.

            I have upgraded imapsync tool version from 1.315 to 1.584 its working good. And i can able to sync all the folders and mails

            Thanks again for your good support.

  9. Hi… I too am stuck a bit. I want to move my rackspace mailboxes to google apps. So this is the command I use: imapsync –host1 imap.gmail.com –port1 993 –user1 you@gmail.com –password1 ****** –ssl1 –host2 imap.gmail.com –port2 993 –user2 you@domain.com –password2 ****** –ssl2 –syncinternaldates –split1 100 –split2 100 –authmech1 LOGIN –authmech2 LOGIN –allowsizemismatch –useheader Message-ID But I have run into an issue. Its telling me that it cannot log in with my account to google apps. Any suggestions?

    1. man I really don’t know…
      maybe it’s a ssl problem and maybe some config must be changed to work these days πŸ˜‰
      thanks for the comment… if I find updates I’ll tell you… regards

  10. Hi,
    I’m still stuck @

    “Can’t locate File/Copy/Recursive.pm in @INC..”

    until I have done:
    1. pt-get install perlbrew
    2. perlbrew install-cpanm
    3. perl -MCPAN -e ‘install File::Copy::Recursive;

    and than ran
    make install
    again
    without success.

    every hint welcome!
    greetz!

  11. thx a lot!
    how did you find out about all dependencies?
    I was fiddling around with the tarball but did not manage to install imapsync correctly.
    your instructions just work!

    thanks again!

    best regards!

    J

  12. Thanks – saved me lots of time, and introduced me to cloud based VPS. I just followed all the steps and got on with migrating my mail…invaluable

  13. hi,
    If I use imap from my site, am I using my own bandwith to transfer de mails?. is there any method to transfer data quickly?

    1. Hi Jhonatan!

      Yes, you are using your own bandwidth. The best method is to use a VPS like the post suggests. With a VPS you can sleep while your email are being transferred :D.

      Best regards.

  14. Hey would you mind letting me know which hosting company
    you’re using? I’ve loaded your blog in 3 completely
    different web browsers and I must say this blog loads
    a lot quicker then most. Can you suggest a good web hosting provider at a fair price?
    Thanks a lot, I appreciate it!

    1. Hi Francisco!

      Actually I&#039m using a VPS (virtual private server) at digitalocean.com. It is not a hosting company but something more particular. There you can install the services you need, just for you.

      If your website is slow you have to check a lot of things. This can be a bad programming code, a database problem, or even if your website is with hight traffic you have to increase memory, cpu or others things.

      If you want contact me by email (me@juniorgrossi.com) and I can help you more.

      Thanks for the comment. Welcome to the blog.

    1. Hi Cecco!

      Welcome! I had problems with old imapsync versions too and the Git version was the best solution.

      Thanks for the comment!

  15. Hi Grossi

    I have been trying for months now to get Outlook PST files into Gmail personal account. I have done the drag and drop folder level and it is not an easy way of going about it. Do you know if It is possible to install my own local IMAP server on my Mac (will use Ubuntu VM to install IMAP Server) and then use imapsync to import into gmail. I am not sure if I need a IMAP server that is properly ‘connected” to the internet (including receiving client connections) etc. for IMAP Sync to work or (hopefully) I can perform some local IMAP installation and log onto it with out worrying about email internet domains. The mail goal would be to import the pst to local non active IMAP installation and then use IMAPSync to copy to gmail.

    Do you perhaps have a view on this and if it will be possible.

    Thank you.
    Kind regards
    Lombard

    1. Hi Lombard!

      I think you can install a IMAP server, but I don’t think it’s the best ideia. You will need that for a very short time, so rent a VPS is fast and cheap. At Digital Ocean you will pay $5/month, but this migration process is about 2 or 3 days so, you will pay almost nothing to do that.

      Thanks for the comment and for the visit!

  16. Hi, Grossi. Do you have examples of how to accelerate the migration of data using imapsync (in Windows machines) ?. I need to run at least 10 processes in parallel by machine.

    Best,
    Daniel

    1. Hi Ali! What’s going wrong? I don’t have experience with Exchange but I think imapsync works with all mail servers that have imap as protocol. I’ve tested it with Yahoo!, Gmail and Google Apps with success but never with Exchange. Sorry about that! Regards.

  17. Btw, I’m just Windows aware. Would you mind showing me how to build an imapsync.exe file from the GitHub downloaded? Thanks in advanced!

    1. Hi Tu9a2!

      Thanks for the comment. I don’t have experience using imapsync in Windows systems. I know you don’t need to “run” imapsync inside Windows. For example you can have a Linux machine where you’ll call your Windows Exchange server as origin and set your destiny to migrate all mail.

      Thanks again!

  18. Thanks groosi. I’m falling deep in sh!t with a case, having to transfer about 100GB mail from internet MDaemon server to internal Exchange 2013 server. Hope this would help me much :v

Leave a Reply

Your email address will not be published. Required fields are marked *