toGMAIL: Export Thunderbird Mailboxes into GMail
Introduction
toGMAIL (aka toIMAP) allows you to export entire mailboxes from Thunderbird to GMail. In fact, you can export email from any mbox-style mailbox into any IMAP-enabled mail system. While you can use Thunderbird itself to copy emails to an IMAP-enabled account, the process has been (for me) error prone, and doesn’t support recovering from export errors.
There are also SMTP-based exporters, but these can be fairly cumbersome to use, and don’t support specifying target IMAP folders (which become lables in GMail).
Downloads and Installation
- For Python users (any operating system) download the script and configuration file. You’ll need Python 2.5. toIMAP may work with 2.6 or 2.4, but I haven’t tested it.
- Windows users without Python can download the SilverProGold2000Y2KTTurboExtremeLanceBassSignatureLimitedEditionDeluxeTrimPackage executable version.
Unzip the files somewhere and edit the ‘user’ and ‘password’ values in toIMAP.cfg to match your gmail user and password. If you’re exporting to an IMAP system other than gmail, change thge host and SSL parameters accordingly. Be sure to preserve the quotes in the host, user and password fields. You should be ready to go.
Running toIMAP
To get quick instructions, run toIMAP.py (or toIMAP.exe) with no parameters:
/home/mikey/# toIMAP.py
INFO toIMAP Starting...
Usage:
toIMAP -m <mailboxfile> [-f <folder>] [-p] [-s id]
-m Specify mailbox file -f Target folder (INBOX default) -p Pretend -- don't actually send up messages -s Start at -- start importing from specified ID
The -m option is the only required option. Use the -f option to specify a target folder (label in gmail) to export into. For example:
c:\toIMAP\toIMAP.exe -m “Work Messages”
will export all messages in mailbox “Work Messages” to the INBOX folder. To export to folder (or label) “work”:
c:\toIMAP\toIMAP.exe -m “Work Messages” -f work
To simulate but not actually make any changes to the remote mailbox, use “pretend” mode with the -p flag:
c:\toIMAP\toIMAP.exe -m “Work Messages” -f work -p
Messages in the local mailbox are identified by sequentially by a number. To restart the export with a specific email id (or to bypass a message that cannot be imported) use the -s (start) option. This will cause toIMAP to start exporting with the messge identified by the specified id.
c:\toIMAP\toIMAP.exe -m “Work Messages” -f work -s 100
toIMAP will create a logfile (toIMAP.log) in the current directory so you have a record of what messages were processed. The logfile is appended to, and will be recreated if deleted.
heya,
This seems like an awesome tool, and I’ve been waiting for something like this for ages.
However, it does seem to choke up on certain messages.
My mbox file is around 850 Mb, and contains around 12,000 messages. When I run toIMAP, it loads the mailbox, and memory usage is around 1100 Mb. That’s fine though, as it seems to start uploading.
However, on certain messages (particularly mailer-daemon “failure delivery” messages, but also on some others) toIMAP seems to stall, the CPU usages jumps to 50%, and eventually it will print out:
ERROR
Traceback (most recent call last):
File “toIMAP.py”, line 138, in main
r = server.append(folder, None, email.utils.parsedate(v['Date']),str(v))
File “/usr/lib/python2.6/imaplib.py”, line 318, in append
return self._simple_command(name, mailbox, flags, date_time)
File “/usr/lib/python2.6/imaplib.py”, line 1059, in _simple_command
return self._command_complete(name, self._command(name, *args))
File “/usr/lib/python2.6/imaplib.py”, line 889, in _command_complete
typ, data = self._get_tagged_response(tag)
File “/usr/lib/python2.6/imaplib.py”, line 990, in _get_tagged_response
self._get_response()
File “/usr/lib/python2.6/imaplib.py”, line 907, in _get_response
resp = self._get_line()
File “/usr/lib/python2.6/imaplib.py”, line 1000, in _get_line
line = self.readline()
File “/usr/lib/python2.6/imaplib.py”, line 1171, in readline
line.append(char)
MemoryError
ERROR Failed to upload message 104
ERROR You can retry by rerunning with the -s and the failed message id.
Any suggestions on what might be causing this, or workarounds? (I can send you those emails it chokes on, if that would help).
Cheers,
March 29th, 2009 at 12:31 amVictor
Hello. Great job. I did not expect this on a Wednesday. This is a great story. Thanks!
April 7th, 2009 at 3:01 amswampgas, I’ve never met you, but I want to offer to bear your child (I don’t have a uterus, so this could be difficult to actually acheive) !!!
i’ve spent two days hunting for your solution. It’s the only “migrate to GMAIL” solution that I’ve seen that allows real email migration- maintaining date and ‘folder’ in the form of a label.
AWESOME! thank you for making this work!
the only snag that I hit is that on my ISP, python was unable to find one of the email libraries that’s required. But that’s not your fault
April 15th, 2009 at 5:54 pmVictor: I suspect python has simply run out of memory; the problem is that I’m using the python mbox library to load the mailbox. Mbox doesn’t do a lazy load, and instead tries to load the entire mailbox into memory. In your case, I suspect this leaves nothing over to actually perform the imap operation.
Really, I should write my own mbox parser that lazily loads each message — but this was a quick and dirty tool for my own personal use. You *could* try running on a machine with more memory. I managed to load a 10k message box on a machine with 2 gig.
June 6th, 2009 at 8:52 amHi, interest post. I’ll write you later about few questions!
June 13th, 2009 at 7:56 pmYou are my hero! I’ve been fooling around for *months* trying to migrate my mail to Gmail, hitting all kinds of walls, but your solution works like a charm!
Many thanks,
June 15th, 2009 at 3:40 amGreg
Original post by Dmitri Gromov
June 15th, 2009 at 5:46 amI made a couple of small tweaks to your code to support long-options on the cmdline, log debug-level only to the logfile, and cmdline specify file locations.
http://www.alexjb.com/blog/?p=20
If you’d rather I didn’t host the files on my site, just let me know and I’ll take em down.
Thanks again for making the initial effort!
August 18th, 2009 at 2:32 pman additional note – I spoke too soon in terms of having this run with Python 2.4. The mailbox module in 2.4 has a very different structure, and the code in this script attempts to instantiate a class that doesn’t exist for Python 2.4. The problem doesn’t manifest until the code actually attempts to load a mailbox file.
August 18th, 2009 at 3:31 pmWow, what a great tool. I tried the trick of mapping gmail in Thunderbird, but it didn’t work. Then I tried using GLoader, but the SMTP was wrong… and using my own SMTP for 1000 messages in a day got me flagged as a spammer.
I had one hiccup where it failed, but I was able to use the -s flag to pick up where I left off and it worked like a champ. Sweet!
November 21st, 2009 at 3:20 pmHello,
I’ve searched 2 days for a smoothless way to transfer my Thunderbird achives to my Gmail account. Theoretically TB must to the job because IMAP is supported. So drag and drop to my GMAIL account should work… but it doesn’t. TB stalls and stops transferring without noticing.
This is a wonderfull application: set op the config-file. Locate your archives-file. Run a command-line an hit the road! For now my laptop is finally busy transferring each of the 7000+ mails to my Gmail-account. It does 770 messages in a half an hour – including the read-out of the folder, which means that the job will be done in max. 5 hours or something.
THANK YOU for sharing!!!!!!!!!
January 24th, 2010 at 2:21 pmThank you very much. The code for Sent Mail etc. is “[Google Mail]/Sent Mail”
March 7th, 2010 at 3:42 pm