How to Backup Your Fastmail & Gmail Accounts with isync (2024)

isync is a command line application that can synchronize a remote IMAP account (like a Fastmail or Gmail account) with localmaildirs. This post will show you how to use isync to make a local backup of your Fastmail and Gmailaccounts.

isync is quick to install and configure and worksgreat.

The project is called isync but the actual command line application is mbsyncbecause massive changes were once made to the userinterface.

isync can do two-way synchronization of your IMAP and local maildirs.It even allows fine-grained control of which types of changes get synced(new messages and folders, deletions and flag changes)in which direction(s) and on a per-folder basis.See man mbsync for the details and for an explanation of the“account”, “store” and “channel” concepts that you’ll see in the .mbsyncrc filebelow.

In this post we’re going to be using isync to make a read-only backup of your IMAP accounts:changes will be synced from your Fastmail and Gmail accounts to your local filesystem but not the other way round, so there’s no chance of isync accidentally deletingmail from your Fastmail or Gmail account if something goes wrong with the localcopy.

Installisync

On Ubuntu it’sjust:

$ sudo apt install isync

Configureisync

You need to create an ~/.mbsyncrc file before isync will work.Here’s an ~/.mbsyncrc to backup a Fastmail account and a Gmail account to a local ~/Mail folder:

# ~/.mbsyncrcCopyArrivalDate yes # Don't mess up message timestamps when moving them between folders.Sync Pull # Download changes only, don't sync local changes up to the server.Create Near # Automatically create new folders in the local copy.Remove Near # Automatically remove deleted folders from the local copy.Expunge Near # Expunge deleted messages from the local copy.IMAPAccount fastmailHost imap.fastmail.comUser YOUR_USERNAME@fastmail.comPassCmd "pass mbsync/fastmail"SSLType IMAPSIMAPStore fastmail-remoteAccount fastmailMaildirStore fastmail-localPath ~/Mail/Fastmail/Inbox ~/Mail/Fastmail/InboxSubFolders VerbatimChannel fastmailFar :fastmail-remote:Near :fastmail-local:Patterns *SyncState *IMAPAccount gmailHost imap.gmail.comUser YOUR_USERNAME@gmail.comPassCmd "pass mbsync/gmail"SSLType IMAPSPipelinedepth 1Timeout 60IMAPStore gmail-remoteAccount gmailMaildirStore gmail-localPath ~/Mail/Gmail/Inbox ~/Mail/Gmail/InboxSubFolders VerbatimChannel gmailFar :gmail-remote:Near :gmail-local:Patterns *SyncState *

The Pipelinedepth 1 slows isync down by preventing it from having multipleIMAP commands in flight at once. This is necessary to prevent isync fromhitting Gmail’s bandwidth quotas and triggering this errormessage:

IMAP error: unexpected BYE response: [OVERQUOTA] Account exceeded command or bandwidth limits.

I found that I only needed this when downloading a large amount of email at once,such as during the initial download of my whole Gmail account. I was able toremove the Pipelinedepth 1 after that and it has been working fine. RemovingPipelinedepth 1 doesn’t seem to make mbsync gmail run any faster for methough: without knowing how isync works internally I’m guessing this might bebecause I have a lot of maildirs but not much new mail to download in any onemaildir.

The Timeout 60 increases isync’s IMAP connect and data timeouts from thedefault of 20s. I found that this got rid of some timeout errors I was seeingfromGmail.

Create Fastmail and Gmail apppasswords

You need to create a Fastmail app password for isync to access to your account with.In Fastmail’s web interface go to /settings/security/devicekeys/, unlock the interface, and click New App Password.You can create an app password with IMAP access only and with read-only access as an extra protection against isync accidentally deleting your mail.See Fastmail’s docs for more on creating apppasswords.

How to Backup Your Fastmail & Gmail Accounts withisync (1)

For Gmail you have to go to myaccount.google.com/security/ and click on App passwords to create one.See Google’s docs on app passwords.You also need to select Enable IMAP in your Gmail settings before isync will work,see Gmail’s IMAP docs fordetails.

How to Backup Your Fastmail & Gmail Accounts withisync (2)

Install `pass` and add the app passwords toit

We’re going to use pass to store the Fastmail and Gmail app passwords in encrypted files for isync toread:

Install pass:
```
#! console$ sudo apt install pass
```
Create a GPG key that’ll be used to encrypt thepasswords:
```
$ gpg --full-generate-key
```
Initialize pass with the GPGkey:
```
$ pass init <GPG_KEY_ID>
```

If you’re not sure what the GPG key’s ID is you can run gpg --list-secret-keys tosee.

You can now optionally run:
```
$ pass git init
```

If you have Git installed and want pass to automatically keep history in a gitrepo.

Add the Fastmail app password to pass:
```
$ pass insert mbsync/fastmail
```

Paste in the password whenasked.

Add the Gmail app password to pass:
```
$ pass insert mbsync/gmail
```

You should now be able to read the app passwords with pass mbsync/fastmail or pass mbsync/gmail. These are the commands that the .mbsyncrc file above usesto get thepasswords.

Create the empty localmaildirs

isync won’t create these so you have to do itmanually:

$ mkdir -p ~/Mail/Fastmail ~/Mail/Gmail

Run `mbsync`

Now to download all of your email justrun:

$ mbsync -a

You can re-run the command at any time to update the local copy.You can also download only Fastmail or only Gmail with mbsync fastmail or mbsync gmail.

The command’s output looks likethis:

C: 0/2 B: 5/173 F: +0/0 *0/0 #0/0 N: +23/422 *0/0 #0/0

The C: 0/2 is the number of channels that it has finished syncing. (So 0/2 means it has finished 0 channels so far, so it’s currently syncing the firstchannel.)
The B: 5/173 means that it has so far finished syncing 5 out of 173 mailboxes. (173 is the count of all mailboxes to be synced, across allchannels.)
The F: +0/0 *0/0 #0/0 counts the number of messages synced up to the far copy: +0/0 is the number of messages added, *0/0 is the number of messages whose flags have been updated, and #0/0 is the number of messages deleted. These are all 0/0 because we’re downloading changes only, we’re not syncing changes up to theserver.
N: +23/422 *0/0 #0/0 is the same three counts of message changes synced down to the local copy: +23/422 messages added, *0/0 messages with flags updated, and #0/0 messages deleted sofar.

The totals change as isync runs.For example when isync begins syncing the first channel and connects to thefirst IMAP account it finds out about the number of mailboxes in that count so itdisplays B: 0/173.When it finishes syncing those 173 mailboxes isync starts syncing the nextchannel and connects to the next IMAP account and finds out about the numberof mailboxes in that account so the B section changes to B: 173/502.The F and N totals behave similarly as mailboxes are opened and newmessages to be synced arediscovered.

If you run mbsync with --verbose it’ll print more detailed informationabout what’s happening. As well as log messages about connecting and logging into IMAP accounts it also prints several lines for each mailbox synced.For example this shows it creating the lists/isync-devel folder in the local(“near”) maildir and downloading 422 messages intoit:

Opening far side box lists/isync-devel...Opening near side box lists/isync-devel...Creating near side box lists/isync-devel...Maildir notice: no UIDVALIDITY, creating new.Loading far side box...Loading near side box...near side: 0 messages, 0 recentfar side: 422 messages, 0 recentSynchronizing...C: 0/1 B: 16/176 F: +0/0 *0/0 #0/0 N: +422/422 *0/0 #0/0

This gives you detailed information about what was synced up and down for each folder.But if you have a lot of folders it produces a lot of output which can hide any errors orwarnings.

Errors fromGmail

I got a couple of errors during the initial big download of my Gmail account.This kepthappening:

IMAP error: unexpected BYE response: System Error

If you restart the mbsync command it’ll continue for a while longer and then the error will happen again,so you can just keep restarting it and eventually it’ll finish thedownload.

Having isync crash and restarting it seems to produce this warning aswell:

Warning: lost track of 39544 pulled message(s)

I was told on the isync mailing list (back in 2016)that this most likely doesn’t mean anything and at worst could mean someduplicatemails.

After getting through the initial download I haven’t had any of these errorsfrom Gmail again. I never got the errors at all withFastmail.

Read the local copy withMutt

Once you’ve made a local copy of your email you can read the ~/Mail folder with a local mail client like Thunderbird or Mutt.

To install Muttrun:

$ sudo apt install mutt

You can now just run mutt -Rf ~/Mail/Fastmail/Inbox/ to open your Fastmail inbox in read-only mode.But it can be convenient to create a ~/.muttrc file to avoid having to pass arguments on the command line.Here’s a minimalexample:

# ~/.muttrcset read_onlyset spoolfile=+Fastmail/Inboxset mbox=+Fastmail/Archiveset record=+Fastmail/Sentset header_cache=~/.muttcacheset message_cachedir=~/.muttcachemacro index,pager G "!mbsync -a\n"

The set read_only tells Mutt to work in read-only mode (for example Mutt won’t let you delete messages).Since isync isn’t going to sync any changes up to your Fastmail and Gmail accounts it makes sense to tell Mutt not to make any changes to the localcopy.

The spoolfile, mbox and record settings tell Mutt where to find your inbox, archive, and sent folders.For example Mutt will open ~/Mail/Fastmail/Inbox by default when you launch it. Of course Mutt isn’t goingto be adding any messages to the archive or sent folders when in read-only mode but you canstill use Mutt’s mailbox shortcuts for these special folders toopen them easily. For example you can summon the Open mailbox prompt (c) and enter!, > or < to open your inbox, archive folder or sentfolder.

The header_cache and message_cachedir settings tell Mutt to use a ~/.muttcache dir to speed up re-opening and searching large folders.You should create this directory before launching Mutt for the first time: mkdir ~/.muttcache.

Finally the macro at the end binds capital G to run mbsync -a from withinMutt.

Now you can just launch mutt without command linearguments:

$ mutt