SpamSieve + Apple Mail + GMail

I almost gave up on Apple Mail because spam kept “reappearing”. Like the undead, I could not kill it. I would delete spam, and fifteen minutes later it would be back again. It turns out that Apple Mail and GMail have very different ideas about how to organize email, and this disagreement can cause lots of problems.

GMail has a unique view of how mail is organized. This view has been called a “hack”. Basically, the idea is that you have a flow of email, and each email can have tags to organize it. You can tag an email “Important”, “Urgent”, and “From my spouse”–this would be an email with three tags. When this is downloaded to Apple Mail, 3 copies would be made in three mailboxes.

GMail tags all incoming email with “Inbox” so that it shows up in your Apple Mail In Box. And if you delete it from your In Box in Apple Mail, and don’t otherwise tag it, then the email is untagged, or “archived”–being in your email stream, but not tagged. In GMail, you could find it your “All Mail” mailbox.

If you set up GMail to expose “All Mail” to email clients, then these mails will be visible in Apple Mail. This means that any email in “Inbox” will have two copies in Apple Mail–one in “All Mail” and one in “Inbox”.

However, for some reason, Apple Mail cannot delete the mail from the "All Mail mailbox. See https://discussions.apple.com/message/16609690 “Gmail ‘All Mail’ folder. Why can I not delete messages?” for more. What happens is that if you expose the “All Mail” folder in GMail, then Apple Mail thinks that it can delete mail from that folder. But if you move a mail to the trash, a copy is moved to the trash. And when you empty the trash, the copy is deleted, but the original is still in your email stream. However, Apple Mail thinks that it did delete the email, so it removes the copy on your computer, so it looks like it did go away. You can prove that it did not by going to GMail, where you will find the supposedly deleted email. And, about 20 minutes later, Apple Mail will “discover” the email and it will again show up in your All Mail folder.

I spent a few hours deleting about 1,000 spams from my All Mail folder using Apple Mail–and every 20 minutes or so they would all “come back from the dead”, until I figured this out.

This is a SpamSieve issue because SpamSieve’s directions (file:///Applications/SpamSieve.app/Contents/Resources/SpamSieve%20Help/setting-up-apple-mail.html) suggest that you set up the Spam folder as a local folder, and set up a filter that moves all email to that folder so it can be evaluated by SpamSieve. This rule works fine for new email (mail in your Inbox). As long as the mail has a tag, then Apple Mail can handle it correctly–the mail is copied to a local mailbox, deleted from GMail, and evaluated. If SpamSieve thinks that the mail is not spam, then it is copied back to the GMail Inbox for you to read.

The problem is that if you try to find spam in your “All Mail” box using “Train as Spam” or “Apply Rules”. Then what happens is that SpamSieve tries to copy it to your local folder and delete it from “All Mail”–but it can’t delete it. Then it evaluates the mail in the local folder. If SpamSieve thinks that the mail is spam, then it leaves it in the (local) spam box. But, the copy online has not been deleted.

So, I am going to give you a step by step solution on how to solve this.

First, you need to set up Apple Mail correctly. Follow the instructions at https://discussions.apple.com/message/16843068 “How do I set up mailbox for gMail address in Mail version 4.5?” to set up Apple Mail for your GMail accounts. This will map Apple Mail’s spam folder, trash folder, and inbox folder to GMail’s tags “Spam”, “Trash” and “Inbox”.

Next, decide if the issues I am describing are a problem for you. Go to Gmail, and look in your Inbox and in your “All Mail” folders. If all of the mail that you care about is in your in box, and if you are not going to archive it by moving it out of the Inbox, then Apple Mail will work fine with the default settings in SpamSieve. But, if you like to archive your mail by removing all tags, or if, like me, you have a bunch of important mail that is not tagged (but you are not sure how that happened), and a lot of spam mixed into that untagged mail, you need to set up Spam Sieve differently.

First map the Inbox, Spam and Trash folders so that Mail understands them–following the directions at https://discussions.apple.com/message/16843068, for each GMail account.

Then make “All Mail” visible to Mail: In GMail, choose settings (in the gear menu), and then choose labels. Then check “Show in IMAP” for All Mail. When you return to Apple Mail, a new mailbox should appear. This has every email in your account–both received and sent. Do this for every GMail account that you have. You may need to wait a few hours for Mail to pull down all the mail in this new mailbox.

Now follow the instructions at <http://c-command.com/spamsieve/manual-ah/separate-spam-mailboxes> to set up SpamSieve so that there is a separate spam folder for each GMail account. You can also set up SpamSieve so that training is dealt with correctly <http://c-command.com/spamsieve/manual-ah/change-settings-apple>. (There is a problem with the second step that I will explain below).

At this point, SpamSieve filtering should be working. Every new piece of email should be moved to the GMail spam folder if SpamSieve thinks that it is spam. And, if you select a bunch of emails in your “All Mail” folder and choose “Apply Rules” the spam emails should also be moved to the Spam folder and non-spam will be moved back to your Inbox. To delete your spam, just wait 30 days (when GMail will delete it), or delete the contents of the spam folder and empty the trash.

Why does this work, when the default setup does not? It all is because of the fact (a bug, I guess), that Apple Mail can only delete mail that has a tag associated with it. If you set up SpamSieve to move your email to a local folder, then it tries to delete the email from GMail. It can’t do this for emails that are only in the “All Mail” folder, and it ends up moving a copy, and deleting that copy. But if your Spam folder is on line (at GMail), then moving an email to the spam folder means that you add the “Spam” tag to it. Once the email has a tag, then Mail can handle it, and move it to the trash (delete tag “Spam”, add tag “Trash”), and then delete it (purge folder/tag “Trash”).

A consequence of making your “All Mail” folder visible to Mail is that every email in your inbox will show up twice–once in All Mail and once in your In Box. You can fiddle around with smart folders to hide this fact, or you can just live with it. And, of course, when you delete the mail from Inbox, it will still be in All Mail (archived), but that was true whether or not you made All Mail visible to Apple Mail.

The final “gotcha” is what happens when you select an email and choose “Train As Spam”. What SpamSieve wants to do is to delete the email from your Inbox, and move it to the Spam folder. (Or, in GMail terms, it wants to delete the “Inbox” tag and add the “Spam” tag). However, due to somebody’s confusion, it doesn’t always move the message to the correct spam folder. If you have one GMail account, it works great. But if you have two or more GMail accounts, SpamSieve asks Mail to move the message to the Spam folder of the first account listed. This is a problem, because rather than adding a “Spam” tag, this would mean deleting the message from the account it is in, and then copying the message to the other account, where you would add a “Spam” tag to move it into that account’s spam folder. But, as we have said, Mail cannot delete any message that does not have a tag. This means that “Train as Spam” will not delete any untagged email, unless the email is in the first account in the left-hand panel of Apple Mail.

I am sorry that this is so complicated. But I haven’t found a simpler solution to allow Apple Mail, GMail, and SpamSieve to peacefully coexist.

I think for the vast majority of Apple Mail users (with SpamSieve or otherwise), it’s best to hide the All Mail mailbox from Mail.

SpamSieve works this way by design—originally due to limitations of Apple Mail. I will investigate whether it is now possible for SpamSieve to determine the proper per-account Spam mailbox.

I have been thinking about this on my morning walk. The issue is that if you set up SpamSieve with a local Spam box (as recommended by the manual), then spam that is identified by Google will be deleted from All Mail (call it your mailstream), but spam that is identified by SpamSieve will not (but it will be deleted from your InBox). This means that searches in GMail (online) can return spam that you thought you deleted. If you can live with spam being retained in your mailstream, then follow Michael’s advice.

This behavior has answered something that has caused me to scratch my head all year. I kept finding mail from various accounts in my personal account. The “To” field would indicate that it should be in another account, but there it was. I suspect now that these messages were good messages I found in my spam folder, that I would select and “Train as good” to keep that kind of email from being incorrectly filtered. Now I know that I either have to rearrange the order of my mailboxes before using this option, or just move the email back into the correct Inbox by hand (and not fix SpamSieve’s corpus).

A

I was referring to training messages as spam. If you train messages as good, SpamSieve will be able to figure out the proper inbox if the message’s To or Cc contains an address that matches one of the addresses you’ve given the account in Mail’s preferences.

Ah, I see–I use various emails that go to a catch-all address. So this problem is probably a special case for those of us who (a) use multiple GMail accounts, and (b) have various identities in those accounts. Not a large population, and I can live with the issue. Thanks for your clarity on this.

You can list the addresses for your catch-all (separated by commas) in the “Email Address” field in Mail’s Accounts preferences.

This is addressed in SpamSieve 2.9.