Apple Mail Lion - Train as Spam moves to "Junk E-Mail" folder instead of "Spam" folde

[kmoreau893]

I am getting a lot of spam getting through Spamsieve lately, I'm not sure when this started but it seems I'm spending a lot of time manually training the same spam emails as spam. I'm still not sure why this is happening lately.

When i was attempting to troubleshoot it I was noticing all the "Train as Spam" emails were winding up in my "Junk E-Mail" folder rather than going to the Spamsieve automatically-filtered "Spam" folder. I don't know if this is normal or not, I suspect not.

I have a huge corpus, I think, with 29781 messages, 1271402 words, I'm wondering if this is making Spamsieve malfunction?

I also have a huge whitelist, not sure why, but it has about 10000 enabled rules, this doesn't seem right, and I actually don't know how most of them got there, I'm sure I added some of them using the 'train as good' but seems too large. Anyway, any advice is appeciated.

[Michael Tsai]

I am getting a lot of spam getting through Spamsieve lately, I'm not sure when this started but it seems I'm spending a lot of time manually training the same spam emails as spam. I'm still not sure why this is happening lately.

I recommend first following the Why is SpamSieve not catching my spam? page to check for a setup problem.

When i was attempting to troubleshoot it I was noticing all the "Train as Spam" emails were winding up in my "Junk E-Mail" folder rather than going to the Spamsieve automatically-filtered "Spam" folder. I don't know if this is normal or not, I suspect not.

The trained spam messages go to the Spam mailbox unless you’ve used the SpamSieve - Change Settings command to pick a different mailbox.

I have a huge corpus, I think, with 29781 messages, 1271402 words, I'm wondering if this is making Spamsieve malfunction?

It’s possible, but best to first check the setup and the log to see whether/why SpamSieve is predicting the spam messages to be good.

I also have a huge whitelist, not sure why, but it has about 10000 enabled rules, this doesn't seem right, and I actually don't know how most of them got there, I'm sure I added some of them using the 'train as good' but seems too large.

If you’ve received good messages from 5,000 unique senders, it’s normal for the auto-training feature to have created 10,000 whitelist rules (for each name and each address).

[kmoreau893]

I recommend first following the Why is SpamSieve not catching my spam? page to check for a setup problem.

I have read that.

The trained spam messages go to the Spam mailbox unless you’ve used the SpamSieve - Change Settings command to pick a different mailbox.

I have not changed that setting, it's set to "Spam" not "Junk E-Mail" That's why I'm confused.

It’s possible, but best to first check the setup and the log to see whether/why SpamSieve is predicting the spam messages to be good.

It seemed some of the senders that were assuredly spam were on my whitelist, for example, "Costco Gift Card." Others, not sure why. So not sure why that would be there. But I would rather not do so much detective work if I can help it.

If you’ve received good messages from 5,000 unique senders, it’s normal for the auto-training feature to have created 10,000 whitelist rules (for each name and each address).

OK, good to know. Thanks for your quick response, Michael.

[Michael Tsai]

I have not changed that setting, it's set to "Spam" not "Junk E-Mail" That's why I'm confused

When a spam message gets through to the inbox, are you using the “SpamSieve - Train as Spam” menu command? Or a keyboard shortcut or button?

It seemed some of the senders that were assuredly spam were on my whitelist, for example, "Costco Gift Card." Others, not sure why. So not sure why that would be there.

If SpamSieve thinks a message is good, it will add the sender to the whitelist. If it turns out that the message was spam, when you train it as spam SpamSieve will disable the whitelist rule.

But I would rather not do so much detective work if I can help it.

Well, you need to determine why the spam messages were uncaught. You can either figure this out by reading the log or send in a report so that I can help you.

[kmoreau893]

When a spam message gets through to the inbox, are you using the “SpamSieve - Train as Spam” menu command? Or a keyboard shortcut or button?

I usually use the keyboard shortcut, but the menu item also results in the message being put in my Junk E-Mail folder instead of the spam folder. I also have to add that I occasionally use Outlook 2011 for Mac and have a Junk E-Mail folder(s) there and the behavior of manually train as spam is the same as mail.

If SpamSieve thinks a message is good, it will add the sender to the whitelist. If it turns out that the message was spam, when you train it as spam SpamSieve will disable the whitelist rule.

OK it makes sense then that SpamSieve is automatically adding new email addresses to my whitelist, mystery solved here.

Well, you need to determine why the spam messages were uncaught. You can either figure this out by reading the log or send in a report so that I can help you.

After a little whitelist cleanup yesterday, it seems there are much fewer spams getting through, so something clicked in. So maybe I can hold off on the troubleshooting until it gets severe. Just the Junk E-Mail spam folder is still an issue that I'd like resolved if possible.

[Michael Tsai]

I usually use the keyboard shortcut, but the menu item also results in the message being put in my Junk E-Mail folder instead of the spam folder. I also have to add that I occasionally use Outlook 2011 for Mac and have a Junk E-Mail folder(s) there and the behavior of manually train as spam is the same as mail.

Are Apple Mail and Outlook connecting to the same mail account? With Outlook, SpamSieve uses the “Junk E-mail” mailbox, so maybe there’s no problem with Apple Mail at all.

If the “Change Settings” command says that it’s set to use a mailbox called “Spam”, I can’t see any reason why it wouldn’t be doing that, but I suppose you could change it to, say, “Spam2” and if that has any effect.

After a little whitelist cleanup yesterday, it seems there are much fewer spams getting through, so something clicked in.

If you’ve been properly correcting all the mistakes, there should be no incorrect whitelist rules that are enabled, so I would not expect cleaning out the whitelist to have any effect.

[kmoreau893]

Are Apple Mail and Outlook connecting to the same mail account? With Outlook, SpamSieve uses the “Junk E-mail” mailbox, so maybe there’s no problem with Apple Mail at all.

Outlook I use occasionally when I have to and then quit it. It's not running all the time, I realize it uses that Junk E-Mail mailbox. But the 'Train as Spam in Apple Mail is going to the Junk E-mail mailbox.

If the “Change Settings” command says that it’s set to use a mailbox called “Spam”, I can’t see any reason why it wouldn’t be doing that, but I suppose you could change it to, say, “Spam2” and if that has any effect.

Created Spam2 folder on my Mac, used the Train As Spam, wound up in Junk E-Mail.

If you’ve been properly correcting all the mistakes, there should be no incorrect whitelist rules that are enabled, so I would not expect cleaning out the whitelist to have any effect.

OK, I'll see how it all works and keep you posted, thanks for all your quick replies and help.

[Michael Tsai]

Created Spam2 folder on my Mac, used the Train As Spam, wound up in Junk E-Mail.

Try deleting the Spam2 mailbox and using “SpamSieve - Change Settings” to tell it that you want to use Spam2.

[kmoreau893]

Try deleting the Spam2 mailbox and using “SpamSieve - Change Settings” to tell it that you want to use Spam2.

Already did that. Auto filtered spam winds up in Spam2, manually trained spam winds up in Junk E-Mail. Spam2 was auto created, it appeared with the manually trained spam.

[Michael Tsai]

Auto filtered spam winds up in Spam2

It should still be going to Spam. In the future, please only make the exact changes that I suggest. I can’t help you if there’s other stuff going on that I don’t know about.

At this point, I guess we need to back up and check assumptions. Please confirm that you have the SpamSieve rule set up exactly as shown here, that it’s at the top of the rule list, and that there are no other junk/spam rules. I advise not opening Outlook at all until the Apple Mail stuff is sorted out.

Also, please post or e-mail a screenshot showing the “Junk E-mail” mailbox in your mailbox list.

Spam2 was auto created, it appeared with the manually trained spam.

This is more evidence that the training command is using the correct mailbox. Is the “Spam2” mailbox stored “On My Mac” or in your mail account?

[kmoreau893]

It should still be going to Spam. In the future, please only make the exact changes that I suggest. I can’t help you if there’s other stuff going on that I don’t know about.

At this point, I guess we need to back up and check assumptions. Please confirm that you have the SpamSieve rule set up exactly as shown here, that it’s at the top of the rule list, and that there are no other junk/spam rules. I advise not opening Outlook at all until the Apple Mail stuff is sorted out.

Done

Also, please post or e-mail a screenshot showing the “Junk E-mail” mailbox in your mailbox list.

Attached

This is more evidence that the training command is using the correct mailbox. Is the “Spam2” mailbox stored “On My Mac” or in your mail account?

Sorry Michael, I mistyped, the Spam2 box contained the AUTOMATICALLY caught spam once I configured Mail rules to use Spam2 as I though you'd suggested. The "Junk E-Mail" folder is still getting the manually trained spam.
Anyway, changed it back to the "Spam" folder as you recommended in here and now automatically caught spam is going there as is expected. Manually trained spam is still going to "Junk E-Mail" which is stored on the iMap server. Spam and Spam2 folders reside on my mac.

Thanks again for all your quick assistance.

[Michael Tsai]

Sorry Michael, I mistyped

So are you saying that the “Spam2” mailbox was not auto-created when you trained a message as spam? If you don’t remember, please delete it and see.

The "Junk E-Mail" folder is still getting the manually trained spam.

It seems unlikely that SpamSieve is doing this because it won’t even look at the server mailboxes by default. When you use the “Change Settings” command, what does it say for the second question about “a local spam mailbox (rather than on the server)”?

What happens if you delete the “Junk E-mail” mailbox?

[kmoreau893]

So are you saying that the “Spam2” mailbox was not auto-created when you trained a message as spam? If you don’t remember, please delete it and see.

No. After I created the Spam2 folder, as directed by you, and choosing it in the rules and noticing that auto filtered spam was going into Spam2, as directed by you, I deleted Spam2 from the sidebar in Apple Mail. However, as I thought you had asked, I made sure I still had it selected in the Apple Mail Rules. It seems it was auto created somehow by Mail because it was there again after I deleted it. I deleted it again, selected the Spam folder instead of Spam2, and it was no longer auto created.

Regarding the "Junk-Email" folder residing on the server and getting the trained spam from SpamSieve:

It seems unlikely that SpamSieve is doing this because it won’t even look at the server mailboxes by default. When you use the “Change Settings” command, what does it say for the second question about “a local spam mailbox (rather than on the server)”?

Please clarify which "Change Settings" command your referring to?

What happens if you delete the “Junk E-mail” mailbox?

Did it, now the manually trained spam correctly goes into the local "Spam" folder, as it should. However, I'm concerned that this might affect my Outlook SpamSieve operation, is this correct or do I just need to adjust the rules in Outlook? I don't anticipate using Outlook with SpamSieve in the future, but just in case I'd like to know what to do.

Again, thanks for your persistent help and all the time you've taken with me.

[Michael Tsai]

Please clarify which "Change Settings" command your referring to?

I’m referring to the SpamSieve - Change Settings command in Mail’s Message menu. Had you not used this before to make sure the settings were correct?

Did it, now the manually trained spam correctly goes into the local "Spam" folder, as it should. However, I'm concerned that this might affect my Outlook SpamSieve operation, is this correct or do I just need to adjust the rules in Outlook? I don't anticipate using Outlook with SpamSieve in the future, but just in case I'd like to know what to do.

The Outlook rule is set to use a fake folder that (in the user interface) is called “Junk E-mail”, however the actual folder (on the server or on your Mac) that this maps to is controlled in the Accounts settings. In my experience, Outlook figures out how to do the right thing, and you don’t need to change this setting.

[kmoreau893]

I’m referring to the SpamSieve - Change Settings command in Mail’s Message menu. Had you not used this before to make sure the settings were correct?

No, I had not used this before. I think this was the culprit all along. When I used "Change Settings" it asked where trained spam messages should be stored, and filled in was the "Junk E-Mail". As directed by the dialog, I changed this to "Deleted Items". Now when I train spam it winds up in my "Trash" On My Mac, which is where other deleted items wind up. This is good. Not sure why, but I complete missed this in the manual, and I think this is the first time you brought it up as well, perhaps that was too obvious, but maybe not for me. :)

The Outlook rule is set to use a fake folder that (in the user interface) is called “Junk E-mail”, however the actual folder (on the server or on your Mac) that this maps to is controlled in the Accounts settings. In my experience, Outlook figures out how to do the right thing, and you don’t need to change this setting.

Thanks for all your help, I think problem is solved. False negatives seem less as well, though many are still getting through. Wonder if I should adjust the Spam Catching strategy to more aggressive, right now it set to the middle. I am getting 20-30 false negatives a day, it looks like from the False Negatives folder. I'm going to set it to 75% and see what happens.

[Michael Tsai]

No, I had not used this before. I think this was the culprit all along. When I used "Change Settings" it asked where trained spam messages should be stored, and filled in was the "Junk E-Mail". As directed by the dialog, I changed this to "Deleted Items". Now when I train spam it winds up in my "Trash" On My Mac, which is where other deleted items wind up. This is good. Not sure why, but I complete missed this in the manual, and I think this is the first time you brought it up as well, perhaps that was too obvious, but maybe not for me. :)

I brought this up in my very first reply, and you said that it was set to “Spam”. But I guess you were actually looking at a different setting—hence all the confusion above. SpamSieve always uses the “Spam” mailbox unless otherwise directed, so I think you must have changed this setting long ago and forgotten about it.

False negatives seem less as well, though many are still getting through. Wonder if I should adjust the Spam Catching strategy to more aggressive, right now it set to the middle. I am getting 20-30 false negatives a day, it looks like from the False Negatives folder. I'm going to set it to 75% and see what happens.

Such a large number of false negatives probably means that there’s something wrong with the training. Have you been correcting all the mistakes? At this point, it would probably be a good idea to reset the corpus and re-train SpamSieve.

[kmoreau893]

I brought this up in my very first reply, and you said that it was set to “Spam”. But I guess you were actually looking at a different setting—hence all the confusion above. SpamSieve always uses the “Spam” mailbox unless otherwise directed, so I think you must have changed this setting long ago and forgotten about it.

Yep, you had it from the beginning. This had to do with my carelessness in really reading and understanding what you directed me to do. Sorry about that and your wasted time, however I feel I have a better handle on how SpamSieve works, and that you have a pretty good product able to deal with the complexities of spam.

Such a large number of false negatives probably means that there’s something wrong with the training. Have you been correcting all the mistakes? At this point, it would probably be a good idea to reset the corpus and re-train SpamSieve.

OK, I'll give it a try.