Accuracy statistics are static?

After several years of usage I seem to be noticing rough edges in SpamSieve performance and wonder if I need to remove it entirely and start over? Recently posted regarding inability to catch the lates spam that arrives with a .gif attachment displaying the spammers message as a visual text. Michael recommended resetting the corpus and retraining which I did but seems to have not made much difference, the .gifs still come through. I also noted that the accuracy statistics seem stuck, ie not changing. They show 0 false positives and 3 false negatives and do not change as I process additional mail using the scripts via keyboard shortcuts as recommended? I have pasted my entire statistics below, perhaps Michael can offer a recomendation? I notice that resetting the corpus had no impact on the high number of rules and perhaps that may be part of the problem?

Filtered Mail
23 Good Messages
566 Spam Messages (96%)
38 Spam Messages Per Day

SpamSieve Accuracy
0 False Positives
3 False Negatives
99.5% Correct

Corpus
229 Good Messages
608 Spam Messages (73%)
39326 Total Words

Rules
4678 Blocklist Rules
2163 Whitelist Rules

Showing Statistics Since
9/30/06 12:54 PM

There’s no particular benefit to removing everything and starting over. Are the Filtered Mail statistics updating when you receive new mail? One reason the number of false negatives might not be updating is that the messages in question might not actually be false negatives. You can check SpamSieve’s log to make sure that SpamSieve had actually predicted these messages to be good. If you see only “Trained” log entries for the mistakes, then there might be a setup problem. Another reason the statistics might not be updating is if the file:

/Users/<username>/Library/Application Support/SpamSieve/History.db

is damaged. In that case, quitting SpamSieve and renaming it to HistoryOld.db would help.

Resetting the corpus does not affect the number of rules, and having lots of rules is not a problem.

Logs attached
Good Morning Michael, sent a pdf of my logs attached, perhaps you can see a pattern there?

I see some “Trained: Spam (Manual)” entries with no corresponding “Predicted: Good” entries, so my guess is that there’s a setup problem. You can try to resolve it by moving the SpamSieve rule higher in the list. This page has some more information. I’d need to know more about your mail program’s setup to help you beyond that.

On Rules
Hi Michael:
Operating system is Mac OS-X 10.4.8 and MS office 2004 edition currently Entourage is at version 11.2.5, and the database was compacted and rebuilt within the last month. I had also used the page you referenced to clean up my rules, currently only rules in Entourage are the two you specify, in the correct order. See attached screen shots to verify I duplicated what you recommend.

That’s actually not the correct order. “SpamSieve - Move If Spam” should be the first rule.

Do you have any Mailing List Manager rules? Those might be interfering.

Also, if you select a spam message that got through and choose “Message > Apply Rule > All Rules,” does that add a “Predicted:” entry to the bottom of SpamSieve’s log?

Reply

  1. My mistake, changed the order.
  2. Much to my surprise I found ten mailing list rules but they look to be very specific and were added to allow specific software and shareware authors to email updates, do I need to delete them?
  3. I will have to wait for some spam to come through and try it, will post back.

I’ve found that mailing list manager rules can sometimes prevent Entourage from applying the regular rules (and thus, SpamSieve) even if the MLM rules are disabled or the criteria don’t match the messages in question. So, yes, if everything else looks to be setup properly, I think it’s worth re-creating them as regular rules and deleting the MLM rules.

OK Michael, thanks for the help, I will try recreating them as regular rules (I am assuming those would just be Entourage rules) placed in order below the two SpamSieve rules correct?

Right.

Interesting
Well deleting all those list managers rules had the interesting byproduct of shooting this email notification from your forum to the junk folder (previously it was going straight to the inbox with no problem), so I guess that proves your theory that somehow the list manager rules were messing with things. I told SpamSieve that mail from your forum was good mail and we shall see how things go from here.
You must have the patience of a saint to keep fighting this scourge, I am one of the many who appreciate your efforts and would not be without your excellent product. I am a dual platform consultant and recommend your product to my Macintosh clients and use ChoiceMail for the PC crowd.

Success finally!!!
Hi Michael;
Thanks again for all the assistance yesterday on a Sunday no less. The preliminary results seem to be spectacular and SpamSieve seems to have regained a level of accuracy that it has been missing for some time now. It happened gradually so I did not notice it but assisting a client that recently purchased the program on my recommendation renewed my interest when I realized her new install was far more accurate than my existing copy with years of training.
Deleted all the mailing list rules and created corresponding Entourage rules placed down the list from the SpamSieve rules. Also renamed the database history to create a new one, and the false start yesterday with false positives such as the notification from this forum was a careless mistake due to a new Entourage rule where I accidently set the execute parameter to “unless all criteria are met” instead of “if all criteria are met” needless to say that made a mess quickly but was easy fo find and correct. Now with all the changes you recommended SpamSieve seems to be deadly accurate and the logs are looking good with expected results for most messages. Well worth the effort to once again have the clean inbox that David Pogue spoke of in his glowing recommendation of your excellent program. David is not easily impressed so his recommdation speaks highly of the value of your program to manage the volume of email he deals with. Thanks again for your great support and a great program that saves many hours of wasted time dealing with trash in our inboxes.

tortoise, can you give me an example of a “regular rule” that you created from a “mailing list rule”? I had the same problem with mailing list rules in Entourage. I deleted all my mailing list rules but have not re-created them as regular rules yet.

Les

Per your request
Here is one Lestere, I have about ten rules in the list below the two primary SpamSieve rules and everything seems to be running much better than it has in a long time. Getting occasional false positive on folks in my address book but once corrected all seems well again. Michael’s guess that the mailing list rules gum things up seems to be correct.

That shouldn’t happen. (Or do you mean false negatives, i.e. spam messages from addresses in your address book?) Be sure to turn on SpamSieve’s Use Mac OS X Address Book preference and to turn on syncing in Entourage so that it stays up to date. Then messages from people in the address book will never be marked as spam.

Also, I don’t recommend using that “Change status - Not junk E-mail” action, as that will interact poorly with SpamSieve’s auto-training. Instead, I recommend doing one of the following:

  • Let the occasional list message be marked as spam. If it actually is spam, great. If not, then you’ll be able to see that SpamSieve thought it was spam and use the Train Good command.
  • Put the list rules above the SpamSieve rules. Then SpamSieve won’t filter them, and so none of them will be marked as spam.

Reply

Not sure of correct terminology, some of my regular clients were marked as SPAM and sent to junk folder, I retrained and further messages came in fine? So you are saying that even though I don’t use Apple mail or address book I should set up the Apple address book and syncing to keep it all up to date.

As far as these rules are concerned I am beginning to think they are more trouble than they are worth? Seem to cause a lot of trouble and if I were to eliminate all the rules and just monitor my junk folder and train SpamSieve on any mistakes and add address of mistakes to address book then eventually SpamSieve will incorporate the rules that were deleted if I try that, correct?

I did turn on the items you suggested and will observe and see how it all goes now, continued problems and I may just delete all the rules except your two main ones and see if SpamSieve is happier that way.

I probably didn’t give you enough information… I’m interested in creating a regular rule that will allow the automatic downloading of pictures in an HTML newsletter, say MacWorld, a newsletter that I subscribe to. This is easily accomplished in a mailing list rule, but as we both know, these rules don’t play nice with SpamSieve.

Michael, feel free to chime in here!!!

Les

Les, what is the connection between mailing list messages and downloading the pictures? I don’t see how this is different when using regular rules. I think that, even if you tell Entourage to display the images, it will hide them for the messages that SpamSieve marks as spam. What is it that you want to work differently?

My problem is that I don’t know how to write a regular rule that will allow me to automatically display images in specific message. The spam message I am not worried about… I want Entourage to block all other images in messages that I don’t have a specific regular rule for.

Les

OK, now I understand. How did you make your list rule show the images before? I don’t see an option for that. Unfortunately, I don’t know of a way, via a rule or AppleScript, to make Entourage show the images for a particular message if the images are turned off in the preferences.