Frequently asked questions about Cleanfeed

  1. What kind of servers does Cleanfeed work with?
  2. I upgraded to INN 2.3, and now Cleanfeed doesn't work.
  3. My server is small. Won't filtering use too many resources?
  4. Is Cleanfeed still being developed?
  5. Does Cleanfeed employ content-based filters?
  6. How does Cleanfeed compare with Spam Hippo?
  7. I'm a spammer, and Cleanfeed is blocking my posts. What can I do?
  8. I'm not a spammer, but Cleanfeed is blocking my posts anyway. What can I do?
  9. I set something in cleanfeed.conf, but Cleanfeed doesn't seem to be recognizing it. What's wrong?
  10. Can I use that Cleanfeed logo graphic on my website?
  11. I'm not a news administrator, I'm just a user. Can I use Cleanfeed? What can I do?

Q: What kind of servers does Cleanfeed work with?

A: Cleanfeed is known to work with INN, Cyclone, Typhoon, Twister, Diablo, DNews, and NNTPRelay.

In addition, with third party help it is possible to run Cleanfeed on CNews. See this site for more information.


Q: I upgraded to INN 2.3, and now Cleanfeed doesn't work.

A: In INN 2.3, the way the message body gets passed to the filter was changed. This results in better performance (by not doing a memory copy), but also results in the body being passed as a read-only value. Unfortunately, the part of Cleanfeed that counts the lines in the message body is seen by Perl as a write operation, despite the fact that it doesn't actually change anything. The result is an error when running under INN 2.3. This error should only occur if you are running my (older) version of Cleanfeed.

The solution is to apply this patch.


Q: My server is small. Won't filtering use too many resources?

A: Back in the good old days, the answer to this question was "no". But Usenet has grown since then. Filtering a full feed these days requires a non-trivial amount of CPU.

My (older) Cleanfeed release will have more problems coping with a full feed than the current one. Install Marco's newer version for improved performance.

If you are finding that you don't have enough resources to filter the feed, it's likely that you don't have enough resources to take a full newsfeed anyway. Consider cutting down on your resource usage by asking your peers not to send you the large multipart binary groups.


Q: Is Cleanfeed still being developed?

A: Yes. Marco D'Itri has taken over maintenance of the software, as I had gotten to a point where I had little time to devote to the project. There were considerable changes I made to Cleanfeed before handing it over to Marco, but he has done some significant work on it since, including cleaning up some of my ugly code.


Q: Does Cleanfeed employ content-based filtering?

A: Yes, Cleanfeed contains some filters which examine message content to identify possible spam. The filters are based on types of content that correlate highly with spam. They are not designed to censor any ideas or people.

The reason for including content-based filters is because spammers like to play tricks to defeat attempts to filter their garbage, and some of these techniques can defeat a strictly-non-content-based multiposting filter. In order to better catch this spam, certain content-based filters are imposed, and tested to ensure that they in fact catch what they are supposed to catch.

All content-based filters can be disabled with a single configuration option.


Q: How does Cleanfeed compare with Spam Hippo?

A: Cleanfeed and Spam Hippo are different products with different approaches. Hippo is a decent filter which I understand works quite well (I have never used it myself). However, it does seem to have a rather high false-positive rate. In particular, many multipart binary posts are apparently considered spam by the Hippo. The spam statistics at newsadmin.com are generated by Hippo, and include detailed listings of what was considered spam, and a lot of it is quite obviously not spam. For this reason, despite not having actually used Hippo, I am unable to recommend its use.

As for particular differences between the programs, here are some:

  • Cleanfeed contains some content-based filters. (These filters can all be turned off with a single configuration option.) Hippo is non-content-based.
  • Hippo can use a larger database, meaning a longer memory, which makes it better at catching "slow" spam.
  • Cleanfeed can block binaries from non-binary newsgroups.
  • Cleanfeed also contains some protection against control message attacks.
  • If you hate HTML posts, it can get rid of those for you too.
  • Cleanfeed is configurable to the point of absurdity. It also contains hooks for local code in a cleanfeed.local file, so you can add your own filters and have them survive an upgrade of the main program.
  • Cleanfeed is open-source, Hippo is not.

Note that, on INN, it is possible to run both Cleanfeed and Hippo on the same server. Running standalone on Highwind servers, you'll have to choose one or the other. Some people run one filter on a transit server, and the other on a reader server. A filter multiplexer was planned for Cyclone before the software was sold to a different company, but I'm not sure if the new owners of the software intend to pursue the idea.


Q: I'm a spammer, and Cleanfeed is blocking my posts. What can I do?

A: Stop spamming.

If you don't understand what spam is, or believe that what you are doing is not spam, please contact your news administrator or upstream ISP for more information. They should be your first stop, since it is your contract with them that you are very probably breaking.

You can also ask about the situation in the newsgroup news.admin.net-abuse.usenet. If you do, please include enough information for people to research the situation on their own. At a minimum, they will need to know the address you used on the posts which were blocked as spam, and sample subject lines. Please be polite, but understand that there are people in this group who will flame you no matter what you say. Listen to the reasonable ones.

If you understand what you have done, but feel that the rules or procedures or policies are somehow wrong, the place to talk about it is news.admin.net-abuse.policy.

Please do not contact me directly on this subject unless you really have to. Try to understand that I am not blocking your articles, your ISP is; I do not force anyone to run my software or tell them how to configure it. In addition, since Cleanfeed is easy to modify, it is entirely possible that the piece of code that is catching your posts wasn't even mine. Finally, I wrote this program, so if you're spamming you probably won't find much sympathy here.

Please note that being "on-topic" or "relevant" has exactly nothing to do with whether your posts were spam. Spam is not about topic. Usenet is not your free advertising medium.

If you were spamming, but you've stopped, and you just want to start over and play by the rules, but Cleanfeed continues to block your posts, post to news.admin.net-abuse.usenet.


Q: I'm not a spammer, but Cleanfeed is blocking my posts anyway. What can I do?

A: Contact your news administrator first. You may not feel that you are spamming, but he might disagree.

If you really aren't spamming, contact your news administrator anyway. He is the one who installed Cleanfeed, and it is entirely possible that your articles are being rejected due to a local modification made by your newsadmin, possibly an honest mistake. They may also be blocked because of a simple misconfiguration. If the rejection is due to local policy enforcement, he would also be the best person to explain this to you.

Please don't contact me about this if you're not happy with the response you get from your news admin or ISP. If your ISP's server is filtering your articles, I have no way to gain access to the server logs or the configuration files, so I have literally no way to find out why your articles are being filtered. Additionally, of course, I am no longer the maintainer of the software, so I am in even less of a position to help you.

If you've contacted your news administrator, and your non-spam posts are still being blocked, and you can't get the newsadmin to change his mind, and you basically think you'd be better off somewhere else, it is possible to get your Usenet access from someone other than your actual ISP. You might be interested in my page about Premium Usenet providers.


Q: I set something in cleanfeed.conf, but Cleanfeed doesn't seem to be recognizing it. What's wrong?

A: The most common cause of this is file permissions. Cleanfeed needs to be able to read the cleanfeed.conf file in order to load it up. Make sure that file is readable by your news user. Many people install Cleanfeed as root, and, depending on the umask, the files may end up owned by root with permissions of 600. Since the server usually isn't running as root, it can't read the file at run-time.

Also, since cleanfeed.conf is evaluated as Perl code, check the file by typing perl -cw cleanfeed.conf to make sure there are no compilation errors. You will get a few warnings about variables being used only once, but you should see cleanfeed.conf syntax OK. If not, fix the error and try again.


Q: Can I use that Cleanfeed logo graphic on my website?

A: Sure, go ahead. As long as you're using Cleanfeed, feel free to use the logo on your news statistics or information pages. You can make it a link back to my Cleanfeed page if you want.

If your page is black, feel free to reverse the colors to match.


Q: I'm not a news administrator, I'm just a user. Can I use Cleanfeed? What can I do?

A: Cleanfeed is a program that runs on news servers. You cannot run it on your home computer to filter what you see in your newsreader, unless you also run a news server locally (and even then, it would be far more effective to have Cleanfeed running on your ISP's server).

If you're getting tired of all the spam you're seeing in your favorite newsgroups, get in touch with your ISP or Usenet provider and find out what they are doing about it. Ask them if they are filtering. If not, ask why not. It's possible they just never thought about it, because no one brought it up with them before. Ask them to run Cleanfeed. Give them the URL. They can also call on the wisdom and experience of the regulars at news.software.nntp, who include some of the top news administrators on the net.

Whatever you do, one thing is important -- don't keep quiet, assuming they're hearing about it from someone else. The more people they hear it from, the better. Make your concerns known. If your ISP refuses to filter, leave, and let them know why. Find an ISP or Usenet provider who is willing to help Usenet survive. In the end, it's your monthly fees that matter.


Cleanfeed page

Jeremy Nixon - updated 16 August 2002 Filtering by Cleanfeed