Guidance for Blacklisting and Watching

Blacklisting or watching a keyword or a web site address cause the spam detection bot SmokeDetector to trigger an alert whenever that keyword or web site address appears in a post. In other words, it basically says that any post containing this expression is spam, or at least suspicious.

Blacklist and watchlist types

Blacklists

Website Blacklist

The website blacklist consists of a list of websites associated with known spam that automatically raise suspicion when posted anywhere on Stack Exchange.

Blacklisting a website makes SmokeDetector report every post that is posted or modified with a link to the website (formatted as a link or otherwise) in its text.

The website blacklist is maintained in the SmokeDetector GitHub repository, specifically in the file blacklisted_websites.txt.

Keyword Blacklist

The keyword blacklist consists of a list of regular expressions which are frequently seen in spam, and only very rarely outside of spam posts.

Blacklisting a “keyword” (which can actually be a regular expression matching a phrase or a more-complex expression with alternatives, like find (?:true )?love which matches either of “find love” or “find true love”) causes any post which matches it to be reported as probable spam by SmokeDetector. Matches are not reported in the middle of a word; the keyword expression “dog” does not match “doggone” or “endogenous”.

The keyword blacklist is maintained in the SmokeDetector GitHub repository, specifically in the file bad_keywords.txt.

Numbers Blacklist

Similar to blacklisted keywords, but for numbers which the spammer intends to be interpreted by a human as a number (i.e. not for numbers used in URLs, where it is primarily interpreted by a machine). This list is primarily intended for phone numbers, but is not exclusive to only phone numbers. These are not regular expressions. They are just text. Entries are tested both verbatim and with a moderate amount of obfuscation intended to obfuscate numbers removed from the posts.

Searches on metasmoke in order to determine TP and FP should generally use the search that results from the Loose Number Search on MS bookmarklet. However, that search is imperfect, so arguments can be made for including more results or excluding some of the results, if the results wouldn’t be detected by the numbers detection.

Numbers which are substantially repetitive or which contain incrementing or decrementing patterns should not be added to this list, as they can result in too many FP. Such numbers should be added to the keyword lists in forms that are more restrictive than the numbers detection uses. The real-world examples we have of such problematic numbers are 222333444, 53535353, and +91 123456789 which ended up being 31 Total / 3 TP (9.68% TP) / 28 FP / 0 NAA compared to the 99.61% TP of all other entries that have been on the numbers watchlist.

Blacklisted Users

Each SmokeDetector instance maintains a very dynamic list of users which are “blacklisted”. Users are added to the user blacklist when a post they created receives tpu feedback. Users are removed from the user blacklist when a post they created is given fp feedback (not to be confused with fpu feedback which adds a user to the username whitelist). Users can also be manually added to the user blacklist with !!/adblu, or removed with !!/rmblu.

A user being on the user blacklist causes any post they authored to be reported when it’s scanned, but they will be immediately removed from the user blacklist if one of those reports gets fp feedback. A user should be added to the user blacklist when it is substantially likely that posts by them which are scanned in the future will be true positive. In general, users should not be added to the user blacklist for vandalism.

Watchlists

Watched Expressions

“Watching” an expression causes SmokeDetector to report it just like a blacklist expression, but the rule weight is kept at one, in order to prevent matches from triggering autoflagging. The watchlist is considered an “experimental” detection reason. Posts which are detected for only “experimental” reasons are reported into only a limited set of chat rooms, because each room has to opt-in to receiving reports with only experimental detections. All reports are always sent to Charcoal HQ. That means you can use !!/watch to try out different patterns experimentally, just to get an idea of what sorts of posts match a particular expression.

The list of watched expressions is maintained in the SmokeDetector GitHub repository, specifically in the file watched_keywords.txt. The format is slightly different from the other similar files; each entry is a tab-delimited record which includes a date stamp (expressed as Unix epoch, i.e. seconds since midnight Jan 1 1970 UTC), the user name of the person who added the expression, and the regular expression to watch for.

Watched Numbers

Similar to watched expressions, but for numbers which the spammer intends to be interpreted by a human as a number (i.e. not for numbers used in URLs, where it is primarily interpreted by a machine). This list is primarily intended for phone numbers, but is not exclusive to only phone numbers. These are not regular expressions. They are just text. Entries are tested both verbatim and with a moderate amount of obfuscation intended to obfuscate numbers removed from the posts.

See Numbers Blacklist above for detailed information about searching on MS for these numbers and about the fact that numbers which are substantially repetitive or which contain incrementing or decrementing patterns should not be added to this list.

Rules for Blacklisting and Watching

We have established the following rules for watching and blacklisting.

General true positive and false positive requirements

A primary goal of the blacklists is to have a very high percentage of true positive (TP) detections. While it’s impossible to know what will happen in the future, the following general criteria for existing TP and false positive (FP) detections are used to make it much more likely that entries which are added to the blacklists will be seen only in TP posts in the future.

• At least one of the following for the entry is true:
  • There are at least five true positive hits on metasmoke, with no false positive hits on metasmoke; and at least one of those true positive hits is both below the default autoflagging threshold weight (currently, 280) and is not older than six months.
  • There are at least three true positive hits on metasmoke, with no false positives on metasmoke, and it is part of a preexisting campaign. (See below.)
  • There are more than twenty true positive hits on metasmoke in the last six months, and no false positives.
  • There are recent true positive hits on metasmoke, and more than 30 true positive hits on metasmoke overall, and no false positives on metasmoke.

The general target for %TP is in 95%+ range (with some detection reasons below this). Existing blacklist entries which have a %TP rate notably below the overall %TP for the detection reason in which they are included should be either A) adjusted to improve their %TP (i.e. reject more FP posts), or B) moved to the watchlist.

Requirements for specific blacklists

Blacklisted websites:

• Reserved for sites which we are highly confident are used only in spam. You may add a site to this list if the following is true:
  • The entry for the site must meet the general true positive and false positive requirements (above).

Blacklisted keywords:

• Reserved for phrases which we are highly confident are used only in spam. You may add a phrase to this list if all the following are true:
  • The phrase has been used repeatedly in recent spam, has no false positives in metasmoke, and searching on Stack Exchange indicates that it is not a common phrase on any site in the network. When searching on Stack Exchange, you need to search in text, in URLs, and in code.
  • The entry must meet the general true positive and false positive requirements (above).

Blacklisted numbers:

• Reserved for human-interpreted numbers which we are highly confident are used only in spam. You may add a number to this list if it is not a repetitive pattern or an incrementing or decrementing sequence (see above for examples) and at least one of the following is true:
  • The entry for the number meets the general true positive and false positive requirements (above).
  • The entry has at least two true positive hits on metasmoke, with no false positive hits on metasmoke; and at least one of those true positive hits is both below the default autoflagging threshold weight (currently, 280) and is not older than six months; and there is no occurrence of the number on Stack Exchange in reasonable searches.
  • There are more than five true positive hits on metasmoke in the last six months, and no false positives on metasmoke; and there is no occurrence of the number on Stack Exchange in reasonable searches.
  • Phone numbers used in posts which are part of an actual spam wave may be blacklisted upon first true positive occurrence, as long as there are no false positives on metasmoke; and there is no occurrence of the number on Stack Exchange in reasonable searches.

Blacklisted usernames:

• Reserved for usernames which we are highly confident that are used only in spam. You may add a username to this list if all the following are true:
  • The username has been used repeatedly in recent spam which use multiple accounts, or the same account on multiple SE sites, has no false positives in metasmoke, and searching on Stack Exchange indicates there are not a notable number of users which use that name and it’s not likely to be used by new users (e.g. it’s not likely to be someone’s real name).
  • The entry for the username must at least meet the general true positive and false positive requirements (above). In general, username blacklist entries should substantially exceed these requirements. Blacklisting (or even watching) a username should be substantially more rare than adding entries for other types of content. It is quite harmful to innocent users if their posts are routinely detected just because of the username which they have chosen. When an entries targeting usernames gets a false positive, it should be reviewed to see if it can be adjusted to not detect the FP. If it can’t be adjusted, then the report should get fpu feedback in order to whitelist the user from username detections. However, fpu feedback does not fully exempt users from such detections. It only prevents reporting if the only thing the post was detected for were detections in the username.

Recommendations before Watching an item:

Unless a post is obvious spam, we would recommend to check a few things before you add a watch. It can be hard to keep up with new terms, libraries and new projects (nuget, chocolatey, mustache, etc.).

• Search for the URL or keyword on Stack Exchange.
  • Use url:example.com in the search bar for a website.
  • Search a keyword on its own or use "key word" if it’s multiple words, otherwise it will search for all words, which can give unfortunate results.
• Check currently open PRs and see if your watch might already be added.

Watched keywords:

• Anything is game, but be prepared to have it removed if circumstances require it.
  • We will be removing patterns periodically; you can reduce the risk of having useful patterns removed by proactively removing patterns you no longer are interested in, or which produce very uncertain value.
  • Autoflagging weight for this reason is technically forced to stay at 1.
  • SmokeDetector will regard these rules as “experimental”. Reports which are only detected for “experimental” reasons will only be posted in rooms that have specifically opted-in to receive reports for “experimental” reasons. Charcoal HQ does receive “experimental” reports.

Preexisting Campaign

The criteria for blacklisting an entry is relaxed (see above) when the entry is promoted in spam posts which we identify as being part of a “preexisting campaign”. This means the spam posts containing the proposed new blacklist entry can be directly tied to other recent spam posts which contain campaign-specific blacklisted entries which already fulfill the stricter blacklisting criteria for at least one entry which is specific to that campaign (i.e. not just generic blacklisted terms like “keto”). In practice, this helps us trigger blacklisting early for sites or products which are clearly part of a series of spam from the same, or closely associated, spammers, where we can be reasonably sure that the only purpose of the entry is to have a different URL, or slightly different wording, than the other sites used in the preexisting campaign. Note that what is considered a “preexisting campaign” should be narrowly construed, and does not mean everything in an overall type of spam or spam which is for an overall type of product. (This is called “snowshoe spamming” – the tactic is to spread your footprint across many sites or slightly different keywords, so as to evade trivial duplicate detection.)

Some common criteria for establishing a campaign include:

• The same product name has been promoted recently by a known spammer.
  • Obviously, generic promotions for common products are not sufficient. But, for example a product which was unknown until recently and is now visible in posts from several accounts qualify.
• The same IP address is connected exclusively to spam domains.
  • Note that some IP addresses (e.g. Shopify, AWS) are shared between a large number of domains; it is not sufficient that a few of those are also spam domains. But, a common spam hosting pattern is that a single IP address is only seen in spam posts with several different domain names.
• The same spam post (or user account on Stack Exchange, or etc) also links to an already blacklisted domain name.

The intent is to enable us to fast-track blacklisting when there is clear evidence that the new domain name (or product name, etc.) is a sockpuppet of an already known spammer, i.e. we can reasonably establish that it was basically created to evade any blacklisting or other negative reputation associated with another account.

How to Blacklist or Watch Something

You will want to test that the expression you want to blacklist or watch isn’t already covered by one of the existing patterns.

You can use !!/bisect <string to test> to test to see if the text is matched by an existing watch or blacklist entry, and find out what that entry is. !!/bisect only tests against the patterns contained in the lists maintained with the !!/watch and !!/blacklist-* commands. You can test the string for matching all detections by using the !!/test <string to test> command (or !!/test-a <string to test> to test as an answer). The !!/test commands will also test against some blacklist and watchlist entries which are contained in findspam.py.

Everyone with SmokeDetector privileges (if you don’t have those and would like them, read up on how to get them) can blacklist a website, though this will need to be approved by someone with code privileges if you don’t already have them. Additions to the blacklist must be valid regular expressions (regex). In reality that means for largely exact matches (like the website blacklist) that you ensure that special characters (like .) are escaped. (Example: thisisspam\.com)

There are two methods to add a website to a watch list or blacklist:

• Propose a change to the relevant file on GitHub and create a pull request specifying why you want to blacklist the website.

• Use the pertinent chat command in any of the Smoke Detector chatrooms. This will create a pull request containing your change for you.

  • !!/blacklist-website <regex> to blacklist a web site
  • !!/blacklist-keyword <regex> to blacklist a keyword expression
  • !!/blacklist-number <number> to blacklist a phone number (not things like URL path fragments, which can’t be obfuscated; use !!/blacklist-keyword for those)
  • !!/blacklist-username <regex> to blacklist a username (display name)
  • !!/watch <regex> to add something to the watch list (keywords, web sites, and usernames)
  • !!/watch-number <number> to add a phone number to the watch list (not things like URL path fragments, which can’t be obfuscated; use !!/watch for those)

If you’re blacklisting or watching a complex regex to match a whole bunch of different stuff, it’s probably better off in the pattern-matching section of findspam.py. You’ll need to propose a change to the file on GitHub for this; ask for help if you’re unsure what to do. However, if you think something might be appropriate for the pattern-matching section of findspam.py, it’s reasonable to put it on the watchlist first, so we can collect data as to how accurate it is, without giving extra weight to something that might catch a lot of false positive posts.