From 6ff67be0f6e79ec403e08c69717ee8c89451c70e Mon Sep 17 00:00:00 2001 From: Eugen Rochko Date: Sat, 13 Jul 2019 16:45:50 +0200 Subject: Add a spam check (#11217) * Add a spam check * Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance * Add more tests * Add exemption when the message is a reply to something that mentions the sender * Use Nilsimsa Compare Value instead of Levenshtein distance * Use MD5 for messages shorter than 10 characters * Add message to automated report, do not add non-public statuses to automated report, add trust level to accounts and make unsilencing raise the trust level to prevent repeated spam checks on that account * Expire spam check data after 3 months * Add support for local statuses, reduce expiration to 1 week, always create a report * Add content warnings to the spam check and exempt empty statuses * Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check * Add all matched statuses into automatic report --- Gemfile.lock | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'Gemfile.lock') diff --git a/Gemfile.lock b/Gemfile.lock index 340bbcdd8..c3198b7d9 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -12,6 +12,13 @@ GIT specs: http_parser.rb (0.6.1) +GIT + remote: https://github.com/witgo/nilsimsa + revision: fd184883048b922b176939f851338d0a4971a532 + ref: fd184883048b922b176939f851338d0a4971a532 + specs: + nilsimsa (1.1.2) + GEM remote: https://rubygems.org/ specs: @@ -704,6 +711,7 @@ DEPENDENCIES microformats (~> 4.1) mime-types (~> 3.2) net-ldap (~> 0.10) + nilsimsa! nokogiri (~> 1.10) nsa (~> 0.2) oj (~> 3.7) -- cgit