about summary refs log tree commit diff
path: root/app/services
diff options
context:
space:
mode:
authorEugen Rochko <eugen@zeonfederated.com>2019-07-13 16:45:50 +0200
committerGitHub <noreply@github.com>2019-07-13 16:45:50 +0200
commit6ff67be0f6e79ec403e08c69717ee8c89451c70e (patch)
tree1746025bdf7b3113ddb373e9fb137877952d2503 /app/services
parent402302776c82c3853e723fe0c0c4dc99c69da3d9 (diff)
Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
Diffstat (limited to 'app/services')
-rw-r--r--app/services/remove_status_service.rb5
1 files changed, 5 insertions, 0 deletions
diff --git a/app/services/remove_status_service.rb b/app/services/remove_status_service.rb
index 6311971ff..a69fce8b8 100644
--- a/app/services/remove_status_service.rb
+++ b/app/services/remove_status_service.rb
@@ -23,6 +23,7 @@ class RemoveStatusService < BaseService
         remove_from_hashtags
         remove_from_public
         remove_from_media if status.media_attachments.any?
+        remove_from_spam_check
 
         @status.destroy!
       else
@@ -142,6 +143,10 @@ class RemoveStatusService < BaseService
     redis.publish('timeline:public:local:media', @payload) if @status.local?
   end
 
+  def remove_from_spam_check
+    redis.zremrangebyscore("spam_check:#{@status.account_id}", @status.id, @status.id)
+  end
+
   def lock_options
     { redis: Redis.current, key: "distribute:#{@status.id}" }
   end