Clean up reblog tracking keys, related improvements (#5428)

* Clean up reblog-tracking sets from FeedManager

Builds on #5419, with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.

1 files changed, 23 insertions, 5 deletions

diff --git a/app/workers/scheduler/feed_cleanup_scheduler.rb b/app/workers/scheduler/feed_cleanup_scheduler.rb
index 222f5ed84..cfa2d31a4 100644
--- a/app/workers/scheduler/feed_cleanup_scheduler.rb
+++ b/app/workers/scheduler/feed_cleanup_scheduler.rb
@@ -5,18 +5,36 @@ class Scheduler::FeedCleanupScheduler
   include Sidekiq::Worker
 
   def perform
+    reblogged_id_sets = {}
+    feedmanager = FeedManager.instance
+
+    redis.pipelined do
+      inactive_user_ids.each do |account_id|
+        redis.del(feedmanager.key(:home, account_id))
+        reblog_key = feedmanager.key(:home, account_id, 'reblogs')
+        # We collect a future for this: we don't block while getting
+        # it, but we can iterate over it later.
+        reblogged_id_sets[account_id] = redis.zrange(reblog_key, 0, -1)
+        redis.del(reblog_key)
+      end
+    end
+
+    # Remove all of the reblog tracking keys we just removed the
+    # references to.
     redis.pipelined do
-      inactive_users.each do |account_id|
-        redis.del(FeedManager.instance.key(:home, account_id))
-        redis.del(FeedManager.instance.key(:home, account_id, 'reblogs'))
+      reblogged_id_sets.each do |account_id, future|
+        future.value.each do |reblogged_id|
+          reblog_set_key = feedmanager.key(:home, account_id, "reblogs:#{reblogged_id}")
+          redis.del(reblog_set_key)
+        end
       end
     end
   end
 
   private
 
-  def inactive_users
-    @inactive_users ||= User.confirmed.inactive.pluck(:account_id)
+  def inactive_user_ids
+    @inactive_user_ids ||= User.confirmed.inactive.pluck(:account_id)
   end
 
   def redis