about summary refs log tree commit diff
path: root/app/workers
diff options
context:
space:
mode:
authorThibG <thib@sitedethib.com>2019-02-28 15:22:21 +0100
committerEugen Rochko <eugen@zeonfederated.com>2019-02-28 15:22:21 +0100
commit9d3c6f1849120e732a9230959cb302575765ea8c (patch)
tree9fea88d303fe39036ae87698bb723c44a0146811 /app/workers
parent6e8743d17a92438bd2332e97f824bc212fa4b96e (diff)
Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
Diffstat (limited to 'app/workers')
-rw-r--r--app/workers/activitypub/fetch_replies_worker.rb12
-rw-r--r--app/workers/concerns/exponential_backoff.rb11
-rw-r--r--app/workers/fetch_reply_worker.rb12
-rw-r--r--app/workers/thread_resolve_worker.rb5
4 files changed, 36 insertions, 4 deletions
diff --git a/app/workers/activitypub/fetch_replies_worker.rb b/app/workers/activitypub/fetch_replies_worker.rb
new file mode 100644
index 000000000..bf466db54
--- /dev/null
+++ b/app/workers/activitypub/fetch_replies_worker.rb
@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+class ActivityPub::FetchRepliesWorker
+  include Sidekiq::Worker
+  include ExponentialBackoff
+
+  sidekiq_options queue: 'pull', retry: 3
+
+  def perform(parent_status_id, replies_uri)
+    ActivityPub::FetchRepliesService.new.call(Status.find(parent_status_id), replies_uri)
+  end
+end
diff --git a/app/workers/concerns/exponential_backoff.rb b/app/workers/concerns/exponential_backoff.rb
new file mode 100644
index 000000000..f2b931e33
--- /dev/null
+++ b/app/workers/concerns/exponential_backoff.rb
@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+module ExponentialBackoff
+  extend ActiveSupport::Concern
+
+  included do
+    sidekiq_retry_in do |count|
+      15 + 10 * (count**4) + rand(10 * (count**4))
+    end
+  end
+end
diff --git a/app/workers/fetch_reply_worker.rb b/app/workers/fetch_reply_worker.rb
new file mode 100644
index 000000000..f7aa25e81
--- /dev/null
+++ b/app/workers/fetch_reply_worker.rb
@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+class FetchReplyWorker
+  include Sidekiq::Worker
+  include ExponentialBackoff
+
+  sidekiq_options queue: 'pull', retry: 3
+
+  def perform(child_url)
+    FetchRemoteStatusService.new.call(child_url)
+  end
+end
diff --git a/app/workers/thread_resolve_worker.rb b/app/workers/thread_resolve_worker.rb
index c18a778d5..8bba9ca75 100644
--- a/app/workers/thread_resolve_worker.rb
+++ b/app/workers/thread_resolve_worker.rb
@@ -2,13 +2,10 @@
 
 class ThreadResolveWorker
   include Sidekiq::Worker
+  include ExponentialBackoff
 
   sidekiq_options queue: 'pull', retry: 3
 
-  sidekiq_retry_in do |count|
-    15 + 10 * (count**4) + rand(10 * (count**4))
-  end
-
   def perform(child_status_id, parent_url)
     child_status  = Status.find(child_status_id)
     parent_status = FetchRemoteStatusService.new.call(parent_url)