about summary refs log tree commit diff
path: root/app/workers/pubsubhubbub
AgeCommit message (Collapse)Author
2017-09-14Revert unique retry job (#4937)abcang
* Revert "Enable UniqueRetryJobMiddleware even when called from sidekiq worker (#4836)" This reverts commit 6859d4c0289e767955aac3f345074220fe200604. * Revert "Do not execute the job with the same arguments as the retry job (#4814)" This reverts commit be7ffa2d7539d5a1946a3933cb9d242b9fac0ddc.
2017-09-06Switch to static URIs, new URI format in both protocols for new statuses (#4815)Eugen Rochko
* Decouple Status#local? from uri being nil * Replace on-the-fly URI generation with stored URIs - Generate URI in after_save hook for local statuses - Use static value in TagManager when available, fallback to tag format - Make TagManager use ActivityPub::TagManager to understand new format - Adjust tests * Use other heuristic for locality of old statuses, do not perform long query * Exclude tombstone stream entries from Atom feed * Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker * Fix URI not being saved (#4818) * Add more specs for Status * Save generated uri immediately and also fix method order to minimize diff. * Fix alternate HTML URL in Atom * Fix tests * Remove not-null constraint from statuses migration to speed it up
2017-09-05Do not execute the job with the same arguments as the retry job (#4814)abcang
2017-08-29Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)Eugen Rochko
Requires moving Atom rendering from DistributionWorker (where `stream_entry.status` is already nil) to inline (where `stream_entry.status.destroyed?` is true) and distributing that. Unfortunately, such XML renderings can no longer be easily chained together into one payload of n items.
2017-08-24Add configuration to disable private status federation over PuSH (#4582)Eugen Rochko
2017-08-21ActivityPub migration procedure (#4617)Eugen Rochko
* ActivityPub migration procedure Once one account is detected as going from OStatus to ActivityPub, invalidate WebFinger cache for other accounts from the same domain * Unsubscribe from PuSH updates once we receive an ActivityPub payload * Re-subscribe to PuSH unless already unsubscribed, regardless of protocol
2017-07-31Change RuboCop rules to loose (#4464)Yamagishi Kazutoshi
2017-07-28When PuSH subscribe attempts are exhausted, unsubscribe (#4422)Eugen Rochko
2017-07-27Add callback_url/acct information for Sidekiq PuSH workers Exception. (#4281)Clworld
* Add destination informations to exception on SubscribeWorker and DeliveryWorker. * Simplify delivery error message. * Prevent changing Exception type... * fix typo.
2017-07-20Fix webfinger retries (#4275)Eugen Rochko
* Do not raise unretryable exceptions in ResolveRemoteAccountService * Removed fatal exceptions from ResolveRemoteAccountService Exceptions that cannot be retried should not be raised. New exception class for those that can be retried (Mastodon::UnexpectedResponseError)
2017-07-19Correct OStatus inflection (Ostatus -> OStatus) (#4255)Eugen Rochko
2017-07-18Introduce Ostatus name space (#4164)Akihiko Odaki
* Wrap methods of ProcessFeedService::ProcessEntry in classes This is a change same with 425acecfdb15093a265b191120fb2d4e4c4135c4, except that it has the following changes: * Revert irrelevant change in find_or_create_conversation * Fix error handling for RemoteActivity * Introduce Ostatus name space
2017-07-15Optimize uri normalization (#4212)ThibG
* Add dependency on idn-ruby to speed up URI normalization * Use normalized_host instead of normalize.host when applicable When we are only interested in the normalized host, calling normalized_host avoids normalizing the other components of the URI as well as creating a new object
2017-07-14Fix #2672 - Connect signed PuSH subscription requests to instance domain (#4205)Eugen Rochko
* Fix #2672 - Connect signed PuSH subscription requests to instance domain Resolves #2739 * Fix return of locate_subscription * Fix tests
2017-07-14HTTP signatures (#4146)Eugen Rochko
* Add Request class with HTTP signature generator Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06 * Add HTTP signature verification concern * Add test for SignatureVerification concern * Add basic test for Request class * Make PuSH subscribe/unsubscribe requests use new Request class Accidentally fix lease_seconds not being set and sent properly, and change the new minimum subscription duration to 1 day * Make all PuSH workers use new Request class * Make Salmon sender use new Request class * Make FetchLinkService use new Request class * Make FetchAtomService use the new Request class * Make Remotable use the new Request class * Make ResolveRemoteAccountService use the new Request class * Add more tests * Allow +-30 seconds window for signed request to remain valid * Disable time window validation for signed requests, restore 7 days as PuSH subscription duration (which was previous default due to a bug)
2017-07-10Limit total subscribe retries to 10, but space them out more (#4142)Eugen Rochko
Since there is little point in retrying so often when a service is down or does not exist anymore. Subscriptions are renewed 1 day before they should expire, so retrying in 30 minutes, then 2 hours, then 12 hours is fine. If even after that, the remote server does not work, there is little sense in retrying more often than once a day Also, uniqueness of the job should ensure that failed retries will not result in multiple retries for the same endpoint when the next resubscription cycle comes
2017-06-22Some minor change and spec for Account (#3813)Akihiko Odaki (@fn_aki@pawoo.net)
* Introduce domains method to Account relation Account had followers_domains method, which was excessively specific. Let relation of Account have domains method instead. * Move follow_mapping in Account to AccountInteractions * Introduce shared examples for AccountAvatar inclusion * Cover Account more
2017-06-14Batched remove status service (#3735)Eugen Rochko
* Make Pubsubhubbub::DistributionWorker handle both single stream entry arguments, as well as arrays of stream entries * Add BatchedRemoveStatusService, make SuspendAccountService use it * Improve method names * Add test * Add more tests * Use PuSH payloads of 100 to have a clear mapping of 1000 input statuses -> 10 PuSH payloads It was nice while it lasted
2017-05-14Do not cancel PuSH subscriptions after encountering "permanent" error… (#3046)Eugen Rochko
* Do not cancel PuSH subscriptions after encountering "permanent" error response After talking with MMN about it, turns out some servers/php setups do return 4xx errors while rebooting, so this anti-feature that was meant to take load off of the hub is doing more harm than good in terms of breaking subscriptions * Update delivery_worker.rb
2017-05-12Spec coverage and refactor for pubsub/delivery worker (#3021)Matt Jankowski
* Framework for delivery worker spec * Refactor of pubsub delivery worker
2017-05-11Pubsub confirmation worker spec (#2974)Matt Jankowski
* Add specs for valid challenge response from pubsub confirmation worker * Refactor the pubsub confirmation worker
2017-05-10Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items (#2954)Eugen Rochko
to all subscribers. IDN-normalize callback URLs for subscriptions on insert.
2017-05-09add content-type in pubsubhubhub request header (#2943)haoyayoi
* add content-type in pubsubhubhub request header * fix type
2017-05-06Fix Scheduler::SubscriptionsScheduler (#2834)Eugen Rochko
* Fix Scheduler::SubscriptionsScheduler, add worker test for it * Change production log level of Sidekiq to "warn" instead of "info"
2017-05-05More robust PuSH subscription refreshes (#2799)Eugen Rochko
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron Fix an issue where / in domain would raise exception in TagManager#normalize_domain PuSH subscriptions refresh done in a round-robin way to avoid hammering a single server's hub in sequence. Correct handling of failures/retries through Sidekiq (see also #2613). Optimize Account#with_followers scope. Also, since subscriptions are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire refreshing operation halfway through Fix #2702 - Correct user agent header on outgoing http requests * Add test for SubscribeService * Extract #expiring_accounts into method * Make mastodon:push:refresh no-op * Queues are now defined in sidekiq.yml * Queues are now in sidekiq.yml
2017-04-25Punycode URI normalization (#2370)Eugen
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI * Add test for IDN request in FetchLinkCardService * Perform IDN normalization on domains before they are stored in the DB
2017-04-24Followers-only post federation (#2111)Eugen
* Make private toots get PuSHed to subscription URLs that belong to domains where you have approved followers * Authorized followers controller, stub for bulk action * Soft block in the background * Add simple test for new controller * Rename Settings::FollowersController to Settings::FollowerDomainsController, paginate results, rename "private" post setting to "followers-only", fix pagination style, improve post privacy preferences style, improve warning style * Extract compose form warnings into own container, show warning when posting to followers-only with unlocked account
2017-04-12Quick best practice cleanup of views/helpers (#1546)Matt Jankowski
* Remove trailing whitespace * Use query methods instead of explicit .blank? checks
2017-04-07Rewrite Atom generation from stream entries to use Ox instead of Nokogiri ↵Eugen
(#1124) * Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder StreamEntry is now limited to only statuses, which allows some optimization. Removed extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer and PubSubHubbub::DistributionWorker PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker instead. All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri * All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account
2017-04-05Remove PuSH subscriptions when delivery is answered with a 4xx errorEugen Rochko
2017-02-14Disable PuSH for blocked domainsEugen Rochko
2017-02-12Take out private toot distribution out of PuSH againEugen Rochko
2017-02-11Make PuSH only distribute to subscriptions that match follower domainsEugen Rochko
Allow PuSH to distribute private toots with that condition
2017-01-05Improve background jobs params and error handlingEugen Rochko
2017-01-03Fix accidental distribution of hidden stream entries to PuSH subscribersEugen Rochko
2016-12-19Adding some rescues to workers, e.g. don't fail removal job when status ↵Eugen Rochko
can't be found in the first place (already removed)
2016-12-12Restoring old async behaviour of thread resolving as it proved to be more robustEugen Rochko
2016-11-30Track successful PuSH deliveriesEugen Rochko
2016-11-29Adding rack timeout of 30sec, PuSH jobs moved to push queue so theyEugen Rochko
can be processed separately
2016-11-29Fix URLs in inline-rendered XMLEugen Rochko
2016-11-28Fix setting of confirmed=true on successful confirmationEugen Rochko
2016-11-28More logging to ConfirmationWorkerEugen Rochko
2016-11-28Adding embedded PuSH serverEugen Rochko