about summary refs log tree commit diff
path: root/app/workers
AgeCommit message (Collapse)Author
2022-01-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `Gemfile.lock`: Upstream-updated lib textually too close to glitch-soc-only dep. Updated like upstream.
2022-01-28Fix Sidekiq warnings about JSON serialization (#17381)Claire
* Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.
2022-01-19Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-01-19Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/lib/activitypub/activity/create.rb`: Upstream refactored how `Create` activities are handled and how values are extracted from `Create`d objects. This conflicted with how glitch-soc supported the `directMessage` flag to explicitly distinguish between limited and direct messages. Ported glitch-soc's changes to latest upstream changes. - `app/services/fan_out_on_write_service.rb`: Upstream largely refactored that file and changed some of the logic. This conflicted with glitch-soc's handling of the direct timeline and the options to allow replies and boosts in public feeds. Ported those glitch-soc changes on top of latest upstream changes. - `app/services/process_mentions_service.rb`: Upstream refactored to move mention-related ActivityPub deliveries to `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check to not send local-only toots to remote mentioned users. Took upstream's version, as the check is not needed anymore, since it is performed at the `ActivityPub::DeliveryWorker` call site already. - `app/workers/feed_insert_worker.rb`: Upstream added support for `update` toot events, while glitch-soc had support for an extra timeline support, `direct`. Ported upstream changes and extended them to the `direct` timeline. Additional changes: - `app/lib/activitypub/parser/status_parser.rb`: Added code to handle the `directMessage` flag and take it into account to compute visibility. - `app/lib/feed_manager.rb`: Extended upstream's support of `update` toot events to glitch-soc's `direct` timeline.
2022-01-19Fix error when using raw distribution worker (#17334)Eugen Rochko
Regression from #16697
2022-01-19Add support for editing for published statuses (#16697)Eugen Rochko
* Add support for editing for published statuses * Fix references to stripped-out code * Various fixes and improvements * Further fixes and improvements * Fix updates being potentially sent to unauthorized recipients * Various fixes and improvements * Fix wrong words in test * Fix notifying accounts that were tagged but were not in the audience * Fix mistake
2022-01-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/reports/show.html.haml`: Conflicts due to glitch-soc's theming system.
2022-01-17Add notifications for statuses deleted by moderators (#17204)Eugen Rochko
2022-01-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/controllers/accounts_controller.rb`: Upstream introduced support for private pinned toots, but glitch-soc's query was a bit different as it filtered out local-only toots. Used upstream's query, while adding local-only filtering back. - `app/controllers/activitypub/collections_controller.rb`: Same thing with regards to local-only posts. - `app/validators/status_pin_validator.rb`: Not a real conflict, but the line below was different in glitch-soc due to the configurable pinned toots limit.
2022-01-17Add support for private pinned posts (#16954)Claire
* Add support for private pinned toots * Allow local user to pin private toots * Change wording to avoid "direct message"
2022-01-16Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `README.md`: Upstream added some text, but our README is completely different. Kept our README unchanged.
2022-01-16Remove IP tracking columns from users table (#16409)Eugen Rochko
2021-12-18Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-12-17Add ability to purge undeliverable domains from admin interface (#16686)Claire
* Add ability to purge undeliverable domains from admin interface * Add tests
2021-12-16Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/pending_accounts/index.html.haml`: Removed upstream, while it had glitch-soc-specific changes to accomodate for glitch-soc's theming system. Removed the file. Additional changes: - `app/views/admin/accounts/index.html.haml': Accomodate for glitch-soc's theming system.
2021-12-13Fix follow recommendation biased towards older accounts (#17126)Takeshi Umeda
2021-11-25Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/tags/index.html.haml`: Removed upstream while it had changes in glitch-soc to accomodate for the theming system. Additional changes to accomodate for the theming system: - `app/views/admin/trends/links/preview_card_providers/index.html.haml` - `app/views/admin/trends/links/index.html.haml` - `app/views/admin/trends/tags/index.html.haml` - `app/views/admin/tags/show.html.haml`
2021-11-25Add trending links (#16917)Eugen Rochko
* Add trending links * Add overriding specific links trendability * Add link type to preview cards and only trend articles Change trends review notifications from being sent every 5 minutes to being sent every 2 hours Change threshold from 5 unique accounts to 15 unique accounts * Fix tests
2021-11-06Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-11-06Fix AccountNote not having a maximum length (#16942)Claire
2021-08-20Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-08-11Fix followers synchronization mechanism not working when URI has empty path ↵Claire
(#16510) * Fix followers synchronization mechanism not working when URI has empty path To my knowledge, there is no current implementation on the fediverse that can use bare domains (e.g., actor is at https://example.org instead of something like https://example.org/actor) that also plans to support the followers synchronization mechanism. However, Mastodon's current implementation would exclude such accounts from followers list. Also adds tests and rename them to reflect the proper method names. * Move url prefix regexp to its own constant
2021-08-09Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `Gemfile.lock`: Not a real conflict, upstream-updated dependency (redis) textually too close to glitch-soc-only dependecy. Updated redis gem like upstream did.
2021-08-09Add feature to automatically delete old toots (#16529)Claire
* Add account statuses cleanup policy model * Record last inspected toot to delete to speed up successive calls to statuses_to_delete * Add service to cleanup a given account's statuses within a budget * Add worker to go through account policies and delete old toots * Fix last inspected status id logic All existing statuses older or equal to last inspected status id must be kept by the current policy. This is an invariant that must be kept so that resuming deletion from the last inspected status remains sound. * Add tests * Refactor scheduler and add tests * Add user interface * Add support for discriminating based on boosts/favs * Add UI support for min_reblogs and min_favs, rework UI * Address first round of review comments * Replace Snowflake#id_at_start with with_random parameter * Add tests * Add tests for StatusesCleanupController * Rework settings page * Adjust load-avoiding mechanisms * Please CodeClimate
2021-08-08Fix when MoveWorker cannot get locale from remote account (#16576)Takeshi Umeda
2021-06-24Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-06-23Fix newlines in accout notes added by the Move handler (#16415)Claire
* Fix newlines in account notes added by the move handler * Make MoveWorker more robust
2021-06-23Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-06-21Add authentication history (#16408)Eugen Rochko
2021-05-22Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-05-19Fix some RedisLocks auto-releasing too fast (#16276)Claire
* Fix Delete and Create-related locks expiring too fast Fixes #16238 By default, RedisLock expires after 10 seconds, which may not be enough to process statuses, especially when those have attached media files. This commit extends those 10 seconds to 15 minutes, which should be plenty enough to handle any status, while being short enough to not waste many sidekiq job retries in the exceedingly rare case in which a sidekiq process would crash when processing a `Create` or `Delete`. * Fix other RedisLock autorelease durations Fixes #15645 - things that only perform a few simple database queries (e.g. finding and saving a record) have been left unchanged, so they'll still use the default 10s duration - things that perform significantly more complex database queries have been changed to a 5 minutes timeout - things that perform multiple HTTP queries have been changed to a 15 minutes timeout
2021-05-12Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-05-11Fix to be able to redownload avatar and header (#16190)Takeshi Umeda
* Fix to reset if header and avatar download fails * Add RedownloadAvatarWorker and RedownloadHeaderWorker
2021-05-07Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `app/views/statuses/_simple_status.html.haml`: Small markup change in glitch-soc, on a line that has been modified by upstream. Ported upstream changes.
2021-05-06Add Ruby 3.0 support (#16046)Claire
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0 Also improve the Terrapin monkey-patch for the stderr/stdout issue. * Fix keyword argument handling throughout the codebase * Monkey-patch Paperclip to fix keyword arguments handling in validators * Change validation_extensions to please CodeClimate * Bump microformats from 4.2.1 to 4.3.1 * Allow Ruby 3.0 * Add Ruby 3.0 test target to CircleCI * Add test for admin dashboard warnings * Fix admin dashboard warnings on Ruby 3.0
2021-05-05Fix media redownload worker retrying on unexpected response codes (#16111)Eugen Rochko
2021-05-05Improve performance of follow recommendation scheduler (#16159)Claire
Express follow_recommendations in terms of account_summaries rather than accounts, integrate filters that are unconditionally used, and materialize the resulting view. This should result in the bulk of the computation being performed only once instead of **once per recommendation language**.
2021-05-04Fix empty home feed before first follow has finished processing (#16152)Eugen Rochko
Change queue of merge worker from pull to default
2021-04-26Fix thread resolve worker retrying when status no longer exists (#16109)Eugen Rochko
2021-04-24Fix nil error when removing status caused by race condition (#16099)Eugen Rochko
2021-04-20Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `README.md`: Upstream updated copyright year, we don't mention it so kept our version. - `app/controllers/admin/dashboard_controller.rb`: Not really a conflict, upstream change (removing the spam checker) too close to glitch-soc changes. Ported upstream changes. - `app/models/form/admin_settings.rb`: Same. - `app/services/remove_status_service.rb`: Same. - `app/views/admin/settings/edit.html.haml`: Same. - `config/settings.yml`: Same. - `config/environments/production.rb`: Not a real conflict, upstream added a default HTTP header, but we have extra headers in glitch-soc. Added the header.
2021-04-12Change Web Push API deliveries to use request pooling (#16014)Eugen Rochko
2021-04-12Add cold-start follow recommendations (#15945)Eugen Rochko
2021-03-15Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-03-15Fix configuration for sidekiq-unique-jobs after 7.x upgrade (#15908)Eugen Rochko
Remove locks from scheduled jobs
2021-01-06Merge branch 'master' into glitch-soc/merge-upstreamClaire
Conflicts: - `config/webpack/configuration.js`: Upstream updated the `js-yaml` dependency, which changed how to call it. Those changes conflicted because that code is pretty different in glitch-soc which has to deal with its more complex theming system. Proceeded to the same compatibility changes in glitch-soc's code. - `package.json` and `yarn.lock`: Not really a conflict, just glitch-soc-specific dependencies textually too close to some dependencies updated upstream.
2020-12-26Fix being able to import more than allowed number of follows (#15384)ThibG
* Fix being able to import more than allowed number of follows Without this commit, if someone tries importing a second list of accounts to follow before the first one has been processed, this will queue imports for the two whole lists, even if they exceed the account's allowed number of outgoing follows. This commit changes it so the individual queued imports aren't exempt from the follow limit check (they remain exempt from the rate-limiting check though). * Catch validation errors to not re-queue failed follows Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-23Merge branch 'master' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/lib/feed_manager.rb`: Not a real conflict, glitch-soc-only DM-related method too close to changed upstream stuff. Ported upstream changes. - `app/services/batched_remove_status_service.rb`: Additional logic in glitch-soc to clear DMs from timelines. Ported upstream changes and fixed the DM TL clearing logic. - `app/workers/scheduler/feed_cleanup_scheduler.rb`: Additional code in glitch-soc to clear DM timelines. Ported upstream changes.
2020-12-22Improve account deletion performances further (#15407)ThibG
* Delete status records by batches of 50 * Do not precompute values that are only used once * Do not generate redis events for removal of public toots older than two weeks * Filter reported toots a priori for polls and status deletion * Do not process reblogs when cleaning up public timelines As in Mastodon proper, reblogs don't appear in public TLs * Clean the deleted account's own feed in one go * Refactor Account#clean_feed_manager and List#clean_feed_manager * Delete instead of destroy a few more associations * Fix preloading Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-21Merge branch 'master' into glitch-soc/merge-upstreamClaire