about summary refs log tree commit diff
path: root/app/workers/scheduler
AgeCommit message (Collapse)Author
2022-07-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `.env.production.sample`: Our sample config file is very different from upstream since it is much more complete. Upstream added documentation for a few env variables. Copied the new variables/documentation from upstream. - `app/lib/feed_manager.rb`: Upstream added a timeline type (hashtags), while glitch-soc already had an extra one (direct messages). Not really a conflict but textually close changes. Ported upstream's changes. - `app/models/custom_emoji.rb`: Upstream upped the custom emoji size limit, while glitch-soc had configurable limits. Upped the default limits accordingly. - `streaming/index.js`: Upstream reworked how hastags were normalized. Minor conflict due to glitch-soc's handling of instance-local posts. Ported upstream's changes.
2022-07-07ip_cleanup_scheduler: Make IP and session retention configurable (#18757)Jeremy Kescher
2022-05-24Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-05-18Change search indexing to use batches to minimize resource usage (#18451)Eugen Rochko
2022-05-01Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/javascript/packs/admin.js`: Conflicts due to glitch-soc's theming system. Upstream changes have been ported to `app/javascript/core/admin.js` - `app/models/trends/statuses.rb`: Minor conflict due to glitch-soc's option to allow CWed toots in trends. Ported upstream changes.
2022-04-29Remove IP matching from e-mail domain blocks (#18190)Eugen Rochko
Clear out e-mail domain blocks created from automatically resolved DNS records
2022-04-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-04-28Fix single Redis connection being used across all threads (#18135)Eugen Rochko
* Fix single Redis connection being used across all Sidekiq threads * Fix tests
2022-03-22Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `.github/dependabot.yml`: Upstream modified it, but we deleted it in glitch-soc. Keep it deleted.
2022-03-19Fix clean_discarded_statuses (#17822)Jeong Arm
2022-03-02Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-03-01Change authorized applications page (#17656)Eugen Rochko
* Change authorized applications page * Hide revoke button for superapps and suspended accounts * Clean up db/schema.rb
2022-02-26Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-02-26Fix crash in EmailDomainBlockRefreshScheduler (#17649)Claire
2022-02-26Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/models/account.rb`: Not a real conflict, just upstream getting rid of unused constants too close to glitch-soc-specific contents. Removed unused constants like upstream did. - `app/models/trends.rb`: Conflict because glitch-soc disabled email notifications for trending links. Upstream has refactored this quite a bit and added trending posts. Took upstream code, but disabling the extra trending stuff will come in another commit. - `app/views/admin/trends/links/index.html.haml`: Conflict due to glitch-soc's theming system. Ported upstream changes accordingly.
2022-02-25Add trending statuses (#17431)Eugen Rochko
* Add trending statuses * Fix dangling items with stale scores in localized sets * Various fixes and improvements - Change approve_all/reject_all to approve_accounts/reject_accounts - Change Trends::Query methods to not mutate the original query - Change Trends::Query#skip to offset - Change follow recommendations to be refreshed in a transaction * Add tests for trending statuses filtering behaviour * Fix not applying filtering scope in controller
2022-02-24Change e-mail domain blocks to block IPs dynamically (#17635)Eugen Rochko
* Change e-mail domain blocks to block IPs dynamically * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> * Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-02-08Fix localization of cold-start follow recommendations (#17479)Eugen Rochko
2022-02-07Fix Ruby 2.5 incompatibility (#17465)Claire
2022-01-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `Gemfile.lock`: Upstream-updated lib textually too close to glitch-soc-only dep. Updated like upstream.
2022-01-28Fix Sidekiq warnings about JSON serialization (#17381)Claire
* Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.
2022-01-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/reports/show.html.haml`: Conflicts due to glitch-soc's theming system.
2022-01-17Add notifications for statuses deleted by moderators (#17204)Eugen Rochko
2022-01-16Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `README.md`: Upstream added some text, but our README is completely different. Kept our README unchanged.
2022-01-16Remove IP tracking columns from users table (#16409)Eugen Rochko
2021-12-16Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/pending_accounts/index.html.haml`: Removed upstream, while it had glitch-soc-specific changes to accomodate for glitch-soc's theming system. Removed the file. Additional changes: - `app/views/admin/accounts/index.html.haml': Accomodate for glitch-soc's theming system.
2021-12-13Fix follow recommendation biased towards older accounts (#17126)Takeshi Umeda
2021-11-25Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/tags/index.html.haml`: Removed upstream while it had changes in glitch-soc to accomodate for the theming system. Additional changes to accomodate for the theming system: - `app/views/admin/trends/links/preview_card_providers/index.html.haml` - `app/views/admin/trends/links/index.html.haml` - `app/views/admin/trends/tags/index.html.haml` - `app/views/admin/tags/show.html.haml`
2021-11-25Add trending links (#16917)Eugen Rochko
* Add trending links * Add overriding specific links trendability * Add link type to preview cards and only trend articles Change trends review notifications from being sent every 5 minutes to being sent every 2 hours Change threshold from 5 unique accounts to 15 unique accounts * Fix tests
2021-08-09Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `Gemfile.lock`: Not a real conflict, upstream-updated dependency (redis) textually too close to glitch-soc-only dependecy. Updated redis gem like upstream did.
2021-08-09Add feature to automatically delete old toots (#16529)Claire
* Add account statuses cleanup policy model * Record last inspected toot to delete to speed up successive calls to statuses_to_delete * Add service to cleanup a given account's statuses within a budget * Add worker to go through account policies and delete old toots * Fix last inspected status id logic All existing statuses older or equal to last inspected status id must be kept by the current policy. This is an invariant that must be kept so that resuming deletion from the last inspected status remains sound. * Add tests * Refactor scheduler and add tests * Add user interface * Add support for discriminating based on boosts/favs * Add UI support for min_reblogs and min_favs, rework UI * Address first round of review comments * Replace Snowflake#id_at_start with with_random parameter * Add tests * Add tests for StatusesCleanupController * Rework settings page * Adjust load-avoiding mechanisms * Please CodeClimate
2021-06-23Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-06-21Add authentication history (#16408)Eugen Rochko
2021-05-07Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `app/views/statuses/_simple_status.html.haml`: Small markup change in glitch-soc, on a line that has been modified by upstream. Ported upstream changes.
2021-05-05Improve performance of follow recommendation scheduler (#16159)Claire
Express follow_recommendations in terms of account_summaries rather than accounts, integrate filters that are unconditionally used, and materialize the resulting view. This should result in the bulk of the computation being performed only once instead of **once per recommendation language**.
2021-04-20Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `README.md`: Upstream updated copyright year, we don't mention it so kept our version. - `app/controllers/admin/dashboard_controller.rb`: Not really a conflict, upstream change (removing the spam checker) too close to glitch-soc changes. Ported upstream changes. - `app/models/form/admin_settings.rb`: Same. - `app/services/remove_status_service.rb`: Same. - `app/views/admin/settings/edit.html.haml`: Same. - `config/settings.yml`: Same. - `config/environments/production.rb`: Not a real conflict, upstream added a default HTTP header, but we have extra headers in glitch-soc. Added the header.
2021-04-12Add cold-start follow recommendations (#15945)Eugen Rochko
2021-03-15Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-03-15Fix configuration for sidekiq-unique-jobs after 7.x upgrade (#15908)Eugen Rochko
Remove locks from scheduled jobs
2020-12-23Merge branch 'master' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/lib/feed_manager.rb`: Not a real conflict, glitch-soc-only DM-related method too close to changed upstream stuff. Ported upstream changes. - `app/services/batched_remove_status_service.rb`: Additional logic in glitch-soc to clear DMs from timelines. Ported upstream changes and fixed the DM TL clearing logic. - `app/workers/scheduler/feed_cleanup_scheduler.rb`: Additional code in glitch-soc to clear DM timelines. Ported upstream changes.
2020-12-22Improve account deletion performances further (#15407)ThibG
* Delete status records by batches of 50 * Do not precompute values that are only used once * Do not generate redis events for removal of public toots older than two weeks * Filter reported toots a priori for polls and status deletion * Do not process reblogs when cleaning up public timelines As in Mastodon proper, reblogs don't appear in public TLs * Clean the deleted account's own feed in one go * Refactor Account#clean_feed_manager and List#clean_feed_manager * Delete instead of destroy a few more associations * Fix preloading Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15Merge branch 'master' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/models/form/admin_settings.rb`: New setting added upstream. Ported it. - `app/views/statuses/_simple_status.html.haml`: Upstream removed RTL classes. Did the same. - `config/settings.yml`: New setting added upstream. Ported it.
2020-12-14Fix performance on instances list in admin UI (#15282)Eugen Rochko
- Reduce duplicate queries - Remove n+1 queries - Add accounts count to detailed view - Add separate action log entry for updating existing domain blocks
2020-10-21Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `.github/dependabot.yml`: Updated upstream, we deleted it to not be flooded by Depandabot. Kept deleted. - `Gemfile.lock`: Puma updated on both sides, went for the most recent version. - `app/controllers/api/v1/mutes_controller.rb`: Upstream updated the serializer to support timed mutes, while glitch-soc added a custom API ages ago to get information that is already available elsewhere. Dropped the glitch-soc-specific API, went with upstream changes. - `app/javascript/core/admin.js`: Conflict due to changing how assets are loaded. Went with upstream. - `app/javascript/packs/public.js`: Conflict due to changing how assets are loaded. Went with upstream. - `app/models/mute.rb`: 🤷 - `app/models/user.rb`: New user setting added upstream while we have glitch-soc-specific user settings. Added upstream's user setting. - `config/settings.yml`: Upstream added a new user setting close to a user setting we had changed the defaults for. Added the new upstream setting. - `package.json`: Upstream dependency updated “too close” to a glitch-soc-specific dependency. No real conflict. Updated the dependency.
2020-10-12Add IP-based rules (#14963)Eugen Rochko
2020-09-28Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `Gemfile.lock`: Not a real conflict, upstream updated dependencies that were too close to glitch-soc-only ones in the file. - `app/controllers/oauth/authorized_applications_controller.rb`: Upstream changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's theming system. Ported upstream changes. - `app/controllers/settings/base_controller.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's theming system. Ported upstream changes. - `app/controllers/settings/sessions_controller.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's theming system. Ported upstream changes. - `app/models/user.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc not preventing moved accounts from logging in. Ported upstream changes while keeping the ability for moved accounts to log in. - `app/policies/status_policy.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's local-only toots. Ported upstream changes. - `app/serializers/rest/account_serializer.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's ability to hide followers count. Ported upstream changes. - `app/services/process_mentions_service.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's local-only toots. Ported upstream changes. - `package.json`: Not a real conflict, upstream updated dependencies that were too close to glitch-soc-only ones in the file.
2020-09-15Change account suspensions to be reversible by default (#14726)Eugen Rochko
2020-06-09Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/activitypub/collections_controller.rb`: Conflict due to glitch-soc having to take care of local-only pinned toots in that controller. Took upstream's changes and restored the local-only special handling. - `app/controllers/auth/sessions_controller.rb`: Minor conflicts due to the theming system, applied upstream changes, adapted the following two files for glitch-soc's theming system: - `app/controllers/concerns/sign_in_token_authentication_concern.rb` - `app/controllers/concerns/two_factor_authentication_concern.rb` - `app/services/backup_service.rb`: Minor conflict due to glitch-soc having to handle local-only toots specially. Applied upstream changes and restored the local-only special handling. - `app/views/admin/custom_emojis/index.html.haml`: Minor conflict due to the theming system. - `package.json`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file. - `yarn.lock`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file.
2020-06-02Add E2EE API (#13820)Eugen Rochko