about summary refs log tree commit diff
path: root/app/workers/scheduler
AgeCommit message (Collapse)Author
2020-08-30[Bug] AccountConversation already covered by Conversation pruning in ↵Fire Demon
DatabaseCleanupScheduler
2020-08-30[Database] Add task to prune empty and orphaned itemsFire Demon
2020-08-30[Spam Prevention] Limit the community digest's boost candidates the last 18 ↵Fire Demon
months of history
2020-08-30[Feature] Add community digest from Monsterfork v1Fire Demon
2020-06-09Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/activitypub/collections_controller.rb`: Conflict due to glitch-soc having to take care of local-only pinned toots in that controller. Took upstream's changes and restored the local-only special handling. - `app/controllers/auth/sessions_controller.rb`: Minor conflicts due to the theming system, applied upstream changes, adapted the following two files for glitch-soc's theming system: - `app/controllers/concerns/sign_in_token_authentication_concern.rb` - `app/controllers/concerns/two_factor_authentication_concern.rb` - `app/services/backup_service.rb`: Minor conflict due to glitch-soc having to handle local-only toots specially. Applied upstream changes and restored the local-only special handling. - `app/views/admin/custom_emojis/index.html.haml`: Minor conflict due to the theming system. - `package.json`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file. - `yarn.lock`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file.
2020-06-02Add E2EE API (#13820)Eugen Rochko
2020-04-02Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/javascript/mastodon/features/compose/components/poll_form.js`: Upstream bumped poll option character limit, but we already had a higher one, kept ours. - `app/validators/poll_validator.rb`: Upstream bumped poll option character limit, but we already had a higher one, kept ours. - `config/initializers/content_security_policy.rb`: Upstream added a rule, the way we compute ours is different, but that added rule has been ported. - `package.json`: No real conflict, dependency update. Performed the same update. - `yarn.lock`: No real conflict, dependency update. Performed the same update.
2020-03-31Fix background jobs not using locks like they are supposed to (#13361)Eugen Rochko
Also: - Fix locks not being removed when jobs go to the dead job queue - Add UI for managing locks to the Sidekiq dashboard - Remove unused Sidekiq workers Fix #13349
2020-01-27Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/serializers/rest/account_serializer.rb`: Upstream added code too close to glitch-soc-specific followers-hiding code. Ported upstream changes.
2020-01-26Fix expired announcements being re-published (#12964)Eugen Rochko
2020-01-24Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/statuses_controller.rb`: Minor conflict due to theming system
2020-01-23Add announcements (#12662)Eugen Rochko
* Add announcements Fix #11006 * Add reactions to announcements * Add admin UI for announcements * Add unit tests * Fix issues - Add `with_dismissed` param to announcements API - Fix end date not being formatted when time range is given - Fix announcement delete causing reactions to send streaming updates - Fix announcements container growing too wide and mascot too small - Fix `all_day` being settable when no time range is given - Change text "Update" to "Announcement" * Fix scheduler unpublishing announcements before they are due * Fix filter params not being passed to announcements filter
2019-09-18Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - app/controllers/auth/sessions_controller.rb Minor conflict due to glitch-soc's theming code
2019-09-17Improve IP cleanup query (#11871)abcang
2019-09-05Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
2019-09-02Change trending hashtags to not disappear instantly after midnight (#11712)Eugen Rochko
2019-07-28Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
2019-07-28Add `tootctl preview_cards remove` (#11320)mayaeh
* Add `tootctl preview_cards remove` * fix code style * Remove `Scheduler::PreviewCardsCleanupScheduler` file * fix code style again Add exclude case where image_file_name is blank * Added a function to output confirmation if the specified number of days is less than 2 weeks
2019-07-23Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - app/controllers/application_controller.rb - app/controllers/auth/confirmations_controller.rb - app/controllers/auth/sessions_controller.rb - app/controllers/settings/deletes_controller.rb - app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
2019-07-21Remove WebSub subscriptions (#11303)Eugen Rochko
2019-07-19Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - Gemfile.lock - app/controllers/accounts_controller.rb - app/controllers/admin/dashboard_controller.rb - app/controllers/follower_accounts_controller.rb - app/controllers/following_accounts_controller.rb - app/controllers/remote_follow_controller.rb - app/controllers/stream_entries_controller.rb - app/controllers/tags_controller.rb - app/javascript/packs/public.js - app/lib/sanitize_config.rb - app/models/account.rb - app/models/form/admin_settings.rb - app/models/media_attachment.rb - app/models/stream_entry.rb - app/models/user.rb - app/serializers/initial_state_serializer.rb - app/services/batched_remove_status_service.rb - app/services/post_status_service.rb - app/services/process_mentions_service.rb - app/services/reblog_service.rb - app/services/remove_status_service.rb - app/views/admin/settings/edit.html.haml - config/locales/simple_form.pl.yml - config/settings.yml - docker-compose.yml
2019-07-15Add periodic removal of older thumbnails for preview cards (#11304)Eugen Rochko
2019-07-06Remove Salmon and PubSubHubbub (#11205)Eugen Rochko
* Remove Salmon and PubSubHubbub endpoints * Add error when trying to follow OStatus accounts * Fix new accounts not being created in ResolveAccountService
2019-06-27Use a redis-cached feed for the DM timelineThibaut Girka
2019-02-02Create Redisable#redis (#9633)ysksn
* Create Redisable * Use #redis instead of Redis.current
2019-01-06Fix PublishScheduledStatusWorker being queued with no arguments (#9724)Eugen Rochko
2019-01-05Add scheduled statuses (#9706)Eugen Rochko
Fix #340
2019-01-02Improve e-mail digest (#9689)Eugen Rochko
- Reduce time-to-digest from 20 to 7 days - Fetch mentions starting from +1 day since last login - Fix case when last login is more recent than last e-mail - Do not render all mentions, only 40, but show number in subject - Do not send digest to moved accounts - Do send digest to silenced accounts
2018-10-09Track historical space stats in PgHero to determine PostgreSQL growth (#8906)Eugen Rochko
2018-08-25Disable retries on all scheduled tasks (#8413)Eugen Rochko
2018-08-21Get rid of all batch order warnings (#8334)Eugen Rochko
2018-08-19Keep scheduler jobs unique until they're done (#8287)Eugen Rochko
2018-05-02Slightly reduce RAM usage (#7301)Eugen Rochko
* No need to re-require sidekiq plugins, they are required via Gemfile * Add derailed_benchmarks tool, no need to require TTY gems in Gemfile * Replace ruby-oembed with FetchOEmbedService Reduce startup by 45382 allocated objects * Remove preloaded JSON-LD in favour of caching HTTP responses Reduce boot RAM by about 6 MiB * Fix tests * Fix test suite by stubbing out JSON-LD contexts
2018-04-04Adjust privacy policy (#6666)Eugen Rochko
* Adjust privacy policy to be more specific to Mastodon Fix #6613 * Change data retention of IP addresses from 5 years to 1 year * Add even more information * Remove all (now invalid) translations of the privacy policy * Add information about archive takeout, remove pointless consent section * Emphasis on DM privacy * Improve wording * Add line about data use for moderation purposes
2018-02-21Account archive download (#6460)Eugen Rochko
* Fix #201: Account archive download * Export actor and private key in the archive * Optimize BackupService - Add conversation to cached associations of status, because somehow it was forgotten and is source of N+1 queries - Explicitly call GC between batches of records being fetched (Model class allocations are the worst offender) - Stream media files into the tar in 1MB chunks (Do not allocate media file (up to 8MB) as string into memory) - Use #bytesize instead of #size to calculate file size for JSON (Fix FileOverflow error) - Segment media into subfolders by status ID because apparently GIF-to-MP4 media are all named "media.mp4" for some reason * Keep uniquely generated filename in Paperclip::GifTranscoder * Ensure dumped files do not overwrite each other by maintaing directory partitions * Give tar archives a good name * Add scheduler to remove week-old backups * Fix code style issue
2018-01-15Move e-mail digest task to sidekiq, reduce workload, improve hint (#6252)Eugen Rochko
2017-12-05Lists redis clean-up (#5886)Eugen Rochko
* When list is deleted, remove feed from redis * Clean up list feeds of inactive users
2017-10-17Clean up reblog tracking keys, related improvements (#5428)aschmitz
* Clean up reblog-tracking sets from FeedManager Builds on #5419, with a few minor optimizations and cleanup of sets after they are no longer needed. * Update tests, fix multiply-reblogged case Previously, we would have lost the fact that a given status was reblogged if the displayed reblog of it was removed, now we don't. Also added tests to make sure FeedManager#trim cleans up our reblog tracking keys, fixed up FeedCleanupScheduler to use the right loop, and fixed the test for it.
2017-10-16Keep references to all reblogs of a status on home feed (#5419)Eugen Rochko
* Keep references to all reblogs of a status on home feed When inserting reblog: Add to set of reblogs of this status on the feed, if original status was present in the feed, add it to that set as well. When removing a reblog: Remove it from that set. Take random remaining item from the set. If one exists, re-insert it into feed, otherwise do not re-insert anything. Fix #4210 * When original is removed, toss out reblog references
2017-09-14Add scheduled worker to purge old user IPs (#4951)sdukhovni
* Add scheduled worker to purge old user IPs * Use ruby 1.9 hash syntax
2017-08-21Periodically remove expired PuSH subscribers (#4654)Eugen Rochko
2017-08-21ActivityPub migration procedure (#4617)Eugen Rochko
* ActivityPub migration procedure Once one account is detected as going from OStatus to ActivityPub, invalidate WebFinger cache for other accounts from the same domain * Unsubscribe from PuSH updates once we receive an ActivityPub payload * Re-subscribe to PuSH unless already unsubscribed, regardless of protocol
2017-08-12Only PuSH-resubscribe to OStatus accounts (#4583)Eugen Rochko
2017-07-24Move clean up of unconfirmed users to sidekiq-scheduler (#4336)Eugen Rochko
* Move clean up of unconfirmed users to sidekiq-scheduler * mastodon:daily is now deprecated
2017-06-25Bind web UI access tokens to sessions (#3940)Eugen Rochko
* Add overview of active sessions * Better display of browser/platform name * Improve how browser information is stored and displayed for sessions overview * Fix test * Fix #2347 - Bind web UI access token to session When you logout, session also destroys the access token, so it's no longer valid. If access token is destroyed some other way, the session is also destroyed, requiring a re-login. Fix #1681 - Add scheduler to remove revoked access tokens and grants * Fix test
2017-06-04change sidekiq queueing to bulk push (#3536)takayamaki
2017-05-22Specs for cleanup workers (#3235)Matt Jankowski
* Add spec files for feed and media cleanup workers * Add coverage for feed and media cleanup schedulers * Clean up feed and media cleanup workers
2017-05-20Replace mastodon:media:clear and mastodon:feeds:clear rake tasks with (#3180)Eugen Rochko
sidekiq-scheduler jobs Resolves #2495
2017-05-06Fix Scheduler::SubscriptionsScheduler (#2834)Eugen Rochko
* Fix Scheduler::SubscriptionsScheduler, add worker test for it * Change production log level of Sidekiq to "warn" instead of "info"
2017-05-05More robust PuSH subscription refreshes (#2799)Eugen Rochko
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron Fix an issue where / in domain would raise exception in TagManager#normalize_domain PuSH subscriptions refresh done in a round-robin way to avoid hammering a single server's hub in sequence. Correct handling of failures/retries through Sidekiq (see also #2613). Optimize Account#with_followers scope. Also, since subscriptions are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire refreshing operation halfway through Fix #2702 - Correct user agent header on outgoing http requests * Add test for SubscribeService * Extract #expiring_accounts into method * Make mastodon:push:refresh no-op * Queues are now defined in sidekiq.yml * Queues are now in sidekiq.yml