about summary refs log tree commit diff
path: root/app/lib/feed_manager.rb
AgeCommit message (Collapse)Author
2022-07-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `.env.production.sample`: Our sample config file is very different from upstream since it is much more complete. Upstream added documentation for a few env variables. Copied the new variables/documentation from upstream. - `app/lib/feed_manager.rb`: Upstream added a timeline type (hashtags), while glitch-soc already had an extra one (direct messages). Not really a conflict but textually close changes. Ported upstream's changes. - `app/models/custom_emoji.rb`: Upstream upped the custom emoji size limit, while glitch-soc had configurable limits. Upped the default limits accordingly. - `streaming/index.js`: Upstream reworked how hastags were normalized. Minor conflict due to glitch-soc's handling of instance-local posts. Ported upstream's changes.
2022-07-17Add ability to follow hashtags (#18809)Eugen Rochko
2022-06-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `.github/workflows/build-image.yml`: Fix erroneous deletion in a previous merge. - `Gemfile`: Conflict caused by glitch-soc-only hCaptcha dependency - `app/controllers/auth/sessions_controller.rb`: Minor conflict due to glitch-soc's theming system. - `app/controllers/filters_controller.rb`: Minor conflict due to glitch-soc's theming system. - `app/serializers/rest/status_serializer.rb`: Minor conflict due to glitch-soc having an extra `local_only` property
2022-06-28Revamp post filtering system (#18058)Claire
* Add model for custom filter keywords * Use CustomFilterKeyword internally Does not change the API * Fix /filters/edit and /filters/new * Add migration tests * Remove whole_word column from custom_filters (covered by custom_filter_keywords) * Redesign /filters Instead of a list, present a card that displays more information and handles multiple keywords per filter. * Redesign /filters/new and /filters/edit to add and remove keywords This adds a new gem dependency: cocoon, as well as a npm dependency: cocoon-js-vanilla. Those are used to easily populate and remove form fields from the user interface when manipulating multiple keyword filters at once. * Add /api/v2/filters to edit filter with multiple keywords Entities: - `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context` `keywords` - `FilterKeyword`: `id`, `keyword`, `whole_word` API endpoits: - `GET /api/v2/filters` to list filters (including keywords) - `POST /api/v2/filters` to create a new filter `keywords_attributes` can also be passed to create keywords in one request - `GET /api/v2/filters/:id` to read a particular filter - `PUT /api/v2/filters/:id` to update a new filter `keywords_attributes` can also be passed to edit, delete or add keywords in one request - `DELETE /api/v2/filters/:id` to delete a particular filter - `GET /api/v2/filters/:id/keywords` to list keywords for a filter - `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a filter - `GET /api/v2/filter_keywords/:id` to read a particular keyword - `PUT /api/v2/filter_keywords/:id` to edit a particular keyword - `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword * Change from `irreversible` boolean to `action` enum * Remove irrelevent `irreversible_must_be_within_context` check * Fix /filters/new and /filters/edit with update for filter_action * Fix Rubocop/Codeclimate complaining about task names * Refactor FeedManager#phrase_filtered? This moves regexp building and filter caching to the `CustomFilter` class. This does not change the functional behavior yet, but this changes how the cache is built, doing per-custom_filter regexps so that filters can be matched independently, while still offering caching. * Perform server-side filtering and output result in REST API * Fix numerous filters_changed events being sent when editing multiple keywords at once * Add some tests * Use the new API in the WebUI - use client-side logic for filters we have fetched rules for. This is so that filter changes can be retroactively applied without reloading the UI. - use server-side logic for filters we haven't fetched rules for yet (e.g. network error, or initial timeline loading) * Minor optimizations and refactoring * Perform server-side filtering on the streaming server * Change the wording of filter action labels * Fix issues pointed out by linter * Change design of “Show anyway” link in accordence to review comments * Drop “irreversible” filtering behavior * Move /api/v2/filter_keywords to /api/v1/filters/keywords * Rename `filter_results` attribute to `filtered` * Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer * Fix systemChannelId value in streaming server * Simplify code by removing client-side filtering code The simplifcation comes at a cost though: filters aren't retroactively applied anymore.
2022-04-27Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `package.json`: Not a real conflict, upstream dependency updated textually too close to a glitch-soc-only dependency. Updated the upstream dependency.
2022-04-26Fix n+1 query on feed merge and populate operations (#18111)Eugen Rochko
2022-04-26Fix UnmergeWorker being exceptionally slow in some conditions (#18110)Claire
2022-04-14Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-04-11FeedManager: skip account when target_account's last status is too old (#18009)dogelover911
Co-authored-by: dogelover911 <dogelover911@localhost>
2022-03-31Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-03-31Fix error MethodError in Chewy::Strategy::Sidekiq::Worker (#17912)Claire
Also refactor a bit to reduce code duplication.
2022-03-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-03-28Refactor account and status formatting (#17877)Claire
* Refactor status formatting * Add account formatting helpers * Remove StatusFormatter * Fixup * Fix copied typo
2022-03-26Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/lib/formatter.rb`: Upstream completely refactored the formatting code and removed that file, while glitch-soc had code for Markdown and HTML toots. Took upstream code, glitch-soc changes will be re-implemented on top of the refactored classes in a later commit. - `app/models/status.rb`: Upstream refactored status edit handling and moved code to `app/models/concerns/status_snapshot_concern.rb`. Applied glitch-soc's changes to that file. - `app/serializers/activitypub/note_serializer.rb`: Not really a conflict, just a line added too close to one modified by glitch-soc. Applied upstream changes while keeping the glitch-soc-modified one. - `app/services/update_status_service.rb`: Not really a conflict, upstream modified a line adjacent to one added by glitch-soc. Applied upstream changes while keeping the glitch-soc line. - `app/views/statuses/_simple_status.html.haml`: Upstream refactored formatting, glitch-soc changed the markup slightly. Applied upstream changes. - `spec/lib/formatter_spec.rb`: Upstream completely refactored the formatting code and removed that file, while glitch-soc had code for Markdown and HTML toots. Took upstream code, glitch-soc changes will be re-implemented on top of the refactored classes in a later commit.
2022-03-26Refactor formatter (#17828)Eugen Rochko
* Refactor formatter * Move custom emoji pre-rendering logic to view helpers * Move more methods out of Formatter * Fix code style issues * Remove Formatter * Add inline poll options to RSS feeds * Remove unused helper method * Fix code style issues * Various fixes and improvements * Fix test
2022-03-10Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/models/status.rb`: Upstream updated media and edit-related code textually close to glitch-soc additions (local-only and content-type). Ported upstream changes. - `app/models/status_edit.rb`: Upstream changes textually close to glitch-soc additions (content-type). Ported upstream changes. - `app/serializers/activitypub/note_serializer.rb`: Upstream changed how media attachments are handled. Not really a conflict, but textually close to glitch-soc additions (directMessage attribute). Ported upstream changes. - `app/services/remove_status_service.rb`: Upstream changed how media attachments are handled. Not really a conflict, but textually close to glitch-soc additions (DM timeline). Ported upstream changes. - `app/services/update_status_service.rb`: Upstream fixed an issue with language selection. Not really a conflict, but textually close to glitch-soc additions (content-type). Ported upstream changes. - `db/schema.rb`: Upstream added columns to the `status_edits` table, the conflict is because of an additional column (`content-type`) in glitch-soc. Ported upstream changes. - `package.json`: Upstream dependency (express) textually adjacent to a glitch-soc-specific one (favico.js) got updated. Updated it as well.
2022-03-09Change how changes to media attachments are stored for edits (#17696)Eugen Rochko
* Change how changes to media attachments are stored for edits Fix not being able to re-order media attachments * Fix not broadcasting updates when polls/media is changed through ActivityPub * Various fixes and improvements * Update app/models/report.rb Co-authored-by: Claire <claire.github-309c@sitedethib.com> * Add tracking of media attachment description changes * Change poll in status edit to have a structure closer to the real one Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-02-23Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `.github/workflows/build-image.yml`: Upstream changed the workflow a bit. Conflict comes from us pushing to ghcr while upstream pushes to dockerhub. Ported the upstream changes while still pushing to ghcr.
2022-02-22Fix various typos (#17621)luzpaz
Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,ro`
2022-02-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `db/schema.rb`: Conflict due to glitch-soc adding the `content_type` column on status edits and thus having a different schema version number. Solved by taking upstream's schema version number, as it is higher than glitch-soc's.
2022-02-16Fix performance of server-side filtering (#17575)Claire
Fixes #17567
2022-01-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `Gemfile.lock`: Upstream-updated lib textually too close to glitch-soc-only dep. Updated like upstream.
2022-01-28Fix Sidekiq warnings about JSON serialization (#17381)Claire
* Fix Sidekiq warnings about JSON serialization This occurs on every symbol argument we pass, and every symbol key in hashes, because Sidekiq expects strings instead. See https://github.com/mperham/sidekiq/pull/5071 We do not need to change how workers parse their arguments because this has not changed and we were already converting to symbols adequately or using `with_indifferent_access`. * Set Sidekiq to raise on unsafe arguments in test mode In order to more easily catch issues that would produce warnings in production code.
2022-01-19Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/lib/activitypub/activity/create.rb`: Upstream refactored how `Create` activities are handled and how values are extracted from `Create`d objects. This conflicted with how glitch-soc supported the `directMessage` flag to explicitly distinguish between limited and direct messages. Ported glitch-soc's changes to latest upstream changes. - `app/services/fan_out_on_write_service.rb`: Upstream largely refactored that file and changed some of the logic. This conflicted with glitch-soc's handling of the direct timeline and the options to allow replies and boosts in public feeds. Ported those glitch-soc changes on top of latest upstream changes. - `app/services/process_mentions_service.rb`: Upstream refactored to move mention-related ActivityPub deliveries to `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check to not send local-only toots to remote mentioned users. Took upstream's version, as the check is not needed anymore, since it is performed at the `ActivityPub::DeliveryWorker` call site already. - `app/workers/feed_insert_worker.rb`: Upstream added support for `update` toot events, while glitch-soc had support for an extra timeline support, `direct`. Ported upstream changes and extended them to the `direct` timeline. Additional changes: - `app/lib/activitypub/parser/status_parser.rb`: Added code to handle the `directMessage` flag and take it into account to compute visibility. - `app/lib/feed_manager.rb`: Extended upstream's support of `update` toot events to glitch-soc's `direct` timeline.
2022-01-19Add support for editing for published statuses (#16697)Eugen Rochko
* Add support for editing for published statuses * Fix references to stripped-out code * Various fixes and improvements * Further fixes and improvements * Fix updates being potentially sent to unauthorized recipients * Various fixes and improvements * Fix wrong words in test * Fix notifying accounts that were tagged but were not in the audience * Fix mistake
2021-05-10Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-05-10Fix blocking someone not clearing up list feeds (#16205)Claire
2021-03-24Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-03-24Update Mastodon to Rails 6.1 (#15910)Claire
* Update devise-two-factor to unreleased fork for Rails 6 support Update tests to match new `rotp` version. * Update nsa gem to unreleased fork for Rails 6 support * Update rails to 6.1.3 and rails-i18n to 6.0 * Update to unreleased fork of pluck_each for Ruby 6 support * Run "rails app:update" * Add missing ActiveStorage config file * Use config.ssl_options instead of removed ApplicationController#force_ssl Disabled force_ssl-related tests as they do not seem to be easily testable anymore. * Fix nonce directives by removing Rails 5 specific monkey-patching * Fix fixture_file_upload deprecation warning * Fix yield-based test failing with Rails 6 * Use Rails 6's index_with when possible * Use ActiveRecord::Cache::Store#delete_multi from Rails 6 This will yield better performances when deleting an account * Disable Rails 6.1's automatic preload link headers Since Rails 6.1, ActionView adds preload links for javascript files in the Links header per default. In our case, that will bloat headers too much and potentially cause issues with reverse proxies. Furhermore, we don't need those links, as we already output them as HTML link tags. * Switch to Rails 6.0 default config * Switch to Rails 6.1 default config * Do not include autoload paths in the load path
2021-01-26Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `CONTRIBUTING.md`: Not a real conflict, glitch-soc quotes the upstream file, which has been changed. Update the quote.
2021-01-22Optimize some regex matching (#15528)luigi
* Use Regex#match? * Replace =~ too * Avoid to call match? from Nil * Keep value of Regexp.last_match
2020-12-23Merge branch 'master' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/lib/feed_manager.rb`: Not a real conflict, glitch-soc-only DM-related method too close to changed upstream stuff. Ported upstream changes. - `app/services/batched_remove_status_service.rb`: Additional logic in glitch-soc to clear DMs from timelines. Ported upstream changes and fixed the DM TL clearing logic. - `app/workers/scheduler/feed_cleanup_scheduler.rb`: Additional code in glitch-soc to clear DM timelines. Ported upstream changes.
2020-12-22Improve account deletion performances further (#15407)ThibG
* Delete status records by batches of 50 * Do not precompute values that are only used once * Do not generate redis events for removal of public toots older than two weeks * Filter reported toots a priori for polls and status deletion * Do not process reblogs when cleaning up public timelines As in Mastodon proper, reblogs don't appear in public TLs * Clean the deleted account's own feed in one go * Refactor Account#clean_feed_manager and List#clean_feed_manager * Delete instead of destroy a few more associations * Fix preloading Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-09Merge branch 'master' into glitch-soc/merge-upstreamClaire
2020-12-09rename replies_policy enumerables (#15304)trwnh
2020-09-08Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - app/controllers/api/v1/timelines/public_controller.rb - app/lib/feed_manager.rb - app/models/status.rb - app/services/precompute_feed_service.rb - app/workers/feed_insert_worker.rb - spec/models/status_spec.rb All conflicts are due to upstream refactoring feed management and us having local-only toots on top of that. Rewrote local-only toots management for upstream's changes.
2020-09-08Refactor feed manager (#14761)Eugen Rochko
2020-09-07Fix multiple boosts of a same toot erroneously appearing in TL (#14759)ThibG
* Check for and record reblog info atomically Instead of using ZREVRANK to determine whether a reblog is a new reblog or not, use ZADD's NX option to perform the check/addition option atomically. * Replace ZREVRANK call with ZSCORE key which is more efficient * Make tests a bit stricter * Fix off-by-one
2020-09-01Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `config/webpack/shared.js`: Upstream has changed how Tesseract.js gets included and dropped a dependency. The conflict is caused by glitch-soc having different code due to its theming system. Ported upstream changes. - `lib/mastodon/version.rb`: Upstream refactor/code style change in a place we replaced upstream's repo URL with ours. Ported upstram changes, keeping our repo URL. - `yarn.lock`: Upstream dropped dependencies, one of which was textually too close to a glitch-soc-specific dependency. Not a real conflict.
2020-09-01Add configuration option to filter replies in lists (#9205)ThibG
* Add database support for list show-reply preferences * Add backend support to read and update list-specific show_replies settings * Add basic UI to set list replies setting * Add specs for list replies policy * Switch "cycling" reply policy link to a set of radio inputs * Capitalize replies_policy strings * Change radio button design to be consistent with that of the directory explorer
2020-09-01Change own direct-visibility statuses to be in the home feed again (#14711)Eugen Rochko
And remove highlighting in web UI Full circle from #8940
2020-07-22Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/javascript/mastodon/actions/compose.js`: Not a “real” conflict, but change too close to a change we made to fix the vanilla WebUI locally pushing authored local-only toots in the public TL view.
2020-07-17Fix boosted toots from blocked account not being retroactively removed from ↵ThibG
TL (#14339) * Fix boosted toots from blocked account not being retroactively removed from TL Fixes #14301 * Add test for clear_from_timeline
2020-07-01Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/javascript/packs/public.js`: Conflict because part of that file has been split to `app/javascript/core/settings.js`. Ported those changes there.
2020-07-01Change Redis#exists calls to Redis#exists? to avoid deprecation warning (#14191)Eugen Rochko
2020-06-24Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/concerns/sign_in_token_authentication_concern.rb`: Conflict due to glitch-soc's theming system. Ported upstream changes. - `app/controllers/concerns/two_factor_authentication_concern.rb`: Conflict due to glitch-soc's theming system. Ported upstream changes.
2020-06-23Fix crash in MergeWorker (#14129)ThibG
Similarly to #12324, the code is passing an Account object where an id is expected.
2020-06-09Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/activitypub/collections_controller.rb`: Conflict due to glitch-soc having to take care of local-only pinned toots in that controller. Took upstream's changes and restored the local-only special handling. - `app/controllers/auth/sessions_controller.rb`: Minor conflicts due to the theming system, applied upstream changes, adapted the following two files for glitch-soc's theming system: - `app/controllers/concerns/sign_in_token_authentication_concern.rb` - `app/controllers/concerns/two_factor_authentication_concern.rb` - `app/services/backup_service.rb`: Minor conflict due to glitch-soc having to handle local-only toots specially. Applied upstream changes and restored the local-only special handling. - `app/views/admin/custom_emojis/index.html.haml`: Minor conflict due to the theming system. - `package.json`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file. - `yarn.lock`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file.
2020-06-09FIX: filters ignore media descriptions (#13837)Ben Lubar
* FIX: filters ignore media descriptions * remove parentheses to make codeclimate happy * combine the text and run the regular expression only once. https://github.com/tootsuite/mastodon/pull/13837#discussion_r431752581 * Fix use of “filter” instead of “compact”, fix coding style issues Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-01-27Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/serializers/rest/account_serializer.rb`: Upstream added code too close to glitch-soc-specific followers-hiding code. Ported upstream changes.