about summary refs log tree commit diff
path: root/streaming
AgeCommit message (Collapse)Author
2022-11-15Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-11-15Fix filters from other users being used in the streaming service (#20719)Claire
2022-11-14Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/views/admin/announcements/edit.html.haml`: Upstream change too close to theming-related glitch-soc change. Ported upstream changes. - `app/views/admin/announcements/new.html.haml` Upstream change too close to theming-related glitch-soc change. Ported upstream changes.
2022-11-13Fix public/local timeline posts not being properly filtered (#20567)Claire
* Fix streaming server using wrong property name for matching filters Late in the PR, the `filter_results` property has been renamed to `filtered`, but the change has not been reflected in the streaming server code. * Fix filter_action attribute being an integer instead of a string
2022-07-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `.env.production.sample`: Our sample config file is very different from upstream since it is much more complete. Upstream added documentation for a few env variables. Copied the new variables/documentation from upstream. - `app/lib/feed_manager.rb`: Upstream added a timeline type (hashtags), while glitch-soc already had an extra one (direct messages). Not really a conflict but textually close changes. Ported upstream's changes. - `app/models/custom_emoji.rb`: Upstream upped the custom emoji size limit, while glitch-soc had configurable limits. Upped the default limits accordingly. - `streaming/index.js`: Upstream reworked how hastags were normalized. Minor conflict due to glitch-soc's handling of instance-local posts. Ported upstream's changes.
2022-07-13Change how hashtags are normalized (#18795)Eugen Rochko
* Change how hashtags are normalized * Fix tests
2022-06-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `.github/workflows/build-image.yml`: Fix erroneous deletion in a previous merge. - `Gemfile`: Conflict caused by glitch-soc-only hCaptcha dependency - `app/controllers/auth/sessions_controller.rb`: Minor conflict due to glitch-soc's theming system. - `app/controllers/filters_controller.rb`: Minor conflict due to glitch-soc's theming system. - `app/serializers/rest/status_serializer.rb`: Minor conflict due to glitch-soc having an extra `local_only` property
2022-06-28Revamp post filtering system (#18058)Claire
* Add model for custom filter keywords * Use CustomFilterKeyword internally Does not change the API * Fix /filters/edit and /filters/new * Add migration tests * Remove whole_word column from custom_filters (covered by custom_filter_keywords) * Redesign /filters Instead of a list, present a card that displays more information and handles multiple keywords per filter. * Redesign /filters/new and /filters/edit to add and remove keywords This adds a new gem dependency: cocoon, as well as a npm dependency: cocoon-js-vanilla. Those are used to easily populate and remove form fields from the user interface when manipulating multiple keyword filters at once. * Add /api/v2/filters to edit filter with multiple keywords Entities: - `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context` `keywords` - `FilterKeyword`: `id`, `keyword`, `whole_word` API endpoits: - `GET /api/v2/filters` to list filters (including keywords) - `POST /api/v2/filters` to create a new filter `keywords_attributes` can also be passed to create keywords in one request - `GET /api/v2/filters/:id` to read a particular filter - `PUT /api/v2/filters/:id` to update a new filter `keywords_attributes` can also be passed to edit, delete or add keywords in one request - `DELETE /api/v2/filters/:id` to delete a particular filter - `GET /api/v2/filters/:id/keywords` to list keywords for a filter - `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a filter - `GET /api/v2/filter_keywords/:id` to read a particular keyword - `PUT /api/v2/filter_keywords/:id` to edit a particular keyword - `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword * Change from `irreversible` boolean to `action` enum * Remove irrelevent `irreversible_must_be_within_context` check * Fix /filters/new and /filters/edit with update for filter_action * Fix Rubocop/Codeclimate complaining about task names * Refactor FeedManager#phrase_filtered? This moves regexp building and filter caching to the `CustomFilter` class. This does not change the functional behavior yet, but this changes how the cache is built, doing per-custom_filter regexps so that filters can be matched independently, while still offering caching. * Perform server-side filtering and output result in REST API * Fix numerous filters_changed events being sent when editing multiple keywords at once * Add some tests * Use the new API in the WebUI - use client-side logic for filters we have fetched rules for. This is so that filter changes can be retroactively applied without reloading the UI. - use server-side logic for filters we haven't fetched rules for yet (e.g. network error, or initial timeline loading) * Minor optimizations and refactoring * Perform server-side filtering on the streaming server * Change the wording of filter action labels * Fix issues pointed out by linter * Change design of “Show anyway” link in accordence to review comments * Drop “irreversible” filtering behavior * Move /api/v2/filter_keywords to /api/v1/filters/keywords * Rename `filter_results` attribute to `filtered` * Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer * Fix systemChannelId value in streaming server * Simplify code by removing client-side filtering code The simplifcation comes at a cost though: filters aren't retroactively applied anymore.
2022-04-25Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-04-19Fix parsing `TRUSTED_PROXY_IP` (#18051)Yamagishi Kazutoshi
2022-03-22Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `.github/dependabot.yml`: Upstream modified it, but we deleted it in glitch-soc. Keep it deleted.
2022-03-21Fix streaming server sometimes silently dropping subscriptions (#17841)Claire
2022-02-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `db/schema.rb`: Conflict due to glitch-soc adding the `content_type` column on status edits and thus having a different schema version number. Solved by taking upstream's schema version number, as it is higher than glitch-soc's.
2022-02-16Change streaming server error messages when failing to parse client input ↵Claire
(#17559) Fixes #17541 - prefix JSON parsing error message by “Error parsing message from …” - output user id if a user is logged in, IP address otherwise - reduce log level from error to warning when a user is logged in, and to silly otherwise
2022-01-07Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-01-07Fix timeline streaming stopping for multiple sessions instead of one (#17259)Claire
* Fix timeline streaming stopping for multiple sessions instead of one Fixes #17256. In updating the code for a newer version of node-redis, #17183 also broke redis subscription management when multiple streaming clients subscribe to the same channel. This commit restores the redis subscription management code. * Let node-redis actually handle the subscriptions
2021-12-27Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `README.md`: We have completely different contents. Kept our version. - `package.json`: Not a real conflict, just an upstream dependency udpated textually too close to a glitch-soc-only dependency. Updated dependencies like upstream. - `streaming/index.js`: Conflict due to code style changes on parts that were modified in glitch-soc to handle local-only toots. Changed style according to upstream.
2021-12-25Finish update of node-redis (#17183)Lerk
* fix streaming redis client * use console.error instead of console.log * follow node-redis migration guide https://github.com/redis/node-redis/blob/master/docs/v3-to-v4.md * fix config options for node-redis * keep indentation * Update streaming/index.js Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2021-10-13Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `package.json`: Not really a conflict, just a glitch-soc-only dependency textually too close to an updated upstream one. - `yarn.lock`: Not really a conflict, just a glitch-soc-only dependency textually too close to an updated upstream one.
2021-10-13fix(streaming): req.scopes can be nullable (#16823)Sasha Sorokin
When checking for required OAuth scopes, an unexpected error could happen due to missing (null-y) req.scopes. This commit fixes that by checking if req.scopes are present before checking if any required scopes are present, otherwise it skips that straight to rejection.
2021-09-26Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `streaming/index.js`: Filtering code for streaming notifications has been refactored upstream, but glitch-soc had similar code for local-only toots in the same places. Ported upstream changes, but did not refactor local-only filtering.
2021-09-26Refactor notifications to go through a separate stream in streaming API (#16765)Eugen Rochko
Eliminate need to have custom notifications filtering logic in the streaming API code by publishing notifications into a separate stream and then simply using the multi-stream capability to subscribe to that stream when necessary
2021-05-07Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `app/views/statuses/_simple_status.html.haml`: Small markup change in glitch-soc, on a line that has been modified by upstream. Ported upstream changes.
2021-05-02Fix "cb is not a function" error in streaming API server (#16134)Eugen Rochko
Third argument of `ping` is the callback Regression from #15932
2021-05-01Change log level of worker start/end to warn in streaming API (#16110)Eugen Rochko
2021-03-24Merge branch 'main' into glitch-soc/merge-upstreamClaire
2021-03-24Switch from deprecated ClusterWS/cws to ws package (#15932)Claire
* Switch from deprecated ClusterWS/cws to ws package Fixes #15184 Co-authored-by: Edho Arief <me@nanaya.pro> * Make bufferutil and utf-8-validate optional dependencies Co-authored-by: Edho Arief <me@nanaya.pro>
2020-11-25Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `Gemfile.lock`: glitch-soc-only dependency textually too close to updated upstream dependencies. Updated to upsteam dependencies.
2020-11-23Fixed ESLint error (#15214)abcang
* eslint --fix * fix consistent-return * fix promise/catch-or-return * ignore import rule
2020-11-14Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/concerns/sign_in_token_authentication_concern.rb`: Conflict caused because of glitch-soc's theming system. Took upstream's new code and applied the theming system changes on top of it. - `app/controllers/concerns/two_factor_authentication_concern.rb`: Conflict caused because of glitch-soc's theming system. Took upstream's new code and applied the theming system changes on top of it.
2020-11-12Fix streaming API allowing connections to persist after access token ↵Eugen Rochko
invalidation (#15111) Fix #14816
2020-09-28Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `Gemfile.lock`: Not a real conflict, upstream updated dependencies that were too close to glitch-soc-only ones in the file. - `app/controllers/oauth/authorized_applications_controller.rb`: Upstream changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's theming system. Ported upstream changes. - `app/controllers/settings/base_controller.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's theming system. Ported upstream changes. - `app/controllers/settings/sessions_controller.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's theming system. Ported upstream changes. - `app/models/user.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc not preventing moved accounts from logging in. Ported upstream changes while keeping the ability for moved accounts to log in. - `app/policies/status_policy.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's local-only toots. Ported upstream changes. - `app/serializers/rest/account_serializer.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's ability to hide followers count. Ported upstream changes. - `app/services/process_mentions_service.rb`: Upstream refactored and changed the logic surrounding suspended accounts. Minor conflict due to glitch-soc's local-only toots. Ported upstream changes. - `package.json`: Not a real conflict, upstream updated dependencies that were too close to glitch-soc-only ones in the file.
2020-09-22Fix crash streaming process when receive invalid json (#14859)fuyu
2020-08-13Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
2020-08-12Fix destructuring error when unsubscribing without subscribing (#14566)Eugen Rochko
2020-08-11Merge branch 'master' into glitch-soc/masterThibaut Girka
Conflicts: - `streaming/index.js`: Upstream entirely refactored it. Ported our changes to upstream's refactor. Hopefuly.
2020-08-11Add support for managing multiple stream subscriptions in a single ↵Eugen Rochko
connection (#14524)
2020-07-07Fix streaming server only working for streams allowing local-only tootsThibaut Girka
2020-07-06Add UI option to show local-only toots in public timelineThibaut Girka
2020-06-26Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
2020-06-24Change streaming server to treat blank redis password as password-less auth ↵ThibG
(#14135) Fixes #14131 Our `mastodon:setup` task defaults to a blank password rather than the absence of password, but some versions of Redis reject blank password authentication when authentication is possible without a password. The Ruby code only uses the Redis password when it's not blank, so do the same for the node.js part.
2020-06-09Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `app/controllers/activitypub/collections_controller.rb`: Conflict due to glitch-soc having to take care of local-only pinned toots in that controller. Took upstream's changes and restored the local-only special handling. - `app/controllers/auth/sessions_controller.rb`: Minor conflicts due to the theming system, applied upstream changes, adapted the following two files for glitch-soc's theming system: - `app/controllers/concerns/sign_in_token_authentication_concern.rb` - `app/controllers/concerns/two_factor_authentication_concern.rb` - `app/services/backup_service.rb`: Minor conflict due to glitch-soc having to handle local-only toots specially. Applied upstream changes and restored the local-only special handling. - `app/views/admin/custom_emojis/index.html.haml`: Minor conflict due to the theming system. - `package.json`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file. - `yarn.lock`: Upstream dependency updated, too close to a glitch-soc-only dependency in the file.
2020-06-02Add E2EE API (#13820)Eugen Rochko
2020-05-10Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - `Gemfile.lock`: Not a real conflict, just a glitch-soc-only dependency too close to a dependency that got updated upstream. Updated as well. - `app/models/status.rb`: Not a real conflict, just a change too close to glitch-soc-changed code for optionally showing boosts in public timelines. Applied upstream changes. - `app/views/layouts/application.html.haml`: Upstream a new, static CSS file, conflict due to glitch-soc's theming system, include the file regardless of the theme. - `config/initializers/content_security_policy.rb`: Upstream dropped 'unsafe-inline' from the 'style-src' directive, but both files are very different. Removed 'unsafe-inline' as well.
2020-05-10Add remote only to public timeline (#13504)Takeshi Umeda
* Add remote only to public timeline * Fix code style
2020-01-25Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
2020-01-24minor server-sent events fixes (#12945)Ben Lubar
* Send output on the server-sent events stream immediately so the client sees that it was successfully opened even if it doesn't have any messages. Fix transparent SSE streaming for the public:local and hashtag:local stream types. * Tell caches to never store server-sent events.
2019-07-30Merge branch 'master' into glitch-soc/merge-upstreamThibaut Girka
Conflicts: - app/controllers/directories_controller.rb - package.json - yarn.lock
2019-07-30Add whitelist mode (#11291)Eugen Rochko
2019-07-16Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (#11302)Eugen Rochko