about summary refs log tree commit diff
path: root/streaming/index.js
AgeCommit message (Collapse)Author
2023-02-11fix metrics format (#23520)emilweth
2023-02-11dot is not allowed (#23519)emilweth
2023-02-09Add information about database pool to metrics endpoint in streaming API ↵Eugen Rochko
(#23469)
2023-02-09Add database pool configuration over `DB_POOL` to streaming API (#23470)Eugen Rochko
Add connection timeout of 15s
2023-02-03Add metrics endpoint to streaming API (#23388)Eugen Rochko
2022-12-19Enable eslint:recommended ruleset (#22433)Nick Schonning
* Enable ESLint recommended ruleset * Disable failing ESLint recommended rules * Remove rules shadowed by eslint:recommended
2022-11-15Fix filters from other users being used in the streaming service (#20719)Claire
2022-11-13Fix public/local timeline posts not being properly filtered (#20567)Claire
* Fix streaming server using wrong property name for matching filters Late in the PR, the `filter_results` property has been renamed to `filtered`, but the change has not been reflected in the streaming server code. * Fix filter_action attribute being an integer instead of a string
2022-07-13Change how hashtags are normalized (#18795)Eugen Rochko
* Change how hashtags are normalized * Fix tests
2022-06-28Revamp post filtering system (#18058)Claire
* Add model for custom filter keywords * Use CustomFilterKeyword internally Does not change the API * Fix /filters/edit and /filters/new * Add migration tests * Remove whole_word column from custom_filters (covered by custom_filter_keywords) * Redesign /filters Instead of a list, present a card that displays more information and handles multiple keywords per filter. * Redesign /filters/new and /filters/edit to add and remove keywords This adds a new gem dependency: cocoon, as well as a npm dependency: cocoon-js-vanilla. Those are used to easily populate and remove form fields from the user interface when manipulating multiple keyword filters at once. * Add /api/v2/filters to edit filter with multiple keywords Entities: - `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context` `keywords` - `FilterKeyword`: `id`, `keyword`, `whole_word` API endpoits: - `GET /api/v2/filters` to list filters (including keywords) - `POST /api/v2/filters` to create a new filter `keywords_attributes` can also be passed to create keywords in one request - `GET /api/v2/filters/:id` to read a particular filter - `PUT /api/v2/filters/:id` to update a new filter `keywords_attributes` can also be passed to edit, delete or add keywords in one request - `DELETE /api/v2/filters/:id` to delete a particular filter - `GET /api/v2/filters/:id/keywords` to list keywords for a filter - `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a filter - `GET /api/v2/filter_keywords/:id` to read a particular keyword - `PUT /api/v2/filter_keywords/:id` to edit a particular keyword - `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword * Change from `irreversible` boolean to `action` enum * Remove irrelevent `irreversible_must_be_within_context` check * Fix /filters/new and /filters/edit with update for filter_action * Fix Rubocop/Codeclimate complaining about task names * Refactor FeedManager#phrase_filtered? This moves regexp building and filter caching to the `CustomFilter` class. This does not change the functional behavior yet, but this changes how the cache is built, doing per-custom_filter regexps so that filters can be matched independently, while still offering caching. * Perform server-side filtering and output result in REST API * Fix numerous filters_changed events being sent when editing multiple keywords at once * Add some tests * Use the new API in the WebUI - use client-side logic for filters we have fetched rules for. This is so that filter changes can be retroactively applied without reloading the UI. - use server-side logic for filters we haven't fetched rules for yet (e.g. network error, or initial timeline loading) * Minor optimizations and refactoring * Perform server-side filtering on the streaming server * Change the wording of filter action labels * Fix issues pointed out by linter * Change design of “Show anyway” link in accordence to review comments * Drop “irreversible” filtering behavior * Move /api/v2/filter_keywords to /api/v1/filters/keywords * Rename `filter_results` attribute to `filtered` * Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer * Fix systemChannelId value in streaming server * Simplify code by removing client-side filtering code The simplifcation comes at a cost though: filters aren't retroactively applied anymore.
2022-04-19Fix parsing `TRUSTED_PROXY_IP` (#18051)Yamagishi Kazutoshi
2022-03-21Fix streaming server sometimes silently dropping subscriptions (#17841)Claire
2022-02-16Change streaming server error messages when failing to parse client input ↵Claire
(#17559) Fixes #17541 - prefix JSON parsing error message by “Error parsing message from …” - output user id if a user is logged in, IP address otherwise - reduce log level from error to warning when a user is logged in, and to silly otherwise
2022-01-07Fix timeline streaming stopping for multiple sessions instead of one (#17259)Claire
* Fix timeline streaming stopping for multiple sessions instead of one Fixes #17256. In updating the code for a newer version of node-redis, #17183 also broke redis subscription management when multiple streaming clients subscribe to the same channel. This commit restores the redis subscription management code. * Let node-redis actually handle the subscriptions
2021-12-25Finish update of node-redis (#17183)Lerk
* fix streaming redis client * use console.error instead of console.log * follow node-redis migration guide https://github.com/redis/node-redis/blob/master/docs/v3-to-v4.md * fix config options for node-redis * keep indentation * Update streaming/index.js Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh> Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2021-10-13fix(streaming): req.scopes can be nullable (#16823)Sasha Sorokin
When checking for required OAuth scopes, an unexpected error could happen due to missing (null-y) req.scopes. This commit fixes that by checking if req.scopes are present before checking if any required scopes are present, otherwise it skips that straight to rejection.
2021-09-26Refactor notifications to go through a separate stream in streaming API (#16765)Eugen Rochko
Eliminate need to have custom notifications filtering logic in the streaming API code by publishing notifications into a separate stream and then simply using the multi-stream capability to subscribe to that stream when necessary
2021-05-02Fix "cb is not a function" error in streaming API server (#16134)Eugen Rochko
Third argument of `ping` is the callback Regression from #15932
2021-05-01Change log level of worker start/end to warn in streaming API (#16110)Eugen Rochko
2021-03-24Switch from deprecated ClusterWS/cws to ws package (#15932)Claire
* Switch from deprecated ClusterWS/cws to ws package Fixes #15184 Co-authored-by: Edho Arief <me@nanaya.pro> * Make bufferutil and utf-8-validate optional dependencies Co-authored-by: Edho Arief <me@nanaya.pro>
2020-11-23Fixed ESLint error (#15214)abcang
* eslint --fix * fix consistent-return * fix promise/catch-or-return * ignore import rule
2020-11-12Fix streaming API allowing connections to persist after access token ↵Eugen Rochko
invalidation (#15111) Fix #14816
2020-09-22Fix crash streaming process when receive invalid json (#14859)fuyu
2020-08-12Fix destructuring error when unsubscribing without subscribing (#14566)Eugen Rochko
2020-08-11Add support for managing multiple stream subscriptions in a single ↵Eugen Rochko
connection (#14524)
2020-06-24Change streaming server to treat blank redis password as password-less auth ↵ThibG
(#14135) Fixes #14131 Our `mastodon:setup` task defaults to a blank password rather than the absence of password, but some versions of Redis reject blank password authentication when authentication is possible without a password. The Ruby code only uses the Redis password when it's not blank, so do the same for the node.js part.
2020-06-02Add E2EE API (#13820)Eugen Rochko
2020-05-10Add remote only to public timeline (#13504)Takeshi Umeda
* Add remote only to public timeline * Fix code style
2020-01-24minor server-sent events fixes (#12945)Ben Lubar
* Send output on the server-sent events stream immediately so the client sees that it was successfully opened even if it doesn't have any messages. Fix transparent SSE streaming for the public:local and hashtag:local stream types. * Tell caches to never store server-sent events.
2019-07-30Add whitelist mode (#11291)Eugen Rochko
2019-07-15Change default interface of web and streaming from 0.0.0.0 to 127.0.0.1 (#11302)Eugen Rochko
2019-05-24Improve streaming server security (#10818)ThibG
* Check OAuth token scopes in the streaming API * Use Sec-WebSocket-Protocol instead of query string to pass WebSocket token Inspired by https://github.com/kubevirt/kubevirt/issues/1242
2019-05-22Migrate from uws to cws (#10805)abcang
2019-03-11Fix streaming API always attempting to use SSL with Postgres (#10231)Eugen Rochko
Fix #10223
2019-03-10Fix SSL configuration regression in streaming API (#10225)Eugen Rochko
* Fix SSL configuration regression in streaming API Fix #10223 * Fix code style issues and integrate #10219 * Fix dumb thing
2019-03-08config: add DB_SSLMODE for managed/remote PG (#10210)Sascha
* config: add DB_SSLMODE for managed/remote PG * streaming: set PG sslmode, defaults to prefer
2018-10-21Fix crash when using UNIX socket (#9036)Gomasy
2018-10-20Check if port/socket is available before forking in Streaming API (#9023)Eugen Rochko
Previously, the server would attempt taking port/socket in worker process, and if it was taken, fail, which made the master process create a new worker. This led to really high CPU usage if the streaming API was started when the port or socket were not available. Now, before clustering (forking) into worker processes, a test server is created and then removed to check if it can be done.
2018-10-11Add check for missing tag param in streaming API (#8955)Eugen Rochko
* Add check for missing tag param in streaming API Fixes error: ``` TypeError: Cannot read property 'toLowerCase' of undefined at app.get (.../streaming/index.js:493:50) ``` * Fix code style issues
2018-10-07Add conversations API (#8832)Eugen Rochko
* Add conversations API * Add web UI for conversations * Add test for conversations API * Add tests for ConversationAccount * Improve web UI * Rename ConversationAccount to AccountConversation * Remove conversations on block and mute * Change last_status_id to be a denormalization of status_ids * Add optimistic locking
2018-08-26Add health endpoint to streaming API (#8441)Eugen Rochko
GET /api/v1/streaming/health Answers with OK. Fix #8337
2018-08-24Support UNIX domain socket for streaming service without using PORT (#8217)MIYAGI Hikaru
* Support UNIX domain socket for streaming service without using PORT The use of UNIX domain socket for streaming service was not officially supported, but it was made unofficial to use by setting a path to PORT. From now on, SOCKET will be used just like setting for puma. * Hundle relative path
2018-07-14Fix streaming API still using filtered instead of chosen languages (#8009)Eugen Rochko
2018-05-21Add media timeline (#6631)Yamagishi Kazutoshi
2018-04-18Direct messages column (#4514)Kaito Sinclaire
* Added a timeline for Direct statuses * Lists all Direct statuses you've sent and received * Displayed in Getting Started * Streaming server support for direct TL * Changes to match other timelines in 2.0
2018-04-17Perform processing that does not use the database before connecting to the ↵abcang
database (#7168)
2017-12-13Make sure call `done();` in the listener of public timeline for anonymous ↵nullkal
connection (#6009)
2017-12-13Fix the condition in streaming listener (#6008)nullkal
2017-12-12Improve error handling in streaming/index.js (#5968)erin
On an unhandled worker exception, we should log the exception and exit with nonzero status, instead of letting workers silently fail and restarting them in an endless loop. Note: we previously tried to handle the `'error'` signal. That's not a signal Node fires; my patch traps `'uncaughtException'`, which is what the code was _trying_ to do.
2017-12-12make it possible to stream public timelines without authorization (#5977)nullkal
* make it possible to stream public timelines without authorization * Fix * Make eslint allow `value == null` * Remove redundant line * Improve style and revert .eslintrc.yml * Fix streamWsEnd * Show IP address instead of (anonymous user) * Add missing semicolon
generated by cgit-pink (git 2.37.1) at 2024-11-22 16:15:12 +0000