about summary refs log tree commit diff
path: root/app/lib
AgeCommit message (Collapse)Author
2022-12-04Fix unbounded recursion in account discovery (#1994)Claire
* Fix trying to fetch posts from other users when fetching featured posts * Rate-limit discovery of new subdomains * Put a limit on recursively discovering new accounts
2022-12-02Use a tree‐based approach for advanced text formatting (#1907)kibigo!
* Use a tree‐based approach for adv. text formatting Sanitizing HTML/Markdown means parsing the content into an HTML tree under‐the‐hood anyway, and it is more accurate to do mention/hashtag replacement on the text nodes in that tree than it is to try to hack it in with regexes et cetera. This undoes the overrides of `#entities` and `#rewrite` on `AdvancedTextFormatter` but also stops using them, instead keeping track of the parsed Nokogiri tree itself and using that in the `#to_s` method. Internally, this tree uses `<mastodon-entity>` nodes to keep track of hashtags, links, and mentions. Sanitization is moved to the beginning, so it should be known that these do not appear in the input. * Also disallow entities inside of `<code>` I think this is generally expected behaviour, and people are annoyed when their code gets turned into links/hashtags/mentions. * Minor cleanup to AdvancedTextFormatter * Change AdvancedTextFormatter to rewrite entities in one pass and sanitize at the end Also, minor refactoring to better match how other formatters are organized. * Add some tests Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-01Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-12-01Fix some performance issues with /admin/instances (#21907)Claire
/admin/instances?availability=failing remains wholly unefficient
2022-11-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `README.md`: Our README is completely different. Discarded upstream changes.
2022-11-27refactor(vacuum statuses): reduce amount of db queries and load for each ↵Kaspar V
query - improve performance (#21487) * refactor(statuses_vacuum): remove dead code - unused Method is not called inside class and private. Clean up dead code. * refactor(statuses_vacuum): make retention_period present test explicit This private method only hides functionality. It is best practice to be as explicit as possible. * refactor(statuses_vacuum): improve query performance - fix statuses_scope having sub-select for Account.remote scope by `joins(:account).merge(Account.remote)` - fix statuses_scope unnecessary use of `Status.arel_table[:id].lt` because it is inexplicit, bad practice and even slower than normal `.where('statuses.id < ?'` - fix statuses_scope remove select(:id, :visibility) for having reusable active record query batches (no re queries) - fix vacuum_statuses! to use in_batches instead of find_in_batches, because in_batches delivers a full blown active record query result, in stead of an array - no requeries necessary - send(:unlink_from_conversations) not to perform another db query, but reuse the in_batches result instead. - remove now obsolete remove_from_account_conversations method - remove_from_search_index uses array of ids, instead of mapping the ids from an array - this should be more efficient - use the in_batches scope to call delete_all, instead of running another db query for this - because it is again more efficient - add TODO comment for calling models private method with send * refactor(status): simplify unlink_from_conversations - add `has_many through:` relation mentioned_accounts - use model scope local instead of method call `Status#local?` - more readable add account to inbox_owners when account.local? * refactor(status): searchable_by way less sub selects These queries all included a sub-select. Doing the same with a joins should be more efficient. Since this method does 5 such queries, this should be significant, since it technically halves the query count. This is how it was: ```ruby [3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain Status Load (1.6ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]] Mention Load (1.5ms) SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]] => EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]] QUERY PLAN ------------------------------------------------------------------------------------------------------------------ Nested Loop (cost=0.15..23.08 rows=1 width=41) -> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8) Filter: (domain IS NULL) -> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41) Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint)) Filter: (NOT silent) (6 rows) ``` This is how it is with this change: ```ruby [4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain Status Load (1.7ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]] Mention Load (0.7ms) SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]] => EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]] QUERY PLAN ------------------------------------------------------------------------------------------------------------------ Nested Loop (cost=0.15..23.08 rows=1 width=41) -> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8) Filter: (domain IS NULL) -> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41) Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint)) Filter: (NOT silent) (6 rows) ```
2022-11-14Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `Gemfile`: Upstream removed blank lines.
2022-11-14Fix emoji substitution not applying only to text nodes in backend code (#20641)Claire
Signed-off-by: Claire <claire.github-309c@sitedethib.com> Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-11-12Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/models/account.rb`: Conflict because we (glitch-soc) have disabled trending of posts without review. Discarded that upstream change. - `app/views/admin/settings/discovery/show.html.haml`: Just an extra setting in glitch-soc. Kept that extra setting.
2022-11-11fixes ArgumentError when proxy is used (#20420)Bearice Ren
* fixes ArgumentError when proxy is used * Update app/lib/request.rb Co-authored-by: Claire <claire.github-309c@sitedethib.com> Co-authored-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-11-10Change incoming activity processing to happen in `ingress` queue (#20264)Eugen Rochko
2022-11-10Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/models/custom_emoji.rb`: Not a real conflict, just upstream changing a line too close to a glitch-soc-specific validation. Applied upstream changes. - `app/models/public_feed.rb`: Not a real conflict, just upstream changing a line too close to a glitch-soc-specific parameter documentation. Applied upstream changes.
2022-11-10Improve performance by avoiding regex construction (#20215)James Tucker
```ruby 10.times { p /#{FOO}/.object_id } 10.times { p FOO_RE.object_id } ```
2022-11-08Change Request connection logic to try both IPv6 and IPv4 when available ↵Claire
(#20108) Fixes #19751
2022-11-08Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-11-08Improve performance by avoiding method cache busts (#19957)James Tucker
Switch to monkey-patching http.rb rather than a runtime extend of each response, so as to avoid busting the global method cache. A guard is included that will provide developer feedback in development and test environments should the monkey patch ever collide.
2022-11-07Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-11-07Fix filter handling in status cache hydration (#19963)Claire
2022-11-06Fix status cache hydration discrepancyClaire
2022-11-06Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/javascript/mastodon/features/compose/components/poll_form.js`: glitch-soc change because of having changed the default number of available poll options. Applied upstream's changes while keeping glitch-soc's default number of poll options. - `public/oops.png`: We had a minor graphics change, probably not worth diverging from upstream. Took upstream version.
2022-11-05Fix users not being able to change their hide_followers_count setting (#1889)Claire
2022-11-04Fix additional issues with status cache hydration (#19747)Claire
* Spare one SQL query when hydrating polls * Improve tests * Fix more discrepancies * Fix possible crash when the status has no application set
2022-11-04Fix various issues with store hydration (#19746)Claire
- Improve tests - Fix possible crash when application of a reblogged post isn't set - Fix discrepancies around favourited and reblogged attributes - Fix discrepancies around pinned attribute - Fix polls not being hydrated
2022-11-04Fix reblogs being discarded after the reblogged status (#19731)Claire
2022-11-04Add caching for payload serialization during fan-out (#19642)Eugen Rochko
2022-10-28Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-10-26Fix improperly checking for blocked domain on followed hashtags (#19472)Eugen Rochko
Fix #19469
2022-10-26Fix vacuum scheduler missing lock, locks never expiring (#19458)Eugen Rochko
Remove vacuuming of orphaned preview cards
2022-10-26Revert "Remove preference to aggregate reblogs in home/list feeds (#18112)" ↵Eugen Rochko
(#19463) This reverts commit af396fa35f589e1f759c7a3a0dad7932f1862309.
2022-10-24Add mention of the translation provider when translating a post (#19433)Claire
2022-10-24Change “Translate” button to only show up when a translation backend is ↵Claire
configured (#19434) * Change “Translate” button to only show up when a translation backend is configured Fixes #19346 * Add `translation` attribute to /api/v2/instance to expose whether the translation feature is enabled Fixes #19328
2022-10-22Add featured tag add/remove activity handler (#19408)Takeshi Umeda
2022-10-20Change public accounts pages to mount the web UI (#19319)Eugen Rochko
* Change public accounts pages to mount the web UI * Fix handling of remote usernames in routes - When logged in, serve web app - When logged out, redirect to permalink - Fix `app-body` class not being set sometimes due to name conflict * Fix missing `multiColumn` prop * Fix failing test * Use `discoverable` attribute to control indexing directives * Fix `<ColumnLoading />` not using `multiColumn` * Add `noindex` to accounts in REST API * Change noindex directive to not be rendered by default before a route is mounted * Add loading indicator for detailed status in web UI * Fix missing indicator appearing while account is loading in web UI
2022-10-09[Glitch] Allow non-logged users to access /webEugen Rochko
Port 43b5d5e38d2b8ad8f1d1ad0911c3c1718159c912 to glitch-soc Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-10-02Revert server-side part of "Add logged-out access to the web UI"Claire
2022-10-02Merge branch 'main' into glitch-soc/merge-upstreamClaire
Conflicts: - `app/controllers/home_controller.rb`: Upstream made it so `/web` is available to non-logged-in users and `/` redirects to `/web` instead of `/about`. Kept our version since glitch-soc's WebUI doesn't have what's needed yet and I think /about is still a much better landing page anyway. - `app/models/form/admin_settings.rb`: Upstream added new settings, and glitch-soc had an extra setting. Not really a conflict. Added upstream's new settings. - `app/serializers/initial_state_serializer.rb`: Upstream added a new `server` initial state object. Not really a conflict. Merged upstream's changes. - `app/views/admin/settings/edit.html.haml`: Upstream added new settings. Not really a conflict. Merged upstream's changes. - `app/workers/scheduler/feed_cleanup_scheduler.rb`: Upstream refactored that part and removed the file. Ported our relevant changes into `app/lib/vacuum/feeds_vacuum.rb` - `config/settings.yml`: Upstream added new settings. Not a real conflict. Added upstream's new settings.
2022-09-29Add logged-out access to the web UI (#18961)Eugen Rochko
2022-09-27Fix auto detect language for translate service (#19244)Yamagishi Kazutoshi
2022-09-27Add retention policy for cached content and media (#19232)Eugen Rochko
2022-09-23Add user content translations with configurable backends (#19218)Eugen Rochko
2022-09-21Refactor ActivityPub handling to prepare for non-Account actors (#19212)Claire
* Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService ActivityPub::FetchRemoteAccountService is kept as a wrapper for when the actor is specifically required to be an Account * Refactor SignatureVerification to allow non-Account actors * fixup! Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService * Refactor ActivityPub::FetchRemoteKeyService to potentially return non-Account actors * Refactor inbound ActivityPub payload processing to accept non-Account actors * Refactor inbound ActivityPub processing to accept activities relayed through non-Account * Refactor how Account key URIs are built * Refactor Request and drop unused key_id_format parameter * Rename ActivityPub::Dereferencer `signature_account` to `signature_actor`
2022-09-20Add ability to filter followed accounts' posts by language (#19095)Eugen Rochko
2022-09-20Improve error reporting and logging when processing remote accounts (#15605)Claire
* Add a more descriptive PrivateNetworkAddressError exception class * Remove unnecessary exception class to rescue clause * Remove unnecessary include to JsonLdHelper * Give more neutral error message when too many webfinger redirects * Remove unnecessary guard condition * Rework how “ActivityPub::FetchRemoteAccountService” handles errors Add “suppress_errors” keyword argument to avoid raising errors in ActivityPub::FetchRemoteAccountService#call (default/previous behavior). * Rework how “ActivityPub::FetchRemoteKeyService” handles errors Add “suppress_errors” keyword argument to avoid raising errors in ActivityPub::FetchRemoteKeyService#call (default/previous behavior). * Fix Webfinger::RedirectError not being a subclass of Webfinger::Error * Add suppress_errors option to ResolveAccountService Defaults to true (to preserve previous behavior). If set to false, errors will be raised instead of caught, allowing the caller to be informed of what went wrong. * Return more precise error when failing to fetch account signing AP payloads * Add tests * Fixes * Refactor error handling a bit * Fix various issues * Add specific error when provided Digest is not 256 bits of base64-encoded data * Please CodeClimate * Improve webfinger error reporting
2022-08-25Remove preference to aggregate reblogs in home/list feeds (#18112)Eugen Rochko
2022-08-25Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-08-25Support "http_hidden_proxy" ENV var for hidden service only proxy (#18427)Jeong Arm
* Support "http_hidden_proxy" ENV var for hidden service only proxy * Fallback to http_proxy if http_hidden_proxy is not set
2022-08-21Merge branch 'main' into glitch-soc/merge-upstreamClaire
2022-08-17Fix ambiguous column names in `tootctl search deploy` (#18993)Mashiro
2022-07-17Merge branch 'main' into glitch-soc/merge-upstreamClaire
- `.env.production.sample`: Our sample config file is very different from upstream since it is much more complete. Upstream added documentation for a few env variables. Copied the new variables/documentation from upstream. - `app/lib/feed_manager.rb`: Upstream added a timeline type (hashtags), while glitch-soc already had an extra one (direct messages). Not really a conflict but textually close changes. Ported upstream's changes. - `app/models/custom_emoji.rb`: Upstream upped the custom emoji size limit, while glitch-soc had configurable limits. Upped the default limits accordingly. - `streaming/index.js`: Upstream reworked how hastags were normalized. Minor conflict due to glitch-soc's handling of instance-local posts. Ported upstream's changes.
2022-07-17Add ability to follow hashtags (#18809)Eugen Rochko
generated by cgit-pink (git 2.37.1) at 2024-11-05 04:02:13 +0000