commit: 64660423c5540761f3924d342ce60656423b7eb4 parent 16027b769c212095fb6a19d7eaa843a31e91b3ce Author: lain <lain@soykaf.club> Date: Thu, 19 Dec 2024 10:47:04 +0000 Merge branch 'mergeback/2.8.0' into 'develop' Mergeback/2.8.0 See merge request pleroma/pleroma!4302Diffstat:
70 files changed, 129 insertions(+), 76 deletions(-)diff --git a/CHANGELOG.md b/CHANGELOG.md@@ -4,6 +4,65 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). +## 2.8.0 + +### Changed +- Metadata: Do not include .atom feed links for remote accounts +- Bumped `fast_html` to v2.3.0, which notably allows to use system-installed lexbor with passing `WITH_SYSTEM_LEXBOR=1` environment variable at build-time +- Dedupe upload filter now uses a three-level sharding directory structure +- Deprecate `/api/v1/pleroma/accounts/:id/subscribe`/`unsubscribe` +- Restrict incoming activities from unknown actors to a subset that does not imply a previous relationship and early rejection of unrecognized activity types. +- Elixir 1.14 and Erlang/OTP 23 is now the minimum supported release +- Support `id` param in `GET /api/v1/statuses` +- LDAP authentication has been refactored to operate as a GenServer process which will maintain an active connection to the LDAP server. +- Fix 'Setting a marker should mark notifications as read' +- Adjust more Oban workers to enforce unique job constraints. +- Oban updated to 2.18.3 +- Publisher behavior improvement when snoozing Oban jobs due to Gun connection pool contention. +- Poll results refreshing is handled asynchronously and will not attempt to keep fetching updates to a closed poll. +- Tuning for release builds to lower CPU usage. +- Rich Media preview fetching will skip making an HTTP HEAD request to check a URL for allowed content type and length if the Tesla adapter is Gun or Finch +- Fix nonexisting user will not generate metadata for search engine opt-out +- Update Oban to 2.18 +- Worker configuration is no longer available. This only affects custom max_retries values for a couple Oban queues. + +### Added +- Add metadata provider for ActivityPub alternate links +- Added support for argon2 passwords and their conversion for migration from Akkoma fork to upstream. +- Respect :restrict_unauthenticated for hashtag rss/atom feeds +- LDAP configuration now permits overriding the CA root certificate file for TLS validation. +- LDAP now supports users changing their passwords +- Include list id in StatusView +- Added MRF.FODirectReply which changes replies to followers-only posts to be direct. +- Add `id_filter` to MRF to filter URLs and their domain prior to fetching +- Added MRF.QuietReply which prevents replies to public posts from being published to the timelines +- Add `group_key` to notifications +- Allow providing avatar/header descriptions +- Added RemoteReportPolicy from Rebased for handling bogus federated reports +- scrubbers/default: Allow "mention hashtag" classes used by Mastodon +- Added dependencies for Swoosh's Mua mail adapter +- Include session scopes in TokenView + +### Fixed +- Verify a local Update sent through AP C2S so users can only update their own objects +- Fixed malformed follow requests that cause them to appear stuck pending due to the recipient being unable to process them. +- Fix incoming Block activities being rejected +- STARTTLS certificate and hostname verification for LDAP authentication +- LDAPS connections (implicit TLS) are now supported. +- Fix /api/v2/media returning the wrong status code (202) for media processed synchronously +- Miscellaneous fixes for Meilisearch support +- Fix pleroma_ctl mix task calls sometimes not being found +- Add a rate limiter to the OAuth App creation endpoint and ensure registered apps are assigned to users. +- ReceiverWorker will cancel processing jobs instead of retrying if the user cannot be fetched due to 403, 404, or 410 errors or if the account is disabled locally. +- Address case where instance reachability status couldn't be updated +- Remote Fetcher Worker recognizes more permanent failure errors +- StreamerView: Do not leak follows count if hidden +- Imports of blocks, mutes, and follows would retry repeatedly due to incorrect error handling and all work executed in a single job +- Make vapid_config return empty array, fixing preloading for instances without push notifications configured + +### Removed +- Remove stub for /api/v1/accounts/:id/identity_proofs (deprecated by Mastodon 3.5.0) + ## 2.7.1 ### Changeddiff --git a/changelog.d/activity-pub-metadata.add b/changelog.d/activity-pub-metadata.add@@ -1 +0,0 @@ -Add metadata provider for ActivityPub alternate linksdiff --git a/changelog.d/argon2-passwords.add b/changelog.d/argon2-passwords.add@@ -1 +0,0 @@ -Added support for argon2 passwords and their conversion for migration from Akkoma fork to upstream.diff --git a/changelog.d/atom-tag.change b/changelog.d/atom-tag.change@@ -1 +0,0 @@ -Metadata: Do not include .atom feed links for remote accountsdiff --git a/changelog.d/bump-lexbor.change b/changelog.d/bump-lexbor.change@@ -1 +0,0 @@ -- Bumped `fast_html` to v2.3.0, which notably allows to use system-installed lexbor with passing `WITH_SYSTEM_LEXBOR=1` environment variable at build-time -\ No newline at end of filediff --git a/changelog.d/ci-git-fetch.skip b/changelog.d/ci-git-fetch.skipdiff --git a/changelog.d/commonapi.skip b/changelog.d/commonapi.skipdiff --git a/changelog.d/debian-install-improve.skip b/changelog.d/debian-install-improve.skip@@ -1 +0,0 @@ -Fixed a formatting issue that had a required commend embedded in a textblock, and change the language to make it a bit more idiomatic. -\ No newline at end of filediff --git a/changelog.d/dedupe-sharding.change b/changelog.d/dedupe-sharding.change@@ -1 +0,0 @@ -Dedupe upload filter now uses a three-level sharding directory structurediff --git a/changelog.d/deprecate-subscribe.change b/changelog.d/deprecate-subscribe.change@@ -1 +0,0 @@ -Deprecate `/api/v1/pleroma/accounts/:id/subscribe`/`unsubscribe` -\ No newline at end of filediff --git a/changelog.d/dialyzer.skip b/changelog.d/dialyzer.skipdiff --git a/changelog.d/docs-fix.skip b/changelog.d/docs-fix.skipdiff --git a/changelog.d/docs-vips.skip b/changelog.d/docs-vips.skipdiff --git a/changelog.d/drop-unwanted.change b/changelog.d/drop-unwanted.change@@ -1 +0,0 @@ -Restrict incoming activities from unknown actors to a subset that does not imply a previous relationship and early rejection of unrecognized activity types.diff --git a/changelog.d/elixir-1.14-docker.skip b/changelog.d/elixir-1.14-docker.skipdiff --git a/changelog.d/elixir.change b/changelog.d/elixir.change@@ -1 +0,0 @@ -Elixir 1.14 and Erlang/OTP 23 is now the minimum supported releasediff --git a/changelog.d/follow-request.fix b/changelog.d/follow-request.fix@@ -1 +0,0 @@ -Fixed malformed follow requests that cause them to appear stuck pending due to the recipient being unable to process them.diff --git a/changelog.d/freebsd-docs.skip b/changelog.d/freebsd-docs.skipdiff --git a/changelog.d/get-statuses-param.change b/changelog.d/get-statuses-param.change@@ -1 +0,0 @@ -Support `id` param in `GET /api/v1/statuses` -\ No newline at end of filediff --git a/changelog.d/hashtag-feeds-restricted.add b/changelog.d/hashtag-feeds-restricted.add@@ -1 +0,0 @@ -Repesct :restrict_unauthenticated for hashtag rss/atom feeds -\ No newline at end of filediff --git a/changelog.d/identity-proofs.remove b/changelog.d/identity-proofs.remove@@ -1 +0,0 @@ -Remove stub for /api/v1/accounts/:id/identity_proofs (deprecated by Mastodon 3.5.0) -\ No newline at end of filediff --git a/changelog.d/incoming-blocks.fix b/changelog.d/incoming-blocks.fix@@ -1 +0,0 @@ -Fix incoming Block activities being rejecteddiff --git a/changelog.d/ldap-ca.add b/changelog.d/ldap-ca.add@@ -1 +0,0 @@ -LDAP configuration now permits overriding the CA root certificate file for TLS validation.diff --git a/changelog.d/ldap-password-change.add b/changelog.d/ldap-password-change.add@@ -1 +0,0 @@ -LDAP now supports users changing their passwordsdiff --git a/changelog.d/ldap-refactor.change b/changelog.d/ldap-refactor.change@@ -1 +0,0 @@ -LDAP authentication has been refactored to operate as a GenServer process which will maintain an active connection to the LDAP server.diff --git a/changelog.d/ldap-tls.fix b/changelog.d/ldap-tls.fix@@ -1 +0,0 @@ -STARTTLS certificate and hostname verification for LDAP authenticationdiff --git a/changelog.d/ldap-warning.skip b/changelog.d/ldap-warning.skipdiff --git a/changelog.d/ldaps.fix b/changelog.d/ldaps.fix@@ -1 +0,0 @@ -LDAPS connections (implicit TLS) are now supported.diff --git a/changelog.d/list-id-visibility.add b/changelog.d/list-id-visibility.add@@ -1 +0,0 @@ -Include list id in StatusView -\ No newline at end of filediff --git a/changelog.d/manifest-icon-size.skip b/changelog.d/manifest-icon-size.skipdiff --git a/changelog.d/mediav2_status.fix b/changelog.d/mediav2_status.fix@@ -1 +0,0 @@ -Fix /api/v2/media returning the wrong status code (202) for media processed synchronouslydiff --git a/changelog.d/meilisearch-misc-fixes.fix b/changelog.d/meilisearch-misc-fixes.fix@@ -1 +0,0 @@ -Miscellaneous fixes for Meilisearch supportdiff --git a/changelog.d/module-search-in-pleroma-ctl.fix b/changelog.d/module-search-in-pleroma-ctl.fix@@ -1 +0,0 @@ -Fix pleroma_ctl mix task calls sometimes not being founddiff --git a/changelog.d/mogrify.skip b/changelog.d/mogrify.skipdiff --git a/changelog.d/mrf-cleanup.skip b/changelog.d/mrf-cleanup.skipdiff --git a/changelog.d/mrf-fodirectreply.add b/changelog.d/mrf-fodirectreply.add@@ -1 +0,0 @@ -Added MRF.FODirectReply which changes replies to followers-only posts to be direct.diff --git a/changelog.d/mrf-id_filter.add b/changelog.d/mrf-id_filter.add@@ -1 +0,0 @@ -Add `id_filter` to MRF to filter URLs and their domain prior to fetching -\ No newline at end of filediff --git a/changelog.d/mrf-quietreply.add b/changelog.d/mrf-quietreply.add@@ -1 +0,0 @@ -Added MRF.QuietReply which prevents replies to public posts from being published to the timelinesdiff --git a/changelog.d/notifications-group-key.add b/changelog.d/notifications-group-key.add@@ -1 +0,0 @@ -Add `group_key` to notifications -\ No newline at end of filediff --git a/changelog.d/notifications-marker.change b/changelog.d/notifications-marker.change@@ -1 +0,0 @@ -Fix 'Setting a marker should mark notifications as read' -\ No newline at end of filediff --git a/changelog.d/oauth-app-spam.fix b/changelog.d/oauth-app-spam.fix@@ -1 +0,0 @@ -Add a rate limiter to the OAuth App creation endpoint and ensure registered apps are assigned to users.diff --git a/changelog.d/oban-recevier-improvements.fix b/changelog.d/oban-recevier-improvements.fix@@ -1 +0,0 @@ -ReceiverWorker will cancel processing jobs instead of retrying if the user cannot be fetched due to 403, 404, or 410 errors or if the account is disabled locally.diff --git a/changelog.d/oban-uniques.change b/changelog.d/oban-uniques.change@@ -1 +0,0 @@ -Adjust more Oban workers to enforce unique job constraints.diff --git a/changelog.d/oban-update.change b/changelog.d/oban-update.change@@ -1 +0,0 @@ -Oban updated to 2.18.3diff --git a/changelog.d/oban_gun_snooze.change b/changelog.d/oban_gun_snooze.change@@ -1 +0,0 @@ -Publisher behavior improvement when snoozing Oban jobs due to Gun connection pool contention.diff --git a/changelog.d/poll-refresh.change b/changelog.d/poll-refresh.change@@ -1 +0,0 @@ -Poll results refreshing is handled asynchronously and will not attempt to keep fetching updates to a closed poll.diff --git a/changelog.d/profile-image-descriptions.add b/changelog.d/profile-image-descriptions.add@@ -1 +0,0 @@ -Allow providing avatar/header descriptions -\ No newline at end of filediff --git a/changelog.d/profile-image-descriptions.skip b/changelog.d/profile-image-descriptions.skipdiff --git a/changelog.d/publisher-reachability.fix b/changelog.d/publisher-reachability.fix@@ -1 +0,0 @@ -Address case where instance reachability status couldn't be updateddiff --git a/changelog.d/release-tuning.change b/changelog.d/release-tuning.change@@ -1 +0,0 @@ -Tuning for release builds to lower CPU usage.diff --git a/changelog.d/remote-object-fetcher.fix b/changelog.d/remote-object-fetcher.fix@@ -1 +0,0 @@ -Remote Fetcher Worker recognizes more permanent failure errorsdiff --git a/changelog.d/remote-report-policy.add b/changelog.d/remote-report-policy.add@@ -1 +0,0 @@ -Added RemoteReportPolicy from Rebased for handling bogus federated reportsdiff --git a/changelog.d/rich-media-no-heads.change b/changelog.d/rich-media-no-heads.change@@ -1 +0,0 @@ -Rich Media preview fetching will skip making an HTTP HEAD request to check a URL for allowed content type and length if the Tesla adapter is Gun or Finchdiff --git a/changelog.d/scrubbers-allow-mention-hashtag.add b/changelog.d/scrubbers-allow-mention-hashtag.add@@ -1 +0,0 @@ -scrubbers/default: Allow "mention hashtag" classes used by Mastodon -\ No newline at end of filediff --git a/changelog.d/se-opt-out.change b/changelog.d/se-opt-out.change@@ -1 +0,0 @@ -Fix nonexisting user will not generate metadata for search engine opt-outdiff --git a/changelog.d/stream-follow-relationships-count.fix b/changelog.d/stream-follow-relationships-count.fix@@ -1 +0,0 @@ -StreamerView: Do not leak follows count if hidden -\ No newline at end of filediff --git a/changelog.d/swoosh-mua.add b/changelog.d/swoosh-mua.add@@ -1 +0,0 @@ -Added dependencies for Swoosh's Mua mail adapterdiff --git a/changelog.d/text-extensions.skip b/changelog.d/text-extensions.skipdiff --git a/changelog.d/todo-cleanup.skip b/changelog.d/todo-cleanup.skipdiff --git a/changelog.d/token-view-scopes.add b/changelog.d/token-view-scopes.add@@ -1 +0,0 @@ -Include session scopes in TokenView -\ No newline at end of filediff --git a/changelog.d/update-oban.change b/changelog.d/update-oban.change@@ -1 +0,0 @@ -Update Oban to 2.18diff --git a/changelog.d/user-factory.skip b/changelog.d/user-factory.skipdiff --git a/changelog.d/user-imports.fix b/changelog.d/user-imports.fix@@ -1 +0,0 @@ -Imports of blocks, mutes, and follows would retry repeatedly due to incorrect error handling and all work executed in a single jobdiff --git a/changelog.d/vapid_keyword_fallback.fix b/changelog.d/vapid_keyword_fallback.fix@@ -1 +0,0 @@ -Make vapid_config return empty array, fixing preloading for instances without push notifications configured -\ No newline at end of filediff --git a/changelog.d/workerhelper.change b/changelog.d/workerhelper.change@@ -1 +0,0 @@ -Worker configuration is no longer available. This only affects custom max_retries values for a couple Oban queues.diff --git a/lib/pleroma/web/activity_pub/activity_pub_controller.ex b/lib/pleroma/web/activity_pub/activity_pub_controller.ex@@ -482,7 +482,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do |> put_status(:forbidden) |> json(message) - {:error, message} -> + {:error, message} when is_binary(message) -> conn |> put_status(:bad_request) |> json(message)diff --git a/lib/pleroma/web/activity_pub/object_validator.ex b/lib/pleroma/web/activity_pub/object_validator.ex@@ -169,7 +169,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do meta = Keyword.put(meta, :object_data, object_data), {:ok, update_activity} <- update_activity - |> UpdateValidator.cast_and_validate() + |> UpdateValidator.cast_and_validate(meta) |> Ecto.Changeset.apply_action(:insert) do update_activity = stringify_keys(update_activity) {:ok, update_activity, meta} @@ -177,7 +177,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do {:local, _} -> with {:ok, object} <- update_activity - |> UpdateValidator.cast_and_validate() + |> UpdateValidator.cast_and_validate(meta) |> Ecto.Changeset.apply_action(:insert) do object = stringify_keys(object) {:ok, object, meta} @@ -207,9 +207,16 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do "Answer" -> AnswerValidator end + cast_func = + if type == "Update" do + fn o -> validator.cast_and_validate(o, meta) end + else + fn o -> validator.cast_and_validate(o) end + end + with {:ok, object} <- object - |> validator.cast_and_validate() + |> cast_func.() |> Ecto.Changeset.apply_action(:insert) do object = stringify_keys(object) {:ok, object, meta}diff --git a/lib/pleroma/web/activity_pub/object_validators/update_validator.ex b/lib/pleroma/web/activity_pub/object_validators/update_validator.ex@@ -6,6 +6,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.UpdateValidator do use Ecto.Schema alias Pleroma.EctoType.ActivityPub.ObjectValidators + alias Pleroma.Object + alias Pleroma.User import Ecto.Changeset import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations @@ -31,23 +33,50 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.UpdateValidator do |> cast(data, __schema__(:fields)) end - defp validate_data(cng) do + defp validate_data(cng, meta) do cng |> validate_required([:id, :type, :actor, :to, :cc, :object]) |> validate_inclusion(:type, ["Update"]) |> validate_actor_presence() - |> validate_updating_rights() + |> validate_updating_rights(meta) end - def cast_and_validate(data) do + def cast_and_validate(data, meta \\ []) do data |> cast_data - |> validate_data + |> validate_data(meta) end - # For now we only support updating users, and here the rule is easy: - # object id == actor id - def validate_updating_rights(cng) do + def validate_updating_rights(cng, meta) do + if meta[:local] do + validate_updating_rights_local(cng) + else + validate_updating_rights_remote(cng) + end + end + + # For local Updates, verify the actor can edit the object + def validate_updating_rights_local(cng) do + actor = get_field(cng, :actor) + updated_object = get_field(cng, :object) + + if {:ok, actor} == ObjectValidators.ObjectID.cast(updated_object) do + cng + else + with %User{} = user <- User.get_cached_by_ap_id(actor), + {_, %Object{} = orig_object} <- {:object, Object.normalize(updated_object)}, + :ok <- Object.authorize_access(orig_object, user) do + cng + else + _e -> + cng + |> add_error(:object, "Can't be updated by this actor") + end + end + end + + # For remote Updates, verify the host is the same. + def validate_updating_rights_remote(cng) do with actor = get_field(cng, :actor), object = get_field(cng, :object), {:ok, object_id} <- ObjectValidators.ObjectID.cast(object),diff --git a/mix.exs b/mix.exs@@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do def project do [ app: :pleroma, - version: version("2.7.51"), + version: version("2.8.0"), elixir: "~> 1.14", elixirc_paths: elixirc_paths(Mix.env()), compilers: Mix.compilers(),diff --git a/test/pleroma/web/activity_pub/activity_pub_controller_test.exs b/test/pleroma/web/activity_pub/activity_pub_controller_test.exs@@ -1644,6 +1644,28 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do assert json_response(conn, 403) end + test "it rejects update activity of object from other actor", %{conn: conn} do + note_activity = insert(:note_activity) + note_object = Object.normalize(note_activity, fetch: false) + user = insert(:user) + + data = %{ + type: "Update", + object: %{ + id: note_object.data["id"] + } + } + + conn = + conn + |> assign(:user, user) + |> put_req_header("content-type", "application/activity+json") + |> post("/users/#{user.nickname}/outbox", data) + + assert json_response(conn, 400) + assert note_object == Object.normalize(note_activity, fetch: false) + end + test "it increases like count when receiving a like action", %{conn: conn} do note_activity = insert(:note_activity) note_object = Object.normalize(note_activity, fetch: false)