logo

pleroma

My custom branche(s) on git.pleroma.social/pleroma/pleroma git clone https://hacktivis.me/git/pleroma.git
commit: 6384d7803520f633a66d1cb9f76e3540863f92fb
parent c45b3bde949bbbd6b8d6de30022c1d1e5b99dc44
Author: Haelwenn <contact+git.pleroma.social@hacktivis.me>
Date:   Mon,  9 Aug 2021 09:37:59 +0000

Merge branch 'simple_policy_reasons_for_instance_specific_policies' into 'develop'

Simple policy reasons for instance specific policies

See merge request pleroma/pleroma!3314

Diffstat:

MCHANGELOG.md1+
Mconfig/description.exs10++++++----
Mdocs/configuration/cheatsheet.md21+++++++++++----------
Mdocs/configuration/mrf.md12++++++------
Mlib/pleroma/config/deprecation_warnings.ex166++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
Mlib/pleroma/web/activity_pub/mrf.ex11+++++++++--
Mlib/pleroma/web/activity_pub/mrf/keyword_policy.ex2++
Mlib/pleroma/web/activity_pub/mrf/reject_non_public.ex2+-
Mlib/pleroma/web/activity_pub/mrf/simple_policy.ex188++++++++++++++++++++++++++++++++++++++++++++++---------------------------------
Mlib/pleroma/web/activity_pub/mrf/user_allow_list_policy.ex2+-
Mlib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex2+-
Mlib/pleroma/web/activity_pub/publisher.ex1+
Mlib/pleroma/web/mastodon_api/views/instance_view.ex15++++++++++++++-
Apriv/repo/migrations/20201005123100_simple_policy_string_to_tuple.exs40++++++++++++++++++++++++++++++++++++++++
Apriv/repo/migrations/20201005124600_quarantained_policy_string_to_tuple.exs61+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Apriv/repo/migrations/20201005132900_transparency_exclusions_string_to_tuple.exs61+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mtest/pleroma/config/deprecation_warnings_test.exs177+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mtest/pleroma/user_test.exs2+-
Mtest/pleroma/web/activity_pub/mrf/simple_policy_test.exs69+++++++++++++++++++++++++++++++++++++--------------------------------
Mtest/pleroma/web/activity_pub/mrf_test.exs9+++++++++
Mtest/pleroma/web/activity_pub/publisher_test.exs74++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mtest/pleroma/web/node_info_test.exs136+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------
Mtest/support/factory.ex32++++++++++++++++++++++++++++++++
23 files changed, 919 insertions(+), 175 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md @@ -19,6 +19,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ### Changed - **Breaking:** Configuration: `:chat, enabled` moved to `:shout, enabled` and `:instance, chat_limit` moved to `:shout, limit` +- **Breaking** Entries for simple_policy, transparency_exclusions and quarantined_instances now list both the instance and a reason. - Support for Erlang/OTP 24 - The `application` metadata returned with statuses is no longer hardcoded. Apps that want to display these details will now have valid data for new posts after this change. - HTTPSecurityPlug now sends a response header to opt out of Google's FLoC (Federated Learning of Cohorts) targeted advertising. diff --git a/config/description.exs b/config/description.exs @@ -687,12 +687,14 @@ config :pleroma, :config_description, [ }, %{ key: :quarantined_instances, - type: {:list, :string}, + type: {:list, :tuple}, + key_placeholder: "instance", + value_placeholder: "reason", description: - "List of ActivityPub instances where private (DMs, followers-only) activities will not be sent", + "List of ActivityPub instances where private (DMs, followers-only) activities will not be sent and the reason for doing so", suggestions: [ - "quarantined.com", - "*.quarantined.com" + {"quarantined.com", "Reason"}, + {"*.quarantined.com", "Reason"} ] }, %{ diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md @@ -39,7 +39,7 @@ To add configuration to your config file, you can copy it from the base config. * `federation_reachability_timeout_days`: Timeout (in days) of each external federation target being unreachable prior to pausing federating to it. * `allow_relay`: Permits remote instances to subscribe to all public posts of your instance. This may increase the visibility of your instance. * `public`: Makes the client API in authenticated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. Note that there is a dependent setting restricting or allowing unauthenticated access to specific resources, see `restrict_unauthenticated` for more details. -* `quarantined_instances`: List of ActivityPub instances where private (DMs, followers-only) activities will not be send. +* `quarantined_instances`: ActivityPub instances where private (DMs, followers-only) activities will not be send. * `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML). * `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes). This will break federation with older software for theses nicknames. @@ -135,15 +135,16 @@ To add configuration to your config file, you can copy it from the base config. Configuring MRF policies is not enough for them to take effect. You have to enable them by specifying their module in `policies` under [:mrf](#mrf) section. #### :mrf_simple -* `media_removal`: List of instances to remove media from. -* `media_nsfw`: List of instances to put media as NSFW(sensitive) from. -* `federated_timeline_removal`: List of instances to remove from Federated (aka The Whole Known Network) Timeline. -* `reject`: List of instances to reject any activities from. -* `accept`: List of instances to accept any activities from. -* `followers_only`: List of instances to decrease post visibility to only the followers, including for DM mentions. -* `report_removal`: List of instances to reject reports from. -* `avatar_removal`: List of instances to strip avatars from. -* `banner_removal`: List of instances to strip banners from. +* `media_removal`: List of instances to strip media attachments from and the reason for doing so. +* `media_nsfw`: List of instances to tag all media as NSFW (sensitive) from and the reason for doing so. +* `federated_timeline_removal`: List of instances to remove from the Federated Timeline (aka The Whole Known Network) and the reason for doing so. +* `reject`: List of instances to reject activities (except deletes) from and the reason for doing so. +* `accept`: List of instances to only accept activities (except deletes) from and the reason for doing so. +* `followers_only`: Force posts from the given instances to be visible by followers only and the reason for doing so. +* `report_removal`: List of instances to reject reports from and the reason for doing so. +* `avatar_removal`: List of instances to strip avatars from and the reason for doing so. +* `banner_removal`: List of instances to strip banners from and the reason for doing so. +* `reject_deletes`: List of instances to reject deletions from and the reason for doing so. #### :mrf_subchain This policy processes messages through an alternate pipeline when a given message matches certain criteria. diff --git a/docs/configuration/mrf.md b/docs/configuration/mrf.md @@ -55,18 +55,18 @@ Servers should be configured as lists. ### Example -This example will enable `SimplePolicy`, block media from `illegalporn.biz`, mark media as NSFW from `porn.biz` and `porn.business`, reject messages from `spam.com`, remove messages from `spam.university` from the federated timeline and block reports (flags) from `whiny.whiner`: +This example will enable `SimplePolicy`, block media from `illegalporn.biz`, mark media as NSFW from `porn.biz` and `porn.business`, reject messages from `spam.com`, remove messages from `spam.university` from the federated timeline and block reports (flags) from `whiny.whiner`. We also give a reason why the moderation was done: ```elixir config :pleroma, :mrf, policies: [Pleroma.Web.ActivityPub.MRF.SimplePolicy] config :pleroma, :mrf_simple, - media_removal: ["illegalporn.biz"], - media_nsfw: ["porn.biz", "porn.business"], - reject: ["spam.com"], - federated_timeline_removal: ["spam.university"], - report_removal: ["whiny.whiner"] + media_removal: [{"illegalporn.biz", "Media can contain illegal contant"}], + media_nsfw: [{"porn.biz", "unmarked nsfw media"}, {"porn.business", "A lot of unmarked nsfw media"}], + reject: [{"spam.com", "They keep spamming our users"}], + federated_timeline_removal: [{"spam.university", "Annoying low-quality posts who otherwise fill up TWKN"}], + report_removal: [{"whiny.whiner", "Keep spamming us with irrelevant reports"}] ``` ### Use with Care diff --git a/lib/pleroma/config/deprecation_warnings.ex b/lib/pleroma/config/deprecation_warnings.ex @@ -20,6 +20,140 @@ defmodule Pleroma.Config.DeprecationWarnings do "\n* `config :pleroma, :instance, mrf_transparency_exclusions` is now `config :pleroma, :mrf, transparency_exclusions`"} ] + def check_simple_policy_tuples do + has_strings = + Config.get([:mrf_simple]) + |> Enum.any?(fn {_, v} -> Enum.any?(v, &is_binary/1) end) + + if has_strings do + Logger.warn(""" + !!!DEPRECATION WARNING!!! + Your config is using strings in the SimplePolicy configuration instead of tuples. They should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, :mrf_simple, + media_removal: ["instance.tld"], + media_nsfw: ["instance.tld"], + federated_timeline_removal: ["instance.tld"], + report_removal: ["instance.tld"], + reject: ["instance.tld"], + followers_only: ["instance.tld"], + accept: ["instance.tld"], + avatar_removal: ["instance.tld"], + banner_removal: ["instance.tld"], + reject_deletes: ["instance.tld"] + ``` + + Is now + + + ``` + config :pleroma, :mrf_simple, + media_removal: [{"instance.tld", "Reason for media removal"}], + media_nsfw: [{"instance.tld", "Reason for media nsfw"}], + federated_timeline_removal: [{"instance.tld", "Reason for federated timeline removal"}], + report_removal: [{"instance.tld", "Reason for report removal"}], + reject: [{"instance.tld", "Reason for reject"}], + followers_only: [{"instance.tld", "Reason for followers only"}], + accept: [{"instance.tld", "Reason for accept"}], + avatar_removal: [{"instance.tld", "Reason for avatar removal"}], + banner_removal: [{"instance.tld", "Reason for banner removal"}], + reject_deletes: [{"instance.tld", "Reason for reject deletes"}] + ``` + """) + + new_config = + Config.get([:mrf_simple]) + |> Enum.map(fn {k, v} -> + {k, + Enum.map(v, fn + {instance, reason} -> {instance, reason} + instance -> {instance, ""} + end)} + end) + + Config.put([:mrf_simple], new_config) + + :error + else + :ok + end + end + + def check_quarantined_instances_tuples do + has_strings = Config.get([:instance, :quarantined_instances]) |> Enum.any?(&is_binary/1) + + if has_strings do + Logger.warn(""" + !!!DEPRECATION WARNING!!! + Your config is using strings in the quarantined_instances configuration instead of tuples. They should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, :instance, + quarantined_instances: ["instance.tld"] + ``` + + Is now + + + ``` + config :pleroma, :instance, + quarantined_instances: [{"instance.tld", "Reason for quarantine"}] + ``` + """) + + new_config = + Config.get([:instance, :quarantined_instances]) + |> Enum.map(fn + {instance, reason} -> {instance, reason} + instance -> {instance, ""} + end) + + Config.put([:instance, :quarantined_instances], new_config) + + :error + else + :ok + end + end + + def check_transparency_exclusions_tuples do + has_strings = Config.get([:mrf, :transparency_exclusions]) |> Enum.any?(&is_binary/1) + + if has_strings do + Logger.warn(""" + !!!DEPRECATION WARNING!!! + Your config is using strings in the transparency_exclusions configuration instead of tuples. They should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, :mrf, + transparency_exclusions: ["instance.tld"] + ``` + + Is now + + + ``` + config :pleroma, :mrf, + transparency_exclusions: [{"instance.tld", "Reason to exlude transparency"}] + ``` + """) + + new_config = + Config.get([:mrf, :transparency_exclusions]) + |> Enum.map(fn + {instance, reason} -> {instance, reason} + instance -> {instance, ""} + end) + + Config.put([:mrf, :transparency_exclusions], new_config) + + :error + else + :ok + end + end + def check_hellthread_threshold do if Config.get([:mrf_hellthread, :threshold]) do Logger.warn(""" @@ -34,20 +168,24 @@ defmodule Pleroma.Config.DeprecationWarnings do end def warn do - with :ok <- check_hellthread_threshold(), - :ok <- check_old_mrf_config(), - :ok <- check_media_proxy_whitelist_config(), - :ok <- check_welcome_message_config(), - :ok <- check_gun_pool_options(), - :ok <- check_activity_expiration_config(), - :ok <- check_remote_ip_plug_name(), - :ok <- check_uploders_s3_public_endpoint(), - :ok <- check_old_chat_shoutbox() do - :ok - else - _ -> - :error - end + [ + check_hellthread_threshold(), + check_old_mrf_config(), + check_media_proxy_whitelist_config(), + check_welcome_message_config(), + check_gun_pool_options(), + check_activity_expiration_config(), + check_remote_ip_plug_name(), + check_uploders_s3_public_endpoint(), + check_old_chat_shoutbox(), + check_quarantined_instances_tuples(), + check_transparency_exclusions_tuples(), + check_simple_policy_tuples() + ] + |> Enum.reduce(:ok, fn + :ok, :ok -> :ok + _, _ -> :error + end) end def check_welcome_message_config do diff --git a/lib/pleroma/web/activity_pub/mrf.ex b/lib/pleroma/web/activity_pub/mrf.ex @@ -33,9 +33,11 @@ defmodule Pleroma.Web.ActivityPub.MRF do %{ key: :transparency_exclusions, label: "MRF transparency exclusions", - type: {:list, :string}, + type: {:list, :tuple}, + key_placeholder: "instance", + value_placeholder: "reason", description: - "Exclude specific instance names from MRF transparency. The use of the exclusions feature will be disclosed in nodeinfo as a boolean value.", + "Exclude specific instance names from MRF transparency. The use of the exclusions feature will be disclosed in nodeinfo as a boolean value. You can also provide a reason for excluding these instance names. The instances and reasons won't be publicly disclosed.", suggestions: [ "exclusion.com" ] @@ -100,6 +102,11 @@ defmodule Pleroma.Web.ActivityPub.MRF do Enum.any?(domains, fn domain -> Regex.match?(domain, host) end) end + @spec instance_list_from_tuples([{String.t(), String.t()}]) :: [String.t()] + def instance_list_from_tuples(list) do + Enum.map(list, fn {instance, _} -> instance end) + end + def describe(policies) do {:ok, policy_configs} = policies diff --git a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex @@ -159,6 +159,8 @@ defmodule Pleroma.Web.ActivityPub.MRF.KeywordPolicy do %{ key: :replace, type: {:list, :tuple}, + key_placeholder: "instance", + value_placeholder: "reason", description: """ **Pattern**: a string or [Regex](https://hexdocs.pm/elixir/Regex.html) in the format of `~r/PATTERN/`. diff --git a/lib/pleroma/web/activity_pub/mrf/reject_non_public.ex b/lib/pleroma/web/activity_pub/mrf/reject_non_public.ex @@ -47,7 +47,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.RejectNonPublic do @impl true def describe, - do: {:ok, %{mrf_rejectnonpublic: Config.get(:mrf_rejectnonpublic) |> Enum.into(%{})}} + do: {:ok, %{mrf_rejectnonpublic: Config.get(:mrf_rejectnonpublic) |> Map.new()}} @impl true def config_description do diff --git a/lib/pleroma/web/activity_pub/mrf/simple_policy.ex b/lib/pleroma/web/activity_pub/mrf/simple_policy.ex @@ -15,7 +15,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_accept(%{host: actor_host} = _actor_info, object) do accepts = - Config.get([:mrf_simple, :accept]) + instance_list(:accept) |> MRF.subdomains_regex() cond do @@ -28,7 +28,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_reject(%{host: actor_host} = _actor_info, object) do rejects = - Config.get([:mrf_simple, :reject]) + instance_list(:reject) |> MRF.subdomains_regex() if MRF.subdomain_match?(rejects, actor_host) do @@ -44,7 +44,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do ) when length(child_attachment) > 0 do media_removal = - Config.get([:mrf_simple, :media_removal]) + instance_list(:media_removal) |> MRF.subdomains_regex() object = @@ -68,7 +68,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do } = object ) do media_nsfw = - Config.get([:mrf_simple, :media_nsfw]) + instance_list(:media_nsfw) |> MRF.subdomains_regex() object = @@ -85,7 +85,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_ftl_removal(%{host: actor_host} = _actor_info, object) do timeline_removal = - Config.get([:mrf_simple, :federated_timeline_removal]) + instance_list(:federated_timeline_removal) |> MRF.subdomains_regex() object = @@ -112,7 +112,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_followers_only(%{host: actor_host} = _actor_info, object) do followers_only = - Config.get([:mrf_simple, :followers_only]) + instance_list(:followers_only) |> MRF.subdomains_regex() object = @@ -137,7 +137,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_report_removal(%{host: actor_host} = _actor_info, %{"type" => "Flag"} = object) do report_removal = - Config.get([:mrf_simple, :report_removal]) + instance_list(:report_removal) |> MRF.subdomains_regex() if MRF.subdomain_match?(report_removal, actor_host) do @@ -151,7 +151,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_avatar_removal(%{host: actor_host} = _actor_info, %{"icon" => _icon} = object) do avatar_removal = - Config.get([:mrf_simple, :avatar_removal]) + instance_list(:avatar_removal) |> MRF.subdomains_regex() if MRF.subdomain_match?(avatar_removal, actor_host) do @@ -165,7 +165,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_banner_removal(%{host: actor_host} = _actor_info, %{"image" => _image} = object) do banner_removal = - Config.get([:mrf_simple, :banner_removal]) + instance_list(:banner_removal) |> MRF.subdomains_regex() if MRF.subdomain_match?(banner_removal, actor_host) do @@ -185,12 +185,17 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do defp check_object(object), do: {:ok, object} + defp instance_list(config_key) do + Config.get([:mrf_simple, config_key]) + |> MRF.instance_list_from_tuples() + end + @impl true def filter(%{"type" => "Delete", "actor" => actor} = object) do %{host: actor_host} = URI.parse(actor) reject_deletes = - Config.get([:mrf_simple, :reject_deletes]) + instance_list(:reject_deletes) |> MRF.subdomains_regex() if MRF.subdomain_match?(reject_deletes, actor_host) do @@ -253,14 +258,42 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do @impl true def describe do - exclusions = Config.get([:mrf, :transparency_exclusions]) + exclusions = Config.get([:mrf, :transparency_exclusions]) |> MRF.instance_list_from_tuples() - mrf_simple = + mrf_simple_excluded = Config.get(:mrf_simple) - |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end) - |> Enum.into(%{}) + |> Enum.map(fn {rule, instances} -> + {rule, Enum.reject(instances, fn {host, _} -> host in exclusions end)} + end) - {:ok, %{mrf_simple: mrf_simple}} + mrf_simple = + mrf_simple_excluded + |> Enum.map(fn {rule, instances} -> + {rule, Enum.map(instances, fn {host, _} -> host end)} + end) + |> Map.new() + + # This is for backwards compatibility. We originally didn't sent + # extra info like a reason why an instance was rejected/quarantined/etc. + # Because we didn't want to break backwards compatibility it was decided + # to add an extra "info" key. + mrf_simple_info = + mrf_simple_excluded + |> Enum.map(fn {rule, instances} -> + {rule, Enum.reject(instances, fn {_, reason} -> reason == "" end)} + end) + |> Enum.reject(fn {_, instances} -> instances == [] end) + |> Enum.map(fn {rule, instances} -> + instances = + instances + |> Enum.map(fn {host, reason} -> {host, %{"reason" => reason}} end) + |> Map.new() + + {rule, instances} + end) + |> Map.new() + + {:ok, %{mrf_simple: mrf_simple, mrf_simple_info: mrf_simple_info}} end @impl true @@ -270,70 +303,67 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do related_policy: "Pleroma.Web.ActivityPub.MRF.SimplePolicy", label: "MRF Simple", description: "Simple ingress policies", - children: [ - %{ - key: :media_removal, - type: {:list, :string}, - description: "List of instances to strip media attachments from", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :media_nsfw, - label: "Media NSFW", - type: {:list, :string}, - description: "List of instances to tag all media as NSFW (sensitive) from", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :federated_timeline_removal, - type: {:list, :string}, - description: - "List of instances to remove from the Federated (aka The Whole Known Network) Timeline", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :reject, - type: {:list, :string}, - description: "List of instances to reject activities from (except deletes)", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :accept, - type: {:list, :string}, - description: "List of instances to only accept activities from (except deletes)", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :followers_only, - type: {:list, :string}, - description: "Force posts from the given instances to be visible by followers only", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :report_removal, - type: {:list, :string}, - description: "List of instances to reject reports from", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :avatar_removal, - type: {:list, :string}, - description: "List of instances to strip avatars from", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :banner_removal, - type: {:list, :string}, - description: "List of instances to strip banners from", - suggestions: ["example.com", "*.example.com"] - }, - %{ - key: :reject_deletes, - type: {:list, :string}, - description: "List of instances to reject deletions from", - suggestions: ["example.com", "*.example.com"] - } - ] + children: + [ + %{ + key: :media_removal, + description: + "List of instances to strip media attachments from and the reason for doing so" + }, + %{ + key: :media_nsfw, + label: "Media NSFW", + description: + "List of instances to tag all media as NSFW (sensitive) from and the reason for doing so" + }, + %{ + key: :federated_timeline_removal, + description: + "List of instances to remove from the Federated (aka The Whole Known Network) Timeline and the reason for doing so" + }, + %{ + key: :reject, + description: + "List of instances to reject activities from (except deletes) and the reason for doing so" + }, + %{ + key: :accept, + description: + "List of instances to only accept activities from (except deletes) and the reason for doing so" + }, + %{ + key: :followers_only, + description: + "Force posts from the given instances to be visible by followers only and the reason for doing so" + }, + %{ + key: :report_removal, + description: "List of instances to reject reports from and the reason for doing so" + }, + %{ + key: :avatar_removal, + description: "List of instances to strip avatars from and the reason for doing so" + }, + %{ + key: :banner_removal, + description: "List of instances to strip banners from and the reason for doing so" + }, + %{ + key: :reject_deletes, + description: "List of instances to reject deletions from and the reason for doing so" + } + ] + |> Enum.map(fn setting -> + Map.merge( + setting, + %{ + type: {:list, :tuple}, + key_placeholder: "instance", + value_placeholder: "reason", + suggestions: [{"example.com", "Some reason"}, {"*.example.com", "Another reason"}] + } + ) + end) } end end diff --git a/lib/pleroma/web/activity_pub/mrf/user_allow_list_policy.ex b/lib/pleroma/web/activity_pub/mrf/user_allow_list_policy.ex @@ -37,7 +37,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.UserAllowListPolicy do def describe do mrf_user_allowlist = Config.get([:mrf_user_allowlist], []) - |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end) + |> Map.new(fn {k, v} -> {k, length(v)} end) {:ok, %{mrf_user_allowlist: mrf_user_allowlist}} end diff --git a/lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex b/lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex @@ -39,7 +39,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicy do @impl true def describe, - do: {:ok, %{mrf_vocabulary: Pleroma.Config.get(:mrf_vocabulary) |> Enum.into(%{})}} + do: {:ok, %{mrf_vocabulary: Pleroma.Config.get(:mrf_vocabulary) |> Map.new()}} @impl true def config_description do diff --git a/lib/pleroma/web/activity_pub/publisher.ex b/lib/pleroma/web/activity_pub/publisher.ex @@ -112,6 +112,7 @@ defmodule Pleroma.Web.ActivityPub.Publisher do quarantined_instances = Config.get([:instance, :quarantined_instances], []) + |> Pleroma.Web.ActivityPub.MRF.instance_list_from_tuples() |> Pleroma.Web.ActivityPub.MRF.subdomains_regex() !Pleroma.Web.ActivityPub.MRF.subdomain_match?(quarantined_instances, host) diff --git a/lib/pleroma/web/mastodon_api/views/instance_view.ex b/lib/pleroma/web/mastodon_api/views/instance_view.ex @@ -95,7 +95,20 @@ defmodule Pleroma.Web.MastodonAPI.InstanceView do {:ok, data} = MRF.describe() data - |> Map.merge(%{quarantined_instances: quarantined}) + |> Map.put( + :quarantined_instances, + Enum.map(quarantined, fn {instance, _reason} -> instance end) + ) + # This is for backwards compatibility. We originally didn't sent + # extra info like a reason why an instance was rejected/quarantined/etc. + # Because we didn't want to break backwards compatibility it was decided + # to add an extra "info" key. + |> Map.put(:quarantined_instances_info, %{ + "quarantined_instances" => + quarantined + |> Enum.map(fn {instance, reason} -> {instance, %{"reason" => reason}} end) + |> Map.new() + }) else %{} end diff --git a/priv/repo/migrations/20201005123100_simple_policy_string_to_tuple.exs b/priv/repo/migrations/20201005123100_simple_policy_string_to_tuple.exs @@ -0,0 +1,40 @@ +defmodule Pleroma.Repo.Migrations.SimplePolicyStringToTuple do + use Ecto.Migration + + alias Pleroma.ConfigDB + + def up, do: ConfigDB.get_by_params(%{group: :pleroma, key: :mrf_simple}) |> update_to_tuples + def down, do: ConfigDB.get_by_params(%{group: :pleroma, key: :mrf_simple}) |> update_to_strings + + defp update_to_tuples(%{value: value}) do + new_value = + value + |> Enum.map(fn {k, v} -> + {k, + Enum.map(v, fn + {instance, reason} -> {instance, reason} + instance -> {instance, ""} + end)} + end) + + ConfigDB.update_or_create(%{group: :pleroma, key: :mrf_simple, value: new_value}) + end + + defp update_to_tuples(nil), do: {:ok, nil} + + defp update_to_strings(%{value: value}) do + new_value = + value + |> Enum.map(fn {k, v} -> + {k, + Enum.map(v, fn + {instance, _} -> instance + instance -> instance + end)} + end) + + ConfigDB.update_or_create(%{group: :pleroma, key: :mrf_simple, value: new_value}) + end + + defp update_to_strings(nil), do: {:ok, nil} +end diff --git a/priv/repo/migrations/20201005124600_quarantained_policy_string_to_tuple.exs b/priv/repo/migrations/20201005124600_quarantained_policy_string_to_tuple.exs @@ -0,0 +1,61 @@ +defmodule Pleroma.Repo.Migrations.QuarantainedStringToTuple do + use Ecto.Migration + + alias Pleroma.ConfigDB + + def up, + do: + ConfigDB.get_by_params(%{group: :pleroma, key: :instance}) + |> update_quarantined_instances_to_tuples + + def down, + do: + ConfigDB.get_by_params(%{group: :pleroma, key: :instance}) + |> update_quarantined_instances_to_strings + + defp update_quarantined_instances_to_tuples(%{value: settings}) do + settings |> List.keyfind(:quarantined_instances, 0) |> update_to_tuples + end + + defp update_quarantined_instances_to_tuples(nil), do: {:ok, nil} + + defp update_to_tuples({:quarantined_instances, instance_list}) do + new_value = + instance_list + |> Enum.map(fn + {v, r} -> {v, r} + v -> {v, ""} + end) + + ConfigDB.update_or_create(%{ + group: :pleroma, + key: :instance, + value: [quarantined_instances: new_value] + }) + end + + defp update_to_tuples(nil), do: {:ok, nil} + + defp update_quarantined_instances_to_strings(%{value: settings}) do + settings |> List.keyfind(:quarantined_instances, 0) |> update_to_strings + end + + defp update_quarantined_instances_to_strings(nil), do: {:ok, nil} + + defp update_to_strings({:quarantined_instances, instance_list}) do + new_value = + instance_list + |> Enum.map(fn + {v, _} -> v + v -> v + end) + + ConfigDB.update_or_create(%{ + group: :pleroma, + key: :instance, + value: [quarantined_instances: new_value] + }) + end + + defp update_to_strings(nil), do: {:ok, nil} +end diff --git a/priv/repo/migrations/20201005132900_transparency_exclusions_string_to_tuple.exs b/priv/repo/migrations/20201005132900_transparency_exclusions_string_to_tuple.exs @@ -0,0 +1,61 @@ +defmodule Pleroma.Repo.Migrations.TransparencyExclusionsStringToTuple do + use Ecto.Migration + + alias Pleroma.ConfigDB + + def up, + do: + ConfigDB.get_by_params(%{group: :pleroma, key: :mrf}) + |> update_transparency_exclusions_instances_to_tuples + + def down, + do: + ConfigDB.get_by_params(%{group: :pleroma, key: :mrf}) + |> update_transparency_exclusions_instances_to_strings + + defp update_transparency_exclusions_instances_to_tuples(%{value: settings}) do + settings |> List.keyfind(:transparency_exclusions, 0) |> update_to_tuples + end + + defp update_transparency_exclusions_instances_to_tuples(nil), do: {:ok, nil} + + defp update_to_tuples({:transparency_exclusions, instance_list}) do + new_value = + instance_list + |> Enum.map(fn + {v, r} -> {v, r} + v -> {v, ""} + end) + + ConfigDB.update_or_create(%{ + group: :pleroma, + key: :mrf, + value: [transparency_exclusions: new_value] + }) + end + + defp update_to_tuples(nil), do: {:ok, nil} + + defp update_transparency_exclusions_instances_to_strings(%{value: settings}) do + settings |> List.keyfind(:transparency_exclusions, 0) |> update_to_strings + end + + defp update_transparency_exclusions_instances_to_strings(nil), do: {:ok, nil} + + defp update_to_strings({:transparency_exclusions, instance_list}) do + new_value = + instance_list + |> Enum.map(fn + {v, _} -> v + v -> v + end) + + ConfigDB.update_or_create(%{ + group: :pleroma, + key: :mrf, + value: [transparency_exclusions: new_value] + }) + end + + defp update_to_strings(nil), do: {:ok, nil} +end diff --git a/test/pleroma/config/deprecation_warnings_test.exs b/test/pleroma/config/deprecation_warnings_test.exs @@ -11,6 +11,183 @@ defmodule Pleroma.Config.DeprecationWarningsTest do alias Pleroma.Config alias Pleroma.Config.DeprecationWarnings + describe "simple policy tuples" do + test "gives warning when there are still strings" do + clear_config([:mrf_simple], + media_removal: ["some.removal"], + media_nsfw: ["some.nsfw"], + federated_timeline_removal: ["some.tl.removal"], + report_removal: ["some.report.removal"], + reject: ["some.reject"], + followers_only: ["some.followers.only"], + accept: ["some.accept"], + avatar_removal: ["some.avatar.removal"], + banner_removal: ["some.banner.removal"], + reject_deletes: ["some.reject.deletes"] + ) + + assert capture_log(fn -> DeprecationWarnings.check_simple_policy_tuples() end) =~ + """ + !!!DEPRECATION WARNING!!! + Your config is using strings in the SimplePolicy configuration instead of tuples. They should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, :mrf_simple, + media_removal: ["instance.tld"], + media_nsfw: ["instance.tld"], + federated_timeline_removal: ["instance.tld"], + report_removal: ["instance.tld"], + reject: ["instance.tld"], + followers_only: ["instance.tld"], + accept: ["instance.tld"], + avatar_removal: ["instance.tld"], + banner_removal: ["instance.tld"], + reject_deletes: ["instance.tld"] + ``` + + Is now + + + ``` + config :pleroma, :mrf_simple, + media_removal: [{"instance.tld", "Reason for media removal"}], + media_nsfw: [{"instance.tld", "Reason for media nsfw"}], + federated_timeline_removal: [{"instance.tld", "Reason for federated timeline removal"}], + report_removal: [{"instance.tld", "Reason for report removal"}], + reject: [{"instance.tld", "Reason for reject"}], + followers_only: [{"instance.tld", "Reason for followers only"}], + accept: [{"instance.tld", "Reason for accept"}], + avatar_removal: [{"instance.tld", "Reason for avatar removal"}], + banner_removal: [{"instance.tld", "Reason for banner removal"}], + reject_deletes: [{"instance.tld", "Reason for reject deletes"}] + ``` + """ + end + + test "transforms config to tuples" do + clear_config([:mrf_simple], + media_removal: ["some.removal", {"some.other.instance", "Some reason"}] + ) + + expected_config = [ + {:media_removal, [{"some.removal", ""}, {"some.other.instance", "Some reason"}]} + ] + + capture_log(fn -> DeprecationWarnings.warn() end) + + assert Config.get([:mrf_simple]) == expected_config + end + + test "doesn't give a warning with correct config" do + clear_config([:mrf_simple], + media_removal: [{"some.removal", ""}, {"some.other.instance", "Some reason"}] + ) + + assert capture_log(fn -> DeprecationWarnings.check_simple_policy_tuples() end) == "" + end + end + + describe "quarantined_instances tuples" do + test "gives warning when there are still strings" do + clear_config([:instance, :quarantined_instances], [ + {"domain.com", "some reason"}, + "somedomain.tld" + ]) + + assert capture_log(fn -> DeprecationWarnings.check_quarantined_instances_tuples() end) =~ + """ + !!!DEPRECATION WARNING!!! + Your config is using strings in the quarantined_instances configuration instead of tuples. They should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, :instance, + quarantined_instances: ["instance.tld"] + ``` + + Is now + + + ``` + config :pleroma, :instance, + quarantined_instances: [{"instance.tld", "Reason for quarantine"}] + ``` + """ + end + + test "transforms config to tuples" do + clear_config([:instance, :quarantined_instances], [ + {"domain.com", "some reason"}, + "some.tld" + ]) + + expected_config = [{"domain.com", "some reason"}, {"some.tld", ""}] + + capture_log(fn -> DeprecationWarnings.warn() end) + + assert Config.get([:instance, :quarantined_instances]) == expected_config + end + + test "doesn't give a warning with correct config" do + clear_config([:instance, :quarantined_instances], [ + {"domain.com", "some reason"}, + {"some.tld", ""} + ]) + + assert capture_log(fn -> DeprecationWarnings.check_quarantined_instances_tuples() end) == "" + end + end + + describe "transparency_exclusions tuples" do + test "gives warning when there are still strings" do + clear_config([:mrf, :transparency_exclusions], [ + {"domain.com", "some reason"}, + "somedomain.tld" + ]) + + assert capture_log(fn -> DeprecationWarnings.check_transparency_exclusions_tuples() end) =~ + """ + !!!DEPRECATION WARNING!!! + Your config is using strings in the transparency_exclusions configuration instead of tuples. They should work for now, but you are advised to change to the new configuration to prevent possible issues later: + + ``` + config :pleroma, :mrf, + transparency_exclusions: ["instance.tld"] + ``` + + Is now + + + ``` + config :pleroma, :mrf, + transparency_exclusions: [{"instance.tld", "Reason to exlude transparency"}] + ``` + """ + end + + test "transforms config to tuples" do + clear_config([:mrf, :transparency_exclusions], [ + {"domain.com", "some reason"}, + "some.tld" + ]) + + expected_config = [{"domain.com", "some reason"}, {"some.tld", ""}] + + capture_log(fn -> DeprecationWarnings.warn() end) + + assert Config.get([:mrf, :transparency_exclusions]) == expected_config + end + + test "doesn't give a warning with correct config" do + clear_config([:mrf, :transparency_exclusions], [ + {"domain.com", "some reason"}, + {"some.tld", ""} + ]) + + assert capture_log(fn -> DeprecationWarnings.check_transparency_exclusions_tuples() end) == + "" + end + end + test "check_old_mrf_config/0" do clear_config([:instance, :rewrite_policy], []) clear_config([:instance, :mrf_transparency], true) diff --git a/test/pleroma/user_test.exs b/test/pleroma/user_test.exs @@ -480,7 +480,7 @@ defmodule Pleroma.UserTest do ) test "it sends a welcome chat message when Simple policy applied to local instance" do - clear_config([:mrf_simple, :media_nsfw], ["localhost"]) + clear_config([:mrf_simple, :media_nsfw], [{"localhost", ""}]) welcome_user = insert(:user) clear_config([:welcome, :chat_message, :enabled], true) diff --git a/test/pleroma/web/activity_pub/mrf/simple_policy_test.exs b/test/pleroma/web/activity_pub/mrf/simple_policy_test.exs @@ -33,7 +33,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "has a matching host" do - clear_config([:mrf_simple, :media_removal], ["remote.instance"]) + clear_config([:mrf_simple, :media_removal], [{"remote.instance", "Some reason"}]) media_message = build_media_message() local_message = build_local_message() @@ -46,7 +46,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "match with wildcard domain" do - clear_config([:mrf_simple, :media_removal], ["*.remote.instance"]) + clear_config([:mrf_simple, :media_removal], [{"*.remote.instance", "Whatever reason"}]) media_message = build_media_message() local_message = build_local_message() @@ -70,7 +70,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "has a matching host" do - clear_config([:mrf_simple, :media_nsfw], ["remote.instance"]) + clear_config([:mrf_simple, :media_nsfw], [{"remote.instance", "Whetever"}]) media_message = build_media_message() local_message = build_local_message() @@ -81,7 +81,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "match with wildcard domain" do - clear_config([:mrf_simple, :media_nsfw], ["*.remote.instance"]) + clear_config([:mrf_simple, :media_nsfw], [{"*.remote.instance", "yeah yeah"}]) media_message = build_media_message() local_message = build_local_message() @@ -115,7 +115,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "has a matching host" do - clear_config([:mrf_simple, :report_removal], ["remote.instance"]) + clear_config([:mrf_simple, :report_removal], [{"remote.instance", "muh"}]) report_message = build_report_message() local_message = build_local_message() @@ -124,7 +124,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "match with wildcard domain" do - clear_config([:mrf_simple, :report_removal], ["*.remote.instance"]) + clear_config([:mrf_simple, :report_removal], [{"*.remote.instance", "suya"}]) report_message = build_report_message() local_message = build_local_message() @@ -159,7 +159,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do |> URI.parse() |> Map.fetch!(:host) - clear_config([:mrf_simple, :federated_timeline_removal], [ftl_message_actor_host]) + clear_config([:mrf_simple, :federated_timeline_removal], [{ftl_message_actor_host, "uwu"}]) local_message = build_local_message() assert {:ok, ftl_message} = SimplePolicy.filter(ftl_message) @@ -180,7 +180,10 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do |> URI.parse() |> Map.fetch!(:host) - clear_config([:mrf_simple, :federated_timeline_removal], ["*." <> ftl_message_actor_host]) + clear_config([:mrf_simple, :federated_timeline_removal], [ + {"*." <> ftl_message_actor_host, "owo"} + ]) + local_message = build_local_message() assert {:ok, ftl_message} = SimplePolicy.filter(ftl_message) @@ -203,7 +206,9 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do ftl_message = Map.put(ftl_message, "cc", []) - clear_config([:mrf_simple, :federated_timeline_removal], [ftl_message_actor_host]) + clear_config([:mrf_simple, :federated_timeline_removal], [ + {ftl_message_actor_host, "spiderwaifu goes 88w88"} + ]) assert {:ok, ftl_message} = SimplePolicy.filter(ftl_message) refute "https://www.w3.org/ns/activitystreams#Public" in ftl_message["to"] @@ -232,7 +237,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "activity has a matching host" do - clear_config([:mrf_simple, :reject], ["remote.instance"]) + clear_config([:mrf_simple, :reject], [{"remote.instance", ""}]) remote_message = build_remote_message() @@ -240,7 +245,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "activity matches with wildcard domain" do - clear_config([:mrf_simple, :reject], ["*.remote.instance"]) + clear_config([:mrf_simple, :reject], [{"*.remote.instance", ""}]) remote_message = build_remote_message() @@ -248,7 +253,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "actor has a matching host" do - clear_config([:mrf_simple, :reject], ["remote.instance"]) + clear_config([:mrf_simple, :reject], [{"remote.instance", ""}]) remote_user = build_remote_user() @@ -256,7 +261,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "reject Announce when object would be rejected" do - clear_config([:mrf_simple, :reject], ["blocked.tld"]) + clear_config([:mrf_simple, :reject], [{"blocked.tld", ""}]) announce = %{ "type" => "Announce", @@ -268,7 +273,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "reject by URI object" do - clear_config([:mrf_simple, :reject], ["blocked.tld"]) + clear_config([:mrf_simple, :reject], [{"blocked.tld", ""}]) announce = %{ "type" => "Announce", @@ -322,7 +327,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do |> URI.parse() |> Map.fetch!(:host) - clear_config([:mrf_simple, :followers_only], [actor_domain]) + clear_config([:mrf_simple, :followers_only], [{actor_domain, ""}]) assert {:ok, new_activity} = SimplePolicy.filter(activity) assert actor.follower_address in new_activity["cc"] @@ -350,7 +355,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "is not empty but activity doesn't have a matching host" do - clear_config([:mrf_simple, :accept], ["non.matching.remote"]) + clear_config([:mrf_simple, :accept], [{"non.matching.remote", ""}]) local_message = build_local_message() remote_message = build_remote_message() @@ -360,7 +365,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "activity has a matching host" do - clear_config([:mrf_simple, :accept], ["remote.instance"]) + clear_config([:mrf_simple, :accept], [{"remote.instance", ""}]) local_message = build_local_message() remote_message = build_remote_message() @@ -370,7 +375,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "activity matches with wildcard domain" do - clear_config([:mrf_simple, :accept], ["*.remote.instance"]) + clear_config([:mrf_simple, :accept], [{"*.remote.instance", ""}]) local_message = build_local_message() remote_message = build_remote_message() @@ -380,7 +385,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "actor has a matching host" do - clear_config([:mrf_simple, :accept], ["remote.instance"]) + clear_config([:mrf_simple, :accept], [{"remote.instance", ""}]) remote_user = build_remote_user() @@ -398,7 +403,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "is not empty but it doesn't have a matching host" do - clear_config([:mrf_simple, :avatar_removal], ["non.matching.remote"]) + clear_config([:mrf_simple, :avatar_removal], [{"non.matching.remote", ""}]) remote_user = build_remote_user() @@ -406,7 +411,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "has a matching host" do - clear_config([:mrf_simple, :avatar_removal], ["remote.instance"]) + clear_config([:mrf_simple, :avatar_removal], [{"remote.instance", ""}]) remote_user = build_remote_user() {:ok, filtered} = SimplePolicy.filter(remote_user) @@ -415,7 +420,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "match with wildcard domain" do - clear_config([:mrf_simple, :avatar_removal], ["*.remote.instance"]) + clear_config([:mrf_simple, :avatar_removal], [{"*.remote.instance", ""}]) remote_user = build_remote_user() {:ok, filtered} = SimplePolicy.filter(remote_user) @@ -434,7 +439,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "is not empty but it doesn't have a matching host" do - clear_config([:mrf_simple, :banner_removal], ["non.matching.remote"]) + clear_config([:mrf_simple, :banner_removal], [{"non.matching.remote", ""}]) remote_user = build_remote_user() @@ -442,7 +447,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "has a matching host" do - clear_config([:mrf_simple, :banner_removal], ["remote.instance"]) + clear_config([:mrf_simple, :banner_removal], [{"remote.instance", ""}]) remote_user = build_remote_user() {:ok, filtered} = SimplePolicy.filter(remote_user) @@ -451,7 +456,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "match with wildcard domain" do - clear_config([:mrf_simple, :banner_removal], ["*.remote.instance"]) + clear_config([:mrf_simple, :banner_removal], [{"*.remote.instance", ""}]) remote_user = build_remote_user() {:ok, filtered} = SimplePolicy.filter(remote_user) @@ -464,7 +469,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do setup do: clear_config([:mrf_simple, :reject_deletes], []) test "it accepts deletions even from rejected servers" do - clear_config([:mrf_simple, :reject], ["remote.instance"]) + clear_config([:mrf_simple, :reject], [{"remote.instance", ""}]) deletion_message = build_remote_deletion_message() @@ -472,7 +477,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "it accepts deletions even from non-whitelisted servers" do - clear_config([:mrf_simple, :accept], ["non.matching.remote"]) + clear_config([:mrf_simple, :accept], [{"non.matching.remote", ""}]) deletion_message = build_remote_deletion_message() @@ -481,10 +486,10 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end describe "when :reject_deletes is not empty but it doesn't have a matching host" do - setup do: clear_config([:mrf_simple, :reject_deletes], ["non.matching.remote"]) + setup do: clear_config([:mrf_simple, :reject_deletes], [{"non.matching.remote", ""}]) test "it accepts deletions even from rejected servers" do - clear_config([:mrf_simple, :reject], ["remote.instance"]) + clear_config([:mrf_simple, :reject], [{"remote.instance", ""}]) deletion_message = build_remote_deletion_message() @@ -492,7 +497,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end test "it accepts deletions even from non-whitelisted servers" do - clear_config([:mrf_simple, :accept], ["non.matching.remote"]) + clear_config([:mrf_simple, :accept], [{"non.matching.remote", ""}]) deletion_message = build_remote_deletion_message() @@ -501,7 +506,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end describe "when :reject_deletes has a matching host" do - setup do: clear_config([:mrf_simple, :reject_deletes], ["remote.instance"]) + setup do: clear_config([:mrf_simple, :reject_deletes], [{"remote.instance", ""}]) test "it rejects the deletion" do deletion_message = build_remote_deletion_message() @@ -511,7 +516,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do end describe "when :reject_deletes match with wildcard domain" do - setup do: clear_config([:mrf_simple, :reject_deletes], ["*.remote.instance"]) + setup do: clear_config([:mrf_simple, :reject_deletes], [{"*.remote.instance", ""}]) test "it rejects the deletion" do deletion_message = build_remote_deletion_message() diff --git a/test/pleroma/web/activity_pub/mrf_test.exs b/test/pleroma/web/activity_pub/mrf_test.exs @@ -63,6 +63,15 @@ defmodule Pleroma.Web.ActivityPub.MRFTest do end end + describe "instance_list_from_tuples/1" do + test "returns a list of instances from a list of {instance, reason} tuples" do + list = [{"some.tld", "a reason"}, {"other.tld", "another reason"}] + expected = ["some.tld", "other.tld"] + + assert MRF.instance_list_from_tuples(list) == expected + end + end + describe "describe/0" do test "it works as expected with noop policy" do clear_config([:mrf, :policies], [Pleroma.Web.ActivityPub.MRF.NoOpPolicy]) diff --git a/test/pleroma/web/activity_pub/publisher_test.exs b/test/pleroma/web/activity_pub/publisher_test.exs @@ -267,6 +267,80 @@ defmodule Pleroma.Web.ActivityPub.PublisherTest do end describe "publish/2" do + test_with_mock "doesn't publish a non-public activity to quarantined instances.", + Pleroma.Web.Federator.Publisher, + [:passthrough], + [] do + Config.put([:instance, :quarantined_instances], [{"domain.com", "some reason"}]) + + follower = + insert(:user, %{ + local: false, + inbox: "https://domain.com/users/nick1/inbox", + ap_enabled: true + }) + + actor = insert(:user, follower_address: follower.ap_id) + + {:ok, follower, actor} = Pleroma.User.follow(follower, actor) + actor = refresh_record(actor) + + note_activity = + insert(:followers_only_note_activity, + user: actor, + recipients: [follower.ap_id] + ) + + res = Publisher.publish(actor, note_activity) + + assert res == :ok + + assert not called( + Pleroma.Web.Federator.Publisher.enqueue_one(Publisher, %{ + inbox: "https://domain.com/users/nick1/inbox", + actor_id: actor.id, + id: note_activity.data["id"] + }) + ) + end + + test_with_mock "Publishes a non-public activity to non-quarantined instances.", + Pleroma.Web.Federator.Publisher, + [:passthrough], + [] do + Config.put([:instance, :quarantined_instances], [{"somedomain.com", "some reason"}]) + + follower = + insert(:user, %{ + local: false, + inbox: "https://domain.com/users/nick1/inbox", + ap_enabled: true + }) + + actor = insert(:user, follower_address: follower.ap_id) + + {:ok, follower, actor} = Pleroma.User.follow(follower, actor) + actor = refresh_record(actor) + + note_activity = + insert(:followers_only_note_activity, + user: actor, + recipients: [follower.ap_id] + ) + + res = Publisher.publish(actor, note_activity) + + assert res == :ok + + assert called( + Pleroma.Web.Federator.Publisher.enqueue_one(Publisher, %{ + inbox: "https://domain.com/users/nick1/inbox", + actor_id: actor.id, + id: note_activity.data["id"] + }) + ) + end + test_with_mock "publishes an activity with BCC to all relevant peers.", Pleroma.Web.Federator.Publisher, [:passthrough], diff --git a/test/pleroma/web/node_info_test.exs b/test/pleroma/web/node_info_test.exs @@ -150,37 +150,127 @@ defmodule Pleroma.Web.NodeInfoTest do ) end - test "it shows MRF transparency data if enabled", %{conn: conn} do - clear_config([:mrf, :policies], [Pleroma.Web.ActivityPub.MRF.SimplePolicy]) - clear_config([:mrf, :transparency], true) + describe "Quarantined instances" do + setup do + clear_config([:mrf, :transparency], true) + quarantined_instances = [{"example.com", "reason to quarantine"}] + clear_config([:instance, :quarantined_instances], quarantined_instances) + end - simple_config = %{"reject" => ["example.com"]} - clear_config(:mrf_simple, simple_config) + test "shows quarantined instances data if enabled", %{conn: conn} do + expected_config = ["example.com"] - response = - conn - |> get("/nodeinfo/2.1.json") - |> json_response(:ok) + response = + conn + |> get("/nodeinfo/2.1.json") + |> json_response(:ok) + + assert response["metadata"]["federation"]["quarantined_instances"] == expected_config + end + + test "shows extra information in the quarantined_info field for relevant entries", %{ + conn: conn + } do + clear_config([:mrf, :transparency], true) - assert response["metadata"]["federation"]["mrf_simple"] == simple_config + expected_config = %{ + "quarantined_instances" => %{ + "example.com" => %{"reason" => "reason to quarantine"} + } + } + + response = + conn + |> get("/nodeinfo/2.1.json") + |> json_response(:ok) + + assert response["metadata"]["federation"]["quarantined_instances_info"] == expected_config + end end - test "it performs exclusions from MRF transparency data if configured", %{conn: conn} do - clear_config([:mrf, :policies], [Pleroma.Web.ActivityPub.MRF.SimplePolicy]) - clear_config([:mrf, :transparency], true) - clear_config([:mrf, :transparency_exclusions], ["other.site"]) + describe "MRF SimplePolicy" do + setup do + clear_config([:mrf, :policies], [Pleroma.Web.ActivityPub.MRF.SimplePolicy]) + clear_config([:mrf, :transparency], true) + end - simple_config = %{"reject" => ["example.com", "other.site"]} - clear_config(:mrf_simple, simple_config) + test "shows MRF transparency data if enabled", %{conn: conn} do + simple_config = %{"reject" => [{"example.com", ""}]} + clear_config(:mrf_simple, simple_config) - expected_config = %{"reject" => ["example.com"]} + expected_config = %{"reject" => ["example.com"]} - response = - conn - |> get("/nodeinfo/2.1.json") - |> json_response(:ok) + response = + conn + |> get("/nodeinfo/2.1.json") + |> json_response(:ok) + + assert response["metadata"]["federation"]["mrf_simple"] == expected_config + end - assert response["metadata"]["federation"]["mrf_simple"] == expected_config - assert response["metadata"]["federation"]["exclusions"] == true + test "performs exclusions from MRF transparency data if configured", %{conn: conn} do + clear_config([:mrf, :transparency_exclusions], [ + {"other.site", "We don't want them to know"} + ]) + + simple_config = %{"reject" => [{"example.com", ""}, {"other.site", ""}]} + clear_config(:mrf_simple, simple_config) + + expected_config = %{"reject" => ["example.com"]} + + response = + conn + |> get("/nodeinfo/2.1.json") + |> json_response(:ok) + + assert response["metadata"]["federation"]["mrf_simple"] == expected_config + assert response["metadata"]["federation"]["exclusions"] == true + end + + test "shows extra information in the mrf_simple_info field for relevant entries", %{ + conn: conn + } do + simple_config = %{ + media_removal: [{"no.media", "LEEWWWDD >//<"}], + media_nsfw: [], + federated_timeline_removal: [{"no.ftl", ""}], + report_removal: [], + reject: [ + {"example.instance", "Some reason"}, + {"uwu.owo", "awoo to much"}, + {"no.reason", ""} + ], + followers_only: [], + accept: [], + avatar_removal: [], + banner_removal: [], + reject_deletes: [ + {"peak.me", "I want to peak at what they don't want me to see, eheh"} + ] + } + + clear_config(:mrf_simple, simple_config) + + clear_config([:mrf, :transparency_exclusions], [ + {"peak.me", "I don't want them to know"} + ]) + + expected_config = %{ + "media_removal" => %{ + "no.media" => %{"reason" => "LEEWWWDD >//<"} + }, + "reject" => %{ + "example.instance" => %{"reason" => "Some reason"}, + "uwu.owo" => %{"reason" => "awoo to much"} + } + } + + response = + conn + |> get("/nodeinfo/2.1.json") + |> json_response(:ok) + + assert response["metadata"]["federation"]["mrf_simple_info"] == expected_config + end end end diff --git a/test/support/factory.ex b/test/support/factory.ex @@ -142,6 +142,11 @@ defmodule Pleroma.Factory do } end + def followers_only_note_factory(attrs \\ %{}) do + %Pleroma.Object{data: data} = note_factory(attrs) + %Pleroma.Object{data: Map.merge(data, %{"to" => [data["actor"] <> "/followers"]})} + end + def audio_factory(attrs \\ %{}) do text = sequence(:text, &"lain radio episode #{&1}") @@ -267,6 +272,33 @@ defmodule Pleroma.Factory do |> Map.merge(attrs) end + def followers_only_note_activity_factory(attrs \\ %{}) do + user = attrs[:user] || insert(:user) + note = insert(:followers_only_note, user: user) + + data_attrs = attrs[:data_attrs] || %{} + attrs = Map.drop(attrs, [:user, :note, :data_attrs]) + + data = + %{ + "id" => Pleroma.Web.ActivityPub.Utils.generate_activity_id(), + "type" => "Create", + "actor" => note.data["actor"], + "to" => note.data["to"], + "object" => note.data, + "published" => DateTime.utc_now() |> DateTime.to_iso8601(), + "context" => note.data["context"] + } + |> Map.merge(data_attrs) + + %Pleroma.Activity{ + data: data, + actor: data["actor"], + recipients: data["to"] + } + |> Map.merge(attrs) + end + def note_activity_factory(attrs \\ %{}) do user = attrs[:user] || insert(:user) note = attrs[:note] || insert(:note, user: user)