SkyChatFilter >> Similarity, memorization, patterns 1.0.0

Lightweight yet powerful chat filter to guard against spam, swearing, caps and advertising

  1. MyzelYam
    SkyChatFilter is a lightweight yet powerful bungeecord chat filter which guards against spam, swearing, caps and advertising.

    There are many chat message filtering solutions, however most of them are either easily bypassed or have lots of false positives. SkyChatFilter combines multiple techniques using unconventional algorithms which all work together to create an amazingly powerful chat filter with little annoyance for players and staff. Most significant features:

    Meticulous repetition detection
    • Similar chat messages cannot be repeated by the same player in a certain period of time. What makes this special is that it can't be circumvented easily. Common methods to circumvent it are stopped using the following algorithms:
    • You can't repeat the same chat message in a certain period of time (message equality check)
    • You also can't append a few numbers and try to send it anyway (message suffix check)
      [​IMG]
    • You also can't send it if it's similar to the last message in any way (message similarity check using Levenshtein edit distance)
      [​IMG]
    • You also cannot switch-spam messages since by default the last 5 chat messages are kept in memory and all of the above applies to them as well (configurable)
    It is very hard and very frustrating to try to get around this since every failed attempt will also be kept in memory.
    There are also leniency messures to stop this from affecting normal users:

    • Only applies to long messages. Short replies such as "Yes" or "xD" will not be detected and are instead handled by the flood protection.
    • Message memory expiry. Messages that weren't sent recently will not be detected so you can't accidentally trigger the detection by repeating yourself after coming back from being AFK.
    You can configure precisely how aggressive or lenient the repetition checks are and you can also define how similar messages have to be to count as a repetition.

    Confirmation-driven advertising detection
    With advert filters there has always been the issue of the filter having too many false positives or being easily bypassed through character substitution. Manual measures by staff members often come too late and the chat can't be cleared fast enough.

    The solution to this is a combined approach: Messages that are detected as advertising will be put on hold to be confirmed by a staff member.

    This means that they will not be sent and the staff members will get an alert asking them to either approve the message or issue a direct punishment. Approving the message or issuing a preset punishment suggestion can be done by simply clicking a button in the chat.
    [​IMG]
    Once a punishment has been selected and automatically issued (all of them are configurable) you cannot select another option so staff members don't accidentally punish a player twice.

    There are various message pattern checks which use regular expressions and have been created with common and less common bypass techniques in mind. Tricks such as replacing the dot with spaces or other characters will still be detected with little false positives.

    Using this approach players will never be punished unfairly and they also dont have to (try to) repeat their chat messages. Furthermore, issuing punishments for advertising has never been faster and doesn't even require the staff members to issue a command.

    Other features:
    • Flood protection. Players can't send too many chat messages in a short period of time. The threshold can be lower for messages sent directly after joining the server.
    • Caps filter. Messages can be adjusted or cancelled if the percentage of uppercase letters is too big. Short messages can be excluded.
    • Swearword filter. Messages are checked against a customizable list of swearwords.
    • Full customizability. All thresholds and features can be changed or turned off. All messages are customizable and can be different depending on the player's server. There are exempt permissions for every check.
    Since all the filters work together a malicious new player trying to advertise other servers would have to completely bypass the advertising check and the caps check and also cant send that message more than one time due to the repetition (combined with flood) checks.

    If you'd like to change the patterns of the AntiAd detection you will need to watch a few tutorials for regular expressions first. However, the default ones are quite powerful and only need minor adjustments normally. I recommend RegExr for testing and modifying regular expressions. I cannot and will not assist with regular expressions since there are tutorials that are much better at explaining and testing regex than I am.

    Code (Text):
    # SkyChatFilter - Config

    # Commands that are filtered just like chat messages (Includes plugin:cmd syntax automatically)
    IncludedCommands:
      - ch
      - herochat
      - global
      - g
      - me
      - eme
      - action
      - eaction
      - describe
      - edescribe
    # You can disable features if you dont need them
    EnabledModules:
      - AntiAdvertising
      - AntiSpam/Flood
      - AntiSpam/Repetition
      - AntiSwear
      - CapsFilter

    # Use '/skychatfilter reload' to reload the config files. Reload permission: skychatfilter.reload

    # ### AntiSpam settings ###

    # Flood detection. This makes sure that players dont send too many messages ('MaxMessagesPerPeriod') in a
    # short period of time ('ClearPeriod'). You can define a different maximum of chat messages ('AfterJoin')
    # for players that joined the server recently ('TimeUntilNoMoreJoin'). A player will be kicked from the
    # server if the applying 'MaxMessages' threshold has been exceeded severely ('KickAfterXOverMax').
    # Bypass permission: skychatfilter.bypassflood
    FloodMaxMessagesPerPeriodWithoutJoin: 6
    FloodClearPeriodInSeconds: 4
    FloodMaxMessagesPerPeriodAfterJoin: 3
    FloodTimeUntilNoMoreJoinInSeconds: 30
    FloodKickAfterXOverMax: 10

    # Repetition detection. This makes sure that players dont send the same messages or similar messages
    # ('Similarity'-rate between 0 (not similar) and 1 (very similar)) over and over again. Messages that are
    # short ('MinLength') are excepted from this detection. The last few chat messages ('SaveAmount') of a
    # player will be saved for this check and will trigger the repetition check if they were sent recently
    # ('ExpiryInSeconds'). Bypass permission: skychatfilter.bypassrepetition
    SameMessageSimilarity: 0.8
    SameMessageMinLength: 5
    SameMessageSaveAmount: 5
    SameMessageExpiryInSeconds: 30

    # ### AntiCaps settings ###

    # Chat messages that contain mostly ('CapsMinPercentage') uppercase letters can trigger AntiCaps if
    # they aren't short ('MinLength'). They will either be adjusted ('AdjustMessage') to only contain lowercase
    # characters (except first character) or blocked.
    # Bypass permission: skychatfilter.bypasscaps
    CapsMinPercentage: 0.5
    CapsMinLength: 4
    CapsAdjustMessage: true

    # ### AntiSwear info ###

    # The plugin will create a separate blacklist.txt file. You can put banned words/sentences into
    # this file (separated by newline)

    # ### AntiAdvertising settings ###

    # Chat messages that match any of the following regular expressions will trigger the filter.
    # You can use https://regexr.com/ to build, modify and test these expressions and make them suit your needs.
    # There are many tutorials for RegEx online and I recommend that you dont modify the pre-defined expressions
    # if you aren't proficient in RegEx. Bypass permission: skychatfilter.bypassad
    BlacklistRegEx:
      # General URLs
      - '(http(s)?:\/\/.)?(www\.)?[[email protected]:%._\+~#=]{2,256}(\.| ?\(?dot\)? ?| ?\(?punkt\)? ?)[a-z]{2,6}\b([[email protected]:%_\+.~#?&//=]*)'
      # IP Addresses
      - '(?:[0-9]{1,3}( ?\. ?|\(?dot\)?)){3}[0-9]{1,3}'
      # Suspicious Subdomain + TLD
      - '(mc|play|http:\/|https:\/)(\/www)?[\W_]{1,3}(\w){3,20}[\W_]{1,3}(net|com|de|eu|us|org|me|ch|cc|co.uk|uk)'
      # Suspicious DOT + TLD
      - '([^\w\d\u00e4\u00f6\u00fc\u00df ]|_|dot|punkt)( )?(cc|net\.br|me|us|gg|co|at|de|eu|net|nl|se|com|org|jp|tk|ch|io|xyz|uk|ck|au|ru|cn\.in|co\.uk|co\.jp|fr|com\.br|be)((?!\S)|\/|:)'
      # Placeholders for your enemy server names
      - 'examplecraftpvpmc|godpvperstestmc'

    # Exclusions from the filter (own server IP, own websites, popular websites, etc).
    # The following regular expressions will be removed from the chat message prior to checking the blacklist.
    OverridingWhitelistRegEx:
      - 'myserver\.eu|mysecondserver\.eu|mywebsite\.eu|google\..+'

    # The alert message that staff members will receive when a chat message triggers the filter.
    # Here you can configure the options a staff member has to punish the sender or to allow the message quickly
    # if it turns out to be a false positive. Staff permission: skychatfilter.receivealert
    AdStaffAlerter:
      # The alert which a staff member receives (0: player, 1: chat message)
      AlertMessage: "&7[Advert] &c{0}: {1} &3"
      # The possible punishments (0: offender's player name, <allow> sends the chat message)
      Punishments:
        "[AdBanP]": "ban {0} Advertising"
        "[AdMute1h]": "mute {0} 1h Advertising"
        "[OK]": "<allow>"

    # Localized messages, configured per server
    Language:
      English:
        FloodTriggered: "&cYou cannot send that many messages in such a short time!"
        TextTooSimilar: "&cYou cannot send the same message twice!"
        NoAdvertising: "&cYou are not allowed to mention other servers or websites!"
        ActionAlreadyPerformed: "&cAn action for this message has already been performed!"
        NoSwearing: "&cSwearing and circumventing the filter is prohibited!"
        AllowMessage: "Allow message"
        MessageAllowed: "&aWe've reconsidered your message and approved it. Sorry about that!"
        AntiCaps: "&cYour message contained too many uppercase characters and has been adjusted!"
      German:
        FloodTriggered: "&cDu kannst nicht so viele Nachrichten in so kurzer Zeit senden!"
        TextTooSimilar: "&cDu kannst nicht dieselbe Nachricht nochmal senden!"
        NoAdvertising: "&cEs ist dir nicht erlaubt andere Server oder Webseiten zu erwähnen!"
        ActionAlreadyPerformed: "&cEs wurde für diese Nachricht bereits eine Aktion ausgeführt!"
        NoSwearing: "&cDie Benutzung von Schimpfwörtern sowie die Umgehung des Filters ist verboten!"
        AllowMessage: "Nachricht erlauben"
        MessageAllowed: "&aWir haben deine Nachricht überprüft und nachträglich zugelassen. Sorry!"
        AntiCaps: "&cDeine Nachricht enthielt zu viele Großbuchstaben und wurde angepasst!"
      # Which servers should use which locale? Non-configured servers use the 'Default' locale
      Servers:
        Default: German
        LobbyUS: English
        SkyBlockUS: English

    This is a bungeecord plugin and needs to be installed inside the bungee's plugins folder. There are no dependencies.
    SkyChatFilter contains the java-string-similarity library by Thibault Debatty which is released under the MIT license which you can obtain here.
    Please private message me if you encounter any bugs.
    SkyChatFilter is released under this open-source license.
    Valentina_pro likes this.

Recent Reviews

  1. Valentina_pro
    Valentina_pro
    5/5,
    Version: 1.0.0
    Very cool and original chat filter, it's rare to see something this cool. <3
  2. _4ch
    _4ch
    4/5,
    Version: 1.0.0
    Thanks pal for this plugin is quite convenient but I didn’t understand how the blacklist.txt file should look like
  3. Goro922
    Goro922
    5/5,
    Version: 1.0.0
    ________________________________

    i love this plugin, more update please
    ________________________________