Date   

locked Re: Spam filter

 

On Sun, Oct 2, 2016 at 11:49 PM, Shal Farley <shals2nd@...> wrote:

I thought the advice on the ARC list was to let the receiving services deal with distinguishing the replays from the real. I'm not so sure it is yet worth panicking over one receiving service making a dumb error.


Regardless of ARC, I feel like something needs to be done to address what's happening right now. The same guy is creating group after group (~50 so far today; hundreds since this started last week) of spam lists (each using a unique Yahoo email as owner). I don't want to host this junk and I don't want it cluttering up the directory. And I have to believe on some level it'd hurt our email reputation if we did host it, regardless of replay attacks.

I'm with you on the downsides of spam filters. Maria's idea of requiring a credit card to start a group would probably curtail this (with exceptions for people moving their existing groups from other services), but would also probably prevent a lot of legit people from using Groups.io. Or I could continue to require that I approve all groups before they are allowed to post messages, with the obvious downsides of that approach. I'm open to suggestions. 

A maze of twisty passages....

Thanks,
Mark


locked Re: Spam filter

Maria
 

On Mon, Oct 3, 2016 at 05:38 am, Linda wrote:
FYI: I still haven't figured out how to ensure that Gmail and Windows Live Mail never put mail from my own, moderated groups into the Spam/Junk folder. I'd welcome ideas for a solution.

Is this happening a lot? I think that if you ask GMAIL to remember to not put groups.io stuff in to the spam/promo/social folder that it remembers that behavior? Are you not seeing that?

Are others getting complaints from members re: group emails going to spam? The groups I am subscribed to on groups.io are usually in my inbox just fine.


Maria


locked Re: Spam filter

Linda
 

Hi Mark, you wrote:
"That implies not allowing moderators the opportunity to approve messages that have been marked as spam (like what Y! Groups apparently used to let people do)."

But that assumes that your spam filter will never make a mistake...

FYI: I still haven't figured out how to ensure that Gmail and Windows Live Mail never put mail from my own, moderated groups into the Spam/Junk folder. I'd welcome ideas for a solution.

Thanks,
Linda


locked Re: Spam filter

 

Mark,

I think that we need to try to prevent the opportunity for the replay
attacks we've been having this past week. That is, I think it's
probably more important than I've thought for the emails that
Groups.io sends out to not be spam, even with groups being opt-in and
people wanting to receive the messages.
I thought the advice on the ARC list was to let the receiving services deal with distinguishing the replays from the real. I'm not so sure it is yet worth panicking over one receiving service making a dumb error.

Unless there are more now, but even so - if this type of attack is becoming more common (with other lists, not just with Groups.io) the email services will be motivated to figure it out. There are certainly many clues in what the replay scammer is doing, starting with the fact that his envelope From is different than yours and hence not aligned with the Return-Path field nor with the DKIM d parameter.

That means preventing the scenario where one person creates a group
and then sends a spam message to that group (which only contains
himself). That implies not allowing moderators the opportunity to
approve messages that have been marked as spam (like what Y! Groups
apparently used to let people do).
I'm concerned this will lead you down a dark path.

The essence of the problem is that spam remains "in the eye of the beholder", despite everyone's belief that they know it when they see it. I have very little faith that a content filter can be devised which will reliably distinguish a message destined for replay from those used legitimately by groups of broad interests.

My understanding is that the more successful email services mix a lot of behavioral measurements in with the content discriminator. That may be a way to mitigate the harm this path might do. Such extrinsic factors as the age and size of the group, the age of the group moderator's account, etc. might be used to discriminate whether a message marked as spam is allowed to be approved or not.

Right now I'm preventing all this by having to approve all new groups
before they can post messages. But there are downsides to that
approach.
Yeah, doesn't scale is just the beginning. Circumvention will no doubt crop up one way or another.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked Re: Spam filter

 

On Fri, Sep 30, 2016 at 10:23 PM, Shal Farley <shals2nd@...> wrote:

Effectiveness aside, the UI for it is similar to what one would expect in an email UI: each group has a Spam folder (or Pending Spam list) and messages diverted there can be examined by moderators with the primary options to "approve" or "delete". That's one difference versus a typical email UI: the approve choice posts the message; it doesn't merely move the message from the spam list to the pending list.
I think the other pending list operations (reject with reply to sender and edit) are also available.

 
Thanks for the overview. I'm not sure what we should do for Groups.io. I think that we need to try to prevent the opportunity for the replay attacks we've been having this past week. That is, I think it's probably more important than I've thought for the emails that Groups.io sends out to not be spam, even with groups being opt-in and people wanting to receive the messages. That means preventing the scenario where one person creates a group and then sends a spam message to that group (which only contains himself). That implies not allowing moderators the opportunity to approve messages that have been marked as spam (like what Y! Groups apparently used to let people do).

Right now I'm preventing all this by having to approve all new groups before they can post messages. But there are downsides to that approach.

Thoughts/ideas appreciated.

Thanks,
Mark


locked Re: tweak wording for confirmation of deleting a hashtag #suggestion

 

On Sun, Oct 2, 2016 at 01:46 pm, Mark Fletcher wrote:
(because otherwise what would #SPOON point to?)

That's why I said it would say that you would delete all pointers to it. ;)
--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: tweak wording for confirmation of deleting a hashtag #suggestion

 

Yes, the existing wording says that better. But is that actually what you want it to do? I assumed a user can delete a hashtag without deleting all of its aliases. That's a scary feature. If someone wants to delete all aliases as well, I'd make that a checkbox option.
--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: tweak wording for confirmation of deleting a hashtag #suggestion

 

On Sat, Oct 1, 2016 at 7:01 AM, J_Catlady <j.olivia.catlady@...> wrote:

Currently, confirmation of deleting a hashtag reads: "Are you sure you wish to delete this hashtag and aliases pointing to it?" I deleted a tag this morning and after clicking "yes," had a momentary, delayed-reaction panic that all its aliases would also be deleted. I think what is meant is not "aliases pointing to it" but "pointers to it from aliases."

 
Say you have hashtag #FORK, and there's an alias called #SPOON that points to #FORK. If you delete #FORK, then #SPOON is also deleted (because otherwise what would #SPOON point to?). I think the existing wording says that better, but I'm not a master wordsmith. Thoughts?

Thanks,
Mark


locked make "add tags" dropdown alphabetical #suggestion

 

Be easier if the add-tags dropdown were alphabetical.
--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: tweak top-posting algorithm to allow explicit, intentional, changed, partial quotes #suggestion

 

Shal,

Your "strict" interpretation suits me just fine. It's better than the opposite.

J

On Sat, Oct 1, 2016 at 1:49 PM, Shal Farley <shals2nd@...> wrote:
J,

> There's also still the issue (I think - haven't explicitly checked in
> awhile) that I brought up long ago, about the Groups.io auto-sig (for
> web-post sigs) going below the whole thing and therefore included in
> the ellipses and not showing up.

That may be an issue with posts by email too, depending on how the user's email interface is configured. Some, like Thunderbird (and I think Gmail), give you a choice of placement.

That too could be taken care of by reference to the to the In-Reply-To message: ellipses or elide for only the span of matched text. That would leave showing all material added by the respondent -- above and below the quote.

I imagine the primary complaint with my "strict" idea would be that it may tend to leave in more quoted material than some would prefer. But it would serve my primary purpose, which is to clean up the "tail" of quotes that occur when users don't trim at all.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum





--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: tweak top-posting algorithm to allow explicit, intentional, changed, partial quotes #suggestion

 

J,

There's also still the issue (I think - haven't explicitly checked in
awhile) that I brought up long ago, about the Groups.io auto-sig (for
web-post sigs) going below the whole thing and therefore included in
the ellipses and not showing up.
That may be an issue with posts by email too, depending on how the user's email interface is configured. Some, like Thunderbird (and I think Gmail), give you a choice of placement.

That too could be taken care of by reference to the to the In-Reply-To message: ellipses or elide for only the span of matched text. That would leave showing all material added by the respondent -- above and below the quote.

I imagine the primary complaint with my "strict" idea would be that it may tend to leave in more quoted material than some would prefer. But it would serve my primary purpose, which is to clean up the "tail" of quotes that occur when users don't trim at all.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked Re: tweak top-posting algorithm to allow explicit, intentional, changed, partial quotes #suggestion

 

There's also still the issue (I think - haven't explicitly checked in awhile) that I brought up long ago, about the Groups.io auto-sig (for web-post sigs) going below the whole thing and therefore included in the ellipses and not showing up. I'll check today whether that's still happening. I don't remember any change having been made to this or even whether this issue finally even got across here.
J

Sent from my iPhone

On Oct 1, 2016, at 11:47 AM, J_Catlady <j.olivia.catlady@gmail.com> wrote:

Shal,
Agreed. But it goes further. The algorithm does its thing *even if you quote from a source other than another Groups.io message.* This really goes overboard in my opinion. Makes it very difficult to quote anything. Unless you're savvy to this, you wouldn't think to go underneath the quote and add some token word or character just so readers can see the quote.
J

Sent from my iPhone

On Oct 1, 2016, at 11:36 AM, Shal Farley <shals2nd@gmail.com> wrote:

J,

I think that if possible, the top-posting algorithm should be tweaked
so that the quote is not put into ellipses if either
...
I'd add "(d) is not contained in the In-Reply-To message."

Or perhaps replace (a) through (d) with just: "Does not contain the entirety of the In-Reply-To message." That would be pretty strict, in the sense of covering up the quote only in a true top-post (where the respondent didn't trim the quote at all). My argument in favor of this would be that if the respondent did do some trimming, then he/she probably relied on the partial quote being seen for context.

IIRC I had earlier suggested that the algorithms for hiding or eliding quotes be based (where possible) on comparison to the quoted message. In most cases the quoted message will be available to the algorithm (in the group's Messages).

There are complications with that idea, of course. Comparing only the text is non-trivial in HTML messages, and even in plain-text messages one has to cope with the changes made by the quote itself (inserting > or other quote characters, or leading spaces, or re-wrapping paragraphs, etc.). Still, I'm not sure this is any worse than trying to determine what is and what is not a quote by applying heuristics solely to the current message.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum

--
J

Messages are the sole opinion of the author.

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: tweak top-posting algorithm to allow explicit, intentional, changed, partial quotes #suggestion

 

Shal,
Agreed. But it goes further. The algorithm does its thing *even if you quote from a source other than another Groups.io message.* This really goes overboard in my opinion. Makes it very difficult to quote anything. Unless you're savvy to this, you wouldn't think to go underneath the quote and add some token word or character just so readers can see the quote.
J

Sent from my iPhone

On Oct 1, 2016, at 11:36 AM, Shal Farley <shals2nd@gmail.com> wrote:

J,

I think that if possible, the top-posting algorithm should be tweaked
so that the quote is not put into ellipses if either
...
I'd add "(d) is not contained in the In-Reply-To message."

Or perhaps replace (a) through (d) with just: "Does not contain the entirety of the In-Reply-To message." That would be pretty strict, in the sense of covering up the quote only in a true top-post (where the respondent didn't trim the quote at all). My argument in favor of this would be that if the respondent did do some trimming, then he/she probably relied on the partial quote being seen for context.

IIRC I had earlier suggested that the algorithms for hiding or eliding quotes be based (where possible) on comparison to the quoted message. In most cases the quoted message will be available to the algorithm (in the group's Messages).

There are complications with that idea, of course. Comparing only the text is non-trivial in HTML messages, and even in plain-text messages one has to cope with the changes made by the quote itself (inserting > or other quote characters, or leading spaces, or re-wrapping paragraphs, etc.). Still, I'm not sure this is any worse than trying to determine what is and what is not a quote by applying heuristics solely to the current message.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: tweak top-posting algorithm to allow explicit, intentional, changed, partial quotes #suggestion

 

J,

I think that if possible, the top-posting algorithm should be tweaked
so that the quote is not put into ellipses if either
...
I'd add "(d) is not contained in the In-Reply-To message."

Or perhaps replace (a) through (d) with just: "Does not contain the entirety of the In-Reply-To message." That would be pretty strict, in the sense of covering up the quote only in a true top-post (where the respondent didn't trim the quote at all). My argument in favor of this would be that if the respondent did do some trimming, then he/she probably relied on the partial quote being seen for context.

IIRC I had earlier suggested that the algorithms for hiding or eliding quotes be based (where possible) on comparison to the quoted message. In most cases the quoted message will be available to the algorithm (in the group's Messages).

There are complications with that idea, of course. Comparing only the text is non-trivial in HTML messages, and even in plain-text messages one has to cope with the changes made by the quote itself (inserting > or other quote characters, or leading spaces, or re-wrapping paragraphs, etc.). Still, I'm not sure this is any worse than trying to determine what is and what is not a quote by applying heuristics solely to the current message.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked tweak wording for confirmation of deleting a hashtag #suggestion

 

Currently, confirmation of deleting a hashtag reads: "Are you sure you wish to delete this hashtag and aliases pointing to it?" I deleted a tag this morning and after clicking "yes," had a momentary, delayed-reaction panic that all its aliases would also be deleted. I think what is meant is not "aliases pointing to it" but "pointers to it from aliases." 

--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked tweak top-posting algorithm to allow explicit, intentional, changed, partial quotes #suggestion

 

I think the current algorithm that sweeps quotations into ellipses within an presumed top-post needs to be tweaked. In a recent response to someone, I quoted part of anold post by a third-party member in order to make a specific point. I indented, put quotes around the quote, and bolded sections of it. Yet when my response showed up onlist, the quote was gone, swept into ellipses. The person I was addressing didn't even see it; she only saw the link I'd also included. She thanked me for "the link."

I know that people "should" (as was pointed out when I complained about the top-posting algorithm prior to this) know that they can click on the ellipses. But some people don't, because they might think it's simply the prior message in the thread. Many times, it is not. And if a link is included, all they may notice is the link.

I think that if possible, the top-posting algorithm should be tweaked so that the quote is not put into ellipses if either 

(a) quotation marks are put around it (which, after all, makes it differ from the original); and/or

(b) parts of it are bolded (ditto); and/or

(c) it is indented.

All of these show that the quote was an intentional act and that it is not simply a top-posting situation.

The overzealousness (IMO) of the top-posting algorithm has been a minor but frustrating and continuing annoyance to me. So if any of this can be accomplished, I for one would greatly appreciate it.

Thanks!

--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: Temporarily disabled group creation

 

Mark,

Maybe, but anecdotally, it seems that some services send out FBL
messages in batches. We'll see bunches of FBL messages from more than
one person all at the same time.
I was thinking they might be batched daily, but I thought that would still provide enough resolution I think to distinguish a 30-day auto delete. If the batches are less frequent, or the auto-delete prompter then that could scratch that idea.

Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked Re: Spam filter

 

On Fri, Sep 30, 2016 at 10:23 pm, Shal Farley wrote:
it has been a long time since I've noticed it catch anything. Others have reported that it catches only false positives, and misses actual spam.

Shal, thanks for providing my giggles for the evening. Still literally laughing out loud over this. :-) 
--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: Spam filter

 

Mark,

I am unfamiliar with how Y! Group's spam filter interacts with
groups; can someone clue me in?
I think a lot of people would say "What spam filter?" -- it has been a long time since I've noticed it catch anything. Others have reported that it catches only false positives, and misses actual spam.

But I think the more accurate statement is that for the last several years most (all?) spam is being caught (rejected or dropped) during the server transaction or at any rate well ahead of the forlorn content filter of Y!Groups itself. And I suspect that forlorn content filter has had very little attention over the course of time. It may be little more than a Bayesian discriminator, if even that sophisticated.

Effectiveness aside, the UI for it is similar to what one would expect in an email UI: each group has a Spam folder (or Pending Spam list) and messages diverted there can be examined by moderators with the primary options to "approve" or "delete". That's one difference versus a typical email UI: the approve choice posts the message; it doesn't merely move the message from the spam list to the pending list.
I think the other pending list operations (reject with reply to sender and edit) are also available.

The other side of the coin is in the regular Pending Message list, where the moderator has the option to "Delete as spam". That's also a little different than a normal email UI because it immediately deletes the message, not just move it to the Spam list.

During the 2013 "neo" redesign I railed at them to better coordinate the Pending Message UI and the Pending Spam UI. And neither of those had much in common with the UI for the message archive itself. Instead it was as if they handed loose descriptions of "messages in a list" to distinct teams - the list and message viewing controls are weirdly and confusingly different. That may have gotten better over time, but the paucity of messages in the Spam list means I haven't seen much of that UI in long time.

Moderators have a notification checkbox for whether they wish to be notified of Pending Spam.

There is (or was) separately some kind of filtering on messages sent to the -owner address. That filter had no UI at all: no way to know what false positives had been lost nor to inform the filter of false negatives. As with the primary spam filter, I've not seen any evidence that it is still in operation in quite some time.

Shal


On 9/30/2016 8:54 PM, Mark Fletcher wrote:
Hi All,

Based on the recent spammer incident and some conversations about some
evolving email standards, it's clear that I need to implement a spam
filter sooner rather than later. I am unfamiliar with how Y! Group's
spam filter interacts with groups; can someone clue me in? And are there
any issues with their implementation (the spam filter went in after the
acquisition/after I left)?

Thanks,
Mark
--
Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked Site updates #changelog

 

Changes to the site this week:

  • INTERNAL: A lot of work on changes needed for the Enterprise version.
  • CHANGE: Blocking email addresses from many disposable email address providers.
  • CHANGE: New groups must be approved before they are listed in the directory and appear in search, to hopefully prevent spammers.
  • BUGFIX: Fix for deleting table columns in IE.
  • CHANGE: In the Messages/Expanded Messages view, moved Msg # box to the right to prevent confusion with the Search button.
  • CHANGE: Emailed login links no longer expire after the first time they're clicked.
  • CHANGE: When viewing owner messages, highlight the Members entry in the sidebar.
  • CHANGE: Group home page wording changes.
  • CHANGE: Pulled Groups.io CSS out into its own file.
  • BUGFIX: Sub group direct add sorting wasn't working.

Have a good weekend, everybody!

Mark

18681 - 18700 of 29708