Date   

locked Re: Temporarily disabled group creation

 

On Thu, Sep 29, 2016 at 6:17 PM, Shal Farley <shals2nd@...> wrote:


I wonder if ARC defends against this kind of replay. That is, when you put an ARC-seal on an outbound messages, is there something that makes this copy unique (a time stamp, serial number, or anything else that would invalidate a copied ARC-seal. If not that may be a weakness.

That's an excellent question and thanks for thinking of it. I just posted it to the ARC group, so we'll see.
 

Oh, interesting. I was wondering how much detail you get in the FBL reports. Specifically I was wondering if you were told the user and/or the Message-ID's of "marked" messages. And how they got marked (explict user action versus automation). But that was concerning some discussion in GMF@ about other issues with the FBL reports.

The FBL reports differ a little bit by domain. In general, the email providers try to obscure the email address of the person who received the email. What I look for is the one-click unsubscribe link in the message footer. From that I can figure out the recipient. Message-IDs are not obscured, and (I think for all of them), I get the entire message sent to me. I am not told how they were marked (I wish I was!). 

 
But, silly Yahoo, shouldn't the reports be flung back at the 5321.MailFrom? Sending it back to the From strikes me as akin to backscatter, though it was useful to you in this case. Or is it the (copied) Return-Path they're using? Either way...

Yeah, not sure. But still, pretty stupid of them.

 
> But I'm not sure country blocking isn't a great long term solution
> for this.

I'm reasonably sure it isn't.

For the moment I've blocked Morocco, because I need to actually get some work done. Having that guy around was distracting.

 
Well, I think you have some head-scratching yet to do on this problem.

Yeah. I mean spoofed emails are a fact of life on the Internet and I think (hope) all the major providers recognize the spoofing. So I don't think it'll ding our email sending reputation. It sucks though because I get angry support emails from people demanding to be removed from groups they aren't even on.

We're still small fry in the grand scheme of things, which is nice in a way because I'm able to add defenses over time. But makes us still open to being blocked if things go bad. But hey, the other day we had our first million email day!


Mark


locked Re: Temporarily disabled group creation

 

Mark,

I'm watching them try to register new email addresses in real time
and being denied. They are persistent.
That's the way of ants and other automatons.

What they're doing is creating a group, doing a post, grabbing the
resulting email, and sending that out to people themselves. ... And
our SPF record doesn't apply because they're using a different
envelope domain (I did just change our SPF records from softfail to
fail, but if I understand what's happening, that won't help here).
No, but my understanding is that the misalignment of the 5322.From with the 5321.MailFrom will cause DMARC to fail anyway, for those receiving services that check it. Of course, so do the messages legitimately through Groups.io when you don't rewrite the From.

So if I understand everything correctly, the only way to 'fix' this
would be to set DMARC p=reject and re-write *all* From lines in
emails. I'd obviously rather not do that.
I'd rather you not do that too. The only advantage I can see is that all outbound messages could pass DMARC, instead of only those from the services you rewrite.

I wonder if ARC defends against this kind of replay. That is, when you put an ARC-seal on an outbound messages, is there something that makes this copy unique (a time stamp, serial number, or anything else that would invalidate a copied ARC-seal. If not that may be a weakness.

The reason I was able to see one of these emails is that Yahoo has
sent some FBL reports to us from these forged emails (people who
received these messages marked them as spam, and Yahoo thought they
originated from us).
Oh, interesting. I was wondering how much detail you get in the FBL reports. Specifically I was wondering if you were told the user and/or the Message-ID's of "marked" messages. And how they got marked (explict user action versus automation). But that was concerning some discussion in GMF@ about other issues with the FBL reports.

But, silly Yahoo, shouldn't the reports be flung back at the 5321.MailFrom? Sending it back to the From strikes me as akin to backscatter, though it was useful to you in this case. Or is it the (copied) Return-Path they're using? Either way...

But I'm not sure country blocking isn't a great long term solution
for this.
I'm reasonably sure it isn't.

Well, I think you have some head-scratching yet to do on this problem.

--
Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked Re: Temporarily disabled group creation

 

Oh, and I know the spammer is reading beta@. 

Exciting!

Mark

On Thu, Sep 29, 2016 at 3:56 PM, Mark Fletcher <markf@corp.groups.io> wrote:
On Thu, Sep 29, 2016 at 1:29 PM, Shal Farley <shals2nd@...> wrote:
Mark,

> Most of the groups are being created using disposable email
> addresses. I am in the process of blocking those email providers.

Maybe you need a throttle (and internal alert) on the rate of group creation from unknown email providers. Then if you get an alert for a rash of creations from an unknown you can decide whether to white or black list the provider, or leave it gray ("unknown") pending further information. Sort of a poor-man's reputation system.


I think at least for now just checking against a comprehensive list of disposable email providers seems to be helping a lot. I'm watching them try to register new email addresses in real time and being denied. They are persistent.


> Also, most likely related, someone is forging spam emails from
> Groups.io. So I may need to look into some changes to mitigate that.
> I'll keep you posted.

If they're forging the groups.io domain in the From field then maybe DMARC p=reject is your friend.

What they're doing is creating a group, doing a post, grabbing the resulting email, and sending that out to people themselves. So the DKIM sig still authenticates. And our SPF record doesn't apply because they're using a different envelope domain (I did just change our SPF records from softfail to fail, but if I understand what's happening, that won't help here).

So if I understand everything correctly, the only way to 'fix' this would be to set DMARC p=reject and re-write *all* From lines in emails. I'd obviously rather not do that.

The reason I was able to see one of these emails is that Yahoo has sent some FBL reports to us from these forged emails (people who received these messages marked them as spam, and Yahoo thought they originated from us). 

I've also thought about blocking all IP addresses from Morocco. We currently block all IPs from Afghanistan, because of an earlier incident. But I'm not sure country blocking isn't a great long term solution for this.

(Note for the majority of people on beta@ who probably don't understand most of the above: The email forgery isn't a big deal right now and won't affect your groups).

Thanks,
Mark



locked Re: Temporarily disabled group creation

 

On Thu, Sep 29, 2016 at 1:29 PM, Shal Farley <shals2nd@...> wrote:
Mark,

> Most of the groups are being created using disposable email
> addresses. I am in the process of blocking those email providers.

Maybe you need a throttle (and internal alert) on the rate of group creation from unknown email providers. Then if you get an alert for a rash of creations from an unknown you can decide whether to white or black list the provider, or leave it gray ("unknown") pending further information. Sort of a poor-man's reputation system.


I think at least for now just checking against a comprehensive list of disposable email providers seems to be helping a lot. I'm watching them try to register new email addresses in real time and being denied. They are persistent.


> Also, most likely related, someone is forging spam emails from
> Groups.io. So I may need to look into some changes to mitigate that.
> I'll keep you posted.

If they're forging the groups.io domain in the From field then maybe DMARC p=reject is your friend.

What they're doing is creating a group, doing a post, grabbing the resulting email, and sending that out to people themselves. So the DKIM sig still authenticates. And our SPF record doesn't apply because they're using a different envelope domain (I did just change our SPF records from softfail to fail, but if I understand what's happening, that won't help here).

So if I understand everything correctly, the only way to 'fix' this would be to set DMARC p=reject and re-write *all* From lines in emails. I'd obviously rather not do that.

The reason I was able to see one of these emails is that Yahoo has sent some FBL reports to us from these forged emails (people who received these messages marked them as spam, and Yahoo thought they originated from us). 

I've also thought about blocking all IP addresses from Morocco. We currently block all IPs from Afghanistan, because of an earlier incident. But I'm not sure country blocking isn't a great long term solution for this.

(Note for the majority of people on beta@ who probably don't understand most of the above: The email forgery isn't a big deal right now and won't affect your groups).

Thanks,
Mark


locked Re: Temporarily disabled group creation

 

Mark,

Most of the groups are being created using disposable email
addresses. I am in the process of blocking those email providers.
Maybe you need a throttle (and internal alert) on the rate of group creation from unknown email providers. Then if you get an alert for a rash of creations from an unknown you can decide whether to white or black list the provider, or leave it gray ("unknown") pending further information. Sort of a poor-man's reputation system.

Also, most likely related, someone is forging spam emails from
Groups.io. So I may need to look into some changes to mitigate that.
I'll keep you posted.
If they're forging the groups.io domain in the From field then maybe DMARC p=reject is your friend.

If they're forging a "subscriber's" domain, as if it is a message that passed through a group, I'm not sure. The message would already fail DMARC, but that puts the disposition policy in the hands of the domain that was forged.

Shal

--
Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum


locked Re: Temporarily disabled group creation

 

On Thu, Sep 29, 2016 at 7:00 AM, J_Catlady <j.olivia.catlady@...> wrote:
On Wed, Sep 28, 2016 at 11:00 pm, Christopher Hallsworth wrote:
Good idea, never thought of that one.

That's why Mark makes the big bucks. ;) 


Heh. :-)

The system is working, in that I've deleted 128 groups just so far this morning. Seems to be a determined spammer from Morocco. 

Most of the groups are being created using disposable email addresses. I am in the process of blocking those email providers.

Also, most likely related, someone is forging spam emails from Groups.io. So I may need to look into some changes to mitigate that. I'll keep you posted.

Thanks,
Mark


locked Re: Problem with content outside of the 'charset' #bug

 

Hi Mark,

Thanks very much for getting back to me! Yes, I'm talking about the summary.

Your code works as designed: garbage in garbage out. However, some groups, like mine, have a need to support multiple languages. The use case is for people communicating in several languages, e.g., Japanese students studying Russian. Some mail clients, like outlook.live.com are uni-lingual.

Tests on my test group indicate that the plain text portion contains characters that, when displayed as utf-8 (the charset that I used to send the message), show up correctly in the body of the message but not in the summary. That indicates to me that you are doing something different to the characters when you display the summary than when you display the message contents. In this case, there is nothing wrong with the characters themselves. The only garbage is that their charset has been wrongly declared

Thanks again!

David.


locked Re: Temporarily disabled group creation

 

On Wed, Sep 28, 2016 at 11:00 pm, Christopher Hallsworth wrote:
Good idea, never thought of that one.

That's why Mark makes the big bucks. ;) 
--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Re: Temporarily disabled group creation

christopher hallsworth <challsworth2@...>
 

Mark

Good idea, never thought of that one.

On 28 Sep 2016, at 22:35, Mark Fletcher <markf@corp.groups.io> wrote:

Hi All,

Group creation has been turned back on. Now, when a (listed) group is created, it won't actually appear in the directory or search until I approve it. This is mentioned on the page displayed after the group is created. Also, when a group's name or description is changed, I'm notified as well. In those instances, the changes are made instantly to the directory and search, like before. But since I'm notified, I can retroactively delete the group if it was a spam group in hiding, so to speak.

Thanks,
Mark

On Wed, Sep 28, 2016 at 1:04 PM, HR Tech via Groups.io <m.conway11=yahoo.com@groups.io> wrote:
This is likely a bad idea but I'll share anyway. What about a credit card transaction like other sites use to verify identity? Does nextdoor do that? Or paypal when you set up an account and link it to a bank account? Do they take $1 and then reimburse it...? It's been a while since i needed to set one up.

Not ideal, and no idea if it would help.

Maria





locked Re: Problem with content outside of the 'charset' #bug

 

Hi David,

On Wed, Sep 28, 2016 at 5:33 PM, David Andrews <jdabud@...> wrote:

The good news is that the content of the message is displayed much better than it would have been by Yahoo!Groups. Thank you! But the special characters don't show up correctly in the message listing.

I assume you're referring to the topic summary, the line displayed in the Topics view under each topic subject? For that line, we want plain text, because any sort of HTML formatting would screw up the display. So, we first look for a plain text part of the email, which most emails have. If we find one, we just pull out the summary from that. If there is no plain text part, we use the HTML part and strip it of formatting before we pull the summary out.

With your email, it does have a plain text part, but if you look at it, live.com screwed up the characters, and subbed in a bunch of question marks. We dutifully pull those out for the summary line. So, not our fault. But it does have me wondering if we should always just use the HTML part of the message if there is one.

Thanks,
Mark


locked Re: Problem with content outside of the 'charset' #bug

 

On Wed, Sep 28, 2016 at 05:33 pm, David Andrews wrote:
the special characters don't show up correctly in the message listing

Coincidentally, I noticed the same behavior a day or two ago, where the message snippet in the message list didn't preserve formatting. After reporting it to Mark, I found out that HTML is stripped from message snippets, for some reason having to do with spam filters. 
--
J

Messages are the sole opinion of the author. 

I wish I could shut up, but I can't, and I won't. - Desmond Tutu


locked Problem with content outside of the 'charset' #bug

 


道德經: 道可道,非常道。

Ñ ñ Ā ā Ī ī Ŋ ŋ Ū ū Ḍ ḍ Ḷ ḷ Ṁ ṁ Ṃ ṃ Ṅ ṅ Ṇ ṇ Ṭ ṭ


The characters above are the first few characters of the Dao De Jing (Tao Te Ching) followed by characters used for the Romanization of classical Indian languages. The message was sent by outlook.live.com Webmail. If it works as in my other tests, will be sent out in the inappropriate charset, iso-8859-1.


The good news is that the content of the message is displayed much better than it would have been by Yahoo!Groups. Thank you! But the special characters don't show up correctly in the message listing.


Thanks,


David.


locked Re: Temporarily disabled group creation

Steph Mathews <smathews@...>
 

Okay, thank you Mark, I appreciate it.  Steph

Sent: Wednesday, September 28, 2016 4:51 PM
Subject: Re: [beta] Temporarily disabled group creation

Hi Steph,

On Wed, Sep 28, 2016 at 2:47 PM, Steph Mathews <smathews@...> wrote:
Hi Mark,
I have a question about this.
If our group was created and has been listed before hand.  Is there anything we the group owners need to do to get our group re-approved?  Just wondering.  Steph


Sorry, should have made this clear. No, nothing needs to be done for existing groups. 

Thanks,
Mark 


locked Re: Temporarily disabled group creation

 

Hi Steph,

On Wed, Sep 28, 2016 at 2:47 PM, Steph Mathews <smathews@...> wrote:
Hi Mark,
I have a question about this.
If our group was created and has been listed before hand.  Is there anything we the group owners need to do to get our group re-approved?  Just wondering.  Steph


Sorry, should have made this clear. No, nothing needs to be done for existing groups. 

Thanks,
Mark 


locked Re: Temporarily disabled group creation

Steph Mathews <smathews@...>
 

Hi Mark,
I have a question about this.
If our group was created and has been listed before hand.  Is there anything we the group owners need to do to get our group re-approved?  Just wondering.  Steph

Sent: Wednesday, September 28, 2016 4:35 PM
Subject: Re: [beta] Temporarily disabled group creation

Hi All,

Group creation has been turned back on. Now, when a (listed) group is created, it won't actually appear in the directory or search until I approve it. This is mentioned on the page displayed after the group is created. Also, when a group's name or description is changed, I'm notified as well. In those instances, the changes are made instantly to the directory and search, like before. But since I'm notified, I can retroactively delete the group if it was a spam group in hiding, so to speak.

Thanks,
Mark

On Wed, Sep 28, 2016 at 1:04 PM, HR Tech via Groups.io <m.conway11@...> wrote:

This is likely a bad idea but I'll share anyway. What about a credit card transaction like other sites use to verify identity? Does nextdoor do that? Or paypal when you set up an account and link it to a bank account? Do they take $1 and then reimburse it...? It's been a while since i needed to set one up.

Not ideal, and no idea if it would help.

Maria



locked Re: Temporarily disabled group creation

 

Hi All,

Group creation has been turned back on. Now, when a (listed) group is created, it won't actually appear in the directory or search until I approve it. This is mentioned on the page displayed after the group is created. Also, when a group's name or description is changed, I'm notified as well. In those instances, the changes are made instantly to the directory and search, like before. But since I'm notified, I can retroactively delete the group if it was a spam group in hiding, so to speak.

Thanks,
Mark

On Wed, Sep 28, 2016 at 1:04 PM, HR Tech via Groups.io <m.conway11@...> wrote:

This is likely a bad idea but I'll share anyway. What about a credit card transaction like other sites use to verify identity? Does nextdoor do that? Or paypal when you set up an account and link it to a bank account? Do they take $1 and then reimburse it...? It's been a while since i needed to set one up.

Not ideal, and no idea if it would help.

Maria



locked Re: Temporarily disabled group creation

Maria
 

This is likely a bad idea but I'll share anyway. What about a credit card transaction like other sites use to verify identity? Does nextdoor do that? Or paypal when you set up an account and link it to a bank account? Do they take $1 and then reimburse it...? It's been a while since i needed to set one up.

Not ideal, and no idea if it would help.

Maria


locked Re: Temporarily disabled group creation

christopher hallsworth <challsworth2@...>
 

Mark

How about some kind of challenge like a captcha but instead of typing characters in an image or numbers in an audio file ask questions only we humans know the answer to and not both humans and bots. A good example is a masths question.

On 28 Sep 2016, at 17:51, Mark Fletcher <markf@corp.groups.io> wrote:

Hi All,

I just temporarily disabled new group creation (you can still create subgroups). I did this because there's at least one persistent spammer creating new spam groups faster than I can delete them. You can look at the newest groups list right now to see a bit of the cesspool.

I need to clean out all the spam groups and put in place a system to prevent new ones. I'm thinking something like I need to approve a group before it appears in the search directory. Other suggestions are appreciated.

Thanks,
Mark


locked Re: Temporarily disabled group creation

Joseph Hudson <jhud7789@...>
 

Oh no Mark, this is not good. I agree. Definitely needs to be a system to approve groups before they're made available to the public. I will go ahead and do the same for subgroups just in case an owner or a moderator of another group, decides to start making spam subgroups on top of the parent group just to be on the safe side.

On Sep 28, 2016, at 11:51 AM, Mark Fletcher <markf@corp.groups.io> wrote:

Hi All,

I just temporarily disabled new group creation (you can still create subgroups). I did this because there's at least one persistent spammer creating new spam groups faster than I can delete them. You can look at the newest groups list right now to see a bit of the cesspool. 

I need to clean out all the spam groups and put in place a system to prevent new ones. I'm thinking something like I need to approve a group before it appears in the search directory. Other suggestions are appreciated.

Thanks,
Mark


locked Re: Temporarily disabled group creation

 

Mark,

I need to clean out all the spam groups and put in place a system to
prevent new ones. I'm thinking something like I need to approve a
group before it appears in the search directory. Other suggestions
are appreciated.
Victim of your own success. Alas.

If the spammers are using the group home page to promote something then keeping them out of the directory makes sense, but it may not be enough to discourage them from creating the groups. They may be posting links to those groups wherever they can.

And if their primary purpose is to use the group to send email spam it won't likely have much impact on them at all. But you have insight into what they're doing that I don't (I seem to have missed the "right now" window).

Spammers with botnets were the beginning of the end for Y!Groups, I recall when Gordon Strause mentioned that the bots were creating new accounts and new groups by the millions per day (not exaggeration). The CAPTCHA they had had in place had been cracked -- first by mechanical turk, then by automation. I sure hope you can find better ways of coping with the onslaught than Yahoo did.

Shal


On 9/28/2016 9:51 AM, Mark Fletcher wrote:
Hi All,

I just temporarily disabled new group creation (you can still create
subgroups). I did this because there's at least one persistent spammer
creating new spam groups faster than I can delete them. You can look at
the newest groups list right now to see a bit of the cesspool.

I need to clean out all the spam groups and put in place a system to
prevent new ones. I'm thinking something like I need to approve a group
before it appears in the search directory. Other suggestions are
appreciated.

Thanks,
Mark
--
Shal
https://groups.io/g/Group_Help
https://groups.io/g/GroupManagersForum

18181 - 18200 of 29183