Topics

moderated Messages to +owner being wrongly routed to the whole group

Peter Martinez <Peter.Martinez@...>
 

Over on the Group Manager's Forum, it was suggested that I raise this problem here. Please read through the topic with the same name as this one on that group to see the full story, but in essence the + in the email commands is causing difficulty because some routes in the internet are stripping-out anything after a + in the local part of an email address. This results in a user who sends an email command finding it appears on the group broadcast, (and the command is not executed). The effect can be embarrassing in the +owner command, where a private message becomes public.

This practice by some intermediate hosts in the internet may well be "illegal" according to RFC 2821 but my intuition tells me it won't go away,so I would ask groups.io to consider changing the format of the email command. This can be done step-by-step, adding the new form alongside the existing + form then later deleting the + form when it becomes clear that there are no users still using it. I suggest something like <mygroups.command@groups.io> - the use of a dot in the local part of an email address is a well-established practice.

regards
Peter

 

On Wed, Sep 18, 2019 at 03:20 PM, Peter Martinez wrote:


This practice by some intermediate hosts in the internet
Not some hosts (plural), but single broken mailserver used by a part of BT users.

Peter Martinez <Peter.Martinez@...>
 

Lena:

This practice (of stripping the localpart after a +) may not be just happening at the single mailserver that we have found (synchronoss.net). .It may be a single TYPE of mailserver equipment or software from a single hardware or software supplier that could be fitted in many places. At the least I would ask groups.io to consider adding an alternative email command format (dot in place of +) so that groups that experience this problem can work around it.

regards
Peter

Peter Martinez <Peter.Martinez@...>
 

Lena:
Its true that we have only found one mailserver with this bug so far, but the type of mailserver with this bug could appear in other locations and I don't think it is safe to continue using a + in the email command.
regards
Peter

Duane
 

On Thu, Sep 19, 2019 at 08:09 AM, Peter Martinez wrote:
I don't think it is safe to continue using a + in the email command
If that's true, then it's not "safe" to use any punctuation in the local part.  Per RFC 2821, they should all be treated the same.

Duane

Dave Wade
 

Duane,

I folks are violating RFC2821 then they are not treating them the same. However, I think only “.” is safe..

Dave

 

From: main@beta.groups.io <main@beta.groups.io> On Behalf Of Duane
Sent: 19 September 2019 19:08
To: main@beta.groups.io
Subject: Re: [beta] Messages to +owner being wrongly routed to the whole group

 

On Thu, Sep 19, 2019 at 08:09 AM, Peter Martinez wrote:

I don't think it is safe to continue using a + in the email command

If that's true, then it's not "safe" to use any punctuation in the local part.  Per RFC 2821, they should all be treated the same.

Duane

Duane
 

On Thu, Sep 19, 2019 at 01:11 PM, Dave Wade wrote:
I folks are violating RFC2821 then they are not treating them the same. However, I think only “.” is safe.
Exactly.  If they're not following the standard for one, what's to say they'll follow the standard for others?  Why do you think that "." is safe?  Since it's part of the same standard, it may not be safe on some services.  It's up to the email services to follow standards so problems like this don't occur.  It shouldn't require a web site (or other service) to "work around" the problem they've created.

Duane

 

On Thu, Sep 19, 2019 at 06:09 AM, Peter Martinez wrote:
the type of mailserver with this bug could appear in other locations and I don't think it is safe to continue using a + in the email command.
Sounds like that's going to extremes. Anything bad can happen at any time. Doesn't mean you avoid doing everything because it's not safe.
 
--
J

Messages are the sole opinion of the author, especially the fishy ones.
My humanity is bound up in yours, for we can only be human together. - Desmond Tutu

Peter Martinez <Peter.Martinez@...>
 

Duane:

Dave says that a dot is safe because dots are used by everybody and they ARE safe - I have one in my own email address. We know that + CAN be unsafe because we found a situation where it IS unsafe. So lets switch from a + to a dot.

The buggy mailserver that we have found at btinternet is supplied and operated by Synchronoss Inc. in the U.S.A, so I think there may be many more of them around the world.

If you see a hole in the highway ahead, you drive round it so you don't break a wheel. You don't drive straight into the hole and risk breaking a wheel just because the regulations say that the highway department must keep the highway free of holes.

regards
Peter

Chris Jones
 

On Fri, Sep 20, 2019 at 07:41 AM, Peter Martinez wrote:
So lets switch from a + to a dot.
I was involved in some of the original investigation into this phenomenon, but I genuinely believe that changing the format of Groups.io email addresses is not the way to go, at least for the moment.

While it would probably be easy for Mark to implement any such change, that change would impact on any / all Groups.io Subscribers who have or may have the current address format in their contacts lists, and I do not see that as acceptable unless it really cannot be avoided, and in my view it is far too early so say that such a change is unavoidable. Furthermore, such a change would (IMHO) be a violation of the Principle of Least Astonishment.

Having said that, although the problem is not one of Mark's creation he might be better placed than any one of us to tackle Synchronoss with the aim of getting that organisation to ensure that their traffic - handling is compliant with the agreed standard.

To continue the motoring analogy, driving around the pothole is all very well as a short - term fix, but does nothing to address the fact that the pothole shouldn't be there is the first place. Circumvent the pothole and before long there will be a second pothole to navigate, and then a third and so on. In other words an unaddressed problem is more likely to spread; it certainly won't repair itself.

What happens when the next service - provider non - compliance arises? Find another work - around?

Mark; if by any chance you are willing to tackle BT / Synchronoss about this incorrect handling of traffic please let us know and one of us can post the exact details of when this problem arises and when it does not, because the current understanding is that it affects end users who are customers of a single (UK) Mail Service Provider, but even then under a single set of circumstances, albeit a very common set.

As Peter has noted previously there is reason to believe (hope!) that some details are working their way through to that MSP (BT) but with no certainty of a satisfactory outcome.

Chris

Dave Wade
 

Chris,

 

Yes it’s a single UK provider, however it’s the biggest UK mail provider, and as far as I can tell it has around 10 million customers….

 

Dave

 

From: main@beta.groups.io <main@beta.groups.io> On Behalf Of Chris Jones via Groups.Io
Sent: 20 September 2019 09:48
To: main@beta.groups.io
Subject: Re: [beta] Messages to +owner being wrongly routed to the whole group

 

On Fri, Sep 20, 2019 at 07:41 AM, Peter Martinez wrote:

So lets switch from a + to a dot.

I was involved in some of the original investigation into this phenomenon, but I genuinely believe that changing the format of Groups.io email addresses is not the way to go, at least for the moment.

While it would probably be easy for Mark to implement any such change, that change would impact on any / all Groups.io Subscribers who have or may have the current address format in their contacts lists, and I do not see that as acceptable unless it really cannot be avoided, and in my view it is far too early so say that such a change is unavoidable. Furthermore, such a change would (IMHO) be a violation of the Principle of Least Astonishment.

Having said that, although the problem is not one of Mark's creation he might be better placed than any one of us to tackle Synchronoss with the aim of getting that organisation to ensure that their traffic - handling is compliant with the agreed standard.

To continue the motoring analogy, driving around the pothole is all very well as a short - term fix, but does nothing to address the fact that the pothole shouldn't be there is the first place. Circumvent the pothole and before long there will be a second pothole to navigate, and then a third and so on. In other words an unaddressed problem is more likely to spread; it certainly won't repair itself.

What happens when the next service - provider non - compliance arises? Find another work - around?

Mark; if by any chance you are willing to tackle BT / Synchronoss about this incorrect handling of traffic please let us know and one of us can post the exact details of when this problem arises and when it does not, because the current understanding is that it affects end users who are customers of a single (UK) Mail Service Provider, but even then under a single set of circumstances, albeit a very common set.

As Peter has noted previously there is reason to believe (hope!) that some details are working their way through to that MSP (BT) but with no certainty of a satisfactory outcome.

Chris

Duane
 

On Fri, Sep 20, 2019 at 04:58 AM, Dave Wade wrote:
it’s the biggest UK mail provider, and as far as I can tell it has around 10 million customers
I would think a provider of that size would follow the RFCs.

Duane

Dave Wade
 

Duane,

One would hope so except It’s the UK equivalent of Ma Bell so its history is that it was the national government telephony provider. As such it created the rules as steps to prevent other competing.

I believe it sees its dominant position and breaking the rules as a way to mess with (foo bar) the competition. It has a history of behaving badly e.g.

 

https://www.bbc.co.uk/news/technology-13015194

 

its e-mail system is a total mess having originally been outsourced Yahoo and now the load is split between Yahoo and a German (I think) provider and frequently goes astray:-

 

https://home.bt.com/pages/email/index.html#targetText=We're%20about%20to%20move,make%20sure%20you're%20ready.

 

Much of the support based in India. A quick google on “BT Indian call centre” will show the problems people have…

… I suspect you have as much chance of getting it fixed as you have of getting Schrodinger’s cat to the moon dead and alive….

 

Dave

G4UGM

 

From: main@beta.groups.io <main@beta.groups.io> On Behalf Of Duane
Sent: 20 September 2019 11:21
To: main@beta.groups.io
Subject: Re: [beta] Messages to +owner being wrongly routed to the whole group

 

On Fri, Sep 20, 2019 at 04:58 AM, Dave Wade wrote:

it’s the biggest UK mail provider, and as far as I can tell it has around 10 million customers

I would think a provider of that size would follow the RFCs.

Duane

Chris Jones
 

On Fri, Sep 20, 2019 at 10:58 AM, Dave Wade wrote:
...as far as I can tell it has around 10 million customers….
In the group I moderate (>2100 Subscribers) 182 have btinternet.com addresses and a further 8 have btconnect.com one.

Dave w also wrote: I suspect you have as much chance of getting it fixed as you have of getting Schrodinger’s cat to the moon dead and alive….

I suspect that is certainly true if we have to go via the normal "call centre" route. :(  In this case Peter has gone via the "BT Community" from which the matter has (reportedly) been escalated.

The odd thing is that if a (btinternet) email is sent to a Groups.io "+" address from an email client then the + and what follows up to the @ is stripped off and the email is misrouted to the Topics page*. If, however, the same email is sent from BT Mail's web UI then it is routed correctly to whatever is after the +. I don't know if similar mishaps arise if the originating address is btconnect; I know of one Subscriber who has such an address so I might ask him if he could try a test message or two.

* If the Sub happens to be unmoderated then of course the message appears immediately.

Chris

Dave Wade
 

Chris,

 

I don’t find that odd at all. I also suspect that because of the way the “btinternet.com” address space is split that the result when sending from webmail may vary depending on which provider is servicing the mail.

I am pretty sure some users are still on the Yahoo server. There are also “btopenworld.com” and perhaps still some “talk21.net” addresses.

 I would also say, from experience that several web-mail based e-mail clients don’t understand that rules for the  left had side of the “@” so the user portion are different to the RHS or domain portion.

GMAIL allows me to ad “+” and an arbitrary string to my e-mail address, which is useful for tracking address leakage, but often its rejected as “invalid” when putting it in forms.

 

Dave

 

From: main@beta.groups.io <main@beta.groups.io> On Behalf Of Chris Jones via Groups.Io
Sent: 20 September 2019 12:21
To: main@beta.groups.io
Subject: Re: [beta] Messages to +owner being wrongly routed to the whole group

 

On Fri, Sep 20, 2019 at 10:58 AM, Dave Wade wrote:

...as far as I can tell it has around 10 million customers….

In the group I moderate (>2100 Subscribers) 182 have btinternet.com addresses and a further 8 have btconnect.com one.

Dave w also wrote: … I suspect you have as much chance of getting it fixed as you have of getting Schrodinger’s cat to the moon dead and alive….

I suspect that is certainly true if we have to go via the normal "call centre" route. :(  In this case Peter has gone via the "BT Community" from which the matter has (reportedly) been escalated.

The odd thing is that if a (btinternet) email is sent to a Groups.io "+" address from an email client then the + and what follows up to the @ is stripped off and the email is misrouted to the Topics page*. If, however, the same email is sent from BT Mail's web UI then it is routed correctly to whatever is after the +. I don't know if similar mishaps arise if the originating address is btconnect; I know of one Subscriber who has such an address so I might ask him if he could try a test message or two.

* If the Sub happens to be unmoderated then of course the message appears immediately.

Chris

 

Late replying to this.

I'm wary (used it right this time Shal!) of changing this, especially to a period. It's not like periods aren't interpreted differently by different mail servers. Gmail, for example, ignores all periods in email addresses. Next time you send an email to a friend with a gmail account, feel free to sprinkle some periods in the left side of their email address. Note that we do not ignore periods, even for Gmail accounts. If I had to do it over, I'd probably special case this to handle it, but it seems to only confuse people a few times a year.

Mark

 

Mark,


I'm wary (used it right this time Shal!) of changing this, ...
Indeed. ;-)

I too do not favor changing it.

Or even aliasing it. I thought of that but realized it wouldn't solve the problem - the people who need to use the alias wouldn't know about it. That way lies only more confusion.

If I had to do it over, I'd probably special case this to handle it,
but it seems to only confuse people a few times a year.
I'm not sure what you're thinking, but I thought perhaps you could notice when receiving a message:

IF (message delivered to Groups.io by btinternet.com) AND
(Received chain includes synchronoss.net)
THEN
Do something different.

The question is what to do. I initially thought you could look to the header To and/or Cc fields to determine if there was a command and what it was, but I think some of the messages I looked at had neither. Probably cases of Bcc, but I'm not sure about that.

So the only two actions I can think of would be to either force the message to Pending, and let the mods deal with it, or reject the message, with a message text saying that btinternet users must post via the web interface, not via SMTP client applications.

But either of those are pretty onerous for the 99.9% use case of a simple message posting. So maybe accept the message for posting if there is a To or CC that includes the group posting address without command.

Shal

Peter Martinez <Peter.Martinez@...>
 

Mark/Shal:

The fact that gmail ignores dots in the leftside of email addresses, is not relevant to this discussion. Gmail ONLY do this on emails coming into the gmail domain, and that's reasonable. This practice doesn't affect emails going to any other domain.

We are talking here only about interpreting emails coming into the groups.io domain. The format of the leftside on such emails is entirely at the discretion of groups.io. If groups.io assign special meaning to a leftside with an embedded + or an embedded dot, that doesn't interact in any way with what gmail might do with their incoming emails.

The problem in this case is that we know there are en-route hosts that (illegally) truncate leftsides of emails in transit, treating a "+" as marking the end of the leftside. OK, the proper solution is to get these en-route hosts to cease this practice, but what we are discussing here now is the possibility that groups.io could do something to avoid this bug (because we have a very low expectation of getting it fixed properly). Groups.io may be the only organisation that has chosen to embed a "+" within leftsides of addresses within it's own domain, so we may get no support for any campaign to get it fixed properly.

It shouldn't be difficult to implement. Just amend the code that looks for a "+ so it will accept a dot as marking the start of a commandword. If there ARE presently leftsides assigned within @groups.io that contain embedded dots, then its not as simple, but Mark will know how to deal with that.

As a trial, you could just ADD a process to detect a dot AS WELL AS "+" in the part of the code that parses for email commands. We can then check that groupname(dot)command works and solves the problem and there are no unexpected side-effects. The rest of the users carry on as before. Then it's just a case of amending the documentation and waiting until there are no remaining users who remember the purpose of a "+", then quietly remove the "+" from the code.

Needless to say I AM progressing this problem with binternet. I have had a fault report reference number for a week now but no-one has come back to me yet ...

I don't like the idea of patching the groups.io server to detect "synchronoss" and/or "btinternet" within headers and attempt a kludge. This could cause no end of confusion if btinternet solve the problem at their end, or change to another subcontractor. They have only been using synchronoss for a couple of years. Prior to that they were using Critical Path. This change may have been the onset of this problem of course!

regards
Peter

Dave Sergeant
 

Um, the idea of btinternet users only posting via the web interface is
surely a non starter. I have no statistics but I sense on the groups I
moderate that probably 80% of users, including a LOT of btinternet
users, only use email and most have never been anywhere near the web
interface.

It all comes down to the decision by Mark to use + as the only
distinguisher of an owners message was a poor one.

Dave

On 25 Sep 2019 at 18:38, Shal Farley wrote:

So the only two actions I can think of would be to either force the
message to Pending, and let the mods deal with it, or reject the
message, with a message text saying that btinternet users must post via
the web interface, not via SMTP client applications.

http://davesergeant.com

 

Peter,

Groups.io may be the only organisation that has chosen to embed a "+"
within leftsides of addresses within it's own domain, ...
Gmail is another.

It uses + in the user part as the lead-in for an alias address. That feature won't work properly with messages sent from a synchronoss afflicted service.

so we may get no support for any campaign to get it fixed properly.
You can add to your leverage by telling them that they're breaking a Gmail feature.
https://support.google.com/mail/answer/22370?hl=en
(the "Use Gmail aliases" section at the bottom of the page)

Shal