I don't think the concern that Reply-To could be used to mailbomb someone would arise much in a reply-to-sender-only group, but if it did, it should be easy for a group owner to track down who did it and deal with them appropriately.
Good point, since the owner/mods will also have received the offending message.
And it brings to mind a pro-active mitigation: force all such messages to be moderated. And of course, make the fact that there's an inbound Reply-To prominently visible in the Pending list and pending message, possibly with the option of allowing the moderator to edit or remove it.
This idea is by analogy to the fact that all non-subscriber posts (if the group allows them) are moderated. Except here the (potentially) non-subscriber address is in the Reply-To field rather than the From field.
But I'll concede that's a bunch of work for a likely to be rarely used feature.