I've devised a solution for this particular situation. The hashtag #poll-notice is moderated so replies via email are queued. Then I use the rejection notice I created (NoEmailVotes) which explains that you can't vote via email. There are usually one or two that try to vote via email on each poll, but they quickly learn.
