Both of those seem problematic to me.
If you go the route of payment before approval I think the refund if rejected would have to be automatic - not subject to the group admin's volition.
But I recognize the difficulty in requiring approval first: that breaks the flow and incurs an arbitrary delay (for a moderator response). How would the applicant resume processing?
That might take something like the Existing Transfers list, where the applicant could come back later to process payments for group memberships as they are approved (we'd have to assume the user could apply to more than one paid group at a time). Maybe that becomes an account page for payments, where the user gets to manage them.
Perhaps "Not Paid", with characteristics similar to "Not Confirmed" - no posting or access. And perhaps also with the ability of the group admins to waive/reduce the required payment.
Eventually there's the whole flow when it is time for the member to renew. That probably also speaks to an account page to manage the paid subscriptions (which renewals are due soon?).