I think this could be a pretty good system, but it would require a far more robust moderation framework than we have right now, as well as a lot of thought. I'm just brainstorming here, but I think at a minimum we would need:

An elegant appeal system for when bad calls are made, maybe even run the appeals back through the same moderation loop as regular reports (with a rate limit to prevent abuse)
A way to (democratically?) remove people with a history of opinionated or incorrect moderation decisions from the system
A better modlog, allowing complete transparency rather than "mod banned this user"
A distribution engine for moderation that does not allow users to pick and choose what they moderate, to avoid the issue pointed out by @ShadowAether@sh.itjust.works
A way to prevent the same few terminally-online user(s) from moderating constantly, to the point that the whole instance slowly warps to their preferences
Perhaps a method of giving accounts with strong reputations more weight in mod decisions? This one's definitely arguable, but I can imagine a future where people create bots or alts to auto-approve their rule-breaking posts