I know what you're talking about. I've been organizing a photography competition for six years now and sometimes I feel like doing it wrong :-).
When it comes to judging photos, I help myself by submitting photos for judging in the form of a poll to a wide audience.
Interesting idea, thanks for that. May look into this idea more when I get some time back.
I'm glad you're interested. When the time is right, you'll find out what it entails :-)