Given that you seem to be viewing this as a first level screening, I think you need to make the 1000+ karma people review 10-20 pending comments before reading any article based on what's in the queue.
This would insures that the queue doesn't pile up. It would actually encourage good comments in the same ways that knowing a blog reads everything does. It would insure that if the system doesn't work or has problem, you'd hear about it immediately from long-term users.
I would implement it as "( ) endorse ( ) favorite" side-by-side so that the net effect is that great comments pop out of pending with better velocity.
Finally, the 1000+ karma rule for reviewers while convenient is probably a bad framing.
Instead, I'd say that 1000+ karma users are required to review and 100-1000 karma can volunteer to review. If you volunteer, your endorsements have to correlate with 1000+ over 95% of the time or they don't count. Obviously, you can randomly test that to whatever statistical significance you like.
Good luck with the new system -- this is the only site where comments are any good in my experience.
This would insures that the queue doesn't pile up. It would actually encourage good comments in the same ways that knowing a blog reads everything does. It would insure that if the system doesn't work or has problem, you'd hear about it immediately from long-term users.
I would implement it as "( ) endorse ( ) favorite" side-by-side so that the net effect is that great comments pop out of pending with better velocity.
Finally, the 1000+ karma rule for reviewers while convenient is probably a bad framing.
Instead, I'd say that 1000+ karma users are required to review and 100-1000 karma can volunteer to review. If you volunteer, your endorsements have to correlate with 1000+ over 95% of the time or they don't count. Obviously, you can randomly test that to whatever statistical significance you like.
Good luck with the new system -- this is the only site where comments are any good in my experience.