you know, sometimes i wonder if my votes are being aggregated, too....it seems unlikely, but everysooften i feel the urge to look over my shoulder... (ok i admit it! i voted up hristov for being candid and having good intentions.)
Your votes are being aggregated, that's what the karma counter is all about.
But 'high karma' as such shouldn't be a factor, it can mean many things. Average karma per comment would be a much better way of judging peoples contributions.