The fact that a lot of people aren't interested enough to vote is just that. They represent the majority of people who care enough to vote. Provided that all competent adults have the right to vote if they want to, I don't see any problem with that. Conditional on what, exactly? "Virtue," "wisdom," "truth" and "knowledge?" Are you aware that your arguments are taken direct from advocates of monarchy or aristocracy?