I think there is something to be said for making a system where everything is allowed except that which is specifically excluded. Otherwise, when you run into a grey area...there's subjectivity.
People yell about their rights a great deal, but what it really boils down to is what are you NOT allowed to do.
For instance...I have "Freedom of Speech"....but I don't really...because laws have been passed limiting my freedom of speech. I can't make a sexist joke at the workplace. There is a law forbidding it. So, really, what does the "Freedom of Speech" give me. It gives me the freedom to speak, save where it is in violation of the law. If I wasn't given that right...I'd still have the freedom to speak, except where violated by law.