Our "social contract" is not a valid contract in any normative sense of the term, since the supposed parties never agreed to it. The standard response is that you agree by choosing to remain in the country. But that only works if the government already owns the country and so is entitled to decide who is permitted to live there, a claim that is itself based on the supposed social contract, which makes the argument circular.

A better metaphor than a contract would be a peace treaty. Different people in a society have different, and sometimes inconsistent, views of what each is entitled to. Since none of us has an unlimited ability to enforce his view, in practice we settle for some compromise, the best outcome we think we can get given the existing balance of forces. Viewed as a contract that is a contract made under duress, so not binding--but then, the same is true of a peace treaty, since the usual alternative to signing it is having people continue to drop bombs on you.

via Questions for Independence Day Evening, Art Carden | EconLog | Library of Economics and Liberty.