It's in the Constitution.
We went over this last summer and I proved you're wrong.
Here is how immigration is included in the Constitution:
Congress has the power to define and punish...Offenses against the Law of Nations. The Law of Nations includes the right of nations to forbid the entrance of foreigners or particular people for any reason.
Article I Section 8 Clause 9:
The Congress shall have Power...To define and punish Piracies and Felonies committed on the high Seas, and
Offenses against the Law of Nations;
Law of Nations: Book II, Chapter VII, Paragraph 94
The sovereign may forbid the entrance of his territory either to foreigners in general or in particular cases, or to certain persons or for certain particular purposes, according as he may think it advantageous to the state. There is nothing in all this that does not flow from the rights of domain and sovereignty: every one is obliged to pay respect to the prohibition; and whoever dares to violate it, incurs the penalty decreed to render it effectual. But the prohibition ought to be known, as well as the penalty annexed to disobedience: those who are ignorant of it, ought to be informed of it when they approach to enter the country. Formerly the Chinese, fearing lest the intercourse of strangers should corrupt the manners of the nation, and impair the maxims of a wise but singular government, forbade all people entering the empire: a prohibition that was not at all inconsistent with justice, provided they did not refuse human assistance to those whom tempest or necessity obliged to approach their frontiers. It was salutary to the nation, without violating the rights of any individual, or even the duties of humanity, which permits us, in case of competition, to prefer ourselves to others.
http://thehill.com/blogs/pundits-blog/immigration/333368-the-constitution-does-indeed-permit-immigration-caps-as-the
https://www.usconstitution.net/xconst_A1Sec8.html
http://tigerboard.com/boards/view.php?message=14958831