One could simply turn this around, and ask, "Why doesn't the State just leave?" The "love it or leave it" bromide begs the underlying question: Who is entitled to occupy this space? Perhaps a hardcore statist would simply assume that the government rightfully owns everything, but anarcho-capitalists reject that assumption, given the State's history of conquest and plunder. We believe rightful property comes from homesteading and voluntary exchange, not conquest. A good anarcho-capitalist response may be, "The State doesn't rightfully own this property; people do."
http://www.ozarkia.net/bill/anarchism/faq.html