Mostly the amount of labour that goes into it. As this article shows (The $482 Difference Between Two Loafers – The Everything Guide to Shoes — New York Magazine), a shoe that costs under $100 can look like a shoe that costs over $500, but the (price) difference is in the details. I liked this article because it gives you a better sense of what goes into the making of a pair of shoes. I am sure the same could be said for most clothes nowadays.
Smart People I Know 
Interesting article, but I have noticed that expensive doesn’t always mean better; especially when it comes to durability. I’ve had relatively inexpensive hiking boots, that outlasted a much more expensive pair. (Also running shoes) I also find that footwear that is made in Brazil is both attractive & very well made. ; )
You are right. I always aim for best value myself.