Why? Because they are big and getting bigger. As this article shows, Marathons Grow in Popularity, but City Streets Have Limitations (NYTimes.com). So big cities are setting limits and rejecting alot of people.
Should you bother? Most definitely. The NYC Marathon is a fantastic experience. Imagine being in a 26 mile ticker tape parade in New York where you are the hero: that’s what it is like to run that city’s marathon. And anyone who qualifies for Boston has shown they are the top tier of marathon runners.
That said, there are lots of other good marathons out there not in big cities, like Columbus, Ohio, that are not as well known but very well run and very much worth attending. Big Sur on California is another. And Kiawah Island in South Carolina had a marathon in December which is a great time and great place to run. There are lots of great marathons to run. You just need to do some planning and you too can have a great experience.