I can’t say if it was a lie. Maybe it was a fairytale. Something too good to be true but something many of us including me wanted to believe in. Whatever your thoughts, I recommend you read this strong critique on it: Where Did the Long Tail Go? by Ted Gioia. If you are a true believer, Gioia will get you rethinking it.
As for me, I think part of the problem is that online services nudge or even push us to the short tail. There are advantages to them when it comes to selling us more of the short part of the curve in red. We need services and aggregators to get our attention to spiral outwards and look at things we never considered before. Spotify still does that to an extent when it builds me playlists.
Another part of the problem is the willingness of people to get out of their comfort zone and explore the long tail. Again, Spotify will recommend music lists to me, but I often find myself sticking with the tried and true. Services need to better encourage people to try new things or make it easier to try new things. Give people options, but in a smart way. I know it can be done. I hope it will be done.