I've lived in high crime areas, and SF doesn't have the kind of high crime that would actually deter me. The monotony of a homogeneous population is enough to keep me away from the Bay for as long as I can; and I know plenty of other talented people that feel the same
I've lived all along the West Coast from Tijuana to Seattle. There is no place along this coast which I would call homogeneous except the Bay. I would challenge your locals statement, but at this point, I've wasted too much time on No True Scotsman fallacies for this lifetime.
Either way, my community was one of the last diverse communities to get priced out of the Bay (around the early 2010s). Yet, you can find my community in any other populated place along the west coast. The bay definitely has a diversity issue when it comes to interesting people.
The thing about being priced out is that it can only happen if you aren't a homeowner. If you are a homeowner (or in such a household), the same effect is making you more than $100k a year richer. So you just have to like living with your parents.