Cool concept but the implementation is terrible. Just give me a button to input my own location. Not to mention the dot on the globe is about 3,000 miles from the street view it gives me.
That is likely to happen if you decline location permission and the IP geolocation fallback is inaccurate. If you try on your phone it should get an accurate GPS location. Adding a way to manually override the location by specifying latitude and longitude is on my to-do list.
If my apartment windows opened I could throw a rock to the spot where the streetview was taken from (though that would probably bother some people). While I agree a custom input location would be cool I suspect it's because you didn't ok the geo permission.