> Even RSA which has stood the test of time since 1977, is only considered 'safe' in so far as there's no algorithm better than brute force for factoring products of large primes

https://en.wikipedia.org/wiki/General_number_field_sieve but most of your point still stands. RSA is edging slowly closer to the abyss.