You're correct on the reasons why "ghoti" cannot be pronounced like "fish," but what your explanation illustrates is that the mapping from English spelling to pronunciation is extremely nuanced - needlessly so.

A more direct phonetic writing system, like many other languages have, would make it much easier to learn how to read and write English.