If the protocol was properly designed, the proprietary nature of the server wouldn't be an issue as we could write a server implementing the protocol too. For instance, I don't have to worry about all of tarsnap being open source, because I know from the way the client is implemented that the server can't do a damned thing to sniff my data.
However, due to how much of an awful botch job the protocol is, even if the server was FLOSS, it would still be untrustable.