I don't think it is just the shell though. As soon as you start trying to do any automatic interchange of filesystem metadata, you are going to run into the same problems. Namely you have to have semantically important sets of bytes, and filesystems which allow arbitrary binary strings can insert semantic data into your exchange system, which necessitates extra escaping on all sides.
So the question is what assumptions we have, and how those should be enforced. Here are some humble recommendations.
1. Disable non-printing characters in filenames (this is encoding dependent and assumes a kernel aware of encoding).
2. Allow system administrators to configure additional rules, like "no leading/training whitespace, no starting with a hyphen, no internal tabs, no SGML special characters, UTF-8 only" and the like.
This would empower application developers to state what assumptions they rely on and eventual standards to emerge.
So the question is what assumptions we have, and how those should be enforced. Here are some humble recommendations.
1. Disable non-printing characters in filenames (this is encoding dependent and assumes a kernel aware of encoding).
2. Allow system administrators to configure additional rules, like "no leading/training whitespace, no starting with a hyphen, no internal tabs, no SGML special characters, UTF-8 only" and the like.
This would empower application developers to state what assumptions they rely on and eventual standards to emerge.