I can edit the terminal input line with full vim-mode in my shell (zsh). In fact, in normal mode I can even hit 'v' and edit the line in a full vim session to zip around however I want, do completions, etc. It's pretty close to the platonic ideal method of input I could imagine.
# edit long commands
autoload -U edit-command-line
bindkey -M vicmd v edit-command-line
That `edit-command-line` autoload script is part of zshcontrib and should be installed by default on any recent zsh distribution. I've never encountered a server I've ssh'ed into that didn't have it. Enjoy :)