sftp and sshfs ask me for my password (and work properly after I enter it), whereas ssh and scp don't ask for a password and just work. When I force the use of public key authentication instead of password authentication, like this
I'm integrating a remote server into my desktop menus.
First thing I did was putting a .ssh/authorized_keys files so the server will recognize my computer and not ask me for a password. (Yes, this is secure, there's public key cryptography involved.)