How does ssh key works when encrypted with ansible vault? How does ansible.builtin.meta: reset_connection task affects the ssh connection

Question

I am using ansible to perform some tasks on remote virtual machine. In my inventory file I have provided the ssh file as

all:
  hosts:
    vm:
      ansible_host: host_ip
      ansible_user: ssh_user_name
      ansible_ssh_private_key_file: path to ansible-vault encrypted ssh private key

If I use the 'ansible_ssh_private_key_file' in decrypted I don't face any error. But in encrypted form I face "Invalid format error".

Load key "path to private ssh key": invalid format

I have read couple of similar errors on internet. From where I understand it could be related to Windows-style (CRLF) line separators. The file must end with a single LF. https://serverfault.com/questions/854208/ssh-suddenly-returning-invalid-format

I have changed the file to Unix style (using notepad++). Now when I run my playbook it works till the reset connection task,

- name: Restart session (simulate log out and log in)
  ansible.builtin.meta: reset_connection

After performing the above task again the "invalid format error" starts showing up and I am not able to proceed. Internally how the ansible-vault encrypted ssh file is working? Why after reset connection task the ssh key file format is becoming invalid?

Answer 1

Q: "How does SSH key work when encrypted with Ansible vault?"

A: You can't encrypt ansible_ssh_private_key_file by Ansible vault. This option is used directly by SSH. SSH is not able to decrypt the Ansible vault. See

shell> ansible-doc -t connection ansible.builtin.ssh
  ...
- private_key_file
        Path to private key file to use for authentication.
        set_via:
          cli:
          - name: private_key_file
            option: --private-key
          env:
          - name: ANSIBLE_PRIVATE_KEY_FILE
          ini:
          - key: private_key_file
            section: defaults
          vars:
          - name: ansible_private_key_file
          - name: ansible_ssh_private_key_file
        default: null

The SSH connection plugin uses this option to create options for SSH. See

shell> man ssh
  ...
     -o option
             Can be used to give options in the format used in the configuration file.
             This is useful for specifying options for which there is no separate com‐
             mand-line flag.  For full details of the options listed below, and their
             possible values, see ssh_config(5).

                   AddKeysToAgent
                   AddressFamily
                   BatchMode
                   ...
                   IdentityFile
                   ...

See the below snippet from the source code

key = self.get_option('private_key_file')
        if key:
            b_args = (b"-o", b'IdentityFile="' + to_bytes(os.path.expanduser(key), errors='surrogate_or_strict') + b'"')
            self._add_args(b_command, b_args, u"ANSIBLE_PRIVATE_KEY_FILE/private_key_file/ansible_ssh_private_key_file set")

The option private_key_file is expanded by the Python method os.path.expanduser and concatenated to the string "IdentityFile=". You can see that there is no decryption involved.

Notes:

• OpenSSH provides you with the option to encrypt the private key with a passphrase. See ssh-keygen: What is the passphrase for?.

• Neither SSH (and as a consequence) nor Ansible provide you with the option of submitting the passphrase as an option or on the command line.

• See SSH agents on how to provide SSH passphrase.