GitHub action push to pantheon

Question

I'm trying to create an action that pushes changes to pantheon on every push so we can have more advanced workflows with GitHub but keep the convenience of dev-sites created on pantheon. I have set up 3 secrets

1. SSH_KEY (private key)
2. HOST_IP
3. PANTHEON_REPO (url)

And came up with this action after some Google-ing:

name: Push to pantheon
on: [push,pull_request]
jobs:
  pusher:
    name: Push to pantheon
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install SSH key
        uses: shimataro/ssh-key-action@v2
        with:
          key: ${{ secrets.SSH_KEY }}
          known_hosts: 'just-a-placeholder-so-we-dont-get-errors'
      - name: Adding Known Hosts
        run: ssh-keyscan -H ${{ secrets.HOST_IP }} >> ~/.ssh/known_hosts
      - name: Push changes
        run: |
          git remote add pantheon ${{ secrets.PANTHEON_REPO }}
          git config --local user.email "..."
          git config --local user.name "Github Action"
          git push pantheon ${GITHUB_REF##*/}

However the action fails at git remote add with the error:

Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I created the ssh key specifically for this and added the public key to pantheon.