Strange issue with terminal getting messed up: vim/coc-nvim/golang/gopls/zsh/gpg

Question

I am having a very annoying issue and I don't know exactly what's the problem. It could have loads of possible causes, so please bear with me trying to fix this.

I use vim to code. I use coc.nvim as plugin for code completion and navigation. Languageserver is set to golang and I use this config:

{
  "suggest.noselect": false,
  "diagnostic.errorSign": "✘",
  "diagnostic.warningSign": "!",
  "diagnostic.infoSign": "?",
  "diagnostic.checkCurrentLine": true,
  "coc.preferences.currentFunctionSymbolAutoUpdate": true,
  "coc.preferences.formatOnSaveFiletypes": [
    "javascript",
    "html",
    "json",
    "css",
    "scss",
    "go"
  ],
  "languageserver": {
    "golang": {
      "command": "gopls",
      "rootPatterns": [
        "go.work",
        "go.mod",
        ".vim/",
        ".git/",
        ".hg/"
      ],
      "trace.server": "verbose",
      "filetypes": [
        "go"
      ],
      "initializationOptions": {
        "usePlaceholders": true
      }
    },
    "golangci-lint-languageserver": {
      "command": "golangci-lint-langserver",
      "filetypes": [
        "go"
      ],
      "initializationOptions": {
        "command": [
          "golangci-lint",
          "run",
          "--out-format",
          "json"
        ]
      }
    }
  }
}

What's happening is that apparently randomly, if I open a .go file (it seems to happen mostly with go files which import libraries), my terminal gets irreparably messed up, usually completely messing up my editing session, not knowing if I was able to save or not my current buffer. Have to kill the shell!

Now, this started happening after at work we had been urged to use ssh keys stored on a yubikey. For this to work, we had to reconfigure the gpg-agent.

This is it's config:

enable-ssh-support
ttyname $GPG_TTY
default-cache-ttl 60
max-cache-ttl 60
pinentry-program /usr/bin/pinentry-curses

What I think is happening, is that during my vim editing session some ssh connection is opened (presumably to download github libs?). At the bottom appears something which looks like a prompt to enter the ssh key - but at this point I have lost the control of the terminal. Nothing works.

I think that is pinentry-curses which triggers that. But why is an ssh connection triggered and why does that mess up my terminal?

I run ubuntu 20.04 with i3, zsh with oh-my-zsh and powerlevel10k. I load these plugins for zsh, including the ssh-agent one which I think is needed:

plugins=(git z zsh-autosuggestions vi-mode ssh-agent)

I am really annoyed, any suggestion is highly appreciated!