I have read in an older GnuPG manual (which I can't retrace) that it is possible to copy gpg.exe to the application's main folder (and pass the correct path in code), so that the users do not need to install GnuPG on their PC to use the application. I just tested(with GpgAPI) this but as expected some of the libraries are missing, as stated in the error messages I got.
So I'm wondering if the concept is nonetheless acceptable and feasible, and which files should I integrate to my app(Winform)?
I suggest you to use a better gpg/pgp library instead of the old and unmaintained gpgapi.
You can use pgpsharp (https://github.com/soukoku/pgpsharp) or starksoft-aspen (https://github.com/bentonstark/starksoft-aspen)