What will happen If I remove some extensions folder to remove false positive CVE?

Question

I'm using Code-Server to create a container image based on Code-Server and other tools. When I use grype scanner, I'm getting a lot of False Positive vulnerabilities related to the extensions folder. These are some of them:

False positive CVE image

I'm wondering if there is a consequence if I decide to remove some of those extensions folder that I don't use, for example perl, ruby, etc.

Thanks !!