Multiple ORIGINS for CORS not working in case of custom authorizer error

Question

I have CORS enabled in the application and have multiple origins, Its working fine with normal requests and error responses. But when I am getting some error in custom authorizer or deny the authorization. The CORS is not working for any status code(4XX or 5XX). I don't want to use * and have to handle multiple origins. I am using serverless

Here is the snippet from serverless.yml

resources:
  Description: My backend
  Resources:
    GatewayResponseDefault4XX:
      Type: 'AWS::ApiGateway::GatewayResponse'
      Properties:
        ResponseParameters:
          gatewayresponse.header.Access-Control-Allow-Origin: "'qwe.xyz.com,abc.xxxx.com,xyz.xxxx.xom'"
          gatewayresponse.header.Access-Control-Allow-Headers: "'qwe.xyz.com,abc.xxxx.com,xyz.xxxx.xom'"
        ResponseType: DEFAULT_4XX
        RestApiId:
          Ref: 'ApiGatewayRestApi'
    GatewayResponseDefault5XX:
      Type: 'AWS::ApiGateway::GatewayResponse'
      Properties:
        ResponseParameters:
          gatewayresponse.header.Access-Control-Allow-Origin: "'qwe.xyz.com,abc.xxxx.com,xyz.xxxx.xom'"
          gatewayresponse.header.Access-Control-Allow-Headers: "'qwe.xyz.com,abc.xxxx.com,xyz.xxxx.xom'"
        ResponseType: DEFAULT_5XX
        RestApiId:
          Ref: 'ApiGatewayRestApi'

Could you please help me to find a solution if I have multiple origins. How to handle multiple origins if default gateway responses for error codes