How do I configure opensearch as a logstash output properly, I am getting a host unreachable error

Question

so I am currently connected to a contabo hosted kubernetes cluster. On there I have running kafka and opensearch/opensearch dashboards deployements. I am trying to run logstash so that I can get the data from a kafka topic to opensearch, https://hub.docker.com/r/opensearchproject/logstash-oss-with-opensearch-output-plugin this is the image that I use for logstash (https://justpaste.it/47676 this is my logstash configuration). And the following is my opensearch configuration https://justpaste.it/a090p When I deploy logstash, I successfully get the data from the kafka topic, so my input plugin is working as expected, but the output is not, I am failing to output data to opensearch from logstash. The following is the logs from the logstash pod: https://justpaste.it/620g4 .

This is the output of "kubectl get services"

NAME                                  TYPE           CLUSTER-IP       EXTERNAL-IP   
PORT(S)                               AGE
dashboards-opensearch-dashboards      ClusterIP      10.96.114.252    <none>        5601/TCP                              5d20h
grafana                               ClusterIP      10.107.83.28     <none>        3000/TCP                              44h
logstash-service                      LoadBalancer   10.102.132.114   <pending>     5044:31333/TCP                        28m
loki                                  ClusterIP      10.99.30.246     <none>        3100/TCP                              43h
loki-headless                         ClusterIP      None             <none>        3100/TCP                              43h
my-cluster-kafka-0                    NodePort       10.101.196.50    <none>        9094:32000/TCP                        53m
my-cluster-kafka-1                    NodePort       10.96.247.75     <none>        9094:32001/TCP                        53m
my-cluster-kafka-2                    NodePort       10.98.203.5      <none>        9094:32002/TCP                        53m
my-cluster-kafka-bootstrap            ClusterIP      10.111.178.24    <none>        9091/TCP,9092/TCP,9093/TCP            53m
my-cluster-kafka-brokers              ClusterIP      None             <none>        9090/TCP,9091/TCP,9092/TCP,9093/TCP   53m
my-cluster-kafka-external-bootstrap   NodePort       10.109.134.74    <none>        9094:32100/TCP                        53m
my-cluster-zookeeper-client           ClusterIP      10.98.157.173    <none>        2181/TCP                              54m
my-cluster-zookeeper-nodes            ClusterIP      None             <none>        2181/TCP,2888/TCP,3888/TCP            54m
opensearch-cluster-master             ClusterIP      10.98.55.121     <none>        9200/TCP,9300/TCP                     19h
opensearch-cluster-master-headless    ClusterIP      None             <none>        9200/TCP,9300/TCP                     19h
prometheus-operated                   ClusterIP      None             <none>        9090/TCP                              25m
prometheus-operator                   ClusterIP      None             <none>        8080/TCP                              50m

What am I doing wrong and how do I establish this connection?

Answer 1

I figured it out. I think that it was expecting an ssl certificate and that is why it was refusing the connection. The way that I "fixed" this (because I don't need the ssl certification for this project for now) is that I changed the logstash configuration in this way.

  logstash.conf: |
    input {
        kafka{
          codec => json
          bootstrap_servers => "10.111.178.24:9092"
          topics => ["t_events"]
        }
    }
    output {
       opensearch {
          hosts       => ["https://10.102.102.109:9200"]
          ssl_certificate_verification => false
          user        => "admin"
          password    => "admin"
          index       => "logstash-logs-%{+YYYY.MM.dd}"
        }
    }

So I have added the "ssl_certificate_verification => false" line to the config and that enabled me to connect from logstash to opensearch and send the data. Now I have the data encryption aspect by using a https protocol but I am lacking the ssl authentication which I am fine with for this project.