Setup: there is an EKS cluster running with 2 worker nodes and there is a separate Elasticsearch cluster as well in the same VPC as the worker nodes. How can I / shall I open a connection between a logstash container in a pod on a worker node to the Elasticsearch cluster? I guess a service is needed for logstash but what type and how to set it up? Thanks for answering!
AWS Elastic Kubernetes Service: how to expose a container/pod to an Elasticsearch cluster inside AWS?
604 Views Asked by Gábor Varga At
1
There are 1 best solutions below
Related Questions in AMAZON-WEB-SERVICES
- S3 integration testing
- How to get content of BLOCK types LAYOUT_TITLE, LAYOUT_SECTION_HEADER and LAYOUT_xx in Textract
- Error **net::ERR_CONNECTION_RESET** error while uploading files to AWS S3 using multipart upload and Pre-Signed URL
- Failed to connect to your instance after deploying mern app on aws ec2 instance when i try to access frontend
- AWS - Tab Schema Conversion don't show up after creating a Migration Project
- Unable to run Bash Script using AWS Custom Lambda Runtime
- Using Amazon managed Prometheus to get EC2 metrics data in Grafana
- AWS Dns record A not navigate to elb
- Connection timed out error with smtp.gmail.com
- AWS Cognito Multi-tenant Integration | Ok to use Client’s Idp?
- Elasticbeanstalk FastAPI application is intermittently not responding to https requests
- Call an External API from AWS Lambda
- Why my mail service api spring isnt working?
- export 'AWSIoTProvider' (imported as 'AWSIoTProvider') was not found in '@aws-amplify/pubsub'
- How to take first x seconds of Audio from a wav file read from AWS S3 as binary stream using Python?
Related Questions in ELASTICSEARCH
- How does Elasticsearch do attribute filtering during knn (vector-based) retrieval?
- Elastic python to extract last 1hr tracing
- Elastic search not giving result when Hyphen is used in search text
- FluentD / Fluent-Bit: Concatenate multiple lines of log files and generate one JSON record for all key-value from each line
- Elasticsearch functional_score with parameter of type string array as input not working
- Elasticsearch - cascading http inputs from Airflow API
- AWS Opensearch - Restore snapshot - Failed to parse object: unknown field [uuid] found
- cluster block exception for system index of kibana
- What settings are best for elasticsearch query to find full word and half word
- OpenSearch - Bulk inserting Million rows from Pandas dataframe
- unable access to kibana
- PySpark elastic load fail with error SparkContext is stopping with exitCode 0
- How to use query combined to KNN with ElasticSearch?
- Facing logstash compatibility issues
- If the same document is ingested at two different times, how to have the same id in Elasticsearch
Related Questions in KUBERNETES
- Golang == Error: OCI runtime create failed: unable to start container process: exec: "./bin": stat ./bin: no such file or directory: unknown
- I can't create a pod in minikube on windows
- Oracle setting up on k8s cluster using helm charts enterprise edition
- Retrieve the Dockerfile configuration from the Kubernetes and also change container Java parameter?
- Summarize pods not running, by Namespace and Reason - I'm having trouble finding the reason
- How to get Java running parameters from Spring Boot running inside container in pod where no ps exist
- How do we configure prometheus server to scrape metrics from a pod with Istio sidecar proxy?
- In rke kube-proxy pod is not present
- problem with edge server registration in Eureka
- Unable to Access Kubernetes LoadBalancer Service from Local Device Outside Cluster
- Kubernetes cluster on GCE connection refused error
- Based on my experience, I've outlined the Kubernetes request flow. Could someone please add or highlight any points I might have overlooked?
- how to define StackGres helm chart "restapi" values to use internal LoadBalancer - AWS EKS
- Python3.11 can't open file [Errno 2] No such file or directory
- Cannot find remote pod service - SERVICE_UNAVAILABLE
Related Questions in AMAZON-EKS
- Terraform: How to add Windows worker nodes to eks cluster?
- SSL with GRPC on AWS EKS and Istio Ingress gives StatusCode.UNAVAILABLE
- Jenkins Helm backup fails with WebIdentityErr
- How to share an AWS NLB between two EKS Services?
- Airflow Unittest.cfg permission problems?
- How to enable automatic mTLS using istio mesh in AWS EKS?
- Cert-Manager dns01 challenge order pending
- Communication Between Pods in Different Cluster in K8s using yaml
- nginx docker connect() failed (110: Connection timed out) while connecting to upstream, same configuration works without docker, what could be wrong?
- AWS Elastic Kubernetes Service: how to expose a container/pod to an Elasticsearch cluster inside AWS?
- kubectl authentication to aws eks cluster
- Adding EKS managed windows node group failed. How to debug?
- Kubernetes AWS NLB Services Create Excessive Amount Of Security Group Rules
- Autoscaling Kubernetes based on number of Jobs on AWS EKS
- EKS LoadBalancer service not returning response outside from EKS
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
As a comment alluded to, you can do this via standard AWS Security Group adjustments. ie- Make sure that your worker nodes' security group allows Outbound connectivity to your Elasticsearch cluster on port 9200 or whatever port you're using, and make sure that your Elasticsearch Cluster Ec2 instances allow Inbound traffic from your Worker Nodes on port 9200. This assumes you're not using AWS's nifty new Security Group per Pod functionality, which allows you to get even more granular with your rules.
And then to test, you can exec into your logstash pod and curl your elasticsearch cluster endpoint. You can install curl if it's not already installed.