telepresence: error: connector.Connect: the traffic manager gRPC API timed out

Question

when I using this command in macOS Monteney 12 to connect to kubernetes(1.23) cluster:

telepresence connect

show error like this:

telepresence: error: connector.Connect: the traffic manager gRPC API timed out.  The current timeout 15s can be configured as "timeouts.trafficManagerAPI" in "/Users/example/Library/Application Support/telepresence/config.yml"

See logs for details (28 errors found): "/Users/example/Library/Logs/telepresence/connector.log"
If you think you have encountered a bug, please run `telepresence gather-logs` and attach the telepresence_logs.zip to your github issue or create a new one: https://github.com/telepresenceio/telepresence/issues/new?template=Bug_report.md .

I have tried to instatll socat in the server host. Still did not fixed this problem. The telepresence version is:

➜  ~ telepresence version
Client: v2.5.8 (api v3)
Root Daemon: v2.5.8 (api v3)
User Daemon: v2.5.8 (api v3)

anyone facing the same issue in the newest version of telepresence? This is the detail output:

2022-05-08 14:42:32.7014 info    connector/session : Connected to context context-reddwarf (https://106.14.183.131:6443)
2022-05-08 14:42:32.7648 info    connector/session : Connecting to traffic manager...
2022-05-08 14:42:32.7648 error   connector/session : unable to get APIKey: GetAPIKey: not logged in
2022-05-08 14:42:32.8477 info    connector/session : Existing Traffic Manager 2.5.8 not owned by cli or does not need upgrade, will not modify
2022-05-08 14:42:33.0033 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-5cb99c9fd6-cvh26_ambassador")
2022-05-08 14:42:34.1164 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-5cb99c9fd6-cvh26_ambassador")
2022-05-08 14:42:35.4853 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-5cb99c9fd6-cvh26_ambassador")
2022-05-08 14:42:37.7611 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-5cb99c9fd6-cvh26_ambassador")
2022-05-08 14:42:42.0067 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-5cb99c9fd6-cvh26_ambassador")
2022-05-08 14:42:47.9749 error   connector/session : Unable to connect to TrafficManager: the traffic manager gRPC API timed out.  The current timeout 15s can be configured as "timeouts.trafficManagerAPI" in "/Users/example/Library/Application Support/telepresence/config.yml"

I checked traffic-manager pod and found it was in the namespace ambassador. the I have already tried this command with kubernetes namespace:

telepresence connect -n ambassador

I also tried to tweak the timeout in config.yml:

timeouts:
  trafficManagerAPI: 90
  trafficManagerConnect: 90

still did not fixed this problem.

Answer 1

Today I am facing the similiar issue Telepresence connect: error: connector.Connect: the port-forward connection to the traffic manager timed out, fisrt I check the log /Users/xiaoqiangjiang/Library/Logs/telepresence/connector.log and found out the error like this:

2023-03-06 02:36:33.5346 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-56489c7cb7-pgdx8_ambassador")
2023-03-06 02:36:36.2675 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-56489c7cb7-pgdx8_ambassador")
2023-03-06 02:36:40.4036 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-56489c7cb7-pgdx8_ambassador")
2023-03-06 02:36:48.1923 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-56489c7cb7-pgdx8_ambassador")
2023-03-06 02:36:58.3995 error   connector/session : Error with k8sPortForwardDialer dial: unable to upgrade connection: pod not found ("traffic-manager-56489c7cb7-pgdx8_ambassador")

then I found this pod was terminating for a long time, then delete the pod like this:

kubectl delete pod traffic-manager-56489c7cb7-pgdx8 -n ambassador --grace-period=0 --force

rerun the connect command like this:

➜  telepresence sudo telepresence connect
Connected to context context-reddwarf (https://10.124.23.121:6443)

problem solved. this is the newest version info of telepresence:

➜  telepresence telepresence version
Client         : v2.11.1
Root Daemon    : v2.11.1
User Daemon    : v2.11.1
Traffic Manager: v2.11.1

hope this could help others.