Azure Sphere is not able to connect to Edge device -- TLS handshake failed

Question

I am trying to configure and setup Azure Sphere device as child device to a IoT Edge device in nested fashion. Following is the procedure

1. Followed this link https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-nested-iot-edge?view=iotedge-2020-11 to forma a hierarchy of Azure IoT Edge devices (Top layer + low layer 1 + low layer 2), total 3 layers

2. Followed https://learn.microsoft.com/en-us/azure-sphere/app-development/setup-iot-edge?tabs=cliv1 to configure SPhere as child device of Top layer.

3. I am using Azure Samples, Azure IoT Example to connect to edge device. Made necessary changes in CMakeLists.txt and app_manifest.json file (Hostname as IP address of Edge Device, certificate as the root certificate (iotedge_config_cli_root.pem) generated at step 1 above)

4. Via Azure portal set the Azure Sphere device as child of edge device (top-layer).

5. Azure Sphere device is manually provisioned by following instructions in link https://learn.microsoft.com/en-us/azure-sphere/app-development/setup-iot-hub?tabs=cliv2beta

6. Azure Edge device, as per Step 6. Provide the tenant CA certificate of the Azure Sphere device to the IoT Edge device, appended tenant CA certificate in PEM format to iotedge_config_cli_root.pem.

7. Restarted the edge device and tried to sphere device tried to connect to edge device. But it was unsuccessful. Following were errors at edgeHub module

   <4> 2021-08-18 13:09:00.901 +00:00 [WRN] - TLS handshake failed., System.AggregateException: One or more errors occurred. (Authentication failed, see inner exception.) ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL. ---> Interop+Crypto+OpenSslCryptographicException: error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate --- End of inner exception stack trace --- at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, Byte[] recvBuf, Int32 recvOffset, Int32 recvCount, Byte[]& sendBuf, Int32& sendCount) at System.Net.Security.SslStreamPal.HandshakeInternal(SafeFreeCredentials credential, SafeDeleteContext& context, ArraySegment1 inputBuffer, Byte[]& outputBuffer, SslAuthenticationOptions sslAuthenticationOptions) --- End of inner exception stack trace --- at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception) at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest) --- End of stack trace from previous location where exception was thrown --- at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsServer(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.<AuthenticateAsServerAsync>b__68_2(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task``1 promise, Boolean requiresSynchronization) --- End of inner exception stack trace ---, 734368d8 <4> 2021-08-18 13:10:01.660 +00:00 [WRN] - TLS handshake failed., System.IO.IOException: Channel is closed, 5cc9b5ea

8. At Sphere side below was error

   IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started. INFO: Azure IoT Hub connection complete. Azure IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started. INFO: Azure IoT Hub connection complete. Azure IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started. /run/daa/41d781ff-519a-406d-b1dd-c4078f372dff [Azure IoT] Using HSM cert at /run/daa/41d781ff-519a-406d-b1dd-c4078f372dff [Azure IoT] Using HSM cert at /run/daa/41d781ff-519a-406d-b1dd-c4078f372dff [Azure IoT] Using HSM cert at /run/daa/41d781ff-519a-406d-b1dd-c4078f372dffINFO: Azure IoT Hub connection complete. Azure IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started. INFO: Azure IoT Hub connection complete. Azure IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started. INFO: Azure IoT Hub connection complete. Azure IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started. INFO: Azure IoT Hub connection complete. Azure IoT connection status: IOTHUB_CLIENT_CONNECTION_NO_NETWORK INFO: Azure IoT Hub connection started.**'

9. As per https://github.com/Azure/azure-sphere-samples/blob/master/Samples/AzureIoT/Troubleshooting.md the error points to These errors may occur if:

10. Inbound port 8883 has not been opened on the IoT Edge device. For more information, see Step 3 in Open IoT Edge gateway device ports for communication.

11. The sample is using an incorrect or invalid IoT Edge device root CA certificate.

12. The edgeAgent and edgeHub modules are not running on the IoT Edge device.

But reason 1 and 3 can be ruled out. Inbound port 8883 is open at edge device and all edge modules are running.

Need your help in resolving this issue. Note that the sphere device was able to connect to IoT hub directly.

Thanks, Jeevan

Answer 1

I hope the below tech-Community tutorial walkthrough might help you with the initial setup. AzureSphere Device-->IoT Edge-->IoT Hub, authored by Steve Patrick

Answer 2

Setting the environment variable 'OptimizeForPerformance' to false in the EdgeHub module has worked for people with the TLS handshake error (https://github.com/MicrosoftDocs/azure-docs/issues/15565 and https://github.com/Azure/iotedge/issues/159)

However i'm also experiencing this error with the OptimizeForPerformance being false, and i have not found a solution yet. I also believe that it could be a problem with the certificate communication between modules. You can try to comment out the Certificate settings section in your config.

I'm also curious what is contained in the certificates that you're using for the ca section?

Answer 3

I resolved this problem by fixing my CA certificates. The problem for me was that there was some invalid extensions in the certificate. I'm no expert, but my invalid certificate had some indication that it could not sign other certificates, even though that it was marked as a CA certificate in all other ways.

Another problem could also be using certificates that expire in 2038 or later in iotedge version 1.1. See https://learn.microsoft.com/en-us/azure/iot-edge/production-checklist?view=iotedge-2018-06&preserve-view=true#install-production-certificates