Trying to use OpenVPN, but "TLS key negotiation failed"

This thread has been locked for further replies. You can start a new thread to share your ideas or ask questions.

Trying to use OpenVPN, but "TLS key negotiation failed"

This thread has been locked for further replies. You can start a new thread to share your ideas or ask questions.
Trying to use OpenVPN, but "TLS key negotiation failed"
Trying to use OpenVPN, but "TLS key negotiation failed"
2023-04-02 01:45:12
Tags: #VPN #OpenVPN
Model: Archer AX50  
Hardware Version: V1
Firmware Version: 1.0.11 Build 20220526 rel.63519(5553)

I'm trying to set up OpenVPN, but when I try to connect I get the following error:

 

Sat Apr  1 22:37:27 2023 WARNING: Compression for receiving enabled. Compression has been used in the past to break encryption. Sent packets are not compressed unless "allow-compression yes" is also set.
Sat Apr  1 22:37:27 2023 Note: --cipher is not set. OpenVPN versions before 2.5 defaulted to BF-CBC as fallback when cipher negotiation failed in this case. If you need this fallback please add '--data-ciphers-fallback BF-CBC' to your configuration and/or add BF-CBC to --data-ciphers.
Sat Apr  1 22:37:27 2023 OpenVPN 2.6.2 [git:v2.6.2/3577442530eb7830] Windows-MSVC [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] [DCO] built on Mar 24 2023
Sat Apr  1 22:37:27 2023 Windows version 10.0 (Windows 10 or greater), amd64 executable
Sat Apr  1 22:37:27 2023 library versions: OpenSSL 3.0.8 7 Feb 2023, LZO 2.10
Sat Apr  1 22:37:27 2023 DCO version: v0
Sat Apr  1 22:37:27 2023 TCP/UDP: Preserving recently used remote address: [AF_INET]179.98.xxx.xx:1194
Sat Apr  1 22:37:27 2023 UDPv4 link local: (not bound)
Sat Apr  1 22:37:27 2023 UDPv4 link remote: [AF_INET]179.98.xxx.xx:1194
Sat Apr  1 22:38:27 2023 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Sat Apr  1 22:38:27 2023 TLS Error: TLS handshake failed

Sat Apr  1 22:38:27 2023 SIGUSR1[soft,tls-error] received, process restarting

 

I've read around that it may be related to time settings on the router, but that's up to date. I tried rebooting the router, enabling and disabling the VPN server, every suggestion I found here in the forum. Nothing worked.

 

I'm using OpenVPN GUI 11.39.0.0/2.6.2.

 

My ISP doesn't have CG-NAT and I have DDNS set up.

 

What should I do to make this work? The router GUI makes it seem so simple...

  0      
  0      
#1
Options
3 Reply
Re:Trying to use OpenVPN, but "TLS key negotiation failed"
2023-04-02 03:43:37 - last edited 2023-04-02 03:46:17

  @julianomnunes 

 

Hi,

 

Out of curiosity I just updated my working OpenVPN 2.5.3 to the latest version OpenVPN 2.6.2 and afterwards it wouldn't work for me as well, although the error message I got was somewhat different.

 

Anyway, after I manually added the line "data-ciphers-fallback AES-128-CBC" to the *.ovpn file that was exported by my AX50 I was able to connect the VPN again.

 

So, you could either try that or downgrade to test with an older version of OpenVPN, which are still available for download on the official website.

 

  2  
  2  
#2
Options
Re:Trying to use OpenVPN, but "TLS key negotiation failed"
2023-04-02 04:02:09

  @woozle I just tried downgrading to version 2.5.9 (I thought I'd try the last revision before 2.6) and it worked! I still had to add the "data-ciphers AES-128-CBC" line, as the .ovpn provided by the router weirdly(?) doesn't have it.

 

One thing I'm going to try now is upgrading again to the latest version and adding "data-ciphers-fallback AES-128-CBC" to the config file.

 

You see, I'm fairly green at this and didn't realize until your reply that I had to actually set a cipher after "fallback". Silly me, I was reading the error message and trying to add only "add-ciphers-fallback" to the config file, only to fail immediately to connect.

 

Now I just have to figure out how to make my remote PC access my local PCs while connected through the VPN. So far I can't even ping them. Something to do with static routing, I think? I don't know, but I'm sure I'll get there! If you know of any tutorials, guides for dummies or something like that, I'd be really grateful :)

 

Thanks so much for your help!

  0  
  0  
#3
Options
Re:Trying to use OpenVPN, but "TLS key negotiation failed"
2023-04-02 17:51:21

  @julianomnunes 

 

The AX50 is running OpenVPN 2.3.8 and there will be more compatibility issues as the gap between the server and client versions widens.

 

If your local PC is running MS Windows with Windows Defender Firewall enabled (which is the default), then you need to make changes to it in order to allow access from remote devices.

For example, if you want to be able to "ping" the local PC through the VPN you have to allow ICMPv4 echo requests from remote IP addresses.

To do that run "wf.msc" (either via right-click [Start] -> Run or from a command line) and then do like illustrated in the screenshot below. (make sure you modify the entry that is actually in use, as indicated by the green check mark)
Whether you allow "any" IP address, a certain subnet (like 10.8.0.0/24) or just the one IP address of the remote PC is up to you.

 

Of course, if you want to access other services than just "ping" on the local PC via the VPN connection, then you will need to modify the "Inbound Rules" entries corresponding to that service in the Windows Firewall as well.

 

In case you have replaced Windows Defender Firewall with something else or don't use MS Windows, then you will have to figure out how to make those changes there.

 

  3  
  3  
#4
Options