Business Community > Gateways > IPSEC VPN not possible with Windows built in client
< Gateways

IPSEC VPN not possible with Windows built in client

IPSEC VPN not possible with Windows built in client

IPSEC VPN not possible with Windows built in client
IPSEC VPN not possible with Windows built in client
a week ago - last edited a week ago
Tags: #VPN #VLAN & Multi-Networks #Firmware Update
Model: ER7412-M2  
Hardware Version: V1
Firmware Version: 1.0.1 Build 20240719 Rel.56423

I am trying to figure out what is wrong here but it seems this GW with this firmware is not capable of setting up simple L2TP/IPSEC connection or IKEv2 connection, with the Windows 11 built in client.

Also this device firmware seems to be more than 1 year old and I can not find any updates for it.

 

I was able to make same VPN working with bigger GW device. So is this one somehow disabled, or just abandoned from support?

 

I am using it in standalone mode. And trying to create simple client to Network encrypted VPN.

 

Some logs samples:

 

1 2025-08-28 19:18:36 IPsec WARNING 2.5G WAN/LAN1: Phase 1 of IKE negotiation failed. (Peers=192.168.100.116<->192.168.1.100, Error=14)
2 2025-08-28 19:18:06 IPsec NOTICE 2.5G WAN/LAN1: IKE negotiation began in responder mode. (Mode=Main Mode, Peers=192.168.100.116<->192.168.1.100)

 

There is no NAT, I am excercising in my local network! But I need to install this device to the client very soon!

 

Seems like Windows 11 uses IKEv2 and the automatic created IPSEC policy for the L2TP server can not be edited. And this makes it incompatible with Windows client settings.

 

If I try to use IKEv2 then I get in troubles as windows 11 support either user/pass or certificate. And tplink support only pre shared key.

 

Before several months I was able to setup the biggest GW model without troubles, and till that day it is working fine.

I do not want to use 3rd party VPN clients. And the reason that I bought this device was to make it working the same way.

  0      
 
  0      
 
#1
Options
1 Accepted Solution
Re:IPSEC VPN not possible with Windows built in client-Solution
a week ago - last edited a week ago

  @Clive_A 

Hello,

Thank you for your reply. I was able to resolve the problem with the most unexpected thing ever .....

Anyway the problem with your advice is - if you do use windows built in client + tp link L2TP/IPSEC you have no options from the omada side.

If you pick from the menu encryption, it automatically creates an IPSEC policy which you can not touch/modify.

Now on the windows side you can touch some things but very limited, close to useless.

So either it works out of the box or fails...

Same story if you try to use IKEv2. Now you have tons of options on the tplink side except you have ONLY pre-shared keys.

And on the Windows side you have almost all supported EXCEPT pre-shared keys.......... so basically they are incompatible ......

Wireshark could be a good option but now not needed.

So after some hours ( like 8 or more ) with experiments I decided to try to connect to another device which I know is working. First of course checked and compared all settings from both sides. And they looked all the same. Then tried to connect to this device and it again failed ......... Then I realized it may be something on the host.

So fastly set up another PC and I was able to connect to the other/remote side that just failed with the first PC.... Then tried the same PC with local tplink which is in question here. Then again it connected right away.....

Then on the first PC which initially failed played some time with firewalls and settings with no good results.

Then just out of nowhere decided to check the IP settings, and found something very interesting. There were some leftover settings with IP address from different networks.

I am on the x.x.100.x network but the IP address was from the x.x.1.x network....... cleared the manual settings and left it by DHCP and it immediately got connected to the VPN.....

So the L2TP without encryption was working ok with these strange settings, but the encryption got broken somehow.

I am on the same local network with the PC and the tplink ( WAN port) all the time with working ping and everything.

So somehow these two IPs on the same adapter from which I was trying to connect got phase one of the IKE broken.

 

Recommended Solution
  0  
  0  
#3
Options
3 Reply
Re:IPSEC VPN not possible with Windows built in client
a week ago - last edited a week ago

  @P2000 

If you can provide a Wireshark result, I might give an answer to the reason or provide some further suggestions.


This log shows that the first IKE key exchange phase has failed to negotiate with the peer. 

Maybe you could try different encryptions to figure out which Windows supports.

Recommend IKEv2. 

  1  
  1  
#2
Options
Re:IPSEC VPN not possible with Windows built in client-Solution
a week ago - last edited a week ago

  @Clive_A 

Hello,

Thank you for your reply. I was able to resolve the problem with the most unexpected thing ever .....

Anyway the problem with your advice is - if you do use windows built in client + tp link L2TP/IPSEC you have no options from the omada side.

If you pick from the menu encryption, it automatically creates an IPSEC policy which you can not touch/modify.

Now on the windows side you can touch some things but very limited, close to useless.

So either it works out of the box or fails...

Same story if you try to use IKEv2. Now you have tons of options on the tplink side except you have ONLY pre-shared keys.

And on the Windows side you have almost all supported EXCEPT pre-shared keys.......... so basically they are incompatible ......

Wireshark could be a good option but now not needed.

So after some hours ( like 8 or more ) with experiments I decided to try to connect to another device which I know is working. First of course checked and compared all settings from both sides. And they looked all the same. Then tried to connect to this device and it again failed ......... Then I realized it may be something on the host.

So fastly set up another PC and I was able to connect to the other/remote side that just failed with the first PC.... Then tried the same PC with local tplink which is in question here. Then again it connected right away.....

Then on the first PC which initially failed played some time with firewalls and settings with no good results.

Then just out of nowhere decided to check the IP settings, and found something very interesting. There were some leftover settings with IP address from different networks.

I am on the x.x.100.x network but the IP address was from the x.x.1.x network....... cleared the manual settings and left it by DHCP and it immediately got connected to the VPN.....

So the L2TP without encryption was working ok with these strange settings, but the encryption got broken somehow.

I am on the same local network with the PC and the tplink ( WAN port) all the time with working ping and everything.

So somehow these two IPs on the same adapter from which I was trying to connect got phase one of the IKE broken.

 

Recommended Solution
  0  
  0  
#3
Options
Re:IPSEC VPN not possible with Windows built in client
a week ago

  @P2000 

P2000 wrote

 

  @Clive_A 

Hello,

Thank you for your reply. I was able to resolve the problem with the most unexpected thing ever .....

Anyway the problem with your advice is - if you do use windows built in client + tp link L2TP/IPSEC you have no options from the omada side.

If you pick from the menu encryption, it automatically creates an IPSEC policy which you can not touch/modify.

Now on the windows side you can touch some things but very limited, close to useless.

So either it works out of the box or fails...

Same story if you try to use IKEv2. Now you have tons of options on the tplink side except you have ONLY pre-shared keys.

And on the Windows side you have almost all supported EXCEPT pre-shared keys.......... so basically they are incompatible ......

Wireshark could be a good option but now not needed.

So after some hours ( like 8 or more ) with experiments I decided to try to connect to another device which I know is working. First of course checked and compared all settings from both sides. And they looked all the same. Then tried to connect to this device and it again failed ......... Then I realized it may be something on the host.

So fastly set up another PC and I was able to connect to the other/remote side that just failed with the first PC.... Then tried the same PC with local tplink which is in question here. Then again it connected right away.....

Then on the first PC which initially failed played some time with firewalls and settings with no good results.

Then just out of nowhere decided to check the IP settings, and found something very interesting. There were some leftover settings with IP address from different networks.

I am on the x.x.100.x network but the IP address was from the x.x.1.x network....... cleared the manual settings and left it by DHCP and it immediately got connected to the VPN.....

So the L2TP without encryption was working ok with these strange settings, but the encryption got broken somehow.

I am on the same local network with the PC and the tplink ( WAN port) all the time with working ping and everything.

So somehow these two IPs on the same adapter from which I was trying to connect got phase one of the IKE broken.

 

I cannot provide a fix or targeted solution without anything in detail. There is no diagram or anything. That was a blind guess suspecting that you have a wrong config. 

Anyway, good to know you find the problem yourself.

 

Your description still indicates a problem with your config and diagram issue. IP subnetting or design. I don't have comment on "IKE broken" that part. 

  0  
  0  
#4
Options

Information

Helpful: 0

Views: 128

Replies: 3

Tags

VPN VLAN & Multi-Networks Firmware Update
About Us
Press
Copyright © TP-Link Systems Inc. All rights reserved.