@NetworkInPlux 

ER605 won't use IPSec VPN as the WAN. So clients in LOCATION B will try to get Intenret through the"Untrust LAN", that's not correct.

You can try L2TP LAN-to-LAN connection on these two TP-Link routers. Then set up a policy routing on Location B router(as VPN client). On Policy routing you can force the clients using VPN to obtaion Intenret(set up VPN tunnel as the clients' WAN).

But you know, I did not really test it. I just "think" why it did not work and how to solve it. Don't blame me if it's not the issue XD

Thanks for your suggestion, @Somnus , based on what you wrote i have been playing around with the two ER605 units a little more.

I have now also set them up as L2TP server on Location A, and client on Location B, while both units have only an IP address and a subnet mask on their WAN port. This also works in part, the units will set up an L2TP VPN nicely. And i am able to ping the internet router through the VPN. DNS requests from hosts in Location B are resoved by the firewall in location A through the VPN. 

However web browsing or reaching anything else on the internet past the Internet Firewall on location A from a client on Location B, on the the far side of the VPN behind one of the ER605, seems to still not work.

I suppose it would be more logical if the WAN port on the unit in Location B directly worked as a dialup for the L2TP connection. But if i have the L2TP setup described above working, and change Location B's ER605 from VPN client, to the WAN port acting directly as a dial up WAN port to the L2TP server on the ER605 in Location A, i can not get it to connect at all. 

It is strange to me, as it seems like a pretty common use for these VPN routers, to extend your private network over an untrusted network, and in my case to share the internet connection infrastructure and existing firewall security from Location A with Location B over it. But somehow i am not managing to get this to work properly with these units.

  @NetworkInPlux 

I am completely lost here. I set up a firewall policy allowing all traffic in all directions on both units.

After this i have tried setting the two units up as L2TP client and server, the tunnel estabishes correctly, but on the client side none of the clients on the LAN can reach anything on the other side.

I have also tried setting up an OpenVPN connection between the two units, with the server side set to full mode. Again the tunnel sets up correctly, but none of the hosts on the lan side of the client unit can reach anything on the server side.

I just need to set up a VPN across a foreign network, to extend our HQ LAN and share our internet infrastructure with the second location. I did not think it would be this much of an issue to accomplish this. Can anyone point me in the right direction, so i can extend my headquarters LAN over a foreign LAN using these VPN routers? Or should i bin them?