Answering my own question, I seems I've worked it out (for connections to AWS, if you use IPSEC you can use serverless AWS components, for any other VPN type you'd have to run a server of your own, and do a good deal extra networking work in AWS).

Just before I start, the TP-link makes some things hard which should be simple:

- There are no useful logs of any activity (or lack thereof) - so debugging is almost impossible (trial and error is a more likely strategy)

- The IP range of the tplink can't be within the IP range of the remote side of the VPN (an unnecessary limitation, and very annoying)

- You can't setup two VPNs with the same tplink subnet IP range (so can't have dual redundant VPNs from the tp-link to AWS or anywhere else) - also an unnecessary and annoying limitation

On a more positive note, NAT traversal is something IPSEC generally struggles with, but the tp-link makes it easy (automatic, actually)

However, settings that work are as follows. You'll need to have downloaded a "VPN config" for the AWS side. There's a "generic" manufacturer which gives you all the information you need. AWS always creates two tunnels, you can use either (but not both):

- Policy Name: <anything>

- Mode: Lan-to-lan

- Remote gateway: The IP of the AWS end of the connection (pick one of the two tunnels AWS always creates)

- WAN: The wan port of your tp-link

- Local subnet: The IP range of your local subnet (sadly, unlike a lot of IPSEC, you can't put 0.0.0.0/0 in here)

- Remote subnet: The IP range on the AWS side (can't contain the local subnet, and can't be 0.0.0.0/0)

- Pre-Shared key: Get this from the AWS VPN config for the tunnel you chose

In the advanced settings:

Phase 1 Settings:

- Proposal: sha1-aes256-dh2

- (other proposals empty)

- Exchange Mode: Main Mode

- Negotiation Mode: Initiator mode

- Local ID type: IP address

- Remote ID type: IP address

- AS lifetime: 28800

- DPD: enable

- DPD interval: 10 (this should match the DPD interval in the AWS config)

Phase 2 Settings:

- Encapsulation Mode: Tunnel

- Proposal: esp-sha1-aes256

- (other proposals empty)

- PFS: dh2 (should match the Perfect Forward Secrecy group in the AWS config)

- S Lifetime: 28800

Notes:

- NAT seems to work without needing to set Local or Remote "types". In my setup it's something like [Tp-link -> Network -> NAT router -> ppoe -> Internet] and it seems to work fine (you do need port forwarding from the Internet back to the tp-link on ports 500 and 4500 though)
- The local and remote subnet things are a pain, and forces a whole load of network design that shouldn't be required. The only real solution is to limit the size of the CIDR in AWS and exclude the tp-link networks (or use different IP ranges for the tp-link networks - eg. 10.x.x.x in AWS and 192.168.x.x on the tp-link). It's a stupid limitation that shouldn't exist, but unless tp-link fix it, we're stuck with the problem.

- On the AWS side, you can't have a "dynamic" VPN, as there's no BGP client on the tp-link. As such, you need static routes to make the VPN work.

- The local and remote subnet options affect routing on the tp-link, but don't affect routing at all on the AWS side (AWS behaviour is better in this regard, it would be better for the tp-link to setup the VPN separately from configuring routing, but you can't have everything I suppose)

- You need to setup routing in AWS to send traffic to the tp-link network "down" the VPN. That's just regular AWS networking stuff, dependent on how you do your networking, and so I won't cover it here