My internet connection is fibre 100mb
From my main router 192.168.0.254 I see 80mb on all devices

I have another router set up with a VPN 192.168.1.200, because I am using a VPN I only see about 12mb on devices connected to that, (not a problem)

but if I copy a file from my NAS (non VPN) or download an update on my non VPN computers the TV on the VPN buffers or stops altogether. The copying dialog box only shows about 6mb/sec or less, which is limited by the drive speed.

As my main router is handling 100mb I would like to setup QOS on the router to try to stop the problems with my TV or is there a better way ?