Hey @BLyon,

I would first recommend that you double-check that all the wires are properly connected and there are connections that may have come loose when installing the switch to the box.

We have seen a similar behavior exhibited in the past whenever the neutral wire becomes loose or disconnected, and it is not uncommon for the connection to become lose when putting the switch back into the box.