you're saying that any changes to EITHER resolution field, window or fullscreen, invoke use of GPU scaling?
That's my guess. I'm pretty sure that if the GPU gets involved in one, it gets involved in the other.
You might try asking on the DOSBox forum about the technical differences. I looked myself, and using 640x400 resolution with no scaler does seem to provide a subtle "filtered" look that normal2x does not.
Furthermore, if the resolution field is not equal to your native res, then going fullscreen will invoke GPU scaling anyway. Only it's your drivers that invoke it instead of DOSBox.
So, let's say you set fullscreen resolution to 800x600, but leave the scaler off. What will happen, if I understand it correctly, is first, DOSBox will make your GPU scale 320x200 up to 800x600, and then try to display 800x600 fullscreen. And then your drivers will make the GPU scale 800x600 up to whatever your native resolution is. Incidentally, this should cause black bars on all four sides, if aspect scaling is enabled both times.