[robespierre]

the command immediately returns to the prompt without causing anything: setmon -x 1280x1024_60

Then that is working as intended:

-x The specified video format will take effect when the X server restarts (when one logs out from an X session) or during the next boot cycle.

To set the mode both immediately, and for future boots, you can use the -w option in place of -x.