Redshift change user password from workbench/j

Question

I have a weird problem when trying to change Redshift user password using SQL workbench/j.

Here is the command I am running: alter user someUser password 'someNewPassword';

It returns response "ALTER USER someUser successful"

However, when I try to login with the new password it fails, while the old password still works. Does anyone know why this could happen?