When we specify "--algorithms=sgd" in vw-hyperopt, does it run with adaptive, normalised and invariant updates?

Question

the confusion is because when we specify --sgd in vw command line, it runs classic sgd, without adaptive, normalised and invariant updates. So, when we specify algorithm as sgd in vw-hyperopt, does it run as classic or with special updates? Is it mandatory to specify algorithm in vw-hyperopt? Which is the default algorithm? Thank you.

Answer 1

Looking at the source code confirms that the meaning of --algorithm sgd here simply leaves the default alone.

This is different than vw --sgd. It doesn't disable the defaults by passing --sgd to vw. IOW: yes, the adaptive, normalized and invariant updates will still be in effect.

Also: you can verify this further by looking at the log file created by vw-hyperopt in the current dir and verify it has no --sgd option in it. This log includes the full vw command line it executes for training and testing, e.g:

2020-09-08 00:58:45,053 INFO  [root/vw-hyperopt:239]: executing the following command (training): vw -d mydata.train -f ./current.model --holdout_off -c ... --loss_function quantile