Complex numbers

[ni-chen]

Hi Jettify,

AdaBound and Adahessian doesn't work for complex numbers, which show the below messages

for AdaBound
File "optimizer.py", line 701, in step step_size.div_(denom).clamp_(lower_bound, upper_bound).mul_( RuntimeError: "clamp_scalar_cpu" not implemented for 'ComplexFloat'

For Adahessian
File "optimizer.py", line 433, in <listcomp> * torch.randint_like( RuntimeError: check_random_bounds handles only integral, floating-point and boolean types

Do you have any ideas to solve these issues? Thanks.

Bests,
Ni