assert not x.requires_grad in Tensor.replace()?

[eugeneteoh]

Any reason why requires_grad needs to be False for replacing a tensor?
https://github.com/tinygrad/tinygrad/blob/master/tinygrad/tensor.py#L143

Here is my use case which doesn't work. It's quite common in RL to copy weights to target networks.

from tinygrad import nn

class Network:
    def __init__(self):
        self.test = nn.Linear(10, 1)

net = Network()
target_net = Network()
tau = 0.9

for param, target_param in zip(nn.state.get_parameters(net), nn.state.get_parameters(target_net)):
    target_param.replace(tau * param + (1 - tau) * target_param)

[eugeneteoh]

Unless there are any other functions that does the same thing as torch.tensor.copy_

[chenyuxyz]

check Tensor.assign and for example the usage in optimizers

tinygrad/tinygrad/nn/optim.py

Line 40 in 0c197b9

self.b[i].assign(self.momentum * self.b[i] + g) # NOTE: self.b[i] is zero on the first run, no if required

oh assign currently checks not required_grad too, maybe it's okay to remove

[eugeneteoh]

On it