Page MenuHomePhabricator

PytorchExperiment crashes when loading optimizer
Closed, ResolvedPublic


Although the test on the experiment logger passes, then loading a checkpoint created by a PytorchExperiment, the loading crashes when using the values loaded for the optimizer with:

File "/home/sebastian/workspace/meddec/meddec/expert_networks/TaskCT_Seb/", line 531, in <module>
File "/home/sebastian/workspace/trixi/trixi/experiment/", line 96, in run
  raise e
File "/home/sebastian/workspace/trixi/trixi/experiment/", line 75, in run
File "/home/sebastian/workspace/meddec/meddec/expert_networks/TaskCT_Seb/", line 213, in train
  current_loss, output_data = self.train_pass(data, epoch)
File "/home/sebastian/workspace/meddec/meddec/expert_networks/TaskCT_Seb/", line 247, in train_pass
File "/home/sebastian/.virtualenvs/pytorch/lib/python3.5/site-packages/torch/optim/", line 92, in step
  exp_avg.mul_(beta1).add_(1 - beta1, grad)

RuntimeError: Expected object of type torch.FloatTensor but found type torch.cuda.FloatTensor for argument #4 'other'
INFO:default-xuyannVZEL:Experiment exited. Checkpoints stored =)

A quick fix is not loading the optimizer parameters by

if "optimizer" in save_types:
    #optimizer_dict = self.get_pytorch_optimizers()


However, this of course does not load the optimizer then.