Can sam-hq perform breakpoint training: continue training on the already trained sam-hq, or continue training on the official sam-hq weights

[YUANMU227]

Can sam-hq perform breakpoint training: continue training on the already trained sam-hq, or continue training on the official sam-hq weights？

Modifying the --start_epoch parameter will not allow continued training on the existing epoch_*.pth.

How to perform breakpoint training?

[bhack]

It seems that currently the restore restore_model logic is available only on eval:

sam-hq/train/train.py

Lines 361 to 373 in 3224888

	else:
	sam = sam_model_registry[args.model_type](checkpoint=args.checkpoint)
	_ = sam.to(device=args.device)
	sam = torch.nn.parallel.DistributedDataParallel(sam, device_ids=[args.gpu], find_unused_parameters=args.find_unused_params)
	if args.restore_model:
	print("restore model from:", args.restore_model)
	if torch.cuda.is_available():
	net_without_ddp.load_state_dict(torch.load(args.restore_model))
	else:
	net_without_ddp.load_state_dict(torch.load(args.restore_model,map_location="cpu"))
	evaluate(args, net, sam, valid_dataloaders, args.visualize)

[bhack]

@lkeab Can you share the sam-hq only network checkpoint so that we could eventually restore it for the train?