source code for VL-LTR #2791

psky1111 · 2023-05-11T07:02:34Z

intial pr for VL-LTR

paddle-bot · 2023-05-11T07:02:39Z

Thanks for your contribution!

CLAassistant · 2023-05-11T07:02:39Z

All committers have signed the CLA.

TingquanGao

代码问题很多，请仔细修改。

代码风格有问题，使用pre-commit格式化后再提交。
无关的改动删掉；
无关的注释、代码删掉；
逻辑一致、近似的复用之前的代码，或对之前代码进行扩展，避免代码冗余；
改动历史代码时，保证向前兼容。

TingquanGao · 2023-06-01T03:06:52Z

ppcls/arch/backbone/model_zoo/VL_LTR/logger.py

和之前的 PaddleClas/ppcls/utils/logger.py 有什么区别吗？为什么在这里添加logger文件。

TingquanGao · 2023-06-01T03:08:37Z

ppcls/arch/backbone/model_zoo/VL_LTR/download.py

这个看起来只是用来下载预训练权重，PaddleClas/ppcls/utils/download.py中已有类似方法可以复用。

TingquanGao · 2023-06-01T03:10:00Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

这个文件是做什么的？

TingquanGao · 2023-06-01T03:12:35Z

ppcls/arch/backbone/model_zoo/VL_LTR/VL_LTR_finetune.py

+        #k = self.wk(self.norm1k(kx)).reshape(Bk, Nk, self.num_heads, C //
+        #                                     self.num_heads).permute(0, 2, 1, 3)


无用的注释删掉。

TingquanGao · 2023-06-01T03:12:52Z

ppcls/arch/backbone/model_zoo/VL_LTR/VL_LTR_finetune.py

+            x = paddle.expand(qx,(qx.shape[0], kx.shape[0], qx.shape[-1]))
+            #x = qx.expand(qx.shape[0], kx.shape[0], qx.shape[-1])
+            x = paddle.concat((x,v),axis=-1)
+            #x = torch.cat((x, v), dim=-1)  # [Bq, Bk, 2*C]


类似无用的注释删掉

TingquanGao · 2023-06-01T03:23:17Z

ppcls/data/preprocess/batch_ops/batch_operators.py

@@ -58,7 +58,49 @@ def _mix_target(self, targets0, targets1, lam):
    def __call__(self, batch):
        return batch

+#######
+class MixupOperatorLT(BatchOperator):


能否在之前MimupOperator类的基础上改一下，复用之前的代码，看起来逻辑基本一致。

合并之前的mixup类

TingquanGao · 2023-06-01T03:25:08Z

ppcls/engine/evaluation/classification.py

+        batch_size = batch[1].shape[0]
+        #batch[0] = paddle.to_tensor(batch[0])


为什么要改这里？

ppcls/engine/train/train.py

TingquanGao · 2023-06-01T03:27:01Z

ppcls/engine/train/train.py

@@ -64,7 +64,7 @@ def train_epoch(engine, epoch_id, print_batch_step):
                for i in range(len(engine.optimizer)):
                    engine.scaler.minimize(engine.optimizer[i], scaled)
        else:
-            loss.backward()
+            loss.backward(retain_graph=True)


这个模型训练时必须保留反向计算图吗？

TingquanGao · 2023-06-01T03:28:12Z

tools/train.py

无关改动删掉

TingquanGao · 2023-06-01T03:43:07Z

这个PR请重新提到 dev_hackathon4 分支

TingquanGao · 2023-06-01T12:28:23Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

+def load_dygraph_pretrain(path=None):
+    if not (os.path.isdir(path) or os.path.exists(path + '.pdparams')):
+        raise ValueError("Model pretrain path {}.pdparams does not "
+                         "exists.".format(path))
+    param_state_dict = paddle.load(path + ".pdparams")
+    return param_state_dict
+
+
+def load_dygraph_pretrain_from_url(pretrained_url, use_ssld=False):
+    if use_ssld:
+        pretrained_url = pretrained_url.replace("_pretrained",
+                                                "_ssld_pretrained")
+    local_weight_path = get_weights_path_from_url(pretrained_url).replace(
+        ".pdparams", "")
+    return load_dygraph_pretrain(path=local_weight_path)


测试代码删掉

TingquanGao · 2023-06-01T12:28:47Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

+        return tensor
+
+
+def trunc_normal_(tensor, mean=0., std=1., a=-2., b=2.):


移到组网代码文件中

TingquanGao · 2023-06-01T12:29:04Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

+    return new_pos_embed
+
+
+def _no_grad_trunc_normal_(tensor, mean, std, a, b):


移到组网代码文件中。

TingquanGao · 2023-06-01T12:29:07Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

+}
+
+
+def interpolate_pos_embed(pos_embed_checkpoint: paddle.Tensor, new_patch_size, num_extra_tokens=1):


移到组网代码文件中。

TingquanGao · 2023-06-01T12:30:11Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

+from ......ppcls.utils.download import get_weights_path_from_url
+
+
+MODEL_URLS = {


移到组网代码文件中。

TingquanGao · 2023-06-01T12:31:35Z

ppcls/arch/backbone/model_zoo/VL_LTR/utils.py

这个文件可以删掉了。

TingquanGao · 2023-06-01T12:31:43Z

ppcls/arch/backbone/model_zoo/VL_LTR/__init__.py

这个文件可以删掉了。

TingquanGao · 2023-06-01T12:33:20Z

ppcls/data/dataloader/bpe_simple_vocab_16e6.txt.gz

不放在repo中

TingquanGao · 2023-06-01T12:36:56Z

ppcls/data/preprocess/batch_ops/batch_operators.py

@@ -58,7 +58,49 @@ def _mix_target(self, targets0, targets1, lam):
    def __call__(self, batch):
        return batch

+#######
+class MixupOperatorLT(BatchOperator):


合并之前的mixup类

source code for VL-LTR

4012c81

intial pr for VL-LTR

paddle-bot bot added contributor status: proposed labels May 11, 2023

update backbone

7ce0cd8

psky1111 mentioned this pull request May 21, 2023

【PaddlePaddle Hackathon 第四期】任务总览 PaddlePaddle/Paddle#51281

Closed

psky1111 added 3 commits May 29, 2023 23:59

update

719e1de

update final version

2b5b45e

update

288951e

TingquanGao reviewed Jun 1, 2023

View reviewed changes

fix pr problem

52e3c86

TingquanGao reviewed Jun 1, 2023

View reviewed changes

Ligoml removed the status: proposed label Sep 20, 2023

paddle-bot bot assigned cuicheng01 Feb 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

source code for VL-LTR #2791

source code for VL-LTR #2791

psky1111 commented May 11, 2023

paddle-bot bot commented May 11, 2023

CLAassistant commented May 11, 2023 •

edited

TingquanGao left a comment

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao commented Jun 1, 2023 •

edited

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

TingquanGao Jun 1, 2023

		#k = self.wk(self.norm1k(kx)).reshape(Bk, Nk, self.num_heads, C //
		# self.num_heads).permute(0, 2, 1, 3)

		batch_size = batch[1].shape[0]
		#batch[0] = paddle.to_tensor(batch[0])

		return tensor


		def trunc_normal_(tensor, mean=0., std=1., a=-2., b=2.):

		return new_pos_embed


		def _no_grad_trunc_normal_(tensor, mean, std, a, b):

		}


		def interpolate_pos_embed(pos_embed_checkpoint: paddle.Tensor, new_patch_size, num_extra_tokens=1):

		from ......ppcls.utils.download import get_weights_path_from_url


		MODEL_URLS = {

source code for VL-LTR #2791

Are you sure you want to change the base?

source code for VL-LTR #2791

Conversation

psky1111 commented May 11, 2023

paddle-bot bot commented May 11, 2023

CLAassistant commented May 11, 2023 • edited

TingquanGao left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TingquanGao commented Jun 1, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CLAassistant commented May 11, 2023 •

edited

TingquanGao commented Jun 1, 2023 •

edited