Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can not reprocduce the performance reported in the paper #2

Open
ty4b112 opened this issue Oct 13, 2020 · 14 comments
Open

Can not reprocduce the performance reported in the paper #2

ty4b112 opened this issue Oct 13, 2020 · 14 comments

Comments

@ty4b112
Copy link

ty4b112 commented Oct 13, 2020

Hi,I run your code with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' to get the average experimental results of 10 iterations of BiGCN model on Twitter15 (running 100 iterations takes too much time).
I got the result 'Total_Test_Accuracy: 0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220', which has a big gap compared with the results reported in your paper '0.886 0.891 0.860 0.930 0.864'.

@zhenyuhe00
Copy link

I run with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' and got the result "Total_Test_Accuracy: 0.8276|NR F1: 0.7847|FR F1: 0.8360|TR F1: 0.8838|UR F1: 0.7941"

@zhenyuhe00
Copy link

As for Twitter16 dataset, I run with 'python ./model/Twitter/BiGCN_Twitter.py Twitter16 10' and got the result "Total_Test_Accuracy: 0.8642|NR F1: 0.7875|FR F1: 0.8585|TR F1: 0.9318|UR F1: 0.8635"

@17853535785
Copy link

I want to ask, you get this code can run successfully? Are there many mistakes and bugs?

@17853535785
Copy link

Hi,I run your code with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' to get the average experimental results of 10 iterations of BiGCN model on Twitter15 (running 100 iterations takes too much time).
I got the result 'Total_Test_Accuracy: 0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220', which has a big gap compared with the results reported in your paper '0.886 0.891 0.860 0.930 0.864'.

Hello, I would like to ask if you have successfully run all the code? If so, is it necessary to configure CUDA auxiliary when configuring the environment? Or just need CPU.

@17853535785
Copy link

As for Twitter16 dataset, I run with 'python ./model/Twitter/BiGCN_Twitter.py Twitter16 10' and got the result "Total_Test_Accuracy: 0.8642|NR F1: 0.7875|FR F1: 0.8585|TR F1: 0.9318|UR F1: 0.8635"

Hello, I would like to ask if you have successfully run all the code? If so, is it necessary to configure CUDA auxiliary when configuring the environment? Or just need CPU.

@zhenyuhe00
Copy link

Hi,I run your code with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' to get the average experimental results of 10 iterations of BiGCN model on Twitter15 (running 100 iterations takes too much time).
I got the result 'Total_Test_Accuracy: 0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220', which has a big gap compared with the results reported in your paper '0.886 0.891 0.860 0.930 0.864'.

Hello, I would like to ask if you have successfully run all the code? If so, is it necessary to configure CUDA auxiliary when configuring the environment? Or just need CPU.

Hi, I successfully ran the code by following main.sh. But one more thing, remember to create Twitter15graph/Twitter16graph/Weibograph dir in the data directory.

@isrczhu1
Copy link

isrczhu1 commented Dec 24, 2020

Hi,I run your code with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' to get the average experimental results of 10 iterations of BiGCN model on Twitter15 (running 100 iterations takes too much time).
I got the result 'Total_Test_Accuracy: 0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220', which has a big gap compared with the results reported in your paper '0.886 0.891 0.860 0.930 0.864'.

In execution getTwitter.py After that, the corresponding documents have been obtained. There are some mistakes . Did you make any other changes after you got the twitter15graph? Besides, is your environment the same as the author's? Thank you!

@isrczhu1
Copy link

isrczhu1 commented Dec 24, 2020

I run with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' and got the result "Total_Test_Accuracy: 0.8276|NR F1: 0.7847|FR F1: 0.8360|TR F1: 0.8838|UR F1: 0.7941"

In execution getTwitter.py After that, the corresponding documents have been obtained. There are some mistakes also after i get graph. Did you make any other changes after you got the twitter15graph? Besides, is your environment the same as the author's? Thank you!

@17853535785
Copy link

Hi,I run your code with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' to get the average experimental results of 10 iterations of BiGCN model on Twitter15 (running 100 iterations takes too much time).
I got the result 'Total_Test_Accuracy: 0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220', which has a big gap compared with the results reported in your paper '0.886 0.891 0.860 0.930 0.864'.

Hello, I would like to ask if you have successfully run all the code? If so, is it necessary to configure CUDA auxiliary when configuring the environment? Or just need CPU.

Hi, I successfully ran the code by following main.sh. But one more thing, remember to create Twitter15graph/Twitter16graph/Weibograph dir in the data directory.

Hello, would you like to ask if the packages you depend on are consistent with the version provided in the original text when you configure the environment?

@wj-gxy
Copy link

wj-gxy commented Feb 22, 2021

I want to ask, you get this code can run successfully? Are there many mistakes and bugs?

Because of the computer configuration, I run in win10 environment, can run.

@gymbeijing
Copy link

Hi, I got a error message saying that

[Errno 2] No such file or directory: '/home/ubuntu/fake_news_detection/BiGCN/data/Twitter15graph/516377014790782977.npz'

Does anyone know what is the npz file?

@Struggle-Forever
Copy link

Hi,I run your code with 'python ./model/Twitter/BiGCN_Twitter.py Twitter15 10' to get the average experimental results of 10 iterations of BiGCN model on Twitter15 (running 100 iterations takes too much time). I got the result 'Total_Test_Accuracy: 0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220', which has a big gap compared with the results reported in your paper '0.886 0.891 0.860 0.930 0.864'.
I ran this code and couldn't reproduce the results in the paper. In particular, the data of weibo is only about 90% accurate.

@rg399000
Copy link

嗨,我用"python ./model/Twitter/BiGCN_Twitter.py Twitter15 10"运行你的代码,以获得在Twitter15上BiGCN模型10次迭代的平均实验结果(运行100次迭代需要太多时间)。我得到的结果'Total_Test_Accuracy:0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220',与论文"0.886 0.891 0.860 0.930 0.864"相比,差距很大。

您好,我想问您是否已成功运行所有代码?如果是这样,在配置环境时是否有必要配置 CUDA 辅助设备?或者只需要 CPU。

嗨,我通过遵循 main.sh 成功运行了代码。但还有一件事,记得在数据目录中创建Twitter15graph/Twitter16graph/Weibograph dir。

您好,您想询问您所依赖的软件包是否与配置环境时原始文本中提供的版本一致吗?

您好,我也想问这个问题,请问您解决了吗?请问可否咨询一下您呢?

@luckyonetwo
Copy link

嗨,我用"python ./model/Twitter/BiGCN_Twitter.py Twitter15 10"运行你的代码,以获得在Twitter15上BiGCN模型10次迭代的平均实验结果(运行100次迭代需要太多时间)。我得到的结果'Total_Test_Accuracy:0.8582|NR F1: 0.8364|FR F1: 0.8537|TR F1: 0.9099|UR F1: 0.8220',与论文"0.886 0.891 0.860 0.930 0.864"相比,差距很大。

您好,我想问您是否已成功运行所有代码?如果是这样,在配置环境时是否有必要配置 CUDA 辅助设备?或者只需要 CPU。

嗨,我通过遵循 main.sh 成功运行了代码。但还有一件事,记得在数据目录中创建Twitter15graph/Twitter16graph/Weibograph dir。

您好,您想询问您所依赖的软件包是否与配置环境时原始文本中提供的版本一致吗?

您好,我也想问这个问题,请问您解决了吗?请问可否咨询一下您呢?

你好,我想问一下 local variable 'fold0_x_test' referenced before assignment 这个报错怎么解决呢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants