Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fandom - Add support for comments #456

Open
upintheairsheep opened this issue Mar 9, 2023 · 2 comments
Open

Fandom - Add support for comments #456

upintheairsheep opened this issue Mar 9, 2023 · 2 comments

Comments

@upintheairsheep
Copy link

Fandom's API has a controller that grabs the comments of a page, which is paged. The comments cannot be accessed traditionally through the "AllPages" section as I have thought. The comments seem to be retained even after moderators turn off comments on a wiki.
Example:
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=8
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=7
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=6
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=5
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=4
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=3
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=2
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false&page=1
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=false
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=0
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=1
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=2
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=3
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=4
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=5
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=6
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=7
https://phobia.fandom.com/wikia.php?controller=ArticleCommentsController&method=getComments&title=Diagraphephobia&namespace=0&hideDeleted=true?page=8

This may be more suitable for a WARC with ArchiveBot than a WikiDump.

@nemobis
Copy link
Member

nemobis commented Jun 19, 2023

Once upon a time, these were just normal pages included in the XML dump, though it took millennia to download them one request at a time. Have you checked?

@nemobis
Copy link
Member

nemobis commented Jun 19, 2023

Some context https://community.fandom.com/f/p/3650285180289026549

The above answers are based on how the old migration script that we used in the past worked. For the upcoming forum retirement, we'll have to update that script

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants