Skip to content

Commit

Permalink
xpath: remove unwanted sections from the body (#520)
Browse files Browse the repository at this point in the history
* Fix remove reader comments xpaths.py

* Removed suggest links

* Removed side stories
  • Loading branch information
felipehertzer committed Mar 11, 2024
1 parent 0d7d5a6 commit 3d0c934
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions trafilatura/xpaths.py
Expand Up @@ -143,7 +143,7 @@
or contains(@id, "premium-") or contains(@id, "paywall")
or contains(@class, "obfuscated") or contains(@class, "blurred")
or contains(@class, " ad ")
or contains(@class, "next-post")
or contains(@class, "next-post") or contains(@class, "side-stories")
or contains(@class, "related-stories") or contains(@class, "most-popular")
or contains(@class, "mol-factbox") or starts-with(@class, "ZendeskForm")
or contains(@class, "message-container") or contains(@id, "message_container")
Expand All @@ -154,8 +154,8 @@
# comment debris + hidden parts
'''.//*[@class="comments-title" or contains(@class, "comments-title") or
contains(@class, "nocomments") or starts-with(@id, "reply-") or starts-with(@class, "reply-") or
contains(@class, "-reply-") or contains(@class, "message")
or contains(@id, "akismet") or contains(@class, "akismet") or
contains(@class, "-reply-") or contains(@class, "message") or contains(@id, "reader-comments")
or contains(@id, "akismet") or contains(@class, "akismet") or contains(@class, "suggest-links") or
starts-with(@class, "hide-") or contains(@class, "hide-print") or contains(@id, "hidden")
or contains(@style, "hidden") or contains(@hidden, "hidden") or contains(@class, "noprint")
or contains(@style, "display:none") or contains(@style, "display: none") or contains(@class, " hidden") or @aria-hidden="true"
Expand Down

0 comments on commit 3d0c934

Please sign in to comment.