Skip to content

A collection of domains used for shortlink-services like bitly

License

Notifications You must be signed in to change notification settings

Bon-Appetit/shorturl-domains

Repository files navigation

Note

Beginning September 1st, 2021, there will be no more DNS checks. A small VPS was used for this, but it has now become too expensive.

Tip

The list is too big to view directly in your web browser and it's hard to search through it there, especially if you want to find specific patterns or domains. For a more convenient search option, you can use the following: https://codealdente.github.io/search/.


List Size Last List Update Commit Activity

Short URL Domains

This collection brings together domains used in shortlink services, similar to bitly. Rather than crafting a new list, we leverage the excellent work of others and consolidate everything into a single list.

ATTENTION

Caution

The repository has seen a significant amount of effort to ensure the validity of the domains. DNS checks are handled cautiously to reduce the risk of false results. However, it's important to note that things can change rapidly. Please bear in mind that there's no warranty regarding the accuracy of this list. Exercise caution, especially for critical use cases.

List of adult domains

block.txt

This compilation contains all domains aggregated from various sources. Each domain has undergone syntax verification, and any duplicates have been eliminated. Additionally, specific domains, such as those from whitelists, have been omitted.

pass.txt

The pass.txt file consolidates all whitelisted domains into a single file. Similar to blacklisted domains, each domain undergoes syntax checking to ensure uniform verification.

Blacklist sources

bl-sources.txt

This file contains a list of website links, one after the other, pointing to domains associated with adult websites.

bl-sources.csv

This file serves as a user-friendly reference for the information in bl-sources.txt, providing details for every repository along with the timestamp of the source's last update. The CSV is generated by parsing GitHub raw file URLs and querying the GitHub API. Its order follows "last_update_to_file" in descending order to display sources with recent activity at the top.

bl-custom.txt

Custom black list: Domains which aren't listed in the sources will be added here to have them included in the block list.

bl-custom

In this directory, you will discover dedicated blacklist files utilized to structure external data or domains in a specific format, facilitating smooth integration and functionality within background processes.

Whitelist sources

wl-sources.txt

This file holds a list of website links, one after the other, leading to domains that we want to keep off the block list (usually to prevent mistakenly blocking legitimate websites).

wl-sources.csv

This file serves as a user-friendly reference for the information in wl-sources.txt, providing details for every repository along with the timestamp of the source's last update. The CSV is generated by parsing GitHub raw file URLs and querying the GitHub API. Its order follows "last_update_to_file" in descending order to display sources with recent activity at the top.

wl-custom.txt

Custom white list: Domains which should be excluded from the block list will be added here.

wl-custom

Within this directory, you'll find specialized whitelist files. These are primarily employed to organize external data or domains into a specific structure or format for seamless integration and functionality in background processes.

Is something missing or incorrect?

If you've got a domain that needs to be added or removed, just open an issue and drop the details there. It'd be awesome if you could include a URL pointing to a file within a GitHub Repository or Gist. Thank you!

Disclaimer

Warning

THIS REPOSITORY RELIES ON EXTERNAL DOMAINS. NO RESPONSIBILITY IS ASSUMED FOR THE ACCURACY OR APPROPRIATENESS OF THEIR CONTENT. ENSURE EXTERNAL SOURCES ALIGN WITH YOUR NEEDS. USAGE OF THIS REPOSITORY IMPLIES AN AGREEMENT TO DIRECT ISSUES WITH EXTERNAL CONTENT TO THE SOURCE, AND NO LIABILITY IS HELD. EXERCISE CAUTION WHEN INTERACTING WITH EXTERNAL CONTENT.