Multiple subdomains wrongly identified as domain

[jonathanvandriessen]

Hi,

First of all, thank you for tldjs, great library!

When I call tldjs.getDomain() on the following urls, tldjs identifies the entire url (without the protocol) as the domain, and no subdomains (tldjs.getSubdomain() returns empty).
This are the examples I have that cause an issue:

http://bntp-assets.global.ssl.fastly.net/
http://cres-wpstatic.freetls.fastly.net/
http://r3engage-live.global.ssl.fastly.net/
http://ticket-magic-ember-herokuapp-com.global.ssl.fastly.net/

Thank you!

[ctavan]

@jonathanvandriessen, this is expected behavior since both of the following are public suffixes:

• global.ssl.fastly.net
• freetls.fastly.net

See https://publicsuffix.org/list/ for reference

The domain as returned by getDomain() is the public suffix plus one more element left to it. This is why you get the full domain.

[jonathanvandriessen]

Thanks for your reply @ctavan !

[remusao]

@jonathanvandriessen To add to the answer from @ctavan, if this is something blocking for your usage of the library, this could be solved by having a way to match only ICANN rules, ignoring the PRIVATE section of the suffix list (since the rules for fastly.net are in the PRIVATE section).

I already have some POC on one of my branches to do something like that. If there is some interest for such feature, we could try to integrate it.

[jonathanvandriessen]

@remusao it is blocking for my usage of the library indeed.

Currently, I'm considering a workaround where I check where:
IF I don't get a subdomain AND I do get a domain that has more than 2 dots in it THEN I consider the url to be having a subdomain. I know this is probably not perfect but it seems to do the trick for the exceptions I know of.

FYI: My use case is the following: given a domain (could include subdomains), I'm trying to determine if I need to add www in front of it to make it a valid url. I add .www if the domain does not seem to have a subdomain.

I'd be happy to test drive your solution!

[ctavan]

@remusao how about simply adding an additional getIcannDomain()? I believe that could be useful. I believe that changing the behavior of the current getDomain() is probably to big of a breaking change.

[remusao]

@ctavan Agreed about not introducing a breaking change. Maybe we could, as you suggest, add a new method in the API + a new attribute in the result of parse to know what kind of rule triggered?

[ZLightning]

Maybe something like the isRegisterable()/getRegisterableDomain() methods in this Java library would be a good way to solve this issue. Those functions would be quite useful for security researchers. Public suffix domians in the private section, as returned by getDomain(), do not have whois data, and the registerable domain's info does not apply to the private subdomains since they likely are not under the direct control of the owner of the registerable domain.

This issue is duplicated in #120 #117 #78 and appears to be the main part of the v3 API considerations in #124