subdomain auto support

[wanghaisheng]

in doc it say i need manually pass subdomain to sitemap_todf

[eliasdabbas]

It's not clear what your question is. Please include full details.

[wanghaisheng]

now

adv.crawl('https://example.com',

          'example_crawl_1.jl',

          follow_links=True

          allowed_domains=['help.example.com', 'example.com', 'community.example.com'])

may be

adv.crawl('https://example.com',

          'example_crawl_1.jl',

          follow_links=True

          allowed_sub_domains=True)

[eliasdabbas]

Crawling all sub-domains is allowed by default already.

Your first comment mentions sitemap_to_df and now you are talking about crawl?

You have to communicate clearly and explain properly what you want. Take time to think what you want and double check what you wrote. I don't have time to try to figure this out for you.

[wanghaisheng]

that is my mistake
right now i use sitemap_to_df at first then I wanto use crawl as plan b
did not aware that

Crawling all sub-domains is allowed by default already.