为Ta荐(TaJian.tv)工作的基于Hero的Node.js爬虫程序,可抓取B站、抖音、快手、西瓜视频播放页、直播页的标题和封面图
-
Updated
May 22, 2024 - JavaScript
为Ta荐(TaJian.tv)工作的基于Hero的Node.js爬虫程序,可抓取B站、抖音、快手、西瓜视频播放页、直播页的标题和封面图
🕸 Generates and delivers RSS feeds via HTTP. Docker image available! Create your own feeds or get started quickly with the included configs.
Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.
Download website to local directory (including all css, images, js, etc.)
Plugin for website-scraper which returns html for dynamic websites using puppeteer
🌱 goClone - clone websites in a matter of seconds
DPULSE - Domain Public Data Collection Service
Introducing NightFall, a cutting-edge tool revolutionizing Open-Source Intelligence. Dive deeper into the vast web with NightFall, unlocking unparalleled data extraction capabilities. NightFall empowers users to explore uncharted territories of the dark web and unearth hidden gems with pinpoint accuracy, courtesy of its advanced keyword extraction.
Sentiment-driven stock market prediction
Plugin for website-scraper which allows to save resources to existing directory
Uscrapper Vanta: Dive deeper into the web with this powerful open-source tool. Extract valuable insights with ease and efficiency, from both surface and deep web sources. Empower your data mining and analysis with Vanta's advanced capabilities. Fast, reliable, and user-friendly, Uscrapper Vanta is the ultimate choice for researchers and analysts.
Scraper for https://marvelsnapzone.com to retrieve metadata of Marvel SNAP cards.
Automatically curates and posts content to LinkedIn. It can optionally use web scraping to gather data, which is then fed to ChatGPT to craft engaging LinkedIn posts.
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON.
Use GPTparser with your OpenAI API to scrape & parse files into structured JSON files.
A robust Image Scraper that leverages OpenAI's GPT Chat Completions to determine the relevant HTML used to Scrape Images from websites.
Transform your website into a dynamic and interactive platform with SiteAssistant AI. Built with Python, Streamlit, LangChain, Openai - GPT 3.5
Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.
ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on the scraped content. The bot utilizes Retrieval Augmented Generation and webscraping to return natural language answers to the user's queries.
Tool for alerting when a website changes
Add a description, image, and links to the website-scraper topic page so that developers can more easily learn about it.
To associate your repository with the website-scraper topic, visit your repo's landing page and select "manage topics."