A LinkedIn Scraper to scrape up to 10k LinkedIn profiles and save their e-mail addresses if available!
It collects 10k profiles from the LinkedIn directory and their details like name, current position/headline, and location information. After all profiles are collected, it starts finding their email addresses. You can narrow down your searches based on location, role, etc.
Scrape public LinkedIn profile data at scale with Proxycurl APIs.
• Scraping Public profiles are battle-tested in court in HiQ VS LinkedIn
case.
• GDPR, CCPA, SOC2 compliant
• High rate Limit- 300 requests/minute
• Fast APIs respond in ~2s
• Fresh data- 88% of data is scraped real-time, other 12% are not older than 29 days
• High accuracy
• Tons of data points returned per profile
Built for developers, by developers
Trouble with the base version? Check the "Cookie Based" version here
Need location filter or role filter? Check this version
- First, download Python software from Python's official website. Python 3.x only is supported. Download from here or for a precise Python version, download this version and scroll to the bottom to download the correct version based on your operating system and make sure to tick on "Add to PATH" during installation in windows machines
- Now, from the start menu (Windows) or Applications list (Linux/Mac), search for Command Prompt (Windows) or terminal (on Mac/Linux) and copy-paste the command written below:
pip3 install requests
This will show some installation progress and will install the library eventually. If you see any pip warning, you may ignore that as that's optional.
- If pip doesn't get recognized as a command, please re-install Python with "Add python to executable path" enabled, or for Mac/Linux, run the command
apt-get install python3-pip
- An account on LinkedIn is a must! You can create temporary profiles if you want.
- Assuming that the Python software and the library required by this project are installed, time for the script execution. First, download the Python script of your choice and put it inside a folder.
- Right-click on the Python script and select the option "Edit with IDLE". If you don't see this option, you have to figure that out yourself to fix the problem but a correct installation will show this option in the right-click menu.
- This option should open up a code window. Locate the linkedin_email and linkedin_password placeholders and put your login details (don't worry, it won't get leaked to anyone). After that, set the desired company profile URL inside the target_company_link placeholder and save the changes by pressing the ctrl+s shortcut.
- Now, locate the Run menu and select Run Module and the automation will start processing. When you see a >>> at the bottom of the output screen, it will mean that the process has finished.
- You can set the configuration of email, password, and company profile link and save the changes. Double-clicking the Python file will also execute it.
- In the terminal, cd to the script folder and type
python3 Random_Scraper.pyOr,
python3 CompanyWise_Leads.py
Random_Scraper.py is the initial development of the scraper that collects up to 10k random LinkedIn profiles from the directory and picks info from their profile
CompanyWise_Leads.py is the revised version of the code to be able to collect company wise employee profiles for more leads information.
If you're having issues with the login-based version(the base script), you can try either the cookie version (doesn't have e-mail scraping ability) or the ProxyCurl API integrated version (Allows searching location, roles and have e-mail scraping capabilities)
- Do not log in from the IP address from where you don't usually login to your LinkedIn account, otherwise, it will trigger their security system and won't let you log in.
- Result is limited to 10,000 records only (this is a limitation from LinkedIn's side)
- First-page data is not collected due to being away from API endpoint capability. But the cookie version will get you covered.
- For searching by location or by role, or if you don't want to use your own LinkedIn account, you have to use the ProxyCurl version as they offer this ability.
Do not flood LinkedIn using your account on a large number to avoid issues with your account. If you want to anonymously scrape LinkedIn Leads for your business, you can check Proxycurl APIs to stay on the safe side (they offers trial access too for FREE!).
Star the repository and share it with your friends who might need this. More variants of the LinkedIn Scraper will come soon. Keep this on watch for more updates!