Skip to content

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

License

Notifications You must be signed in to change notification settings

daandtu/Android-Web-Scraper

Repository files navigation

android-web-scraper

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

Setup

The project is hosted on jCenter. Add the following to your dependencies:

implementation 'com.daandtu:android-web-scraper:1.0.1'

Add internet permission to AndroidManifest.xml:

<uses-permission android:name="android.permission.INTERNET"/>

Sample usage

Initialisation:

WebScraper webScraper = new WebScraper(this);
webScraper.setUserAgentToDesktop(true); //default: false
webScraper.setLoadImages(true); //default: false
webScraper.loadURL("https://www.github.com/");

If you want to see the browser automation in action:

layout.addView(webScraper.getView());

Interact with webpage elements:

Element el1 = webScraper.findElementByXpath("//*[@id=\"search\"]");
Element el2 = webScraper.findElementByName("img",3);
el1.setText("Android");
el2.click();
Element el3 = webScraper.findElementById("result");
String result = el3.getValue();

Setup OnPageLoadedListener:

webScraper.setOnPageLoadedListener(new WebScraper.onPageLoadedListener() {
            @Override
            public void loaded(String URL) {
                //TODO
            }
        });

Other methods:

Bitmap screenshot = webScraper.takeScreenshot(); //Pay attention with big webpages or use
Bitmap screenshot2 = webScraper.takeScreenshot(500,MAX);

String title = webScraper.getWebsiteTitle();

String html = webScraper.getHtml();

webScraper.clearHistory();
webScraper.clearCache();
webScraper.clearCookies();
webScraper.clearAll(); //Clear history, cache and cookies

webScraper.reload();

About

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages