Skip to content

gologinapp/ScrapingBrowser

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

9 Commits
Β 
Β 
Β 
Β 

Repository files navigation

Scraping Browser

Enhance your web scraping experience with cloud-driven headful browsers, ensuring stealthy automation and efficient unblocking.

Quick start

1. Get an API token

Haven't got a Gologin account? πŸ‘‰ We got you covered! Follow this quick Tango to sign up and configure an API token.

Already got an account πŸ‘‰ Go straight to Gologin API Documentation page and copy an API token.

2. Install puppeteer-core

npm i puppeteer-core

or

yarn add puppeteer-core

puppeteer-core is a lightweight version of Puppeteer that doesn't come with a Chromium browser in it (you don't need a local browser in this case).

3. Set the TOKEN variable to your API token from Step 2 in the example script

const puppeteer = require('puppeteer-core');

const TOKEN = 'your-token'; // πŸ‘ˆ change this

(async () => {
    const browser = await puppeteer.connect({
        browserWSEndpoint: `https://cloudbrowser.gologin.com/connect?token=${TOKEN}`,
        ignoreHTTPSErrors: true,
    });

    console.info('Browser connected! Scraping...');

    const page = await browser.newPage();
    await page.goto('https://iphey.com');
    await page.screenshot({ path: 'screenshot.jpeg' });

    console.info('Screenshot taken!');
})();

4. Run the script

node ./examples/quick-start.js

You should now see the logs and the screenshot saved in the root directory

Browser connected! Scraping...
Screenshot taken!

Done!

API parameters

  • token={TOKEN} (required) - API token from your Gologin account.

  • profile={profile_id} (optional) - ID of a specific profile you want to run. If this parameter is specified, a new profile is not auto generated.

  • geolocation={type}:{country_code} (optional) - Gologin geoproxy (floppydata). This parameter adds a specified proxy to an auto generated profile. If the profile parameter is included, the geolocation parameter has no effect.

Examples

βœ… /connect?token=${TOKEN}&profile={profile_id} - run a profile with profile_id

βœ… /connect?token=${TOKEN} - generate a new profile and run it

βœ… /connect?token=${TOKEN}&geolocation=dataCenter:DE - generate a new profile with a German geolocation and run it (check the geolocation script)

❌ /connect?profile={profile_id} - oops! You need to set the token parameter for authorization

πŸŽ‰ Happy scraping! πŸŽ‰

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published