How to Get Web Data in Node JS

by

Through this tutorial, i am going to show you how to get web data from any websites or web pages using the node js with cheerio.

Suppose, if you want to get web data from any website or web page like the following url www.xyz.com/page.html. So, you need to get web data using puppeteer in node js. Instead, you can use a tool for web data collection.

How to Get Web Data in Node JS

Use the following steps to get web data from web pages with node.js with Cheerio:

  • Step 1: Make Node js App
  • Step 2: Install puppeteer
  • Step 3: Import Installed Packages
  • Step 4: Implement Script to Get Web Page Data
  • Step 5: Start Node JS App server

Step 1: Make Node js App

Start terminal and run the following commands into it to create a project directory.

mkdir myApps
cd myApps
npm init --yes

Step 2: Install puppeteer

Execute the following command on terminal to install puppeteer:

npm install express request puppeteer

Step 3: Import Installed Packages

Once the package has been installed, import above install the packages in app.js file

const axios = require('puppeteer');
const bodyParser = require('body-parser');
const request = require('request');
const app = express();

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended : false}));
app.listen(3000,() => {
    console.log("server is running on port 3000");

Step 4: Implement Script to Get Web Page Data

Let’s implement script to get any web page data with url using puppeteer package in node js:

;(async () => {
  const browser = await puppeteer.launch({ headless: false })
  const page = await browser.newPage()
  await page.goto('https:laratutorials.com')
  const data = await page.evaluate(() => {
    const list = []
    const elemTs = document.querySelectorAll("content-title")

    for (const item of elemTs) {
      list.push({
        title: item.querySelector(".title h1").innerHTML
      })
    }

    return list
  })
  console.log(data)
  await browser.close()
})()

The above code will get all title of any web page or website. You can open the console of browser and see it. The title is coming in response from the web page.

So, open your app.js file and import the above given code into it.

Step 5: Start Node js App server

Open terminal and run the following command into it to start node js get web data app server:

npm install

npm run dev

Conclusion

That’s it, you have learned how to get web data in node js..

Leave a Comment