How to Scrape Google Hotel Prices with Node.js

Seasonal changes, fluctuating demand, and promotional activities cause hotel prices to change frequently. Manually tracking these changes is nearly impossible. Instead, automating this process by scraping travel websites and platforms can save time and effort.

In this article, we will show you how to scrape hotel prices from one of the largest aggregators: Google. By scraping Google's hotel data, you can quickly gather extensive information on hotel prices, ratings, and amenities for analysis, price comparison, or dynamic pricing strategies.

Why Scrape Google Hotel Prices?

When you search for hotel-related keywords, Google generates a dedicated hotel section that includes names, images, addresses, ratings, and prices for thousands of hotels. This is because Google aggregates information from millions of travel and hotel websites into a single place.

Travelers, businesses, and analysts can use this data for various purposes:

• Price Comparison: Compare prices across different booking platforms and travel websites to find the best deals.
• Data Analysis: Analysts can use hotel price data to uncover pricing trends, seasonal fluctuations, and competitive opportunities.
• Dynamic Pricing Strategies: Businesses can optimize revenue and occupancy rates by adjusting prices based on demand, availability, and competitor prices.
• Custom Alerts: Monitor price drops to alert customers or for personal use.
• Travel Aggregation Services: Provide users with a consolidated view of hotel prices and options from various sources.
• Budgeting and Planning: Travelers can anticipate accommodation costs and adjust their plans accordingly.

In short, the uses for this data are vast, but before you can gain insights, you need to collect it.

How to Scrape Google Hotel Prices with Node.js

In this tutorial, we will write a script to collect hotel pricing data and sort the hotel list from cheapest to most expensive, focusing on hotels in New York.

1. Prerequisites

To follow this tutorial, you need the following tools installed on your computer:

• Node.js 18+ and NPM
• Basic knowledge of JavaScript and the Node.js API

2. Setting Up the Project

Create a project folder:

mkdir google-hotel-scraper

Next, initialize a Node.js project by running the following command:

cd google-hotel-scraper  
npm init -y

This command will create a package.json file in the folder. Create an index.js file and add a simple JavaScript statement:

touch index.js  
echo "console.log('Hello world!');" > index.js

Run the index.js file using the Node.js runtime:

node index.js

If "Hello world!" is printed in the terminal, your project is up and running.

3. Installing Necessary Dependencies

To build our scraper, we need two Node.js packages:

• Puppeteer: To load Google Hotel pages and download the HTML content.
• Cheerio: To extract hotel information from the HTML downloaded by Puppeteer.

Install these packages with the following command:

npm install puppeteer cheerio

4. Identifying Information to Scrape from the Google Hotel Page

To extract information from a webpage, we first need to identify the DOM selectors that target the desired HTML elements.

Here is a table of the DOM selectors for each piece of relevant data:

5. Scraping the Google Hotel Page

With the DOM selectors identified, let’s use Puppeteer to download the page's HTML. The initial page we are targeting is: https://www.google.com/travel/search.

In some countries (mainly in Europe), a consent page will be displayed before redirecting to the URL. We will add code to click the "Reject all" button, wait for three seconds, and ensure the Google Hotel page is fully loaded.

Update the index.js file with the following code:

const puppeteer = require('puppeteer');

const PAGE_URL = 'https://www.google.com/travel/search';

const waitFor = (timeInMs) => new Promise(r => setTimeout(r, timeInMs));

const main = async () => {
    const browser = await puppeteer.launch({ headless: 'new' });
    const page = await browser.newPage();

    await page.goto(PAGE_URL);

    const buttonConsentReject = await page.$('.VfPpkd-LgbsSe[aria-label="Reject all"]');
    await buttonConsentReject?.click();
    await waitFor(3000);

    const html = await page.content();
    await browser.close();

    console.log(html);
}

void main();

Run the code using node index.js. The terminal will output the HTML content of the page.

6. Extracting Information from HTML

Although we have the page's HTML, extracting valuable data directly from raw HTML is challenging. This is where Cheerio comes in.

The following code loads the HTML and extracts the room price for each hotel:

const cheerio = require("cheerio");

const $ = cheerio.load(html);

$('.uaTTDe').each((i, el) => {
    const priceElement = $(el).find('.kixHKb span').first();

    console.log(priceElement.text());
});

Update the index.js file to extract the content with Cheerio, store it in an array, and sort it by price from lowest to highest:

const cheerio = require("cheerio");
const puppeteer = require("puppeteer");
const { sanitize } = require("./utils");

const waitFor = (timeInMs) => new Promise(r => setTimeout(r, timeInMs));

const GOOGLE_HOTEL_PRICE = 'https://www.google.com/travel/search';

const main = async () => {
  const browser = await puppeteer.launch({ headless: 'new' });
  const page = await browser.newPage();

  await page.goto(GOOGLE_HOTEL_PRICE);

  const buttonConsentReject = await page.$('.VfPpkd-LgbsSe[aria-label="Reject all"]');
  await buttonConsentReject?.click();
  await waitFor(3000);

  const html = await page.content();
  await browser.close();

  const hotelsList = [];
  const $ = cheerio.load(html);

  $('.uaTTDe').each((i, el) => {
    const titleElement = $(el).find('.QT7m7 > h2');
    const priceElement = $(el).find('.kixHKb span').first();
    const reviewsElement = $(el).find('.oz2bpb > span');
    const hotelStandingElement = $(el).find('.HlxIlc .UqrZme');

    const options = [];
    const pictureURLs = [];

    $(el).find('.HlxIlc .XX3dkb').each((i, element) => {
      options.push($(element).find('span').last().text());
    });

    $(el).find('.EyfHd .x7VXS').each((i, element) => {
      pictureURLs.push($(element).attr('src'));
    });

    const hotelInfo = sanitize({
      title: titleElement.text(),
      price: priceElement.text(),
      standing: hotelStandingElement.text(),
      averageReview: reviewsElement.eq(0).text(),
      reviewsCount: reviewsElement.eq(1).text(),
      options,
      pictures: pictureURLs,
    });

    hotelsList.push(hotelInfo);
  });

  const sortedHotelsList = hotelsList.slice().sort((hotelOne, hotelTwo) => {
    if (!hotelTwo.price) {
      return 1;
    }

    return hotelOne.price - hotelTwo.price;
  });

  console.log(sortedHotelsList);
}

void main();

Run the code and see the results. You have just collected information on all hotels on.

In the previous content, we implemented the crawling of Google hotel prices through Node.js and Puppeteer. Although this method can meet basic needs, it requires writing a lot of code and may encounter many challenges when dealing with complex anti-crawling mechanisms.

In order to complete the task more efficiently, we can recommend a simpler method: using Scrapeless Deep SerpAPI.

Use Scrapeless Deep SerpApi to crawl Google hotel information

Deep SerpApi is a dedicated search engine designed for large language models (LLM) and AI agents, aiming to provide real-time, accurate and unbiased information to help AI applications retrieve and process data efficiently.

It can help developers quickly obtain results for more than 20 different scenarios of Google search engines. It supports multiple parameter settings, can customize searches based on region, language and device type, and provides structured JSON data for developers to use directly.

Advantages of Deep SerpAPI

1. Lowest price: Deep SerpAPI is priced as low as $0.1/k. It is the lowest price on the market.

2. Easy to use: No need to write complex code, just get data through API calls.

3. Real-time: Each request can instantly return the latest search results to ensure the timeliness of the data.

4. Global support: Through global IP addresses and browser clusters, ensure that search results are consistent with the experience of real users.

5. Rich data types: Supports more than 20 search types such as Google Search, Google Maps, Google Shopping, etc.

6. High success rate: Provides up to 99.995% service availability (SLA).

How to use Deep SerpAPI playground to scrape Google hotel information

Deep SerpAPI provides a powerful online tool, Playground, which allows developers to quickly scrape Google hotel information without writing code. Playground is a visual interface that can obtain structured search result data through simple parameter settings and click operations. The following are the detailed steps to use Deep SerpAPI Playground to scrape Google hotel information.

How to get the API KEY of Deep SerpAPI:

• After signing up for free on Scrapeless, you You get 20,000 free search queries.
• Navigate to API Key Management. Then click Create to generate a unique API key. Once created, just click on AP to copy it.

1. Sign Up and Access the Playground

• Create an Account: If you haven't already, sign up for a Deep SerpAPI account.
• Access the Deep SerpApi Playground: Once logged in, navigate to the "Deep SerpApi" section.

2. Set search parameters

• In the Playground, enter your search keyword, such as "New York hotels".
• Set other parameters, such as check-in date, check-out date, country, language, etc.

You can also click to view the official API documentation of Scrapeless to learn about the parameters of Google Hotels.

3. Perform a search

• Click the "Start Search" button, and the Playground will send a request to the Deep Serp API and return structured JSON data.
• The returned data will include the hotel's name, brand details, price information, description, rating, facilities, nearby location, hotel rating, etc.

4. View and export data

• Browse the returned JSON data to view detailed information for each hotel.
• If necessary, you can click "Export" in the upper right corner to export the data to CSV or JSON format for further analysis.

5. Integrate into your project

• If you need to integrate the data into your application, Deep SerpAPI provides library support for multiple programming languages, including Python, JavaScript, Ruby, PHP, Java, C#, C++, Swift, Go, and Rust.

Sample code (Python)

import json
import requests

class Payload:
    def __init__(self, actor, input_data):
        self.actor = actor
        self.input = input_data

def send_request():
    host = "api.scrapeless.com"
    url = f"https://{host}/api/v1/scraper/request"
    token = "your_token"

    headers = {
        "x-api-token": token
    }

    input_data = {
    "engine": "google_hotel",
       q: query,
       engine: 'google',
       gl: 'us', 
       hl: 'en'  
    }

    payload = Payload("scraper.google.hotel", input_data)

    json_payload = json.dumps(payload.__dict__)

    response = requests.post(url, headers=headers, data=json_payload)

    if response.status_code != 200:
        print("Error:", response.status_code, response.text)
        return

    print("body", response.text)


if __name__ == "__main__":
    send_request()

Deep SerpAPI Pricing Plan: Affordable and Powerful

Deep SerpAPI provides a cost-effective solution to help developers quickly obtain Google search results page (SERP) data. Its pricing plan is very competitive, with prices as low as $0.1 per 1,000 queries, applicable to more than 20 search results scenarios of Google.

Free Developer Support Program

Deep SerpAPI now also offers a free developer support program to help developers better integrate and use its API. Here are the details of the program:

• Integration support: Integrate Deep SerpAPI into your AI tools, applications, or projects. We already support Dify, and will soon support Langchain, Langflow, FlowiseAI, and other frameworks.
• Free support duration: After integration, developers can get 1-12 months of free developer support by sharing your results on GitHub or social media.
• Usage quota: Up to 500k usage per month, helping developers not to worry about cost issues in the early stages of the project.

You can join Discord to contact Liam to learn about the event participation

Conclusion

In summary, scraping Google Hotel prices with Node.js can be a valuable method for collecting pricing data, but it comes with challenges like handling dynamic content and avoiding detection. While tools like Puppeteer or Playwright can help, they require ongoing maintenance and technical expertise. To streamline the process, we recommend using Scrapeless, which provides a hassle-free, no-code solution for extracting hotel price data efficiently and reliably. With Scrapeless, you can focus on insights rather than the complexities of web scraping. Try it today and simplify your data collection!

Additional Resources

How to Scrape Google News with Python
How to Use Selenium with PowerShell
Scraping Product Details from Google Shopping with Scrapeless