2 min read

How to use gquestions.py and Pycharm

Recently there has been a lot of buzz around a script which allows you to scrape Google’s “People also ask” into either a CSV or a visual HTML file. It looks a little something like this:

Google People Also Ask

This very nice script (gquestions.py) by nittolese, and in this post I’ll show you how to make it executable within Pycharm. I love to use Pycharm when working with Python, but to execute this script within Pycharm you need to make a few changes to the script, which are covered below in our step-by-step guide.

The following steps will assume you have already installed Python 3 and the Pycharm IDE.

How to Scrape ‘People Also Asked’ Boxes with Python


Step 1: Download the ghit hub files and copy to your project Pycharm - GQUESTIONS.PY

Unzip the gquestions-master.zip file into your pycharm project. For me this is in /Users/jack/PycharmProjects/gquestions/gquestions-master


Step 2: Install dependencies

To do this right click on the gquestions-master directory and open a new local terminal (Open in terminal).

Pycharm - GQUESTIONS.PY

This will then launch a second terminal interface. Once open, enter:

pip install -r requirements.txt
pip install -r requirements.txt

Step 3: Test in terminal

Let’s try to find explamples of English questions around “used cars”.

python gquestions.py query "used cars" en --csv
python gquestions.py query "used cars" en --csv

If successful, within your CSV and Driver files there will be new files for “used cars” and you can stop reading this post!

If, like me, this doesn’t work, we need to carry out a few additional steps to allow it to run.


Step 4: Download the latest version of ChromeDriver

Follow this link to ChromeDriver and download the latest stable version for your OS – mine for example is chromedriver_mac64.zip.


Step 5: Unzip ChromeDriver and copy the file path

Unzip your chrome driver file (chromedriver_mac64) and save it to desktop or another memorable place. Once extracted, copy the file path – mine for example is /Users/jack/Desktop/gquestions – ChromeDriver/chromedriver.


Step 6: Edit the gquestions.py file (line 69)

We now need to edit line 69 “executable_path” value from:

#Default executable_path value
executable_path="chrome_path

#New executable_path value from your own file path 
executable_path="/Users/jack/Desktop/gquestions - ChromeDriver/chromedriver"
#Remember to use speech marks to quote a string

Step 7: Test and you’re done!

Execute a command such as python gquestions.py query "used cars" en --csv in the terminal, and this will now create new HTML and CSV files which can be used within your data analysis.

Google People Also Ask

gquestions.py examples:

Search “used cars” in English and export in html

 python gquestions.py query "used cars" en 

Search headlessly “used cars” in English and export in html

 python gquestions.py query "used cars" en --headless 

Search “used cars” in English and export in html and csv

 python gquestions.py query "used cars" en --csv 

Search “used cars” in English and export in html

 python gquestions.py query "used cars" en depth 1 

What is Selenium in the context of Python?

Selenium is a library which can be used to automate web browser functions, so, within this script, we are using it to interact with Google’s search results.


Want to find out more? National Coding Week is coming up on 16th September. Keep an eye on our blog as we’ll be sharing useful tips daily!

Explore Our Services

DIGITAL PR 

Earn authoritative links and drive brand awareness with Digital PR

PAID SEARCH

Deliver instant traffic and revenue through Paid Search and Shopping

SOCIAL ADS

Reach new audiences and retarget existing ones on social channels 

CONTENT

Attract and engage website visitors with a well executed content strategy

Boost Your YouTube SEO with These Expert Tips

3 min read

Boost Your YouTube SEO with These Expert Tips

Discover effective strategies to improve your YouTube SEO and boost your video rankings with these expert tips.

Read More
LinkedIn Conversation Ads: Boost Your Engagement

5 min read

LinkedIn Conversation Ads: Boost Your Engagement

Discover how LinkedIn Conversation Ads can significantly boost your engagement and drive meaningful conversations with your target audience.

Read More
Traditional PR vs Digital PR: Which is More Effective?

3 min read

Traditional PR vs Digital PR: Which is More Effective?

Discover the key differences between traditional PR and Digital PR, and find out which strategy is more effective in today's digital age.

Read More

How to use Python & Pytrends to automate Google Trends data

In this article we’ll look at how we built a script in Python 3, using Pytrends to effectively automate data collection from Google Trends. Also...

Read More

Autopsy of the Death of Google +

Last year Google came under scrutiny over privacy concerns. One of them was a large vulnerability issue on Google+ causing millions of user’s data to...

Read More

2 min read

Adwords PPC Basics – Google Adwords Account Structure

So this week i will be taking it right back to basics and introducing you to Google Adwords and general account structure. This blog will give a...

Read More