misc/web-scraper
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

added a docstring to the WebPage object

Browse Source
This commit is contained in:
Simon Weald
2018-08-31 19:18:00 +01:00
parent c436016e0c
commit abc628106d
1 changed files with 4 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
utils/helpers.py
View File

@@ -33,6 +33,10 @@ class UrlPool(object):
class WebPage(object): class WebPage(object):
'''
Object to manage common operations required to return
the data from each individual page.
'''
headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0'} headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0'}