add more thoughts
This commit is contained in:
8
notes.md
8
notes.md
@@ -13,3 +13,11 @@
|
||||
* investigate using [tldextract](https://github.com/john-kurkowski/tldextract) to match urls
|
||||
* ~~implement parsing of [robots.txt](http://docs.w3cub.com/python~3.6/library/urllib.robotparser/)~~
|
||||
* investigate [gzip encoding](https://stackoverflow.com/questions/36383227/avoid-downloading-images-using-beautifulsoup-and-urllib-request)
|
||||
|
||||
|
||||
```
|
||||
text/html; charset=utf-8
|
||||
application/xhtml+xml
|
||||
'WebPage' object has no attribute 'source'
|
||||
'WebPage' object has no attribute 'discovered_hrefs'
|
||||
```
|
||||
Reference in New Issue
Block a user