attempt to remove base url with trailing slash (if discovered)

This commit is contained in:
2018-09-04 13:57:52 +01:00
parent 6abe7d68e0
commit 1b9b207a28
2 changed files with 10 additions and 1 deletions

View File

@@ -7,4 +7,4 @@
* ~~`WebPage.parse_urls()` needs to compare startswith to base url~~
* ignore any links which aren't to pages
* better url checking to get bare domain
* remove base url from initial urls with and without trailing slash
* ~~remove base url from initial urls with and without trailing slash~~