misc/web-scraper
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

add flags to README

Browse Source
This commit is contained in:
Simon Weald
2018-09-16 15:58:17 +01:00
parent 524f6a45cd
commit 5262c23281
1 changed files with 7 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@@ -16,6 +16,13 @@ Run:
python crawler.py -u https://urltocrawl.com [-c 100] python crawler.py -u https://urltocrawl.com [-c 100]
``` ```
Flags:
- -u/--url https://url.com
- The base URL is required.
- -c/--concurrency 100
- Specifying concurrency value is optional (defaults to 100).
## Results ## Results
The resulting sitemap will be output to the root of this directory as `sitemap.html` The resulting sitemap will be output to the root of this directory as `sitemap.html`