updated readme
This commit is contained in:
22
README.md
22
README.md
@@ -1 +1,21 @@
|
||||
# Concurrent web scraper
|
||||
# Concurrent web scraper
|
||||
|
||||
## Requirements
|
||||
|
||||
This crawler was written in 3.7.0 to take advantage of the latest `asyncio` features.
|
||||
|
||||
Install required modules:
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python crawler.py -u https://urltocrawl.com
|
||||
```
|
||||
|
||||
## Results
|
||||
|
||||
The resulting sitemap will be output in the root of this directory as `sitemap.html`
|
||||
|
||||
Reference in New Issue
Block a user