updated readme

This commit is contained in:
2018-09-10 20:56:12 +01:00
parent b64711973f
commit a6224f9b6a

View File

@@ -1 +1,21 @@
# Concurrent web scraper
# Concurrent web scraper
## Requirements
This crawler was written in 3.7.0 to take advantage of the latest `asyncio` features.
Install required modules:
```bash
pip install -r requirements.txt
```
Run:
```bash
python crawler.py -u https://urltocrawl.com
```
## Results
The resulting sitemap will be output in the root of this directory as `sitemap.html`