From a6224f9b6a416f1e6e66f29ccc795c52f23e0bb9 Mon Sep 17 00:00:00 2001 From: Simon Weald Date: Mon, 10 Sep 2018 20:56:12 +0100 Subject: [PATCH] updated readme --- README.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ee047cc..e8e228e 100644 --- a/README.md +++ b/README.md @@ -1 +1,21 @@ -# Concurrent web scraper \ No newline at end of file +# Concurrent web scraper + +## Requirements + +This crawler was written in 3.7.0 to take advantage of the latest `asyncio` features. + +Install required modules: + +```bash +pip install -r requirements.txt +``` + +Run: + +```bash +python crawler.py -u https://urltocrawl.com +``` + +## Results + +The resulting sitemap will be output in the root of this directory as `sitemap.html`