diff --git a/README.md b/README.md index ee047cc..e8e228e 100644 --- a/README.md +++ b/README.md @@ -1 +1,21 @@ -# Concurrent web scraper \ No newline at end of file +# Concurrent web scraper + +## Requirements + +This crawler was written in 3.7.0 to take advantage of the latest `asyncio` features. + +Install required modules: + +```bash +pip install -r requirements.txt +``` + +Run: + +```bash +python crawler.py -u https://urltocrawl.com +``` + +## Results + +The resulting sitemap will be output in the root of this directory as `sitemap.html`