3 Commits

Author	SHA1	Message	Date
Simon Weald	c436016e0c	remove unecessary function	2018-08-31 19:16:08 +01:00
Simon Weald	0517e5bc56	crawler now initialises and populates crawled pool with urls it finds	2018-08-31 19:02:21 +01:00
Simon Weald	5e0d9fd568	initial commit of crawler skeleton	2018-08-31 18:26:49 +01:00