GitHub
simplecrawler
v1.1.9
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
simple
crawler
spider
cache
queue
simplecrawler
eventemitter
Readme
Versions
Dependencies
Scripts
Dependencies (4)
async
iconv-lite
robots-parser
urijs
Dev dependencies (4)
chai
eslint
jsdoc-to-markdown
mocha
npm i simplecrawler
Source Code
github.com/simplecrawler...
Homepage
github.com/simplecrawler/sim...
Metadata
BSD-2-Clause
>=8
Christopher Giffard
released 4/12/2020
Downloads
Downloads Tab Here
Maintainers
cgiffard
fredrikekelund
kbychkov