GitHub
simplecrawler
v1.1.7
Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.
simple
crawler
spider
cache
queue
simplecrawler
eventemitter
Readme
Versions
Dependencies
Scripts
Dependencies (4)
async
iconv-lite
robots-parser
urijs
Dev dependencies (4)
chai
eslint
jsdoc-to-markdown
mocha
npm i
[email protected]
Source Code
github.com/simplecrawler...
Homepage
github.com/simplecrawler/sim...
Metadata
BSD-2-Clause
>=6.13.0
Christopher Giffard
released 4/9/2019
Downloads
Downloads Tab Here
Maintainers
cgiffard
fredrikekelund
kbychkov