robot-directives
js
Parse robot directives within HTML meta and/or HTTP headers.
Version 0.4.0 License MIT
Keywords
crawlersheaderhtmlhttpmetametadatanofollownoindexrobotsrobots.txtseospiders
INSTALL
Parse robot directives within HTML meta and/or HTTP headers.