← namespaces / @rubygems / gems / cobweb

@rubygems/cobweb

Cobweb is a web crawler that can use resque to cluster crawls to quickly crawl extremely large sites faster than multi-threaded crawlers. It is also a standalone crawler that has a sophisticated statistics monitoring interface to monitor the progress of the crawls.

details

latest
1.2.1
PURL
pkg:gem/gem.coop/rubygems/cobweb@1.2.1
licenses
MIT
ruby
>= 0
rubygems
>= 0
executables
none
native extensions
none
platforms
ruby

links
homepage
http://github.com/stewartmckee/cobweb

versions

  • 1.2.1 Jan 09 2021 gem.coop system
  • 1.2.0 Feb 25 2019 gem.coop system
  • 1.1.0 Nov 06 2015 gem.coop system
  • 1.0.29 Nov 05 2015 gem.coop system
  • 1.0.28 Mar 20 2015 gem.coop system
  • 1.0.27 Mar 18 2015 gem.coop system
  • 1.0.26 Mar 15 2015 gem.coop system
  • 1.0.25 Jan 24 2015 gem.coop system
  • 1.0.24 Jan 22 2015 gem.coop system
  • 1.0.23 Jan 20 2015 gem.coop system
  • 1.0.22 Jan 20 2015 gem.coop system
  • 1.0.21 Nov 05 2014 gem.coop system
  • 1.0.20 Aug 23 2014 gem.coop system
  • 1.0.19 Nov 26 2013 gem.coop system
  • 1.0.18 Oct 17 2013 gem.coop system
  • 1.0.17 Aug 23 2013 gem.coop system
  • 1.0.16 Aug 22 2013 gem.coop system
  • 1.0.15 Aug 19 2013 gem.coop system
  • 1.0.12 Aug 15 2013 gem.coop system
  • 1.0.11 Feb 18 2013 gem.coop system
See raw versions list →

usage

run on your commandline 
bundle add cobweb --source "https://gem.coop/@rubygems"
or, put in your Gemfile 
gem "cobweb", source: "https://gem.coop/@rubygems"