dmoonfire/dosage
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Activity

GoComics doesn't allow spiders, disable them...

Browse source 
This removes 757 comics, including quite popular ones like Calvin and
Hobbes, Garfield, FoxTrot, etc. :(
This commit is contained in:
Tobias Gruetzmacher 2015-07-16 00:36:10 +02:00
parent 7c15ea50d8
commit 472afa24d3
1 changed files with 4 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
dosagelib/plugins/gocomics.py
View file

@ -17,6 +17,10 @@ _prevSearch = compile(tagre("a", "href", r'(/[^"]+/\d+/\d+/\d+)', after="prev"))
_nextSearch = compile(tagre("a", "href", r'(/[^"]+/\d+/\d+/\d+)', after="next")) _nextSearch = compile(tagre("a", "href", r'(/[^"]+/\d+/\d+/\d+)', after="next"))
def add(name, shortname): def add(name, shortname):
# Unfortunately, the whole http://assets.amuniversal.com/ is blocked by
# robots.txt, so we disable GoComics for now...
return
baseUrl = 'http://www.gocomics.com' baseUrl = 'http://www.gocomics.com'
url = baseUrl + shortname url = baseUrl + shortname
classname = 'GoComics_%s' % name classname = 'GoComics_%s' % name