dmoonfire/dosage
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Activity
1404 commits 1 branch 0 tags 31 MiB
Commit graph

69 commits

Author SHA1 Message Date
Tobias Gruetzmacher
ebbb27d05d Move xpath_class to helpers module. 2017-02-13 22:41:17 +01:00
Tobias Gruetzmacher
b57945efd1 Update GoComic modules. 2017-02-12 12:21:01 +01:00
Tobias Gruetzmacher
a183e812ae Update GoComics module for new site layout. 
(fixes #77)
 2017-01-11 02:21:05 +01:00
Tobias Gruetzmacher
46b7a374f6 Small GoComics update. 2016-11-01 02:51:00 +01:00
Tobias Gruetzmacher
9a6a310b76 Fixup copyright years. 2016-10-29 00:21:41 +02:00
Tobias Gruetzmacher
f342a93aa1 Update GoComics module. 2016-10-01 03:39:36 +02:00
Tobias Gruetzmacher
807bee6342 Migrate GoComics to single-class module. 2016-05-23 00:01:10 +02:00
Tobias Gruetzmacher
f29472c143 Make auto-update script more flexible. 2016-05-22 23:06:05 +02:00
Tobias Gruetzmacher
51008a975b Refactor: Introduce generator methods for scrapers 
This allows one comic module class to generate multiple scrapers. This
change is to support a more dynamic module system as described in #42.
 2016-05-21 01:29:36 +02:00
Tobias Gruetzmacher
be1a63da0c Update GoComics comic list. 2016-05-16 18:26:45 +02:00
Tobias Gruetzmacher
c3f32dfef7 Refactor: Make namer a method. 
When #42 is realized, the naming of files might differ between comic
modules, so the namer's logical location is the instance, not the class.
 2016-04-21 08:20:49 +02:00
Tobias Gruetzmacher
1fbc844077 Update GoComics. 2016-04-17 18:40:09 +02:00
Tobias Gruetzmacher
52515b5fc5 Update GoComics. 2016-04-15 00:26:14 +02:00
Tobias Gruetzmacher
db87ed95e7 Use new features to make modules simpler. 2016-04-13 23:28:43 +02:00
Tobias Gruetzmacher
060281e5ff Use concrete scraper objects everywhere. 
This is a first step for #42. Since most access to the scraper classes
is through instances, modules can now dynamically override url and name
(name is now a property).
 2016-04-13 22:17:30 +02:00
Tobias Gruetzmacher
0468f2f31a Refactor: Convert starter to simple method. 2016-04-13 20:01:51 +02:00
Tobias Gruetzmacher
4e2e4ac529 Prevent scraper from moving to a different comic. 2016-04-12 08:10:47 +02:00
Tobias Gruetzmacher
443ab119e9 Refresh GoComics list from online directory. 2016-04-12 00:36:33 +02:00
Tobias Gruetzmacher
0e385a3697 Update GoComics (no change in supported comics) 
- remove make_scraper magic
- switch to _ParserScraper
 2016-04-11 22:42:01 +02:00
Tobias Gruetzmacher
68d4dd463a Revert robots.txt handling. 
This brings us back to only honouring robots.txt on page downloads, not
on image downloads.

Rationale: Dosage is not a "robot" in the classical sense. It's not
designed to spider huge amounts of web sites in search for some content
to index, it's only intended to help users keep a personal archive of
comics he is interested in. We try very hard to never download any image
twice. This fixes #24.

(Precedent for this rationale: Google Feedfetcher:
https://support.google.com/webmasters/answer/178852?hl=en#robots)
 2015-07-17 20:46:56 +02:00
Tobias Gruetzmacher
472afa24d3 GoComics doesn't allow spiders, disable them... 
This removes 757 comics, including quite popular ones like Calvin and
Hobbes, Garfield, FoxTrot, etc. :(
 2015-07-16 00:36:10 +02:00
Tobias Gruetzmacher
e8af5adcb8 Update list of supported GoComics comics. 2015-04-18 02:04:31 +02:00
Manabi
2b98a9023e Added Peanuts Begins & Wizard of Id Classics 2015-04-13 22:26:12 -04:00
Bastian Kleineidam
641daa738b Updated list of comics 2014-07-03 17:12:25 +02:00
Bastian Kleineidam
0ee5c08771 Match zoom image for GoComics pages. 2014-06-08 10:06:34 +02:00
Peter B
124cf99665 Added Poorly Drawn lines replacing GoComic's version. 2014-01-12 19:08:02 -05:00
Bastian Kleineidam
4d63920434 Updated copyright. 2014-01-05 16:50:57 +01:00
Bastian Kleineidam
5c5aa166c7 Fix gocomic image matcher 2013-12-12 22:54:03 +01:00
Bastian Kleineidam
f23aa86a2c Get larger Gocomic images. 2013-12-11 17:53:52 +01:00
Bastian Kleineidam
f6fc604745 Fix GoComics image URL. 2013-11-14 21:30:51 +01:00
Bastian Kleineidam
7760985601 Fix broken comics 2013-11-12 18:33:14 +01:00
Bastian Kleineidam
3e6414e0e5 Updated plugins 2013-11-07 07:28:47 +01:00
Bastian Kleineidam
8d5ae7b1bb Updated plugins. 2013-07-09 22:21:12 +02:00
Bastian Kleineidam
8559184d69 Updated plugins 2013-07-04 12:22:36 +02:00
Bastian Kleineidam
1c1b0aaf18 Comic fixes. 2013-05-25 23:24:33 +02:00
Bastian Kleineidam
ef878eed7c Updated plugins. 2013-05-22 07:19:16 +02:00
Bastian Kleineidam
dcacbf0b9a Fix some comics. 2013-04-28 19:58:38 +02:00
Bastian Kleineidam
f9a48e6cb9 Updated scripted comic plugins. 2013-04-27 07:47:17 +02:00
Bastian Kleineidam
871de6a8ce Prefer GoComics over Creators since they have a better naming scheme. 2013-04-25 21:50:45 +02:00
Bastian Kleineidam
35c031ca81 Fixed some comics. 2013-04-11 18:27:43 +02:00
Bastian Kleineidam
44c3fb9f16 Remove broken scripted plugins. 2013-04-04 18:30:02 +02:00
Bastian Kleineidam
6303a1cb20 Updated scripted plugins. 2013-04-03 20:27:12 +02:00
Bastian Kleineidam
3dd2daf223 Updated scripted plugins. 2013-03-26 17:35:47 +01:00
Bastian Kleineidam
940a04b499 Fix comic searching. 2013-03-25 19:48:01 +01:00
Bastian Kleineidam
c99827935b Updated plugins with scripts. 2013-03-25 19:40:38 +01:00
Bastian Kleineidam
9f08b21a7e Get correct images of gocomic strips. 2013-03-24 14:13:33 +01:00
Bastian Kleineidam
1cc7d39047 Fix some comics. 2013-03-07 23:08:17 +01:00
Bastian Kleineidam
0215ae82af Fix some comics. 2013-03-07 19:54:18 +01:00
Bastian Kleineidam
23c20bfe32 Fix some comics. 2013-03-07 18:22:24 +01:00
Bastian Kleineidam
01177e25f0 Updated generated comic lists. 2013-03-05 19:06:00 +01:00