dmoonfire/dosage
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Activity
1905 commits 1 branch 0 tags 31 MiB
Commit graph

75 commits

Author SHA1 Message Date
Tobias Gruetzmacher
7e040086b6 Try to inform the user about geo-blocks 
Instead of letting the crawler run into "random" error messages, throw a
specific "geoblocked" exception instead.
 2020-09-28 13:11:34 +02:00
Techwolf
711d8d5e23 Fix BlackTapestries and TheRealmOfKaerwyn 2020-08-14 20:51:34 -07:00
Techwolf
c362f39e3a Fix FurryExperience 2020-08-14 20:50:54 -07:00
Tobias Gruetzmacher
7a176b29f2 Replace xpath_class with custom xpath function 2020-07-31 22:56:30 +02:00
Tobias Gruetzmacher
27d28b8eef Update file headers 
The default encoding for source files is UTF-8 since Python 3, so we can
drop all encoding headers. While we are at it, just replace them with
SPDX headers.
 2020-04-18 13:45:44 +02:00
Techwolf
345a1aff30 Fix GrowingTroubles 2020-04-18 11:44:05 +02:00
Tobias Gruetzmacher
44791439a5 Drop Python 2 support: Obsolete future statements 2020-02-04 01:06:19 +01:00
Techwolf
eb69210e58 Add self to authors list, update copyright headers 2020-01-28 00:23:15 +01:00
Techwolf
dc923028d5 Add SmackJeeves/WhatWeRememberTheMost 2020-01-28 00:16:10 +01:00
Techwolf
4219b32074 Fix VerloreGeleentheid 2020-01-12 21:29:47 +01:00
Techwolf
f49dff13be Move AddictiveScience to KemonoCafe 2020-01-12 21:29:47 +01:00
Techwolf
32529da7e6 Add SmackJeeves/GrowingTroubles 2020-01-09 03:20:34 +01:00
Techwolf
6763c105b7 Add StolenGeneration 2020-01-09 01:13:32 +01:00
Tobias Gruetzmacher
addf78fcbc More cleanup after SmackJeeves redesign 2019-12-29 23:19:17 +01:00
Tobias Gruetzmacher
963db7f448 Adopt SmackJeeves to new site design (fixes #144) 
Some things got lost on the way:
- Since there is no comprehensive comic directory anymore, removed
  automatic update script. New comics need to be added manually.
- Some authors used the opportunity to move from SmackJeeves somewhere
  else - some of those got new modules (either standalone or ComicFury)
- Abunch of comics just disappeared...
 2019-12-26 22:03:18 +01:00
Tobias Gruetzmacher
44af7acdbc Clean up ComicFury a bit 
Mostly getting rid of vanished comics, but also cleaning up the matching
expressions a bit.
 2019-12-25 23:05:11 +01:00
Techwolf
013d10a1f2 Add SmackJeeves/FurryExperience 2019-11-03 23:40:58 +01:00
Techwolf
dd6e536a55 Add Kaerwyn and BlackTapestries 2019-11-03 23:26:03 +01:00
Tobias Gruetzmacher
09687c91f4 Fix some SmackJeeves comics. 2017-05-12 00:32:25 +02:00
Tobias Gruetzmacher
20ab279cde Clean up SmackJeeves... 
Currently only covers already existing modules: Removed 11 broken
modules, added 2 and tried to update comic names and the adult and
endOfLife flags from their index. This isn't helped by the fact that
their search seems to skip some comics...
 2017-02-13 01:46:49 +01:00
Tobias Gruetzmacher
51ed898f5d Fix some SmackJeeves comics. 2016-10-30 14:30:45 +01:00
Tobias Gruetzmacher
b6d99945f6 Merge pull request #73 from acaranta/master 
Added several SmackJeeves Comics
 2016-10-30 11:55:17 +01:00
Tobias Gruetzmacher
9a6a310b76 Fixup copyright years. 2016-10-29 00:21:41 +02:00
acaranta
83880a3cbd corrected RainbowMansion 2016-10-27 09:53:34 +02:00
acaranta
0ed823175c Added even more Smackjeeves comics 2016-10-27 06:58:57 +02:00
acaranta
a5c9a3c35c Added several SmackJeeves Comics 2016-10-26 05:25:13 +02:00
Tobias Gruetzmacher
f5dff27b0a Migrate SmackJeeves to single-class module. 2016-05-22 23:54:21 +02:00
Tobias Gruetzmacher
f29472c143 Make auto-update script more flexible. 2016-05-22 23:06:05 +02:00
Tobias Gruetzmacher
51008a975b Refactor: Introduce generator methods for scrapers 
This allows one comic module class to generate multiple scrapers. This
change is to support a more dynamic module system as described in #42.
 2016-05-21 01:29:36 +02:00
Tobias Gruetzmacher
0c1aa9e8bd Move libxml < 2.9.3 workaround to base class. 2016-05-02 23:22:06 +02:00
Tobias Gruetzmacher
1d52d33311 Remove missing SmackJeeves comics. 2016-04-30 00:56:20 +02:00
Tobias Gruetzmacher
8b1ac4eb35 Fix "tagsoup" on SmackJeeves 
Unfortunatly, browsers render < outside of HTML tags differently then
libXML until recently (libXML 2.9.3), so we need to preprocess pages
before parsing them...

(This was fixed in libXML commit 140c25)
 2016-04-26 08:05:38 +02:00
Tobias Gruetzmacher
8ddf553eb4 Fix some more SmackJeeves modules. 2016-04-22 01:04:47 +02:00
Tobias Gruetzmacher
c3f32dfef7 Refactor: Make namer a method. 
When #42 is realized, the naming of files might differ between comic
modules, so the namer's logical location is the instance, not the class.
 2016-04-21 08:20:49 +02:00
Tobias Gruetzmacher
fe51a449df Update SmackJeeves 
- Now uses _ParserScraper, which makes the pattern quite a bit more
  generic and IMHO more readable
- remove make_scraper magic
- No new comics, only fixed existing ones and removed some dead ones.
 2016-04-20 23:36:45 +02:00
Tobias Gruetzmacher
ee99c087d7 Remove prevUrlMatchesStripUrl. 
It was only used for one test.
 2016-04-16 01:14:26 +02:00
Tobias Gruetzmacher
0468f2f31a Refactor: Convert starter to simple method. 2016-04-13 20:01:51 +02:00
Tobias Gruetzmacher
ff21df596b Remove descriptions and genres (closes #9). 
Maintaining the descriptions creates quite a bit of overhead (finding
them, copying them, checking if they are still correct) for a minimal
user benefit.

PS: Viewing this diff should be easier in a difftool that shows changes
in a line, for example kdiff3.
 2015-04-20 20:29:09 +02:00
Tobias Gruetzmacher
2567bd4e57 Convert starters and other helpers to new interface. 
This allows those starters to work with future scrapers.
 2014-07-26 11:28:43 +02:00
Bastian Kleineidam
641daa738b Updated list of comics 2014-07-03 17:12:25 +02:00
Bastian Kleineidam
4d63920434 Updated copyright. 2014-01-05 16:50:57 +01:00
Bastian Kleineidam
3e6414e0e5 Updated plugins 2013-11-07 07:28:47 +01:00
Bastian Kleineidam
8d5ae7b1bb Updated plugins. 2013-07-09 22:21:12 +02:00
Bastian Kleineidam
8559184d69 Updated plugins 2013-07-04 12:22:36 +02:00
Bastian Kleineidam
1c1b0aaf18 Comic fixes. 2013-05-25 23:24:33 +02:00
Bastian Kleineidam
ef878eed7c Updated plugins. 2013-05-22 07:19:16 +02:00
Bastian Kleineidam
dcacbf0b9a Fix some comics. 2013-04-28 19:58:38 +02:00
Bastian Kleineidam
f9a48e6cb9 Updated scripted comic plugins. 2013-04-27 07:47:17 +02:00
Bastian Kleineidam
871de6a8ce Prefer GoComics over Creators since they have a better naming scheme. 2013-04-25 21:50:45 +02:00
Bastian Kleineidam
35c031ca81 Fixed some comics. 2013-04-11 18:27:43 +02:00