Commit graph

1483 commits

Author SHA1 Message Date
Techwolf
f82ee7ca14 Fix Everblue 2022-03-26 00:38:05 -07:00
Techwolf
11dce19af5 Fix Unsounded 2022-03-26 00:38:05 -07:00
Techwolf
8a7476e81d Fix MangaDex 2022-03-26 00:38:05 -07:00
Techwolf
71db0d5f32 Add KemonoCafe/Bethellium 2022-03-26 00:38:05 -07:00
Techwolf
afe36b19b9 Add MangaDex/DoChokkyuuKareshiXKanojo 2022-03-26 00:38:05 -07:00
Techwolf
526947b07e Fix MangaDex 2022-03-26 00:38:05 -07:00
Techwolf
ca11599ed7 Add Shifters 2022-03-26 00:38:05 -07:00
Techwolf
9b6ea0994b Add RaineDog 2022-03-26 00:38:05 -07:00
Techwolf
94339800ad Add DoesNotPlayWellWithOthers 2022-03-26 00:38:04 -07:00
Techwolf
1856546d6f Add WebToons/ClawShot 2022-03-26 00:38:04 -07:00
Techwolf
a8f34077b6 Add WebToons/KissItGoodbye 2022-03-26 00:38:04 -07:00
Techwolf
049bbbef68 Add ProjectFuture/IPanther 2022-03-26 00:38:04 -07:00
Techwolf
dee9fa41b7 Fix Cloudscratcher 2022-03-26 00:38:04 -07:00
Techwolf
a657a12600 Add ComicFury/Alderwood 2022-03-26 00:38:04 -07:00
Techwolf
346941b66b Add StupidFox 2022-03-26 00:38:04 -07:00
Techwolf
c50b872939 Add Hackles 2022-03-26 00:38:04 -07:00
Techwolf
615dee26e7 Add MangaDex/TheNewGate 2022-03-26 00:38:04 -07:00
Techwolf
ade72c28c5 Fix all MangaDex comics 2022-03-26 00:38:04 -07:00
Techwolf
73675fb162 Fix unDivine 2022-03-26 00:38:04 -07:00
Techwolf
29ad39d003 Fix Centralia2050 2022-03-26 00:38:04 -07:00
Techwolf
4ecb05b6b3 Fix StudioKhimera/UberQuest 2022-03-26 00:38:04 -07:00
Techwolf
5eb6cd40db Fix DeerMe 2022-03-26 00:38:04 -07:00
Techwolf
9834ca4caf Add ProjectFuture/Xerian 2022-03-26 00:38:03 -07:00
Techwolf
7440dba84e Mark ProjectFuture as endOfLife 2022-03-26 00:38:03 -07:00
Tobias Gruetzmacher
b72fff5c23 Remove some dead ComicsKingdom comics 2022-03-20 21:24:03 +01:00
Tobias Gruetzmacher
de7d9f5d55 Fix ComicsKingdom image search (again) (fixes #206) 2022-03-20 21:22:42 +01:00
Tobias Gruetzmacher
44c78f4765 Fix ComicsKingdom (fixes #205) 2022-02-15 00:24:15 +01:00
Tobias Gruetzmacher
45162bf7f2 Enhance default session with host-specific options
This makes much more sense then building custom sessions inside specific
modules.
2022-02-15 00:18:48 +01:00
Tobias Gruetzmacher
5ee888d889 Fix TLS verification for ComicsKingdom
It seems they can make the same error multiple times (see #132), so we
are explicit on which certificate we trust now (fixes #204).
2022-02-14 22:24:12 +01:00
Tobias Gruetzmacher
adf1531d4f Merge krisstraub comics into a common module 2021-08-31 00:38:58 +02:00
Tobias Gruetzmacher
05a873a40f Fix more comics using the Internet Archive 2021-08-31 00:11:28 +02:00
toonn
c3e4c914c4
Erfworld: Add Kickstarter Stories 2021-08-18 14:19:08 +02:00
garbled1
e86526f21a Add a few new comics 2021-08-14 08:17:38 -07:00
Tobias Gruetzmacher
2c8bb90d4e Remove ComicSherpa
This legacy site has already been offline for a few weeks now and
is probably not coming back.
2021-04-02 09:28:56 +02:00
Tobias Gruetzmacher
6436bc0c6a Fix some navigation issues in WLP comics 2021-03-20 01:20:09 +01:00
Tobias Gruetzmacher
05a497fa8a Throttle GrrlPower (fixes #179) 2021-03-20 00:52:28 +01:00
Techwolf
0df3db7f5c Add TaleOfTenThousand 2021-03-19 13:49:32 +01:00
Techwolf
1ebdea7aae Fix SMBC 2021-03-19 13:49:32 +01:00
Techwolf
03f6d2632b Add MangaDex/GanbareDoukiChan 2021-03-19 13:49:32 +01:00
Techwolf
436a61ae60 Add WebToons/FutureYou 2021-03-19 13:49:32 +01:00
Techwolf
54b82d91bf Add WebToons/JustFriends 2021-03-19 13:49:32 +01:00
Techwolf
e0d4f7731e Add WebToons/DrawnToYou 2021-03-19 13:49:32 +01:00
Techwolf
c09f8c991b Add WebToons/YouveGottaBeKittenMe 2021-03-19 13:49:32 +01:00
Techwolf
5ede74cf63 Add WebToons/MyAssassinGirlfriend 2021-03-19 13:49:32 +01:00
Techwolf
348b2b7683 Add WebToons/Blackened 2021-03-19 13:49:32 +01:00
Techwolf
119adcfbd3 Add WebToons/MyDragonGirlfriend 2021-03-19 13:49:32 +01:00
Techwolf
707fb1cbf8 Add WebToons/Phase 2021-03-19 13:49:32 +01:00
Techwolf
2381d7d004 Add WebToons/CChansACatgirl 2021-03-19 13:49:32 +01:00
Techwolf
3b50c5738f Add WebToons/SpellsFromHell 2021-03-19 13:49:32 +01:00
Techwolf
55808bd921 Add MarryMe 2021-03-19 13:49:32 +01:00
Techwolf
fe0f86de95 Add DungeonsAndDenizens 2021-03-19 13:49:32 +01:00
Techwolf
555d418ee3 Fix ShivaeStudios comics 2021-03-19 13:49:01 +01:00
Techwolf
73ac10dc84 Fix LazJonesAndTheMayfieldRegulators 2021-03-19 13:40:39 +01:00
Techwolf
c532f80bcd Fix ByTheBook 2021-03-19 13:40:39 +01:00
Techwolf
5d384f311e Add Recursion 2021-03-19 13:40:39 +01:00
Techwolf
5339eec876 Fix Supercell 2021-03-19 13:40:39 +01:00
Techwolf
d6bc0502a7 Add VampireHunterBoyfriends 2021-03-19 13:40:39 +01:00
Techwolf
fb8450e93a Add FalseStart 2021-03-19 13:40:39 +01:00
Techwolf
1a690bdeaf Fix NineToNine, SwordsAndSausages, and CollegeCatastrophe 2021-03-19 13:40:39 +01:00
Techwolf
0fc196eb3e Fix MangaDex endOfLife check 2021-03-19 13:40:39 +01:00
Techwolf
13c612c26e Fix Bethellium and Moonlace 2021-03-19 13:40:39 +01:00
Techwolf
4ccb66684f Add MangaDex/TamenDeGushi 2021-03-19 13:40:39 +01:00
Techwolf
bedda970cc Add MangaDex/SenpaiGaUzaiKouhaiNoHanashi 2021-03-19 13:40:39 +01:00
Techwolf
d024427e83 Add WebToons/CityVamps 2021-03-19 13:40:39 +01:00
Techwolf
f4a537038a Fix ComicFury/MayTheRainCome 2021-03-19 13:40:39 +01:00
Techwolf
258f147bdf Add WebToons/BladesOfFurry 2021-03-19 13:40:39 +01:00
Techwolf
8579f2136c Add WebToons/LoveMeKnot 2021-03-19 13:40:39 +01:00
Techwolf
bd93dc1d78 Add MangaDex/PashiriNaBokuToKoisuruBanchouSan 2021-03-19 13:40:39 +01:00
Techwolf
5655b18374 Add WebToons/CrystalVirus 2021-03-19 13:40:39 +01:00
Techwolf
4d7fc0b9f0 Add MangaDex/FuguushokuKajishiDakedoSaikyouDesu 2021-03-19 13:40:39 +01:00
Techwolf
dda1474ed0 Add MangaDex/BokuNoKokoroNoYabaiYatsu 2021-03-19 13:40:39 +01:00
Techwolf
e96ff4b64e Mark PowerNap as endOfLife 2021-03-19 13:40:39 +01:00
Techwolf
90c23149ad Fix QuantumVibe 2021-03-19 13:40:39 +01:00
Techwolf
93c9548fab Add WebToons/Petrichor 2021-03-19 13:40:39 +01:00
Techwolf
a6a8c327c6 Add WebToons/DatingWithATail 2021-03-19 13:40:39 +01:00
Techwolf
396a526011 Add MangaDex/MyTinySenpaiFromWork 2021-03-19 13:40:39 +01:00
Nick Bronson
eac197afc2 Fixes to avoid redirect and whitespace warnings. 2021-03-18 18:08:39 +11:00
Nick Bronson
a7db3db83d Cyanide & Happiness regex change
Small modification to fix failures on several outlier comics where the URL was set differently.
2021-03-18 00:40:29 +11:00
Nick Bronson
59e7e16100 Added WebToon comics
Dungeon Minis, Game Masters, Omniscient Reader & The Weekly Roll
2021-03-17 21:53:47 +11:00
Nick Bronson
f05697a811 Fix URL bug for Cyanide & Happiness 2021-03-17 20:02:28 +11:00
Tobias Gruetzmacher
5508539ca8 Merge all Shivae Studio comics into one file
And they should all work again.
2021-02-02 00:41:08 +01:00
Tobias Gruetzmacher
667cd17830 Update SnafuComics 2021-02-01 22:02:54 +01:00
Tobias Gruetzmacher
ac0d2d2095 Skip video pages for MonkeyUser 2021-02-01 00:29:36 +01:00
Tobias Gruetzmacher
2ecfcaec17 Update GoComics 2021-02-01 00:19:22 +01:00
Tobias Gruetzmacher
87f4049347 Try to fix some more comics using the Internet Archive 2021-01-31 23:40:21 +01:00
Techwolf
9c87c214b6 Fix VGCats, VGCats/Adventure, VGCats/Super 2021-01-19 01:46:53 +01:00
Tobias Gruetzmacher
c43bc0cef4 Fix duplicate module detection 2021-01-19 01:19:07 +01:00
Tobias Gruetzmacher
6835d490d8 Remove 3 MangaDex modules which were taken down 2021-01-19 01:07:36 +01:00
Tobias Gruetzmacher
0067114110 Remove some ComicFury modules which are gone 2021-01-19 00:55:52 +01:00
Tobias Gruetzmacher
3d05e59c36 Fix some modules (a-c) 2021-01-18 01:25:34 +01:00
Tobias Gruetzmacher
a5bfbdb07b Merge Flowerlark Studios comics into one file 2021-01-18 01:25:34 +01:00
toonn
1040d4a976
Add Bird-boy (#183)
Co-authored-by: Tobias Gruetzmacher <tobias-git@23.gs>
2021-01-17 21:48:40 +01:00
Tobias Gruetzmacher
890f116179 Remove SmackJeeves
It was closed at the end of 2020...
2021-01-10 19:18:45 +01:00
Tobias Gruetzmacher
c3227c33f5 Fix WebToons cookies 2021-01-09 20:33:36 +01:00
Tobias Gruetzmacher
5966e8c523 Show Python version in "--version" output 2020-12-24 14:05:48 +01:00
Arwarld
651bd62267
Fix for Ruthe.de and NichtLustig.de (now joscha.com) (#181)
NichtLustig moved to joscha.com, new layout and image-names.

Co-authored-by: Marius Knabben <marius.knabben@rwth-aachen.de>
2020-12-02 22:37:18 +01:00
Techwolf
740abed621 Add MangaDex/JahySamaWaKujikenai 2020-11-09 00:54:20 +01:00
Techwolf
ed00c29ce8 Add TekMage 2020-11-09 00:54:20 +01:00
Techwolf
e6260b6cc5 Add ComicFury/Jack 2020-11-09 00:54:20 +01:00
Techwolf
5345a5a276 Add KemonoCafe/KnuckleUp 2020-11-09 00:54:20 +01:00
Techwolf
a3eb2ec9f3 Fix BetterDays and OriginalLife 2020-11-09 00:54:20 +01:00
Techwolf
e82c8f0f8d Mark CarryOn alt comics as endOfLife 2020-11-09 00:54:20 +01:00
Techwolf
ba214da8b0 Add MangaDex/Lv2KaraCheatDattaMotoYuushaKouhoNoMattariIsekaiLife 2020-11-09 00:54:20 +01:00
Techwolf
25efa82388 Add MangaDex/MaouNoOreGaDoreiElfWoYomeNiShitandaGaDouMederebaIi 2020-11-09 00:54:20 +01:00
Techwolf
5d62183806 Add WebToons/TokyoThreatDocumentationProject 2020-11-09 00:54:20 +01:00
Techwolf
348434e182 Add MangaDex/SousouNoFrieren 2020-11-09 00:54:20 +01:00
Techwolf
588597837a Add MangaDex/HowToOpenATriangularRiceball 2020-11-09 00:54:20 +01:00
Techwolf
6611b1e430 Fix WebToons completion detection 2020-11-09 00:54:20 +01:00
Techwolf
4caef255b4 Add WebToons/OVERPOWERED 2020-11-09 00:54:20 +01:00
Techwolf
f6eb2f088e Add WebToons/Defects 2020-11-09 00:54:20 +01:00
Techwolf
722df37080 Fix DeerMe 2020-11-09 00:54:20 +01:00
Techwolf
7d36a2e552 Add Unconventional 2020-11-09 00:54:20 +01:00
Tobias Gruetzmacher
2fcd8d92f2 Fix GeeksNextDoor (& switch to ParserScraper) 2020-10-18 20:44:47 +02:00
Tobias Gruetzmacher
fdf8164d22 Fix KuroShouri (switch to ParserScraper) 2020-10-18 20:28:00 +02:00
Tobias Gruetzmacher
e64635e86b Stricter style checking & related style fixes 2020-10-11 20:15:27 +02:00
Ruthalas
e84bbe2667
Add three new comics (#177)
* Add BloopBloopRelationshipComic to Webtoons
* Add Todd Allison & the Petunia Violet to Smack Jeeves
* Add Oversimplified SCP to MangaDex

Co-authored-by: Tobias Gruetzmacher <tobias-git@23.gs>
2020-10-11 11:30:02 +02:00
Tobias Gruetzmacher
d487485815 Read scraper modules from user data directory
This allows users to add scrapers without setting up a complete Python
development environment.
2020-10-04 23:46:57 +02:00
Tobias Gruetzmacher
0bdf3dd94b Allow adding external directories to the plugin package 2020-10-04 22:28:51 +02:00
Tobias Gruetzmacher
3256f9fdc2 Hardcode the "plugins" package name 2020-10-04 22:28:51 +02:00
Tobias Gruetzmacher
9237bd62b2 Convert scraper cache to a class
This should make it easier to extend with additional entries.
2020-10-04 22:28:51 +02:00
Tobias Gruetzmacher
26f63fd994 Remove unused property in Output class 2020-10-04 22:12:34 +02:00
Tobias Gruetzmacher
d93fed567c Distribute PyInstaller hook
This uses a feature introduced in PyInstaller 4.0 to register hooks as
entry points and automatically discover them whenever a module is used.
2020-10-01 15:51:09 +02:00
Tobias Gruetzmacher
7c2c9c3778 Fix VGCats & switch to HTTPS 2020-09-30 21:37:29 +02:00
Tobias Gruetzmacher
bb71adbe72 Remove VictimsOfTheSystem
Domain seems to be gone for some month and archive.org doesn't have
enough to switch.
2020-09-30 21:29:58 +02:00
Tobias Gruetzmacher
23a3e3b101 Switch WastedTalent to ParserScraper 2020-09-30 21:24:01 +02:00
Tobias Gruetzmacher
dfa412c03e Skip videos for WapsiSquare 2020-09-30 21:17:46 +02:00
Tobias Gruetzmacher
bbe3d9f9be Remove WebDesignerCOTW 2020-09-30 21:16:55 +02:00
Tobias Gruetzmacher
437afc685d Add GenshinImpact to Webtoons module 2020-09-30 18:24:33 +02:00
Tobias Gruetzmacher
e4f1f59432 Fix ButtercupFestival 2020-09-29 23:31:38 +02:00
Tobias Gruetzmacher
dc41c318e9 Fix ButterSafe 2020-09-29 23:04:56 +02:00
Tobias Gruetzmacher
f31c0c681d Fix BusinessCat and mark as endOfLife 2020-09-29 23:00:58 +02:00
Tobias Gruetzmacher
2364e8bab3 Fix Brink and mark as endOfLife 2020-09-29 23:00:53 +02:00
Tobias Gruetzmacher
5afab71e7f Fix BoredAndEvil 2020-09-29 22:35:09 +02:00
Tobias Gruetzmacher
5863ddb73b Fix BookOfBiff 2020-09-29 22:27:32 +02:00
Tobias Gruetzmacher
bbac2f99a7 Fix BetweenFailures 2020-09-29 22:22:55 +02:00
Tobias Gruetzmacher
3330162c6b Fix BetterDays with archive.org 2020-09-29 21:35:41 +02:00
Tobias Gruetzmacher
e98a1601ca Remove workaround for libxml2 older 2.9.3 (2015)
This workaround was written in 2016 while that version was still found
on many systems. Addionally, this workaround needs to be enabled by the
developer, who might not even be aware that they need to enable it for a
specific module. We still throw a warning to the user if running with
such an old libxml version.
2020-09-29 21:16:48 +02:00
Tobias Gruetzmacher
c065a820ed Fix BadMachinery and mark as endOfLife 2020-09-29 21:01:33 +02:00
Tobias Gruetzmacher
24efafdf96 Fix BMovieComic 2020-09-29 20:30:44 +02:00
Tobias Gruetzmacher
0fe32e7562 Remove unused f-string
Since we still want to support Python 3.5 for a bit, we should avoid
f-strings until we finally drop support for that.
2020-09-28 22:19:48 +02:00
Tobias Gruetzmacher
7e040086b6 Try to inform the user about geo-blocks
Instead of letting the crawler run into "random" error messages, throw a
specific "geoblocked" exception instead.
2020-09-28 13:11:34 +02:00
Tobias Gruetzmacher
e34a0b539c Don't rethrow RequestException as IOError
Since RequestException already is an IOError, nothing of value is lost.
2020-09-28 12:05:01 +02:00
Tobias Gruetzmacher
8d7fd8b884 Update GoComics modules
The usual: GoComics removed some comics, added some and renamed some...
2020-09-28 01:15:07 +02:00
Tobias Gruetzmacher
b090d86c9b Switch Drowtales to parser scraper (also fixes it) 2020-09-27 20:56:26 +02:00
Tobias Gruetzmacher
a7f9692ffe Add StalkerXStalker to WebToons module (fixes #173) 2020-09-27 19:21:28 +02:00
Tobias Gruetzmacher
ccb4c32c6b Fix DumbingOfAge 2020-09-27 19:16:34 +02:00
Tobias Gruetzmacher
346b3b4abd Fix ShipInABottle 2020-09-27 18:34:36 +02:00
Tobias Gruetzmacher
632433d896 Move CrapIDrewOnMyLunchBreak to WebToons
Unfortunatly, the original website is broken.
2020-09-27 18:25:33 +02:00
Tobias Gruetzmacher
85c40a942f Remove StuffNoOneToldMe 2020-09-27 15:50:48 +02:00
Tobias Gruetzmacher
8cc916b292 Fix CynWolf 2020-09-27 15:50:25 +02:00