dmoonfire/dosage
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Activity
1925 commits 1 branch 0 tags 31 MiB
Commit graph

1925 commits

This branch
This branch
All branches
Author SHA1 Message Date
Techwolf
5d62183806 Add WebToons/TokyoThreatDocumentationProject 2020-11-09 00:54:20 +01:00
Techwolf
348434e182 Add MangaDex/SousouNoFrieren 2020-11-09 00:54:20 +01:00
Techwolf
588597837a Add MangaDex/HowToOpenATriangularRiceball 2020-11-09 00:54:20 +01:00
Techwolf
6611b1e430 Fix WebToons completion detection 2020-11-09 00:54:20 +01:00
Techwolf
4caef255b4 Add WebToons/OVERPOWERED 2020-11-09 00:54:20 +01:00
Techwolf
f6eb2f088e Add WebToons/Defects 2020-11-09 00:54:20 +01:00
Techwolf
722df37080 Fix DeerMe 2020-11-09 00:54:20 +01:00
Techwolf
7d36a2e552 Add Unconventional 2020-11-09 00:54:20 +01:00
Tobias Gruetzmacher
a5d4d041ec Don't mention removed start script 2020-11-06 01:29:45 +01:00
Tobias Gruetzmacher
848f051a89 Test on Python 3.9 2020-10-31 11:49:16 +01:00
Tobias Gruetzmacher
2fcd8d92f2 Fix GeeksNextDoor (& switch to ParserScraper) 2020-10-18 20:44:47 +02:00
Tobias Gruetzmacher
fdf8164d22 Fix KuroShouri (switch to ParserScraper) 2020-10-18 20:28:00 +02:00
Tobias Gruetzmacher
e64635e86b Stricter style checking & related style fixes 2020-10-11 20:15:27 +02:00
Ruthalas
e84bbe2667
Add three new comics (#177) 
* Add BloopBloopRelationshipComic to Webtoons
* Add Todd Allison & the Petunia Violet to Smack Jeeves
* Add Oversimplified SCP to MangaDex

Co-authored-by: Tobias Gruetzmacher <tobias-git@23.gs>
 2020-10-11 11:30:02 +02:00
Tobias Gruetzmacher
d487485815 Read scraper modules from user data directory 
This allows users to add scrapers without setting up a complete Python
development environment.
 2020-10-04 23:46:57 +02:00
Tobias Gruetzmacher
0bdf3dd94b Allow adding external directories to the plugin package 2020-10-04 22:28:51 +02:00
Tobias Gruetzmacher
3256f9fdc2 Hardcode the "plugins" package name 2020-10-04 22:28:51 +02:00
Tobias Gruetzmacher
9237bd62b2 Convert scraper cache to a class 
This should make it easier to extend with additional entries.
 2020-10-04 22:28:51 +02:00
Tobias Gruetzmacher
5ec0710d26 Enhance test to test timestamp output 2020-10-04 22:14:06 +02:00
Tobias Gruetzmacher
26f63fd994 Remove unused property in Output class 2020-10-04 22:12:34 +02:00
Tobias Gruetzmacher
c86e1ad863 Fix stupid error :D 2020-10-01 18:18:31 +02:00
Tobias Gruetzmacher
020b8497c0 Generate start script for PyInstaller from entry point 2020-10-01 16:13:08 +02:00
Tobias Gruetzmacher
d93fed567c Distribute PyInstaller hook 
This uses a feature introduced in PyInstaller 4.0 to register hooks as
entry points and automatically discover them whenever a module is used.
 2020-10-01 15:51:09 +02:00
Tobias Gruetzmacher
d0162cfb32 Add a test for the "dosage @" feature 2020-10-01 14:35:33 +02:00
Tobias Gruetzmacher
d06e642b56 Assume developers are using virtualenvs 
If this is true, we don't have to mess with sys.path or provide a
"convenience" launcher anymore.
 2020-10-01 14:11:11 +02:00
Tobias Gruetzmacher
7c2c9c3778 Fix VGCats & switch to HTTPS 2020-09-30 21:37:29 +02:00
Tobias Gruetzmacher
bb71adbe72 Remove VictimsOfTheSystem 
Domain seems to be gone for some month and archive.org doesn't have
enough to switch.
 2020-09-30 21:29:58 +02:00
Tobias Gruetzmacher
23a3e3b101 Switch WastedTalent to ParserScraper 2020-09-30 21:24:01 +02:00
Tobias Gruetzmacher
dfa412c03e Skip videos for WapsiSquare 2020-09-30 21:17:46 +02:00
Tobias Gruetzmacher
bbe3d9f9be Remove WebDesignerCOTW 2020-09-30 21:16:55 +02:00
Tobias Gruetzmacher
437afc685d Add GenshinImpact to Webtoons module 2020-09-30 18:24:33 +02:00
Tobias Gruetzmacher
e4f1f59432 Fix ButtercupFestival 2020-09-29 23:31:38 +02:00
Tobias Gruetzmacher
dc41c318e9 Fix ButterSafe 2020-09-29 23:04:56 +02:00
Tobias Gruetzmacher
f31c0c681d Fix BusinessCat and mark as endOfLife 2020-09-29 23:00:58 +02:00
Tobias Gruetzmacher
2364e8bab3 Fix Brink and mark as endOfLife 2020-09-29 23:00:53 +02:00
Tobias Gruetzmacher
5afab71e7f Fix BoredAndEvil 2020-09-29 22:35:09 +02:00
Tobias Gruetzmacher
5863ddb73b Fix BookOfBiff 2020-09-29 22:27:32 +02:00
Tobias Gruetzmacher
bbac2f99a7 Fix BetweenFailures 2020-09-29 22:22:55 +02:00
Tobias Gruetzmacher
3330162c6b Fix BetterDays with archive.org 2020-09-29 21:35:41 +02:00
Tobias Gruetzmacher
e98a1601ca Remove workaround for libxml2 older 2.9.3 (2015) 
This workaround was written in 2016 while that version was still found
on many systems. Addionally, this workaround needs to be enabled by the
developer, who might not even be aware that they need to enable it for a
specific module. We still throw a warning to the user if running with
such an old libxml version.
 2020-09-29 21:16:48 +02:00
Tobias Gruetzmacher
c065a820ed Fix BadMachinery and mark as endOfLife 2020-09-29 21:01:33 +02:00
Tobias Gruetzmacher
24efafdf96 Fix BMovieComic 2020-09-29 20:30:44 +02:00
Tobias Gruetzmacher
4d8c4e8d2e Allow proxies for module tests 
This should make it easier to include geo-blocked comics.
 2020-09-29 01:50:02 +02:00
Tobias Gruetzmacher
b6631f5715 Don't install flake8-no-fstring on Python 3.5 :D 2020-09-28 22:52:39 +02:00
Tobias Gruetzmacher
0fe32e7562 Remove unused f-string 
Since we still want to support Python 3.5 for a bit, we should avoid
f-strings until we finally drop support for that.
 2020-09-28 22:19:48 +02:00
Tobias Gruetzmacher
7e040086b6 Try to inform the user about geo-blocks 
Instead of letting the crawler run into "random" error messages, throw a
specific "geoblocked" exception instead.
 2020-09-28 13:11:34 +02:00
Tobias Gruetzmacher
e34a0b539c Don't rethrow RequestException as IOError 
Since RequestException already is an IOError, nothing of value is lost.
 2020-09-28 12:05:01 +02:00
Tobias Gruetzmacher
8d7fd8b884 Update GoComics modules 
The usual: GoComics removed some comics, added some and renamed some...
 2020-09-28 01:15:07 +02:00
Tobias Gruetzmacher
b090d86c9b Switch Drowtales to parser scraper (also fixes it) 2020-09-27 20:56:26 +02:00
Tobias Gruetzmacher
5652b9cb23 Don't install some flake8 plugins on Python 3.5 
They were never available in Python 3.5, so this breaks the CI build.
 2020-09-27 20:38:06 +02:00