Tobias Gruetzmacher
fc4655157a
Move some more modules to HTTPS and fix canonical URL
2020-01-12 22:41:06 +01:00
Tobias Gruetzmacher
5e09b631db
Move EyeOfRamalach to KemonoCafe
2020-01-12 22:41:06 +01:00
Techwolf
b168e05cfd
Add site engine for Derideal
2020-01-12 21:29:47 +01:00
Techwolf
d232d1e43d
Add LifeAsRendered
2020-01-12 21:29:47 +01:00
Techwolf
0aa1ba3c66
Fix CatsAndCameras
2020-01-12 21:29:47 +01:00
Techwolf
4219b32074
Fix VerloreGeleentheid
2020-01-12 21:29:47 +01:00
Techwolf
15ac062dff
Fix DoemainOfOurOwn
2020-01-12 21:29:47 +01:00
Techwolf
947405a50d
Fix Djandora
2020-01-12 21:29:47 +01:00
Techwolf
aa22d05c44
Fix WLP/PeterIsTheWolf
2020-01-12 21:29:47 +01:00
Techwolf
1a938ffa05
Fix Python 2 compatibility for Newshounds
2020-01-12 21:29:47 +01:00
Techwolf
241732bbba
Add common handler for mgsisk's Wordpress Webcomic plugin
2020-01-12 21:29:47 +01:00
Techwolf
c44a5a3b43
Mark endOfLife comics as completed when checking for updates
2020-01-12 21:29:47 +01:00
Techwolf
121e63c069
Add site engine for StudioKhimera
2020-01-12 21:29:47 +01:00
Techwolf
f49dff13be
Move AddictiveScience to KemonoCafe
2020-01-12 21:29:47 +01:00
Techwolf
af44479b5f
Add DesertFox
2020-01-12 21:29:47 +01:00
Techwolf
81b92a8c18
Add KemonoCafe site engine
2020-01-12 21:29:47 +01:00
Techwolf
419c41fb74
Add PeterAndWhitney
2020-01-12 21:19:56 +01:00
Techwolf
6131c9e2b5
Add PeterAndCompany
2020-01-12 21:19:56 +01:00
Techwolf
163b252776
Add ArtificialIncident
2020-01-12 21:19:56 +01:00
Techwolf
e54d153ca7
Add YoshSaga
2020-01-12 21:19:56 +01:00
Techwolf
3503a2bede
Add site engine for RHJunior and migrate existing comics
2020-01-12 21:19:56 +01:00
Techwolf
0fed6c2c36
Add site engine for NamirDeiter comics and migrate existing comics
2020-01-12 21:19:56 +01:00
Techwolf
4795613343
Add site engine for ProjectFuture
2020-01-12 19:12:26 +01:00
Techwolf
6576dd7ec7
Add ShadesOfGray
2020-01-12 18:45:06 +01:00
Tobias Gruetzmacher
e393ccdce1
Fix Qwantz
2020-01-09 23:36:05 +01:00
Tobias Gruetzmacher
290f4167d9
Remove Creators/Recess, it's gone
2020-01-09 23:27:46 +01:00
Tobias Gruetzmacher
f0348d6b90
Remove ComicGenesis/Cerintha, it's broken
...
Additionally, it also exists on SmackJeeves.
2020-01-09 23:23:20 +01:00
Tobias Gruetzmacher
d9045030cc
Remove ComicFury/Khulthagar, it's gone
2020-01-09 23:17:43 +01:00
Tobias Gruetzmacher
0f28770aaa
Remove old ComicSherpa submodules
2020-01-09 23:15:52 +01:00
Tobias Gruetzmacher
3fe40326e0
Fix AbstruseGoose
2020-01-09 22:59:12 +01:00
Tobias Gruetzmacher
42b5aa9321
Fix Unsounded ( fixes #107 )
2020-01-09 22:21:20 +01:00
Tobias Gruetzmacher
cfd7b80053
Fix Exiern ( fixes #119 )
2020-01-09 21:13:54 +01:00
Tobias Gruetzmacher
18546c194a
Switch SomethingPositive to parser ( fixes #128 )
2020-01-09 21:00:24 +01:00
Tobias Gruetzmacher
041d8be7ef
Re-add one comic that got lost
2020-01-09 19:46:24 +01:00
Tobias Gruetzmacher
3c930167c6
Remove some comics which disappeared
2020-01-09 18:04:09 +01:00
Tobias Gruetzmacher
ff3b7f2cbe
Fix AltermetaOld
2020-01-09 17:41:07 +01:00
Tobias Gruetzmacher
752525c3e9
Fix some old modules using the Internet Archive
2020-01-09 17:38:13 +01:00
Tobias Gruetzmacher
275370a835
Drop webcomic.eu (site is gone)
...
Not enough archived history to switch to Internet Archive...
2020-01-09 03:52:40 +01:00
Techwolf
63da9dd0d4
Fix Newshounds & ProjectionEdge split
2020-01-09 03:33:19 +01:00
Techwolf
5da87d5a66
Fix firstStripUrl for ComicFury comics
2020-01-09 03:24:09 +01:00
Techwolf
dfbc0b148a
Fix Meek
2020-01-09 03:20:34 +01:00
Tobias Gruetzmacher
f05a8c33d4
Take Djandora from Internet Archive
2020-01-09 03:20:34 +01:00
Techwolf
d6e255ea4a
Add ProphecyOfTheCircle
2020-01-09 03:20:34 +01:00
Techwolf
f52075d200
Add OffCentaured
2020-01-09 03:20:34 +01:00
Techwolf
32529da7e6
Add SmackJeeves/GrowingTroubles
2020-01-09 03:20:34 +01:00
Techwolf
cffa42a243
Add FoxDad
2020-01-09 03:20:34 +01:00
Techwolf
3df0d6d715
Add MyLifeWithFel
2020-01-09 03:20:34 +01:00
Techwolf
f7886636cc
Add SpaceFurries
2020-01-09 03:20:34 +01:00
Techwolf
e1c7538c44
Restore LatchkeyKingdom
2020-01-09 03:20:34 +01:00
Techwolf
6763c105b7
Add StolenGeneration
2020-01-09 01:13:32 +01:00
Ruthalas
1d6e80f928
Added unDivine ( #147 )
...
Added comic unDivine using ComicControlScraper
2020-01-09 00:17:07 +01:00
Tobias Gruetzmacher
c48d96d5e3
Use imagesize instead of Pillow to get image sizes
2020-01-04 15:59:49 +01:00
Tobias Gruetzmacher
a8ab66f781
Hello, 2020!
2020-01-04 13:41:37 +01:00
Tobias Gruetzmacher
2afc0cb645
Director: Always cleanup jobs & threads
...
This shouldn't make a difference in normal dosage usage (since dosage
exits right after this anyways), but it removes hidden dependencies
between tests and might help when embedding dosage into other Python
software. As a side effect this gets rid of some global state, so
running dosage multiple times in the same process is more reliable...
2020-01-04 13:39:14 +01:00
Ruthalas
65228ec2d0
Added Headless Bliss ( #146 )
2020-01-01 20:53:34 +01:00
Tobias Gruetzmacher
e5cb27ffa4
Minor URL fixes, switched some modules to https
2019-12-31 01:44:19 +01:00
Tobias Gruetzmacher
be41a692ba
Slightly better namer for CloneManga modules
2019-12-31 00:50:22 +01:00
Tobias Gruetzmacher
5a92505606
Fix & test query string parsing
2019-12-31 00:43:46 +01:00
Tobias Gruetzmacher
addf78fcbc
More cleanup after SmackJeeves redesign
2019-12-29 23:19:17 +01:00
Tobias Gruetzmacher
c0ebf9bbe2
Clean up ComicSherpa
2019-12-29 20:53:41 +01:00
Tobias Gruetzmacher
d0bc0572c2
Add a test to ckeck that renamed comics point to existing comics
2019-12-29 20:50:56 +01:00
Tobias Gruetzmacher
58611fe600
Remove missing GoComics submodules
2019-12-29 02:17:52 +01:00
Tobias Gruetzmacher
7e9790c7a8
Remove vanished Arcamax comics
2019-12-28 01:00:53 +01:00
Tobias Gruetzmacher
42b23c4537
Force CommitStrip to always use TLS
2019-12-28 00:16:41 +01:00
Techwolf
83cd91e229
Fix WLP/PeterIsTheWolf
2019-12-27 23:34:07 +01:00
Techwolf
0b675572df
Fix MonsterUnderTheBed
2019-12-27 23:34:07 +01:00
Techwolf
33954aaab6
Add CatNine/TakeTwo
2019-12-27 23:34:07 +01:00
Techwolf
ae7ab251fd
Add CatNine
2019-12-27 23:34:07 +01:00
Techwolf
2fe790f91c
Add VickiFox
2019-12-27 23:34:07 +01:00
Techwolf
0fcad21b55
Add Prototype
2019-12-27 23:34:07 +01:00
Techwolf
e7ebecd03d
Add TwokindsSketches
2019-12-27 23:34:07 +01:00
Techwolf
0f8fe12c10
Add LilithsWord
2019-12-27 23:34:07 +01:00
Techwolf
38a29f631f
Add FarToTheNorth
2019-12-27 23:34:07 +01:00
Techwolf
a6310c8eeb
Fix WhiteNoise
2019-12-27 23:34:07 +01:00
Techwolf
1fdb644cb3
Add TailsAndTactics
2019-12-27 23:34:07 +01:00
Techwolf
31b2c3c6ec
Fix Namesake
2019-12-27 23:34:07 +01:00
Techwolf
93dc03a429
Fix SleeplessDomain
2019-12-27 23:34:07 +01:00
Techwolf
d8ac47f386
Add FurPiled
2019-12-27 23:34:07 +01:00
Techwolf
8f66fd3647
Fix DominicDeegan (no longer endOfLife)
2019-12-27 23:34:07 +01:00
Techwolf
e156da4457
Add ComicFury/ResNulliusCS
2019-12-27 23:34:07 +01:00
Techwolf
7a54278ccd
Add ClanOfTheCats/Reunion
2019-12-27 23:34:07 +01:00
Techwolf
b340403013
Add ClanOfTheCats
2019-12-27 23:34:07 +01:00
Techwolf
3d75493183
Add CarryOn/LegendOfAnneBunny
2019-12-27 22:58:44 +01:00
Techwolf
dd88b7f133
Add CarryOn/AliceBlueAndTheGardensOfQ
2019-12-27 22:58:44 +01:00
Techwolf
95fe4b331c
Add Erfworld
2019-12-27 22:52:43 +01:00
Techwolf
d39055e494
Fix PvPOnline
2019-12-27 22:48:16 +01:00
Techwolf
7e9ca4daba
Add AntiheroForHire
2019-12-27 22:40:27 +01:00
Techwolf
f15f5920c0
Add CrossTimeCafe
2019-12-27 22:30:00 +01:00
Techwolf
dd8200312a
Fix Galaxion
2019-12-27 22:25:27 +01:00
Techwolf
5705b1a3eb
Add SeelPeel
2019-12-27 22:22:08 +01:00
Techwolf
28bf733ca5
Add Ginpu
2019-12-27 22:20:02 +01:00
Techwolf
c5f57288ec
Refactor WP_LATEST_SEARCH into class variable of WordPressScraper
2019-12-27 22:20:02 +01:00
Techwolf
d732767e02
Fix Freefall
2019-12-27 22:20:02 +01:00
Techwolf
18379b8fc0
Add ComicFury/Snowfall
2019-12-27 22:20:02 +01:00
Techwolf
d733baba48
Add TheJunkHyenasDiner
2019-12-27 22:20:02 +01:00
Techwolf
deda610666
Add PowerNap
2019-12-27 22:20:02 +01:00
Techwolf
e4ded6da25
Add Vexxarr
2019-12-27 22:20:02 +01:00
Techwolf
60d37bcc1b
Add CritterCoven
2019-12-27 22:20:02 +01:00
Techwolf
078841055b
Add TinyDickAdventures
2019-12-27 22:20:02 +01:00
Techwolf
c260640e0a
Add NonPlayerCharacter
2019-12-27 22:20:02 +01:00
Techwolf
4d9eb7df4b
Fix LookingForGroup
2019-12-27 22:20:02 +01:00
Techwolf
6364b34233
Add DeerMe
2019-12-27 22:20:02 +01:00
Techwolf
5e4bd6f34a
Add PeanutBerrySundae
2019-12-27 22:20:02 +01:00
Techwolf
44d2198c34
Add AmbersNoBrainers
2019-12-27 22:20:02 +01:00
Techwolf
e6b1895d3b
Fix RealLife
2019-12-27 22:20:02 +01:00
Techwolf
561646c9c4
Add Delve
2019-12-27 22:20:02 +01:00
Techwolf
2857189068
Fix Weregeek
2019-12-27 20:24:03 +01:00
Techwolf
b383a92f65
Add AdventuresOfFifne
2019-12-27 18:41:49 +01:00
Techwolf
afb2275209
Add WereIWolf
2019-12-27 18:38:02 +01:00
Techwolf
24b4d1a391
Add BeyondTheVeil
2019-12-27 18:17:18 +01:00
Techwolf
9e6bd4803a
Add Centralia2050
2019-12-27 18:16:01 +01:00
Techwolf
523a65581c
Fix MareInternum
2019-12-27 18:15:43 +01:00
Techwolf
8f335c15e1
Add TwinDragons
2019-12-27 17:40:00 +01:00
Techwolf
55180babcf
Add Kitfox
2019-12-27 17:38:37 +01:00
Techwolf
de67d4f63b
Fix KiwiBlitz
2019-12-27 17:37:56 +01:00
Techwolf
255e27eec2
Add HeyKitty
2019-12-27 17:36:51 +01:00
Techwolf
5df3cff92b
Add ComicFury/DNA
2019-12-27 17:35:55 +01:00
Techwolf
66e655c074
Add HeyFox
2019-12-27 17:34:58 +01:00
Techwolf
047f023e50
Fix QuestionableContent
2019-12-27 17:34:38 +01:00
Techwolf
5e79ff9d84
Fix DoghouseDiaries
2019-12-27 17:33:53 +01:00
Techwolf
fc280badb6
Add ComicFury/Swashbuckled
2019-12-27 17:32:47 +01:00
Techwolf
eecffa53a7
Add ComicFury/RedSpot
2019-12-27 17:32:26 +01:00
Techwolf
4c495b78f7
Add DarkWhite
2019-12-27 17:26:22 +01:00
Techwolf
0a81948386
Add ComicFury/Saluna
2019-12-27 17:25:44 +01:00
Techwolf
4d8b35182d
Add ComicFury/Threan
2019-12-27 17:25:23 +01:00
Techwolf
1377830917
Add BallerinaMafia
2019-12-27 17:23:44 +01:00
Techwolf
fff73f9a09
Add FurthiaHigh
2019-12-27 17:23:01 +01:00
Techwolf
ea94a62f58
Fix Concession
2019-12-27 17:22:08 +01:00
Techwolf
f6b358c754
Add VirmirWorld
2019-12-27 17:21:05 +01:00
Techwolf
6df6549c97
Split Catena into CatenaCafe & CatenaManor
2019-12-27 17:16:06 +01:00
Techwolf
1fe966102f
Add OzyAndMillie
2019-12-27 16:14:30 +01:00
Techwolf
306a6eca8c
Add CynWolf
2019-12-27 16:12:43 +01:00
Techwolf
15c9426a2d
Add CommanderKitty
2019-12-27 16:12:17 +01:00
Techwolf
f99839f42a
Add BittersweetCandyBowl
2019-12-27 16:11:01 +01:00
Techwolf
48ed091878
Add PoppyOPossum
2019-12-27 16:10:22 +01:00
Techwolf
9aa0845b17
Fix StandStillStaySilent
2019-12-27 16:09:54 +01:00
Techwolf
8831984991
Add MynarskiForest
2019-12-27 16:07:08 +01:00
Techwolf
24573c4793
Fix Shivae and add side comics
2019-12-27 15:59:10 +01:00
Tobias Gruetzmacher
963db7f448
Adopt SmackJeeves to new site design ( fixes #144 )
...
Some things got lost on the way:
- Since there is no comprehensive comic directory anymore, removed
automatic update script. New comics need to be added manually.
- Some authors used the opportunity to move from SmackJeeves somewhere
else - some of those got new modules (either standalone or ComicFury)
- Abunch of comics just disappeared...
2019-12-26 22:03:18 +01:00
Tobias Gruetzmacher
44af7acdbc
Clean up ComicFury a bit
...
Mostly getting rid of vanished comics, but also cleaning up the matching
expressions a bit.
2019-12-25 23:05:11 +01:00
Techwolf
ab488cbbd6
Add TheCyantianChronicles comics
2019-12-17 01:29:25 +01:00
Techwolf
24ee2e2594
Add AbbysAgency
2019-12-17 01:28:02 +01:00
Techwolf
becc3b98f6
Add AlienDice and AlienDice/Legacy
2019-12-17 01:28:02 +01:00
Techwolf
1def5739b1
Add Tamberlane
2019-12-17 01:28:02 +01:00
Techwolf
05854b6ddf
Add DocRat
2019-12-17 01:28:02 +01:00
Techwolf
0d8a803913
Add LittleTales
2019-12-17 01:28:02 +01:00
Techwolf
a3d3ea92db
Add NeoCTC
2019-12-17 01:28:02 +01:00
Techwolf
ab366f12d3
Add FreighterTails
2019-12-17 01:28:02 +01:00
Techwolf
85a7adadc9
Add TheGentleWolf
2019-12-17 01:28:02 +01:00
Techwolf
6e125fb05f
Merge TracyAndTristan into ComicFury list
2019-12-17 01:28:02 +01:00
Techwolf
48e064cf1b
Add ComicFury/Eros
2019-12-17 01:28:02 +01:00
Techwolf
9cca10b0b7
Add Ryugou
2019-12-17 01:28:02 +01:00
Techwolf
478125f621
Fix UrgentTransformationCrisis
2019-12-17 01:28:02 +01:00
Techwolf
56573c41ae
Add Anaria
2019-12-17 01:28:02 +01:00
Techwolf
cad90e80e2
Fix SkinDeep
2019-12-17 01:28:02 +01:00
Techwolf
a259a65d83
Add Cloudscratcher
2019-12-17 01:28:02 +01:00
Techwolf
138f6001db
Fix Flipside
2019-12-17 01:28:02 +01:00
Techwolf
d370ac5612
Add TheProbabilityBomb
2019-12-17 01:28:02 +01:00
Techwolf
bc87413b22
Add TheOldVictorian
2019-12-17 01:28:02 +01:00
Techwolf
6ed242a7b0
Add OffWhite
2019-12-17 01:28:02 +01:00
Techwolf
b6f2000c0b
Fix TheGamerCat
2019-12-17 01:28:02 +01:00
Techwolf
1b8c404a15
Add FoxTails
2019-12-17 01:28:02 +01:00
Techwolf
9eacd96f5f
Add Everblue
2019-12-17 01:28:02 +01:00
Techwolf
52b0e15330
Add ErmaFelnaEDF
2019-12-17 01:28:02 +01:00
Techwolf
b131baa561
Add ComicFury/ProfessorAmazingAndTheIncredibleGoldenFox
2019-12-17 01:28:02 +01:00
Techwolf
23afbd1fb7
Fix Misfile, add Misfile/HellHigh
2019-12-17 01:28:02 +01:00
Techwolf
6f47be87f7
Add WildeLife
2019-12-17 01:28:02 +01:00
Techwolf
0b433d5897
Add Wrongside
2019-12-17 01:28:02 +01:00
Techwolf
6a5823b3fe
Add StarfireAgency
2019-12-17 01:28:02 +01:00
Techwolf
e62bcb6267
Add Nightshift
2019-12-17 01:28:02 +01:00
Techwolf
8e8d5e5723
Add TracesOfThePast (SFW/NSFW)
2019-12-17 01:28:02 +01:00
Techwolf
57046e23ca
Add A&HClub
2019-12-17 01:28:02 +01:00
Techwolf
ae0b702107
Add Bethellium
2019-12-17 01:28:02 +01:00
Techwolf
085e1642fb
Add Moonlace
2019-12-17 01:28:02 +01:00
Techwolf
280f4fdbb8
Add CutLoose
2019-12-17 01:28:02 +01:00
Techwolf
81e4942e7c
Add Replay
2019-12-17 01:28:02 +01:00
Techwolf
7e6d33ddf5
Fix Lackadaisy
2019-12-17 01:28:02 +01:00
Techwolf
c9e3923442
Fix AlphaLuna
2019-12-17 01:28:02 +01:00
Techwolf
6cfde34663
Fix Altermeta
2019-12-17 01:28:02 +01:00
Techwolf
2916b39c98
Add ApartmentForTwo
2019-12-17 01:28:02 +01:00
Techwolf
ae82096e96
Add QuentynQuinnSpaceRanger
2019-12-17 01:28:02 +01:00
Techwolf
5fb39c5faf
Add TalesOfTheQuestor
2019-12-17 01:28:02 +01:00
Techwolf
ff4fa7892d
Add QuantumVibe
2019-12-17 01:28:02 +01:00
Techwolf
7d03d3369b
Add LastResort
2019-12-17 01:28:02 +01:00
Techwolf
cb360bc983
Add Bloodline
2019-12-17 01:28:02 +01:00
Techwolf
9e52bea334
Add SixPackOfOtters
2019-12-17 01:28:02 +01:00
Techwolf
0648bbadc9
Add Kaspall
2019-12-17 01:28:02 +01:00
Techwolf
379f963442
Add TheClassMenagerie
2019-12-17 01:28:02 +01:00
Techwolf
a6aacf0cc7
Add DMFA side stories
...
(this also moves all DMFA modules into their own file)
2019-12-17 01:28:02 +01:00
Techwolf
b79f22fb65
Mark KeenSpot/GeneCatlow completed :(
2019-12-17 00:05:47 +01:00
Techwolf
f79d3c9309
Fix GunnerkriggCourt
2019-12-17 00:04:20 +01:00
Tobias Gruetzmacher
835f484673
Simplify ComicsKingdom extraction a bit
2019-12-16 23:47:14 +01:00
littauer
4d369376c0
Add ComicsKingdom.com (aka King Features) ( #134 )
...
Note that going back more than about seven days is a paid feature and is untested.
2019-12-16 23:18:04 +01:00
Tobias Gruetzmacher
adb3db3d22
Add some tests for the update check
2019-12-05 22:23:48 +01:00
Tobias Gruetzmacher
f5253c7f45
Return a fallback "version" if dosage is not installed
...
Additionally, inform the user on how to fix the problem. Thanks to twb
for noticing this.
2019-12-05 21:41:15 +01:00
Tobias Gruetzmacher
7615ee60d1
Switch from pkg_resources to importlib
...
One dependency less (at least für Python 3.8+) and it should be faster
(not tested).
2019-12-05 21:40:45 +01:00
Tobias Gruetzmacher
4511bab996
Fix xkcd pages where comic is linked
2019-12-05 07:10:49 +01:00
Tobias Gruetzmacher
66f154f074
Add throttling for GoComics ( fixes #90 )
...
Since this was the goal of the whole throttling implementation ;)
2019-12-04 00:28:27 +01:00
Tobias Gruetzmacher
a347bebfe3
Add simple host-based throttling
2019-12-04 00:28:27 +01:00
Tobias Gruetzmacher
e5e7dfacd6
Move basic HTTP setup into a new module
...
We now subclass requests' Session to make further extensions of the HTTP
flow possible.
2019-12-03 23:58:20 +01:00
Tobias Gruetzmacher
d2861d029f
Fix "Extra Fabulous Comics" ( fixes #129 )
2019-12-03 19:50:41 +01:00
Tobias Gruetzmacher
ddba32391b
Add BlondeSunrise ( fixes #142 )
2019-12-03 00:14:57 +01:00
Tobias Gruetzmacher
f5a5106381
Allow combining -n with -c or -a (related to #90 )
...
This allows fetching "all" comics (or catch up until the last existing
one) while setting an upper bound on how many pages to fetch at the same
time.
2019-12-01 23:19:06 +01:00
Jakob Kogler
6fd3282047
Add comic "turnoff" ( closes #139 )
2019-12-01 21:46:00 +01:00
hkocharyan
31309a26d2
fixed oglaf next and previous tags ( #141 )
2019-11-19 20:56:54 +01:00
Tobias Gruetzmacher
00d0201c5f
Fix a bunch of flake8 issues
2019-11-04 00:16:25 +01:00
Techwolf
255fbfa1b4
Add Evon
2019-11-03 23:51:58 +01:00
Techwolf
b230ef31d8
Add Guardia
2019-11-03 23:51:27 +01:00
Techwolf
70223bd38f
Add YouSayItFirst
2019-11-03 23:50:47 +01:00
Techwolf
a5a868c2e7
Add NicoleAndDerek
2019-11-03 23:50:47 +01:00
Techwolf
be6a6722b5
Add UnlikeMinerva
2019-11-03 23:50:47 +01:00
Techwolf
fcdbd563a2
Add HavocInc
2019-11-03 23:44:49 +01:00
Techwolf
a575e5e0f0
Add Dissonance
2019-11-03 23:44:24 +01:00
Techwolf
14a01c3e47
Add OutOfPlacers
2019-11-03 23:43:23 +01:00
Techwolf
edc59a86e3
Add PlushAndBlood
2019-11-03 23:42:46 +01:00
Techwolf
4c7a654dcc
Add WhiteNoiseLee
2019-11-03 23:42:25 +01:00
Techwolf
12b048d449
Add Savestate
2019-11-03 23:41:48 +01:00
Techwolf
4a783c11ec
Add NotAVillain
2019-11-03 23:41:28 +01:00
Techwolf
013d10a1f2
Add SmackJeeves/FurryExperience
2019-11-03 23:40:58 +01:00
Techwolf
2a7d63b7eb
Add CrimsonFlag
2019-11-03 23:39:53 +01:00
Techwolf
e06f31784e
Add Supercell
2019-11-03 23:38:12 +01:00
Techwolf
f4e3c09717
Add CollegeCatastrophe
2019-11-03 23:37:46 +01:00
Techwolf
59c68bc447
Add NineToNine
2019-11-03 23:37:33 +01:00
Techwolf
a444798460
Add SwordsAndSausages
2019-11-03 23:37:13 +01:00
Techwolf
99ee4147f7
Add SuburbanJungleRoughHousing
2019-11-03 23:36:50 +01:00
Techwolf
f564989e36
Add SuburbanJungle
2019-11-03 23:36:31 +01:00
Techwolf
8a987d3d97
Add ButImACatPerson
2019-11-03 23:36:02 +01:00
Techwolf
44b65f9aac
Add OrderOfTheBlackDog
2019-11-03 23:35:40 +01:00
Techwolf
15a5953120
Add ATaleOfTails
2019-11-03 23:35:06 +01:00
Techwolf
323bfc3a6a
Add Dreamkeepers
2019-11-03 23:34:38 +01:00
Techwolf
70e78a87de
Add CarryOn
2019-11-03 23:34:04 +01:00
Techwolf
e565b083be
Add CavesAndCritters
2019-11-03 23:31:17 +01:00
Techwolf
6d76193a9f
Add IslaAukate and IslaAukateColor
2019-11-03 23:29:02 +01:00
Techwolf
dd6e536a55
Add Kaerwyn and BlackTapestries
2019-11-03 23:26:03 +01:00
Techwolf
0eccdf737a
Add Housepets
2019-11-03 21:53:13 +01:00
Techwolf
48ebffc756
Add HowToBeAWerewolf
2019-11-03 21:39:52 +01:00
Techwolf
f5b7b067b7
Switch AGirlAndHerFed to parser scraper
2019-11-03 21:37:05 +01:00
Techwolf
ed3acd2d2f
Fix TheWhiteboard
2019-11-03 21:35:58 +01:00
Techwolf
b055a8574f
Fix DominicDeegan
2019-11-03 21:34:53 +01:00
Techwolf
1b87afad7e
Fix GrrlPower
2019-11-03 21:34:18 +01:00
Techwolf
9796f994e3
Add SSDD
2019-11-03 21:33:03 +01:00
Techwolf
6b319fdda8
Fix SabrinaOnline
2019-11-03 21:31:50 +01:00
Techwolf
b2db51c361
Add OriginalLife
2019-11-03 21:22:56 +01:00
Techwolf
016516e984
Switch BetterDays to parser scraper
2019-11-03 21:22:29 +01:00
Techwolf
5ca5da51fc
Fix Curtailed
2019-11-03 21:20:23 +01:00
Techwolf
79618e2a2f
Add first strip URL for PS238
2019-11-03 21:19:10 +01:00
Techwolf
764a8ce6f6
Fix SlightlyDamned
2019-11-03 21:17:31 +01:00
Tobias Gruetzmacher
e24c0ae557
Simplify voting code
...
Not sure if I keep this feature, but at least I can now see if anybody
is still using it...
2019-11-03 21:04:34 +01:00
Tobias Gruetzmacher
ac9d8db1e8
Make sure user agent is in all HTTP requests
2019-11-03 20:17:27 +01:00
Mikkel Høgh
0dea216851
Fix missing returns in RSS parsing ( #137 )
...
Also added a very basic test for feed parsing.
2019-10-28 16:48:21 +01:00
Tobias Gruetzmacher
328b3cd072
Add new namer "joinPathPartsNamer"
...
Additionally, switch some comics which benefit from it to the new namer.
This fixes #127 .
2019-06-30 20:52:15 +02:00
Mikkel Høgh
b8b488670e
New comic: The Rock Cocks ( #138 )
2019-06-30 19:46:39 +02:00
Mikkel Høgh
f29d14c3b4
Fix Gunnerkrigg Court prevSearch matcher ( #135 )
2019-06-26 23:25:01 +02:00
Mikkel Høgh
78ac7144b2
Fix Girls with Slingshots matchers ( #136 )
...
Domain name and URLs have changed slightly.
Fixes #105 .
2019-06-26 23:22:45 +02:00
Tobias Gruetzmacher
1d910a5bbd
Remove pbr from runtime
2019-06-19 07:31:34 +02:00
Gervásio Júnior
6c8814fe40
Fix multiple imgs for json flag & ZenPencils bouncer ( #133 )
...
When using the JSON output flag, if the page has more than one image,
dictionary indexing cannot be used as list.
For the ZenPencils comic, the bouncer is missing, saving the page url
as the root url.
2019-06-19 07:09:33 +02:00
Arthur Caranta
ffbf494765
Added support for Tripping Over You comic ( #130 )
2019-04-17 20:28:17 +02:00
Tobias Gruetzmacher
9c4a791853
Repair loader for PyInstaller 3.4
2019-03-08 23:46:50 +01:00
Damjan Košir
78e8f05360
added MonkeyUser
2018-08-28 22:13:48 +12:00
erpbridge
e7410ce26b
Update for new LookingForGroup site format ( #123 )
...
Site uses a WordPress build, but does not explicitly match
any existing scrapers. Fixes #120 .
2018-08-12 23:39:51 +02:00
erpbridge
62d3692d3b
Update for ElGoonishShive and ElGoonishShiveNP ( #122 )
...
Strip no longer supports ID numbers after May 21 site revamp per Dan Shive. Code here switched to ComicControl. Tested and verified locally. (fixes #118 )
2018-08-10 09:13:43 +02:00
Tobias Gruetzmacher
49ec3cc3fa
Fix (and simplify) GoComics expressions ( fixes #117 )
2018-07-14 11:00:27 +02:00
Tobias Gruetzmacher
6ba1e49bfd
Kenneth Reitz’s Code Style™
...
See http://docs.python-requests.org/en/master/dev/contributing/#kenneth-reitz-s-code-style
Effectively, this removes "visual" indents.
2018-06-29 19:26:17 +02:00
Tobias Gruetzmacher
fbb3a18c91
Enable warnings and fix some of them
2018-05-23 00:54:40 +02:00
Dirk Reiners
050a0dc97c
MenageA3 naming fix
2018-04-23 08:07:41 +02:00
Dirk Reiners
cba9edbdec
LifeAintNoPontFarm added
2018-04-23 08:06:13 +02:00
Dirk Reiners
01c1b04778
CyanideAndHappiness fix
2018-04-23 07:53:22 +02:00
Dirk Reiners
fbd2ac2246
Handle get_terminal_size() returning 0 ( fixes #106 )
2018-04-23 07:50:28 +02:00
Peter Janes
2a2ff2d545
GoComics no longer has nav on the comic's home page.
2018-04-06 14:09:13 -04:00
Tobias Gruetzmacher
1fe98d2f7f
Use a diferent div class for GoComics ( fixes #102 ).
2018-03-23 00:29:40 +01:00
Tobias Gruetzmacher
2dbd3382f7
Update LeastICouldDo ( fixes #99 )
2017-12-15 00:00:25 +01:00
Tobias Gruetzmacher
75aa7207ea
Some minor fixes to make some modules work again.
2017-11-27 01:04:35 +01:00
Tobias Gruetzmacher
405c4c0b43
Recreate SluggyFreelance module ( fixes #96 ).
2017-11-26 20:23:33 +01:00
Tobias Gruetzmacher
90685d9b0c
Only support modern versions of PyCountry.
2017-11-26 19:29:48 +01:00
Damjan Košir
79a2516c61
deathbulge fix
2017-11-17 21:49:47 +13:00
Tobias Gruetzmacher
d88f6aeee3
Replace online tests with mocks.
...
We want to test our code, not the comic modules.
2017-10-15 14:54:44 +02:00
Tobias Gruetzmacher
f1b83748ed
When testing the command line, call main method.
...
Previously, we were spawning the main binary in a subprocess, which is
fragile and interacts poorly with some testing frameworks...
2017-10-15 14:54:44 +02:00
Tobias Gruetzmacher
ac2ca54570
Remove handlers after director run.
2017-10-15 14:54:44 +02:00
Damjan Košir
24862715d5
realised we have a scraper for CMS MenageA3 uses
2017-10-03 21:47:32 +13:00
Damjan Košir
0e0dcf1f8e
redoing MenageA3 with ParserScraper (previous search regex was broken)
2017-10-02 21:52:40 +13:00
Tobias Gruetzmacher
6369203bc0
Merge pull request #92 from clonejo/feature/commitstrip
...
add a comic plugin for CommitStrip
2017-09-20 22:46:46 +02:00
Damjan Košir
89a902651c
Merge remote-tracking branch 'origin/master'
2017-09-19 22:36:48 +12:00
Damjan Košir
a9d7b4de12
added Deathbulge
2017-09-19 22:36:19 +12:00
clonejo
331faae3ea
add a comic plugin for CommitStrip
2017-09-18 21:31:15 +02:00
glyphy
ad8374d7b8
Fixing the Menagea3 plugin ( #91 )
...
I've changed the menagea3 plugin so it should work with the
new directory structure found on the site.
2017-09-04 21:19:46 +02:00
Tobias Gruetzmacher
7e0adf1d96
Unify more WordPress-based modules.
2017-05-22 01:17:05 +02:00
Tobias Gruetzmacher
42f66c07b0
Random module fixes.
2017-05-22 00:30:31 +02:00
Tobias Gruetzmacher
f8def5b9db
Bugfix: StandardError does not exist in Python 3.
2017-05-21 23:37:09 +02:00
Tobias Gruetzmacher
a99098d5ad
Update GoComics module.
2017-05-21 23:10:32 +02:00
Tobias Gruetzmacher
1400879dc8
Fix another set of modules (e, k).
2017-05-17 00:11:29 +02:00
Tobias Gruetzmacher
4ee99eb196
Merge pull request #85 from sizlo/improveordering
...
Preserve the order of images in multi image strips for ordered symlink folders
2017-05-16 23:09:46 +02:00
Tim Brier
95e48b8d8d
Keep track of the order of images for multi-image strips in the JSON output
2017-05-15 10:56:47 +01:00
Tobias Gruetzmacher
8b90aa5cfb
Some minor style fixes.
2017-05-15 00:54:02 +02:00
Tobias Gruetzmacher
b8484cde50
Fix some more modules.
2017-05-15 00:27:28 +02:00
Tobias Gruetzmacher
ddd3fb418c
Remove some broken comics from ComicFury module.
2017-05-14 22:45:12 +02:00
Tobias Gruetzmacher
09687c91f4
Fix some SmackJeeves comics.
2017-05-12 00:32:25 +02:00
sizlo
a83911aa67
Favour the first image we found when we're not expecting multiple images
2017-04-18 21:59:04 +01:00
sizlo
8d84361de4
Preserve the order we found images in when removing duplicate images
2017-04-18 21:58:12 +01:00
Tobias Gruetzmacher
593975d907
Minor cleanups for new modules (see #84 ).
2017-04-16 01:28:17 +02:00
Tim Brier
233da3e052
Add support for SurvivingTheWorld and TumbleDryComics ( #84 )
2017-04-16 01:11:30 +02:00
Tobias Gruetzmacher
0973570295
Fix a bunch of modules.
2017-04-16 01:06:41 +02:00
Tobias Gruetzmacher
e6f18a2027
Clean up ComicGenesis
2017-02-27 18:20:54 +01:00
Tobias Gruetzmacher
23dccb184e
Replace PyInstaller version hack with something better.
2017-02-14 22:07:52 +01:00
Tobias Gruetzmacher
abb72a3a24
Fix CloneManga modules.
2017-02-13 23:41:45 +01:00
Tobias Gruetzmacher
ebbb27d05d
Move xpath_class to helpers module.
2017-02-13 22:41:17 +01:00
Tobias Gruetzmacher
20ab279cde
Clean up SmackJeeves...
...
Currently only covers already existing modules: Removed 11 broken
modules, added 2 and tried to update comic names and the adult and
endOfLife flags from their index. This isn't helped by the fact that
their search seems to skip some comics...
2017-02-13 01:46:49 +01:00
Tobias Gruetzmacher
83187b0554
Fix ViiviJaWagner.
2017-02-12 20:29:57 +01:00
Tobias Gruetzmacher
657e61811d
Update list of old and removed modules.
2017-02-12 20:17:07 +01:00
Tobias Gruetzmacher
3b6af33ecb
Some small module fixes.
2017-02-12 20:15:25 +01:00
Tobias Gruetzmacher
5359dd8629
Update ComicFury again...
2017-02-12 19:50:51 +01:00
Tobias Gruetzmacher
9895014655
Fix PHD with an ugly hack...
2017-02-12 16:21:36 +01:00
Tobias Gruetzmacher
b57945efd1
Update GoComic modules.
2017-02-12 12:21:01 +01:00
Tobias Gruetzmacher
ebe98bc8ba
Fix some modules.
2017-02-12 02:16:38 +01:00
Tobias Gruetzmacher
20ca5d7fc2
Fix some modules.
2017-02-06 00:05:05 +01:00
gruetzkopf
edb49faa8b
Add support for 'The Monster under the Bed'
2017-01-22 00:11:05 +01:00
Tobias Gruetzmacher
c4a184d173
Remove some vanished modules.
2017-01-12 02:01:10 +01:00
Tobias Gruetzmacher
36ac459bed
Add removed GoComics modules to old list.
2017-01-12 01:22:13 +01:00
Tobias Gruetzmacher
a183e812ae
Update GoComics module for new site layout.
...
(fixes #77 )
2017-01-11 02:21:05 +01:00
Tobias Gruetzmacher
061efaac6e
New module for ComicSherpa (removed from GoComics)
2017-01-11 01:34:52 +01:00
John Safrit
969e633877
Fix pattern for The Devils Panties
2017-01-08 17:39:59 -05:00
Tobias Gruetzmacher
3f9feec041
Allow modules to ignore some HTTP error codes.
...
This is neccessary since it seems some webservers out there are
misconfigured to deliver actual content with an HTTP error code...
2016-11-01 18:25:02 +01:00
Tobias Gruetzmacher
46b7a374f6
Small GoComics update.
2016-11-01 02:51:00 +01:00
Tobias Gruetzmacher
f7f4e130bf
Small fix to the WLP module.
2016-11-01 02:27:29 +01:00
Tobias Gruetzmacher
bc755d09a3
Apply link modifier to all links.
...
This was previously only the "previous link modifier", now it can also
modify "next" and "latest" links. Additionally, the modifier is given
the current URL, so those cases can be distinguished.
2016-11-01 01:50:44 +01:00
Tobias Gruetzmacher
7fc05f75f5
Remove broken PetiteSymphony comics.
2016-10-31 07:16:10 +01:00
Tobias Gruetzmacher
69e6318f87
Remove ScurryAndCover, too much JavaScript.
2016-10-31 07:04:00 +01:00
Tobias Gruetzmacher
47e2502ec7
Fix a bunch of comic modules.
2016-10-31 06:57:47 +01:00
Tobias Gruetzmacher
446b81fc45
Fix Wumo and friends.
2016-10-30 15:28:54 +01:00
Tobias Gruetzmacher
51ed898f5d
Fix some SmackJeeves comics.
2016-10-30 14:30:45 +01:00
Tobias Gruetzmacher
b6d99945f6
Merge pull request #73 from acaranta/master
...
Added several SmackJeeves Comics
2016-10-30 11:55:17 +01:00
Tobias Gruetzmacher
3b9f30affd
Update ComicFury modules.
2016-10-30 11:04:45 +01:00
Tobias Gruetzmacher
a02660a7d3
Replace custom @memoized with stdlib @lru_cache.
2016-10-29 00:46:49 +02:00
Tobias Gruetzmacher
9a6a310b76
Fixup copyright years.
2016-10-29 00:21:41 +02:00
acaranta
83880a3cbd
corrected RainbowMansion
2016-10-27 09:53:34 +02:00
acaranta
0ed823175c
Added even more Smackjeeves comics
2016-10-27 06:58:57 +02:00
acaranta
a5c9a3c35c
Added several SmackJeeves Comics
2016-10-26 05:25:13 +02:00
Peter Brunner
19445a83ae
Fix smbc
2016-10-18 21:28:42 -04:00
Tobias Gruetzmacher
f94caa8a16
Use terminal size calculation from standard library.
2016-10-14 23:55:10 +02:00
Tobias Gruetzmacher
06be2a026b
Move some ex-KeenSpot comics to shorter names.
2016-10-14 14:23:33 +02:00
Tobias Gruetzmacher
b17d6e5f22
Rework/fix KeenSpot modules.
2016-10-14 00:14:53 +02:00
Tobias Gruetzmacher
064e7976ec
Add namer for Extra Fabulous Comics.
2016-10-06 00:42:50 +02:00
mostlyuseful
fce7dfff19
Add "Extra Fabulous Comics" comic
2016-10-04 17:06:50 +02:00
Tobias Gruetzmacher
f342a93aa1
Update GoComics module.
2016-10-01 03:39:36 +02:00
Tobias Gruetzmacher
c0d945a563
Update ComicFury modules.
2016-10-01 02:52:33 +02:00
Tobias Gruetzmacher
98c98ddfab
Fix some more comic modules (c-f).
2016-09-30 00:15:45 +02:00
Tobias Gruetzmacher
b1d2650615
Fix some modules (a&b).
2016-09-29 01:29:01 +02:00
Damjan Košir
c04c62e92b
xkcd now hone with xpaths
2016-08-18 21:28:25 +12:00
Damjan Košir
9ba184eb43
fixing LoadingArtist
2016-08-16 21:20:35 +12:00
Hubert Figuière
afcd19bf5b
Added Prince of Sartar Comic
2016-08-08 09:18:33 -04:00
Hubert Figuière
81821dc450
Added Space Junk Arlia comic
2016-08-08 09:18:33 -04:00
Tobias Gruetzmacher
fb37f946e0
Speed up comic module tests.
...
This fakes an If-Modified-Since header, so most web servers don't need
to send comic images at all. This should also reduce the amount of data
that needs to be fetched for comic module tests.
2016-08-01 00:44:34 +02:00
Tobias Gruetzmacher
4f80016bf0
Change robotparser import to make PyInstaller happy.
2016-06-06 22:42:01 +02:00
Tobias Gruetzmacher
64c8e502ca
Ignore case for comic download directories.
...
Since we already match comics case-insensitive on the command line, this
was a logical step, even if this means changing quite a bit of code that
all tries to resolve the "comic directory" in a slightly different
way...
2016-06-06 00:08:29 +02:00
Tobias Gruetzmacher
215d597573
Remove DrunkDuck for now.
...
- It's been disabled for ages
- Needs a major rework
- I don't want to add that many comics anyways...
- This also gets rid of make_scraper :)
2016-06-05 22:22:17 +02:00
Tobias Gruetzmacher
67d0d38100
Migrate SnafuComics to single-class module.
2016-06-05 22:12:16 +02:00
Tobias Gruetzmacher
125c96e9dc
Remove command to download ALL comics...
2016-06-05 21:57:56 +02:00
Tobias Gruetzmacher
df2048cb34
Keep track of removed and moved comics ( fixes #41 ).
...
I plan on keeping this list for at least ~ 2 releases and then purging
older entries...
2016-06-05 21:47:58 +02:00
Tobias Gruetzmacher
9b755a7e6c
Restore BobWhite.
2016-06-05 18:32:27 +02:00
Tobias Gruetzmacher
603fd62a1e
Fix workaround for PyInstaller...
2016-06-05 16:01:35 +02:00
Tobias Gruetzmacher
295b53a2d3
Fix name overrides (broken by 51008a).
2016-06-05 10:03:29 +02:00
Tobias Gruetzmacher
844bec09ba
Remove another dead comic from ComicFury.
2016-06-05 01:06:04 +02:00
Tobias Gruetzmacher
12123961a4
Fix error in PyInstaller packaged application.
2016-06-05 00:34:16 +02:00
André-Patrick Bubel
2b8e948868
Add String Theory comic
2016-06-01 11:19:17 +00:00
André-Patrick Bubel
192751073c
Add KillSixBillionDemons comic
2016-05-31 07:28:32 +00:00
Tobias Gruetzmacher
807bee6342
Migrate GoComics to single-class module.
2016-05-23 00:01:10 +02:00
Tobias Gruetzmacher
2c8e57bdea
Migrate Creators to single-class module.
2016-05-22 23:56:59 +02:00
Tobias Gruetzmacher
f5dff27b0a
Migrate SmackJeeves to single-class module.
2016-05-22 23:54:21 +02:00
Tobias Gruetzmacher
1ea20e1743
Migrate WebcomicFactory to single-class module.
2016-05-22 23:40:58 +02:00
Tobias Gruetzmacher
c62a7283a2
Migrate ComicFury to single-class module.
2016-05-22 23:31:53 +02:00
Tobias Gruetzmacher
1834bf179f
Migrate Arcamax to single-class module.
2016-05-22 23:17:24 +02:00
Tobias Gruetzmacher
f29472c143
Make auto-update script more flexible.
2016-05-22 23:06:05 +02:00
Tobias Gruetzmacher
e4650d5941
Remove make_scraper from Nitrocosm.
2016-05-21 14:35:53 +02:00
Tobias Gruetzmacher
b6eb8ab8ef
Remove make_scraper from SandraAndWoo
2016-05-21 14:12:11 +02:00
Tobias Gruetzmacher
4630ea047c
Implement Oglaf's strange navigation ( fixes #33 )
...
(also should fix wummel#91)
2016-05-21 02:38:07 +02:00
Tobias Gruetzmacher
51008a975b
Refactor: Introduce generator methods for scrapers
...
This allows one comic module class to generate multiple scrapers. This
change is to support a more dynamic module system as described in #42 .
2016-05-21 01:29:36 +02:00
Tobias Gruetzmacher
89cfd9d310
Add comics from catomix.com.
2016-05-16 23:55:41 +02:00
Tobias Gruetzmacher
a6cf4e7040
Fix some more comic modules.
2016-05-16 23:16:29 +02:00
Tobias Gruetzmacher
be1a63da0c
Update GoComics comic list.
2016-05-16 18:26:45 +02:00
Tobias Gruetzmacher
6d3f74142c
Move command line tool into package.
...
This way we can use the default Python console_scripts install process.
2016-05-16 14:57:47 +02:00
Tobias Gruetzmacher
b9d9564085
Fix Dilbert ( fixes #44 ).
2016-05-16 01:21:23 +02:00
Tobias Gruetzmacher
e9b3c487c0
Remove some dead comics.
2016-05-16 01:10:20 +02:00
Tobias Gruetzmacher
bd60155d9f
Some more ComicFury comics gone...
2016-05-16 00:53:22 +02:00
Tobias Gruetzmacher
849e60e795
Remove make_scraper magic from webcomiceu.
2016-05-07 03:20:01 +02:00
Tobias Gruetzmacher
975d2376bf
Another round of comic module fixes.
2016-05-07 01:50:10 +02:00
Tobias Gruetzmacher
efe1308db2
Replace home-grown Python2/3 compat. with six.
2016-05-05 23:33:48 +02:00
Tobias Gruetzmacher
77ed0218e0
Fix some comic modules.
2016-05-05 20:55:14 +02:00
Tobias Gruetzmacher
bb2ac39639
Fix some URLs.
2016-05-05 10:12:03 +02:00
Tobias Gruetzmacher
d05316e3ac
Seems ComicFury is deleting comics regularly...
...
Well, there's nothing we can do: Remove them.
2016-05-04 08:26:53 +02:00
Tobias Gruetzmacher
0c1aa9e8bd
Move libxml < 2.9.3 workaround to base class.
2016-05-02 23:22:06 +02:00
Tobias Gruetzmacher
b93a8fde65
Move PensAndTales comics and fix them.
2016-05-02 22:32:14 +02:00
Tobias Gruetzmacher
4006ced43d
Move all HijinksEnsue comics into alphabetic files.
2016-05-02 01:25:34 +02:00
Tobias Gruetzmacher
d5f91ecfd2
Fix some modules in m.py.
2016-04-30 01:59:28 +02:00
Tobias Gruetzmacher
1d52d33311
Remove missing SmackJeeves comics.
2016-04-30 00:56:20 +02:00
Tobias Gruetzmacher
d796f3476c
Fix some modules in d.py.
2016-04-30 00:44:18 +02:00
Tobias Gruetzmacher
cc16fea880
Fix some modules in c.py
2016-04-29 00:35:02 +02:00
Tobias Gruetzmacher
1d94439715
Fix some more comic modules.
2016-04-27 00:31:27 +02:00
Tobias Gruetzmacher
8b1ac4eb35
Fix "tagsoup" on SmackJeeves
...
Unfortunatly, browsers render < outside of HTML tags differently then
libXML until recently (libXML 2.9.3), so we need to preprocess pages
before parsing them...
(This was fixed in libXML commit 140c25)
2016-04-26 08:05:38 +02:00
Tobias Gruetzmacher
035d6e94e4
Allow output level for warnings and errors.
2016-04-26 07:53:53 +02:00
Tobias Gruetzmacher
8ddf553eb4
Fix some more SmackJeeves modules.
2016-04-22 01:04:47 +02:00
Tobias Gruetzmacher
fd85c8583a
Unify similar code in fetchUrl and fetchText
2016-04-22 00:42:46 +02:00
Tobias Gruetzmacher
6574997e01
Refactor: All the other class methods.
...
Turns out, it would have been better if all methods had been instance
methods and not class methods. This finished a big chunk of the rework
needed for #42 .
2016-04-21 23:52:31 +02:00
Tobias Gruetzmacher
0d436b8ca9
Refactor: url modifiers to normal methods.
...
As before, to implement #42 these might want to access information from
the instance, so they should be normal methods.
2016-04-21 21:39:25 +02:00
Tobias Gruetzmacher
c3f32dfef7
Refactor: Make namer a method.
...
When #42 is realized, the naming of files might differ between comic
modules, so the namer's logical location is the instance, not the class.
2016-04-21 08:20:49 +02:00
Tobias Gruetzmacher
5bd2a49f48
Add debug output on matched XPath/CSS expression.
2016-04-20 23:51:54 +02:00
Tobias Gruetzmacher
fe51a449df
Update SmackJeeves
...
- Now uses _ParserScraper, which makes the pattern quite a bit more
generic and IMHO more readable
- remove make_scraper magic
- No new comics, only fixed existing ones and removed some dead ones.
2016-04-20 23:36:45 +02:00
Tobias Gruetzmacher
190cd3b063
Convert language & getDisabledReasons to methods.
...
Both are more properties of a webcomic (this is part of the design
changes for #42 )
2016-04-19 23:53:46 +02:00
Tobias Gruetzmacher
df46907f39
Register EXSLT extensions by default.
...
This allows comic module authors to use the full power of regular
expressions in XPath expression, see http://exslt.org/regexp/regexp.html
for usage. Please be aware that these use the prefix re: instead of
regexp: here.
2016-04-19 23:48:14 +02:00
Tobias Gruetzmacher
4204f5f1e4
Send "If-Modified-Since" header for images.
2016-04-19 00:36:50 +02:00
Tobias Gruetzmacher
13a3409854
Remove some comics that are gone or block us.
2016-04-17 19:42:43 +02:00
Tobias Gruetzmacher
1fbc844077
Update GoComics.
2016-04-17 18:40:09 +02:00
Tobias Gruetzmacher
73e958670d
Update ComicFury (again).
2016-04-17 16:19:44 +02:00
Tobias Gruetzmacher
b0481a01f7
Update languages.
2016-04-16 13:14:12 +02:00
Tobias Gruetzmacher
3329027e4b
Update ComicFury.
2016-04-16 13:13:47 +02:00
Tobias Gruetzmacher
ee99c087d7
Remove prevUrlMatchesStripUrl.
...
It was only used for one test.
2016-04-16 01:14:26 +02:00
Tobias Gruetzmacher
92a688457a
Remove useless indirection.
2016-04-15 23:42:24 +02:00
Tobias Gruetzmacher
52515b5fc5
Update GoComics.
2016-04-15 00:26:14 +02:00
Tobias Gruetzmacher
031a523846
Fix SnafuComics.
2016-04-14 23:52:35 +02:00
Tobias Gruetzmacher
7626b1e100
Webcomics Nation is gone.
2016-04-14 22:46:52 +02:00
Tobias Gruetzmacher
497653c448
Remove make_scraper magic from Arcamax.
2016-04-14 00:17:59 +02:00
Tobias Gruetzmacher
db87ed95e7
Use new features to make modules simpler.
2016-04-13 23:28:43 +02:00
Tobias Gruetzmacher
b266e28ae1
Remove debugging prints 😭
2016-04-13 22:59:06 +02:00
Tobias Gruetzmacher
ff3b824311
Fix variable shadowing...
2016-04-13 22:43:34 +02:00
Tobias Gruetzmacher
060281e5ff
Use concrete scraper objects everywhere.
...
This is a first step for #42 . Since most access to the scraper classes
is through instances, modules can now dynamically override url and name
(name is now a property).
2016-04-13 22:17:30 +02:00
Tobias Gruetzmacher
0468f2f31a
Refactor: Convert starter to simple method.
2016-04-13 20:01:51 +02:00
Tobias Gruetzmacher
16004e43e4
Use default bounceStarter for site modules.
2016-04-13 01:24:13 +02:00
Tobias Gruetzmacher
9028724a74
Clean up update helper scripts.
2016-04-13 00:52:16 +02:00
Tobias Gruetzmacher
42e43fa4e6
Read starter parameters from class.
...
This allows to specify starters in a more declarative and dynamic way.
2016-04-12 23:11:39 +02:00
Tobias Gruetzmacher
b865a171f9
Remove some broken comics.
2016-04-12 08:21:06 +02:00
Tobias Gruetzmacher
4e2e4ac529
Prevent scraper from moving to a different comic.
2016-04-12 08:10:47 +02:00
Tobias Gruetzmacher
443ab119e9
Refresh GoComics list from online directory.
2016-04-12 00:36:33 +02:00
Tobias Gruetzmacher
0e385a3697
Update GoComics (no change in supported comics)
...
- remove make_scraper magic
- switch to _ParserScraper
2016-04-11 22:42:01 +02:00
Tobias Gruetzmacher
ad7a297964
Fix WLP comics.
2016-04-11 01:07:21 +02:00
Damjan Košir
af2e57d850
Added comic ScurryAndCover...
...
- Yay, funky JavaScript parsing!
- Start page isn't latest comic...
Updated-by: Tobias Gruetzmacher <tobias-git@23.gs>
2016-04-11 00:09:53 +02:00
Tobias Gruetzmacher
fa98f6ddbf
Move more comics to common WordPressScraper.
2016-04-10 23:04:34 +02:00
Tobias Gruetzmacher
f6e605e146
Fix unicode error in text search.
2016-04-10 13:16:30 +02:00
Tobias Gruetzmacher
bc10bd9a4d
Streamline color output.
...
- Depend on external colorama instead of embedding an old copy.
- Move most output code into output module.
- Convert pager to context manager.
2016-04-10 03:45:00 +02:00
Tobias Gruetzmacher
bb5b6ffcec
Fix comics in module a.py.
2016-04-07 23:21:31 +02:00
Tobias Gruetzmacher
0033a8046b
Fix creators module.
2016-04-07 00:20:03 +02:00
Tobias Gruetzmacher
8768ff07b6
Fix AhoiPolloi, be a bit smarter about encoding.
...
HTML character encoding in the context of HTTP is quite tricky to get
right and honestly, I'm not sure if I did get it right this time. But I
think, the current behaviour matches best what web browsers try to do:
1. Let Requests figure out the content from the HTTP header. This
overrides everything else. We need to "trick" LXML to accept our
decision if the document contains an XML declaration which might
disagree with the HTTP header.
2. If the HTTP headers don't specify any encoding, let LXML guess the
encoding and be done with it.
2016-04-06 22:22:22 +02:00
Tobias Gruetzmacher
183d18e7bc
Skip non-image on xkcd.
2016-04-06 00:50:01 +02:00
Tobias Gruetzmacher
9feaf245f2
Fixed & removed some comics in s.py.
2016-04-06 00:40:13 +02:00
Tobias Gruetzmacher
6bbdcfb341
BloomingFaeries: Don't download every page twice.
...
(Also, simplify namer, switch to _ParserScraper)
2016-04-05 23:58:43 +02:00
Tobias Gruetzmacher
8db6f8e8b7
Fix ZapComics, remove ZebraGirl.
...
- ZebraGirl is now ComicFury/ZebraGirl...
2016-04-04 00:27:11 +02:00
Tobias Gruetzmacher
0bcfb8a82e
Move ComicControl into common module.
...
- Move all comics using ComicControl into alphabetical files.
- Add BalderDash & Picklewhistle
2016-04-04 00:12:53 +02:00
Tobias Gruetzmacher
0d453a6858
Move Flowerlark Studios into alphabetical files.
2016-04-03 22:58:01 +02:00
Tobias Gruetzmacher
a9f0dfdce4
Merge pull request #39 from peterjanes/peterjanes/sherman-fix
...
Fix Sherman's Lagoon
2016-04-03 22:20:04 +02:00
Tobias Gruetzmacher
926439cd14
Every comic need an url.
2016-04-03 22:03:16 +02:00
Tobias Gruetzmacher
2c6decb7f5
Move WebcomicFactory in its own module.
...
Also, add an updater script for it.
2016-04-03 21:31:56 +02:00
Peter Janes
759bd0c360
Fix Sherman's Lagoon
2016-04-03 14:54:41 -04:00
Tobias Gruetzmacher
bb1f20d867
Remove make_scraper for most WordPress comics.
...
- Dropped KatzenfutterGeleespritzer, because robots.txt.
- Move all WordPress/ComicPress scrapers into alphabetical files.
- Move _WordPressScraper & _ComicPress scraper into common.py.
- Some smaller PEP8 fixes.
2016-04-02 00:19:53 +02:00
