dmoonfire/dosage
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Activity
1252 commits 1 branch 0 tags 31 MiB
Commit graph

1252 commits

This branch
This branch
All branches
Author SHA1 Message Date
Tobias Gruetzmacher 92a688457a Remove useless indirection. 2016-04-15 23:42:24 +02:00
Tobias Gruetzmacher 52515b5fc5 Update GoComics. 2016-04-15 00:26:14 +02:00
Tobias Gruetzmacher 031a523846 Fix SnafuComics. 2016-04-14 23:52:35 +02:00
Tobias Gruetzmacher 7626b1e100 Webcomics Nation is gone. 2016-04-14 22:46:52 +02:00
Tobias Gruetzmacher dab5aef094 Refactor update helpers: Remove duplicate code. 2016-04-14 22:22:37 +02:00
Tobias Gruetzmacher 497653c448 Remove make_scraper magic from Arcamax. 2016-04-14 00:17:59 +02:00
Tobias Gruetzmacher db87ed95e7 Use new features to make modules simpler. 2016-04-13 23:28:43 +02:00
Tobias Gruetzmacher b266e28ae1 Remove debugging prints 😭 2016-04-13 22:59:06 +02:00
Tobias Gruetzmacher ff3b824311 Fix variable shadowing... 2016-04-13 22:43:34 +02:00
Tobias Gruetzmacher 060281e5ff Use concrete scraper objects everywhere. 
This is a first step for #42. Since most access to the scraper classes
is through instances, modules can now dynamically override url and name
(name is now a property).
 2016-04-13 22:17:30 +02:00
Tobias Gruetzmacher 0468f2f31a Refactor: Convert starter to simple method. 2016-04-13 20:01:51 +02:00
Tobias Gruetzmacher 1aebdce5d2 Fix test :) 2016-04-13 01:32:15 +02:00
Tobias Gruetzmacher 16004e43e4 Use default bounceStarter for site modules. 2016-04-13 01:24:13 +02:00
Tobias Gruetzmacher 9028724a74 Clean up update helper scripts. 2016-04-13 00:52:16 +02:00
Tobias Gruetzmacher 42e43fa4e6 Read starter parameters from class. 
This allows to specify starters in a more declarative and dynamic way.
 2016-04-12 23:11:39 +02:00
Tobias Gruetzmacher b865a171f9 Remove some broken comics. 2016-04-12 08:21:06 +02:00
Tobias Gruetzmacher 4e2e4ac529 Prevent scraper from moving to a different comic. 2016-04-12 08:10:47 +02:00
Tobias Gruetzmacher 443ab119e9 Refresh GoComics list from online directory. 2016-04-12 00:36:33 +02:00
Tobias Gruetzmacher 0e385a3697 Update GoComics (no change in supported comics) 
- remove make_scraper magic
- switch to _ParserScraper
 2016-04-11 22:42:01 +02:00
Tobias Gruetzmacher ad7a297964 Fix WLP comics. 2016-04-11 01:07:21 +02:00
Damjan Košir af2e57d850 Added comic ScurryAndCover... 
- Yay, funky JavaScript parsing!
- Start page isn't latest comic...

Updated-by: Tobias Gruetzmacher <tobias-git@23.gs>
 2016-04-11 00:09:53 +02:00
Tobias Gruetzmacher fa98f6ddbf Move more comics to common WordPressScraper. 2016-04-10 23:04:34 +02:00
Tobias Gruetzmacher f6e605e146 Fix unicode error in text search. 2016-04-10 13:16:30 +02:00
Tobias Gruetzmacher e3628ee751 Install all dependencies in tox (& flake8 rules). 2016-04-10 03:47:01 +02:00
Tobias Gruetzmacher bc10bd9a4d Streamline color output. 
- Depend on external colorama instead of embedding an old copy.
- Move most output code into output module.
- Convert pager to context manager.
 2016-04-10 03:45:00 +02:00
Tobias Gruetzmacher bb5b6ffcec Fix comics in module a.py. 2016-04-07 23:21:31 +02:00
Tobias Gruetzmacher 0033a8046b Fix creators module. 2016-04-07 00:20:03 +02:00
Tobias Gruetzmacher 8768ff07b6 Fix AhoiPolloi, be a bit smarter about encoding. 
HTML character encoding in the context of HTTP is quite tricky to get
right and honestly, I'm not sure if I did get it right this time. But I
think, the current behaviour matches best what web browsers try to do:

1. Let Requests figure out the content from the HTTP header. This
   overrides everything else. We need to "trick" LXML to accept our
   decision if the document contains an XML declaration which might
   disagree with the HTTP header.
2. If the HTTP headers don't specify any encoding, let LXML guess the
   encoding and be done with it.
 2016-04-06 22:22:22 +02:00
Tobias Gruetzmacher 183d18e7bc Skip non-image on xkcd. 2016-04-06 00:50:01 +02:00
Tobias Gruetzmacher 9feaf245f2 Fixed & removed some comics in s.py. 2016-04-06 00:40:13 +02:00
Tobias Gruetzmacher 6bbdcfb341 BloomingFaeries: Don't download every page twice. 
(Also, simplify namer, switch to _ParserScraper)
 2016-04-05 23:58:43 +02:00
Tobias Gruetzmacher 8db6f8e8b7 Fix ZapComics, remove ZebraGirl. 
- ZebraGirl is now ComicFury/ZebraGirl...
 2016-04-04 00:27:11 +02:00
Tobias Gruetzmacher 0bcfb8a82e Move ComicControl into common module. 
- Move all comics using ComicControl into alphabetical files.
- Add BalderDash & Picklewhistle
 2016-04-04 00:12:53 +02:00
Tobias Gruetzmacher 0d453a6858 Move Flowerlark Studios into alphabetical files. 2016-04-03 22:58:01 +02:00
Tobias Gruetzmacher a9f0dfdce4 Merge pull request #39 from peterjanes/peterjanes/sherman-fix 
Fix Sherman's Lagoon
 2016-04-03 22:20:04 +02:00
Tobias Gruetzmacher 926439cd14 Every comic need an url. 2016-04-03 22:03:16 +02:00
Tobias Gruetzmacher 2c6decb7f5 Move WebcomicFactory in its own module. 
Also, add an updater script for it.
 2016-04-03 21:31:56 +02:00
Peter Janes 759bd0c360 Fix Sherman's Lagoon 2016-04-03 14:54:41 -04:00
Tobias Gruetzmacher bb1f20d867 Remove make_scraper for most WordPress comics. 
- Dropped KatzenfutterGeleespritzer, because robots.txt.
- Move all WordPress/ComicPress scrapers into alphabetical files.
- Move _WordPressScraper & _ComicPress scraper into common.py.
- Some smaller PEP8 fixes.
 2016-04-02 00:19:53 +02:00
Tobias Gruetzmacher a7b6599cd4 Sort ComicFury comics ignoring case. 2016-03-31 23:26:56 +02:00
Tobias Gruetzmacher cfac0f40f1 Encode JSON cache as UTF-8. 2016-03-31 23:25:53 +02:00
Tobias Gruetzmacher 7f1e136d8b Sort comics alphabetically & PEP8 style fixes. 2016-03-31 23:13:54 +02:00
Tobias Gruetzmacher dda920f353 Use py.test's tmpdir fixture. 2016-03-28 16:29:57 +02:00
Tobias Gruetzmacher d6db1d0b81 Fix a conflict with IPython. 2016-03-20 23:57:07 +01:00
Tobias Gruetzmacher 2f61ecb721 Add build helper for PyInstaller (fixes #22). 2016-03-20 21:42:41 +01:00
Tobias Gruetzmacher 90dfceaeb1 Remove dead modules (& format). 2016-03-20 20:48:42 +01:00
Tobias Gruetzmacher 3ffe9f5715 Cache pip downloads on Travis. 2016-03-20 20:32:08 +01:00
Tobias Gruetzmacher f243096d49 Fix GastroPhobia, remove GeneralProtectionFault. 
(& formatting)
 2016-03-20 20:11:21 +01:00
Tobias Gruetzmacher 56e05380ba Move pytest-travis-fold to the right place. 2016-03-20 19:55:34 +01:00
Tobias Gruetzmacher 1ff4902344 Use tox for testing. 2016-03-20 19:43:48 +01:00