Tobias Gruetzmacher
0e385a3697
Update GoComics (no change in supported comics)
...
- remove make_scraper magic
- switch to _ParserScraper
2016-04-11 22:42:01 +02:00
Tobias Gruetzmacher
ad7a297964
Fix WLP comics.
2016-04-11 01:07:21 +02:00
Damjan Košir
af2e57d850
Added comic ScurryAndCover...
...
- Yay, funky JavaScript parsing!
- Start page isn't latest comic...
Updated-by: Tobias Gruetzmacher <tobias-git@23.gs>
2016-04-11 00:09:53 +02:00
Tobias Gruetzmacher
fa98f6ddbf
Move more comics to common WordPressScraper.
2016-04-10 23:04:34 +02:00
Tobias Gruetzmacher
f6e605e146
Fix unicode error in text search.
2016-04-10 13:16:30 +02:00
Tobias Gruetzmacher
e3628ee751
Install all dependencies in tox (& flake8 rules).
2016-04-10 03:47:01 +02:00
Tobias Gruetzmacher
bc10bd9a4d
Streamline color output.
...
- Depend on external colorama instead of embedding an old copy.
- Move most output code into output module.
- Convert pager to context manager.
2016-04-10 03:45:00 +02:00
Tobias Gruetzmacher
bb5b6ffcec
Fix comics in module a.py.
2016-04-07 23:21:31 +02:00
Tobias Gruetzmacher
0033a8046b
Fix creators module.
2016-04-07 00:20:03 +02:00
Tobias Gruetzmacher
8768ff07b6
Fix AhoiPolloi, be a bit smarter about encoding.
...
HTML character encoding in the context of HTTP is quite tricky to get
right and honestly, I'm not sure if I did get it right this time. But I
think, the current behaviour matches best what web browsers try to do:
1. Let Requests figure out the content from the HTTP header. This
overrides everything else. We need to "trick" LXML to accept our
decision if the document contains an XML declaration which might
disagree with the HTTP header.
2. If the HTTP headers don't specify any encoding, let LXML guess the
encoding and be done with it.
2016-04-06 22:22:22 +02:00
Tobias Gruetzmacher
183d18e7bc
Skip non-image on xkcd.
2016-04-06 00:50:01 +02:00
Tobias Gruetzmacher
9feaf245f2
Fixed & removed some comics in s.py.
2016-04-06 00:40:13 +02:00
Tobias Gruetzmacher
6bbdcfb341
BloomingFaeries: Don't download every page twice.
...
(Also, simplify namer, switch to _ParserScraper)
2016-04-05 23:58:43 +02:00
Tobias Gruetzmacher
8db6f8e8b7
Fix ZapComics, remove ZebraGirl.
...
- ZebraGirl is now ComicFury/ZebraGirl...
2016-04-04 00:27:11 +02:00
Tobias Gruetzmacher
0bcfb8a82e
Move ComicControl into common module.
...
- Move all comics using ComicControl into alphabetical files.
- Add BalderDash & Picklewhistle
2016-04-04 00:12:53 +02:00
Tobias Gruetzmacher
0d453a6858
Move Flowerlark Studios into alphabetical files.
2016-04-03 22:58:01 +02:00
Tobias Gruetzmacher
a9f0dfdce4
Merge pull request #39 from peterjanes/peterjanes/sherman-fix
...
Fix Sherman's Lagoon
2016-04-03 22:20:04 +02:00
Tobias Gruetzmacher
926439cd14
Every comic need an url.
2016-04-03 22:03:16 +02:00
Tobias Gruetzmacher
2c6decb7f5
Move WebcomicFactory in its own module.
...
Also, add an updater script for it.
2016-04-03 21:31:56 +02:00
Peter Janes
759bd0c360
Fix Sherman's Lagoon
2016-04-03 14:54:41 -04:00
Tobias Gruetzmacher
bb1f20d867
Remove make_scraper for most WordPress comics.
...
- Dropped KatzenfutterGeleespritzer, because robots.txt.
- Move all WordPress/ComicPress scrapers into alphabetical files.
- Move _WordPressScraper & _ComicPress scraper into common.py.
- Some smaller PEP8 fixes.
2016-04-02 00:19:53 +02:00
Tobias Gruetzmacher
a7b6599cd4
Sort ComicFury comics ignoring case.
2016-03-31 23:26:56 +02:00
Tobias Gruetzmacher
cfac0f40f1
Encode JSON cache as UTF-8.
2016-03-31 23:25:53 +02:00
Tobias Gruetzmacher
7f1e136d8b
Sort comics alphabetically & PEP8 style fixes.
2016-03-31 23:13:54 +02:00
Tobias Gruetzmacher
dda920f353
Use py.test's tmpdir fixture.
2016-03-28 16:29:57 +02:00
Tobias Gruetzmacher
d6db1d0b81
Fix a conflict with IPython.
2016-03-20 23:57:07 +01:00
Tobias Gruetzmacher
2f61ecb721
Add build helper for PyInstaller ( fixes #22 ).
2016-03-20 21:42:41 +01:00
Tobias Gruetzmacher
90dfceaeb1
Remove dead modules (& format).
2016-03-20 20:48:42 +01:00
Tobias Gruetzmacher
3ffe9f5715
Cache pip downloads on Travis.
2016-03-20 20:32:08 +01:00
Tobias Gruetzmacher
f243096d49
Fix GastroPhobia, remove GeneralProtectionFault.
...
(& formatting)
2016-03-20 20:11:21 +01:00
Tobias Gruetzmacher
56e05380ba
Move pytest-travis-fold to the right place.
2016-03-20 19:55:34 +01:00
Tobias Gruetzmacher
1ff4902344
Use tox for testing.
2016-03-20 19:43:48 +01:00
Tobias Gruetzmacher
3c4b3fdfc3
Remove more old stuff.
2016-03-20 18:59:42 +01:00
Tobias Gruetzmacher
cfcfcc2468
Switch plugin loading to pkgutil.
...
This should work with all PEP-302 loaders that implement iter_modules.
Unfortunatly, PyInstaller (which I plan to use for Windows releases)
does not support it, so we don't get around a special case. Anyways,
this should help for #22 .
2016-03-20 15:13:24 +01:00
Tobias Gruetzmacher
065184f1db
Remove unused helper scripts.
2016-03-20 12:06:58 +01:00
Tobias Gruetzmacher
1af022895e
Fix NuklearPower ( fixes #38 ).
...
Also remove make_scraper magic.
2016-03-17 23:19:52 +01:00
Tobias Gruetzmacher
c1a0a7bd50
py.test travis-ci integration.
2016-03-17 21:12:55 +01:00
Tobias Gruetzmacher
e78558fb9b
Make lxml a requirement.
...
With my last commit > 35% of supported comics use it ;)
2016-03-17 00:58:41 +01:00
Tobias Gruetzmacher
552f29e5fc
Update ComicFury comics. (+871, -245)
...
- Remove make_scraper magic
- Switch to HTML parser
- Update parsing of comic listing.
2016-03-17 00:44:06 +01:00
Tobias Gruetzmacher
6727e9b559
Use vendored urllib3.
...
As long as requests ships with urllib3, we can't fall back to the
"system" urllib3, since that breaks class-identity checks.
2016-03-16 23:18:19 +01:00
Damjan Košir
615f094ef3
fixing EdmundFinney
2016-03-14 20:32:18 +13:00
Tobias Gruetzmacher
6532f15f54
Make pbr happy again.
2016-03-14 01:16:17 +01:00
Tobias Gruetzmacher
bb55f13ff4
Add Coveralls to Travis, badges in README.
2016-03-14 01:10:28 +01:00
Tobias Gruetzmacher
c4fcd985dd
Let urllib3 handle all retries.
2016-03-13 21:30:36 +01:00
Tobias Gruetzmacher
78e13962f9
Sort scraper modules (mostly for test stability).
2016-03-13 20:24:21 +01:00
Tobias Gruetzmacher
36cc4e32f4
Allow selection of comics to test via environment.
...
If you need to run test cases for your new comic module, you can call
py.test like this:
TESTCOMICS="ComicFury/" py.test -v tests/test_comics.py
2016-03-13 13:03:41 +01:00
Tobias Gruetzmacher
2ca74d6e6b
Convert all tests to py.test & cleanups.
2016-03-07 01:08:57 +01:00
Tobias Gruetzmacher
b6a6a34a44
Add CodeClimate config.
2016-03-06 00:41:10 +01:00
Tobias Gruetzmacher
41c03af3d9
Don't use SexyLosers for tests.
...
It's currently broken.
2016-03-04 00:28:05 +01:00
Tobias Gruetzmacher
b64e27f0ad
Add CodeClimate to Travis-CI.
2016-03-04 00:18:21 +01:00