Commit Graph

742 Commits

Author SHA1 Message Date
cyian-1756
ecf7a4b623 IG ripper now no longer errors out on last page 2018-03-09 18:06:01 -05:00
cyian-1756
4af4691893 lastseen feature now works with instagram 2018-03-09 13:36:53 -05:00
cyian-1756
6769bbfb7d Now tells the user if they need to fill out a captcha 2018-03-09 10:39:02 -05:00
cyian-1756
e5096736bf Added tsuminoRipper 2018-03-09 09:25:50 -05:00
cyian-1756
da8d6541f5 Fixed xvideos ripper 2018-03-08 07:33:35 -05:00
cyian-1756
48ffcf68d3 Added normalizeUrl 2018-03-05 13:20:12 -05:00
cyian-1756
e6c43bb482 Added the normalizeUrl func, which allows a ripper to normalize a url before adding it to url histroy/check if its in url history 2018-03-05 13:16:19 -05:00
cyian-1756
34e27cd2fa Changed default value for history.end_rip_after_already_seen 2018-02-25 21:14:35 -05:00
cyian-1756
6ed509aa06
Merge pull request #440 from cyian-1756/skip
Added a last seen feature
2018-02-25 20:47:02 -05:00
cyian-1756
c965de9492 Updated regex 2018-02-25 20:26:06 -05:00
cyian-1756
cac3ca1615 Fixed 8muses 2018-02-25 20:22:07 -05:00
cyian-1756
c0a0b9dce8 Removed debug statement; Changed option name 2018-02-25 19:51:28 -05:00
cyian-1756
8c86377a7f Implemented a last seen feature; Added comments 2018-02-25 18:10:38 -05:00
cyian-1756
66739e4c7c Implemented a last seen feature; Added comments 2018-02-25 18:10:30 -05:00
cyian-1756
4b7db493d1 Added support for incase.buttsimthy.com 2018-02-21 18:38:10 -05:00
cyian-1756
8a4e26474c imgurRipper no longer tries to add ?1 to file names 2018-02-11 10:04:29 -05:00
cyian-1756
639a0353a6 Undid saveAs sanitation in album ripper 2018-02-11 10:01:21 -05:00
cyian-1756
678b810828 AlbumRipper.java now removes bad chars from file names 2018-02-11 08:24:14 -05:00
cyian-1756
9afa1a9316 8Muses ripper now supports ripping from pages containing both images and subalbums 2018-02-10 02:36:28 -05:00
cyian-1756
7792f3d99e
Merge pull request #413 from rephormat/PornhubFix
Pornhub fix
2018-02-09 12:38:14 -05:00
cyian-1756
1fb4ad601d
Merge pull request #417 from cyian-1756/8muses
Eigthmuses ripper now uses ASAP ripping
2018-02-09 12:38:03 -05:00
cyian-1756
cf4bb92b5b
Merge pull request #400 from mszkb/fix_case-sensitive
Fix case sensitive directories on unix-based systems
2018-02-09 12:37:00 -05:00
cyian-1756
914327d203
Merge pull request #420 from cyian-1756/imagearn
Removed ripper and tests for imagearn
2018-02-09 12:33:34 -05:00
cyian-1756
1e297c4ac7
Merge pull request #416 from rephormat/CheveretoCleanup
Remove ripper and tests for gwarchives.com and hushpix.com
2018-02-09 12:22:37 -05:00
cyian-1756
ce972af4cc Removed ripper and tests for imagearn as the site is only only returning 403 errors 2018-02-09 11:51:52 -05:00
cyian-1756
3ed1ed6afc Eigthmuses ripper now uses ASAP ripping 2018-02-08 02:04:38 -05:00
rephormat
d36d3bc311 Remove ripper and tests for gwarchives.com and hushpix.com 2018-02-07 16:46:51 -06:00
rephormat
52099926cc Fix regex and skip quality with upsell designation. 2018-02-07 13:59:37 -06:00
cyian-1756
80e746adbb
Merge pull request #313 from cyian-1756/api-changes
Added hasASAPRipping(), which if returns true allows a ripper to not return anything from getURLsFromPage()
2018-02-07 11:07:09 -05:00
mszkb
26a52809da make method to check if directory exists 2018-01-20 15:13:46 +01:00
cyian-1756
f166c65921
Merge pull request #391 from kevin51jiang/master
Added VSCO Ripper
2018-01-18 12:00:25 -05:00
cyian-1756
c957422d8d
Merge pull request #392 from rephormat/Twitch-ripper
Twitch ripper
2018-01-18 11:59:59 -05:00
cyian-1756
8c6f3c4336
Merge pull request #396 from aaether32323/Webtoons_Ripper
Make webtoons ripper download maximum quality images
2018-01-18 11:59:40 -05:00
cyian-1756
671aae2875
Merge pull request #397 from rephormat/Pornhub-fix
Pornhub fix
2018-01-18 11:59:16 -05:00
cyian-1756
afafbbb466
Merge pull request #398 from kevin51jiang/TumblrAPIKeyFix
Tumblr API Key Choosing Fix & More Documentation
2018-01-18 11:58:50 -05:00
Kevin Jiang
4fdde22395
Merge pull request #4 from kevin51jiang/TumblrAPIKeyFix
Tumblr api key fix, plus more Javadoc stuff.
2018-01-17 21:23:39 -05:00
Kevin Jiang
375f9d8943 Tumblr API Key Choosing Fix
Fixes API choosing. Creates Random objects, generates a number, then feeds
that number to API_KEY.

This is as opposed to the previous way where it created the Random object,
generated a number, then fed it all in a single line.
2018-01-17 21:05:02 -05:00
rephormat
961faa9b63 Fixes RipMeApp/ripme#289 and RipMeApp/ripme#187 2018-01-17 19:17:07 -06:00
Aman
d72ef86b92 Make webtoons ripper download maximum quality images 2018-01-17 10:09:21 -05:00
Kevin Jiang
ebb3485c94 Improved Docs & Code Style Fixes
^
2018-01-16 17:48:35 -05:00
rephormat
a496379a9b
Merge branch 'master' into Twitch-ripper 2018-01-16 07:49:33 -06:00
cyian-1756
654f6e38dd
Merge pull request #387 from rephormat/yuvutu-ripper
Yuvutu ripper
2018-01-16 05:53:21 -05:00
slauzon
93785e5e9b User friendly filenames 2018-01-14 16:10:22 -06:00
Kevin Jiang
64a6f9257b VSCO Ripper Single Image
Currently can only support ripping a single Image. More study will be needed to rip member profiles and collections. (Maybe JSON?)
2018-01-14 15:21:20 -05:00
reph
81f804e61f Fixes xhamster.com video ripper. 2018-01-13 21:23:44 -06:00
rephormat
f7ae0e8d96 Add yuvutu.com ripper. 2018-01-12 19:24:40 -06:00
cyian-1756
da469aea4f Instagram ripper no longer 403s on certain images 2018-01-11 16:07:57 -05:00
Kevin Jiang
9dbd566340 VSCO Ripper Start
Just started the VSCO ripper by emulating another ripper and the instructions
from the wiki. Not yet functional, this commit is just to back stuff up.
2018-01-11 01:18:47 -05:00
cyian-1756
ae7ea5f8e6
Merge pull request #377 from kevin51jiang/GfycatFixAndTest
Gfycat fix and test
2018-01-06 14:06:48 -05:00
cyian-1756
1ebaaca823
Merge pull request #378 from cyian-1756/ig-options
instagram ripper can now be made to skip videos
2018-01-06 14:05:57 -05:00
cyian-1756
4632e7ef59 Added gwarchives support to the cheveretoRipper 2018-01-06 12:24:22 -05:00
Kevin Jiang
8960b8a565 Gfycat Tests & Fix for bad reddit submissions
Added sanitization, and appropriate tests.
Task #361 - Some gfycat doesn't work
2018-01-05 17:01:49 -05:00
cyian-1756
9b4137e094
Merge pull request #367 from cyian-1756/hentai-cafe
Fixed hentai.cafe regex
2018-01-04 15:44:03 -05:00
cyian-1756
3f8349a018 instagram ripper can now be made to skip videos 2018-01-02 21:30:53 -05:00
cyian-1756
cff8e754b1 Removed commented out code for pornpicsRipper 2018-01-02 21:16:47 -05:00
cyian-1756
e972a9caac Added basic pornpics.com ripper 2018-01-02 21:09:50 -05:00
cyian-1756
6fab0375b0 Fixed hentai.cafe regex 2017-12-29 04:31:36 -05:00
cyian-1756
97975a6257
Merge pull request #365 from kevin51jiang/MoreDocumentation
3 Metric Tons of Javadoc documentation for core Ripper stuff.
2017-12-29 04:21:26 -05:00
Kevin Jiang
5b49514e3e 3 Metric Tons of Javadoc documentation for Core Ripper stuff.
There should be no changes to the code. Just a ton of documentation.
It should help newcomers get aquainted with code quicker.
2017-12-28 00:04:23 -05:00
cyian-1756
6d0c562180
Merge pull request #348 from cyian-1756/erome
Erome ripper now matchs links without the www
2017-12-27 02:56:15 -05:00
cyian-1756
c08f053b36
Merge pull request #354 from cyian-1756/tumblr
Tumblr
2017-12-27 02:55:48 -05:00
cyian-1756
dfeb0c3616 FuraffinityRipper can now rip non-public albums 2017-12-24 08:39:27 -05:00
cyian-1756
8a80eb02a3 Added 2 new api keys, ripper can now download raw images from tumblr 2017-12-24 06:45:13 -05:00
cyian-1756
201c3fb086 Tumblr ripper can now rip raw images 2017-12-21 03:40:32 -05:00
cyian-1756
d06a0ae11b Erome ripper now matchs links without the www 2017-12-18 14:35:20 -05:00
cyian-1756
44ab7fd29a Tumblr ripper now tells the user if it hits the rate limit 2017-12-17 00:51:42 -05:00
cyian-1756
7c1f2955c4
Merge pull request #332 from GrigoreAlexandru/master
Added dribbble.com ripper
2017-12-16 16:13:09 -05:00
cyian-1756
d3f658222d
Merge pull request #340 from cyian-1756/tumblr-fix
Forced https for tumblr image links
2017-12-16 16:11:50 -05:00
cyian-1756
dbe39a89b8 Forced https for tumblr image links 2017-12-16 00:13:35 -05:00
cyian-1756
c0025a3b29 Fixed imgur album filenames 2017-12-14 22:38:22 -05:00
cyian-1756
57a53688e4 SankakuComplexRipper now downloads full sized images (#328) 2017-12-10 18:59:08 -08:00
Up rising
4d71cb2757 Added dribbble.com ripper 2017-12-10 16:23:25 +02:00
cyian-1756
1b36fa766f
Merge pull request #310 from cyian-1756/reddit-useragent
Updated reddit user agent
2017-12-07 02:15:35 -05:00
cyian-1756
d1f7708366
Added missing / 2017-12-07 02:05:57 -05:00
cyian-1756
61169352cb Moved hasASAPRipping() from AbstractHTMLRipper to AbstractRipper 2017-12-02 19:20:09 -05:00
cyian-1756
761f1e2987 Fixed bug that caused no images to be downloaded 2017-12-02 05:09:22 -05:00
cyian-1756
e89b27dc14 Added hasASAPRipping(), which if returns true allows a ripper to not return anything from getURLsFromPage() 2017-12-02 04:49:53 -05:00
cyian-1756
f82bb53eb7 Fixed E621 Ripper 2017-12-02 04:25:42 -05:00
cyian-1756
08c1b861bc Updated reddit user agent 2017-12-01 20:17:00 -05:00
cyian-1756
2c2159aa32 Oglaf ripper (#309)
* Added OglafRipper

* Better folder naming for Oglaf ripper

* Added oglaf unit test
2017-12-01 11:15:56 -08:00
metaprime
7716c47cc2
Fix #98: Update User Agent to current most common UA (Chrome 61 on Windows 10 x64) (#300) 2017-11-29 23:11:35 -08:00
cyian-1756
6ea6e7eded Reduce hardcoding in Wordpress comics (#299) 2017-11-29 22:37:08 -08:00
metaprime
5c95fedd5c
Merge pull request #296 from metaprime/hushpix-consent
Format CheveretoRipper; implement AGREE_CONSENT cookie for hushpix; add CheveretoRipperTest
2017-11-29 03:37:53 -08:00
metaprime
cea853f38e Format CheveretoRipper; implement AGREE_CONSENT cookie for hushpix. 2017-11-29 03:18:29 -08:00
metaprime
3ff05e171c Fix konradokonski.com/wiory ripper 2017-11-29 02:55:07 -08:00
cyian-1756
1c4a0bb1ab
Merge pull request #275 from cyian-1756/xhamster_rewrite
Rewrote xhamster ripper to use the html only mobile site and not the JS heavy beta site
2017-11-27 17:34:12 -05:00
cyian-1756
3f0f5daff3
Merge pull request #276 from cyian-1756/tapas
Fixed Tapas ripper
2017-11-27 17:27:38 -05:00
cyian-1756
0fc3b1c14c
Merge pull request #274 from cyian-1756/ig_useraccount_slide_show
Instagram ripper now downloads slide shows when ripping user profiles
2017-11-27 17:26:46 -05:00
cyian-1756
872c100ca5
Merge pull request #277 from cyian-1756/AerisDies
Minor improvements to Aerisdies folder naming
2017-11-27 17:26:13 -05:00
EgorGornak
bd6359babd
Add getGid() 2017-11-28 01:11:03 +03:00
EgorGornak
dc66b03c16
Added getGID() 2017-11-28 01:06:38 +03:00
EgorGornak
bf746aadbc
Update ImagearnRipper.java 2017-11-28 01:03:19 +03:00
EgorGornak
bc2e099093 Change folder name 2017-11-27 23:36:32 +03:00
EgorGornak
ef1bf79ddd Change folder name in imagearn 2017-11-27 17:37:04 +03:00
cyian-1756
ef84cb143b Minor improvements to Aerisdies folder naming 2017-11-23 10:56:43 -05:00
cyian-1756
36cd9c713f Minor improvements to Aerisdies folder naming 2017-11-23 10:50:06 -05:00
cyian-1756
5fc50e524b Fixed ripper for tapas.io 2017-11-23 00:51:37 -05:00
cyian-1756
8e2d32a6bc Rewrote xhamster ripper to use mobile site 2017-11-23 00:18:09 -05:00
cyian-1756
47822f612e Better naming output for instagram slide shows 2017-11-22 18:34:08 -05:00
cyian-1756
2e5f8a99b9 Instagram ripper can now downloads slideshows when ripping user profiles 2017-11-22 18:28:00 -05:00
cyian-1756
9d9cf61961 Added support for ripping single pages from instagram (#239)
* Added support for ripping from single pages

* Removed whitespace

* Instagram folder naming improvments

* Added GID tests for instagram single pages

* Added some album download tests for instagram

* Commented out flaky unit test
2017-11-21 13:07:25 -08:00
metaprime
fc32fbe713
Merge pull request #267 from metaprime/test-wordpress
Add WordpressComicRipperTest; remove dead site comics-xxx.com
2017-11-21 13:02:16 -08:00
cyian-1756
0b3f259873 Started on fixing furaffinity ripper 2017-11-21 12:51:00 -08:00
metaprime
9632fcd554 Add test for thisis.delvecomic.com 2017-11-21 12:47:53 -08:00
metaprime
bb0678644c comics-xxx.com no longer exists 2017-11-21 12:47:53 -08:00
metaprime
bbb4770a11 Add WordpressComicRipperTest 2017-11-21 12:47:53 -08:00
metaprime
f630123a6e
Fix WordpressComicRipper's ripper for freeadultcomix.com (#270) 2017-11-21 12:46:58 -08:00
cyian-1756
7b912dfa64 Improved webtoons folder naming (#265) 2017-11-21 12:12:13 -08:00
metaprime
1000fb78e8 Removed SupertangasRipper (no longer any static content to download) 2017-11-21 04:44:45 -08:00
metaprime
d1aad83d28 Removed MediacrushRipper (permanently down) 2017-11-21 04:23:21 -08:00
metaprime
0afe0377f2 Removed FapprovedRipper (permanently down) 2017-11-21 04:19:04 -08:00
metaprime
14b35a0072 Remove DatwinRipper (datw.in permanently down) 2017-11-21 04:13:12 -08:00
metaprime
46e2ce074a Remove CheebyRipper (cheeby.com permanently down) 2017-11-21 04:10:38 -08:00
metaprime
7ee18bea79 Remove ButttoucherRipper (butttoucher.com is permanently down) 2017-11-21 04:07:47 -08:00
metaprime
d64282ed7b Fix #115: Remove GifyoRipper (gifyo.com no longer exists) 2017-11-21 03:45:49 -08:00
metaprime
b296137b93
Fix #242: Remove broken and not valuable FineboxRipper (#245) 2017-11-21 03:07:36 -08:00
metaprime
dffe9a40aa ImageStash.com is permanently down: remove ImagestashRipper 2017-11-21 02:05:29 -08:00
metaprime
5510b407ad
Fix loggers (#235) 2017-11-19 19:42:12 -08:00
cyian-1756
ecf72e517e
Merge pull request #222 from cyian-1756/ig_hashtag
Added support for ripping from IG tags
2017-11-18 09:16:21 -05:00
cyian-1756
bcf3f9d2a7 Instagram ripper now handles urls with and without a trailing / 2017-11-18 09:07:25 -05:00
cyian-1756
73b165c645
Merge pull request #220 from cyian-1756/ig_regex
Improved instagram regex and added sanitizeURL
2017-11-18 08:53:09 -05:00
cyian-1756
e8185aaf3e Added support for ripping from tags 2017-11-18 07:22:25 -05:00
cyian-1756
5c185c05df Improved instagram regex and added sanitizeURL 2017-11-18 03:52:17 -05:00
cyian-1756
9214ce14bb Improved regex for pichunter.com 2017-11-17 17:15:56 -05:00
cyian-1756
d268bb2cee Pichunter galleries support (#209)
* Added support for pichunter galleries

* Added unit test for pichunter galleries
2017-11-17 02:37:47 -08:00
metaprime
fad35d1afb
Fix #211: Add Utils.filesystemSanitized and apply to filename prefix. (#213) 2017-11-17 02:37:02 -08:00
cyian-1756
a2fc44aced Logging fixes (#189) 2017-11-17 02:35:37 -08:00
cyian-1756
6e24a72314
Merge pull request #177 from cyian-1756/instagram_add_date
Change to IG ripper filename format and move from JSON ripper to HTML
2017-11-16 18:19:04 -05:00
cyian-1756
d435b1a638
Merge branch 'master' into pichunter 2017-11-16 18:09:11 -05:00
cyian-1756
0b2ff13c3e
Merge pull request #207 from cyian-1756/theyiffgallery
Added ripper for theyiffgallery
2017-11-16 17:53:00 -05:00
cyian-1756
a6b3f50b73 Added ripper for Theyiffgallery 2017-11-16 16:18:17 -05:00
cyian-1756
9837f893f1 pichunter now supports all album types 2017-11-16 14:35:40 -05:00
cyian-1756
fc5d4b7533 pichunter ripper no longer errors out on last page 2017-11-16 14:09:03 -05:00
cyian-1756
149ab50dc1 Added basic pichunter ripper 2017-11-16 13:58:21 -05:00
cyian-1756
669a78434c Added regex for xchan 2017-11-14 13:39:38 -05:00
cyian-1756
4016f35ab4 Ripper now falls back to getHost() and getGID(), not page title 2017-11-14 13:22:20 -05:00
cyian-1756
07f180ee70 Removed repetitive func 2017-11-12 05:00:39 -05:00
cyian-1756
d1e021f0ef Added single post support to the instagram ripper 2017-11-12 04:43:26 -05:00
cyian-1756
7f57768f3d instagram ripper now puts the date the image was posted in the saved file name 2017-11-12 03:15:56 -05:00
cyian-1756
79958b1342
Merge pull request #71 from cyian-1756/view-comic
Added viewcomic ripper
2017-11-12 02:24:00 -05:00
cyian-1756
22f438f445
Merge pull request #163 from RipMeApp/webtoons
Fixed webtoons malformed url error message
2017-11-12 02:22:52 -05:00
cyian-1756
690abe848f
Merge pull request #172 from cyian-1756/chanRipperFixes
Fixed chan ripper thread title (For real this time)
2017-11-12 02:20:25 -05:00
cyian-1756
ebacc90915
Merge pull request #173 from cyian-1756/ModelMayhamFixes
Fixed Modelmayhem ripper
2017-11-11 07:32:57 -05:00
cyian-1756
0845efe328 Fixed Modelmayhem ripper 2017-11-11 03:10:08 -05:00
cyian-1756
2de9d58cca Fixed chan ripper thread title (For real this time) 2017-11-11 01:56:36 -05:00
cyian-1756
4a5588c06c
Merge pull request #171 from cyian-1756/chanRipperFixes
Chan ripper now falls back on page title if thread title can not be …
2017-11-11 01:41:26 -05:00
cyian-1756
84cf9b1fc5 Chan ripper now falls back on page title if thread title can not be found 2017-11-10 21:26:16 -05:00
cyian-1756
ffb26c2fe0 Chan ripper now falls back on page title if thread title can not be found 2017-11-10 09:29:26 -05:00
cyian-1756
3f2d119347
Merge pull request #162 from RipMeApp/url_history_fix
Remembering url history no longer causes unit tests to fail
2017-11-10 03:58:30 -05:00
cyian-1756
806a198cf0
Merge pull request #157 from RipMeApp/imgur_t_albums
Added support for imgur /t albums
2017-11-10 03:45:32 -05:00