Search found 42 matches

by avalanch
Sun Apr 15, 2018 9:38 pm
Forum: BlackWidow
Topic: Extremely large save file?
Replies: 9
Views: 7060

Re: Extremely large save file?

It can yes, especially when the site has many links pointing to a single link, it creates a lot of referring links. What I can do is create a small program that can remove all but one referrer from the .web file, it will be very small and load fast, or, you can edit the .web file (it's a text file)...
by avalanch
Thu Mar 15, 2018 4:26 pm
Forum: BlackWidow
Topic: Extremely large save file?
Replies: 9
Views: 7060

Re: Extremely large save file?

Sure, I can give that a try.
by avalanch
Tue Feb 06, 2018 4:03 pm
Forum: BlackWidow
Topic: Extremely large save file?
Replies: 9
Views: 7060

Re: Extremely large save file?

Hello, after 2 long hours, blackwidow finally manages to load this thing, is there any way to cut it down to size or make it load faster?

My system is more than capable of multitasking it while doing other stuff.

I've compressed it so you can try it for yourself.
by avalanch
Thu Jan 25, 2018 1:05 am
Forum: BlackWidow
Topic: Extremely large save file?
Replies: 9
Views: 7060

Extremely large save file?

I just got partway done with a site that has a little over 2k pages to download and I notice that the save file for this project is over 2GB?

2.22GB to be exact.

Why? How? Will it even be able to load that save file if I try to open it tomorrow?
2018-01-25.png
2018-01-25.png (96.08 KiB) Viewed 7060 times
by avalanch
Thu Aug 20, 2015 3:43 pm
Forum: BlackWidow
Topic: Need wget rules converted for Blackwidow.
Replies: 1
Views: 8018

Need wget rules converted for Blackwidow.

I would like the following rules for blackwidow, thanks. I've tried but it stops after 1 download, claiming it's "done".
blackwidow-phpbb.jpg
blackwidow-phpbb.jpg (315.67 KiB) Viewed 8018 times
by avalanch
Thu Jun 11, 2015 10:19 pm
Forum: NameWiz
Topic: Remove numbered listing from files?
Replies: 2
Views: 4172

Re: Remove numbered listing from files?

Thanks, I was overthinking this one a bit lol
by avalanch
Sun Jun 07, 2015 4:03 pm
Forum: NameWiz
Topic: Remove numbered listing from files?
Replies: 2
Views: 4172

Remove numbered listing from files?

Remove numbered listing from files. What settings should I use to remove the numbers in front of the titles? I have over 2400 titles to rename this way, thanks! 0001 - F-Zero for Game Boy Advance (J).zip 0002 - Super Mario Advance (J).zip 0003 - Battle Network Rockman EXE (J).zip 0004 - Kurukuru Kur...
by avalanch
Mon Jan 20, 2014 2:28 pm
Forum: NameWiz
Topic: Need to batch rename multiple images.
Replies: 3
Views: 6808

Re: Need to batch rename multiple images.

Hi thanks, is it possible to setup macros? Like for instance I type in what I want to be removed in a macro and with a press of a button it's entered into the replace/remove fields.
by avalanch
Sun Jan 19, 2014 9:02 pm
Forum: NameWiz
Topic: Need to batch rename multiple images.
Replies: 3
Views: 6808

Need to batch rename multiple images.

I need to try to replace the space characters with -'s. Example:

Rayman 2 - The Great Escape (USA) (En,Fr,De,Es,It)
TO
Rayman-2-The Great Escape-(USA)-(En,Fr,De,Es,It)

Also in this case I noticed it would have put in two -'s instead of one, any tips?
by avalanch
Sun Jan 05, 2014 11:57 am
Forum: BlackWidow
Topic: Feature suggestion thread
Replies: 1
Views: 4852

Feature suggestion thread

Have an idea for a new feature for blackwidow, please keep them all in this thread thanks! I think a good option to be added (from httrack) would be a option to follow robots.txt rules or not. *Detection of if the scan is crashing the site's server or not & proper slowdowns applied if this is the ca...
by avalanch
Sat Jan 04, 2014 10:25 pm
Forum: BlackWidow
Topic: How to download images only from gamefaqs.com
Replies: 8
Views: 12713

Re: How to download images only from gamefaqs.com

I was wrong, it is downloading them properly as you said. :mrgreen:
by avalanch
Sat Jan 04, 2014 9:59 pm
Forum: BlackWidow
Topic: How to download images only from gamefaqs.com
Replies: 8
Views: 12713

Re: How to download images only from gamefaqs.com

ok thanks. Is it possible to make it crawl the subdomain of img.gamefaqs.net? Tricky one there... at first I thought it ended in gamefaqs.com

nvm it seems to be grabbing from the img.gamefaqs.net
by avalanch
Sat Jan 04, 2014 9:47 pm
Forum: BlackWidow
Topic: How to download images only from gamefaqs.com
Replies: 8
Views: 12713

Re: How to download images only from gamefaqs.com

Hi thanks, it doesn't seem to be adding any yet... I can report that I had to slowdown the scan as it tried to become unresponsive. And yes my system is more than capable of handling it... but why nothing added? http://img.photobucket.com/albums/v48/scylla/specs_zpsa0149f8d.png http://i.imgur.com/Vs...
by avalanch
Sat Jan 04, 2014 9:10 pm
Forum: BlackWidow
Topic: How to download images only from gamefaqs.com
Replies: 8
Views: 12713

Re: How to download images only from gamefaqs.com

Also it's interesting to note that some images on gamefaqs are hosted on gamespot.. like so... http://img.gamefaqs.net/screens/1/1/2/gfs_67605_2_3_thm.jpg http://image.gamespotcdn.net/gamespot/images/box/5/1/7/928517_75786_thumb.jpg If it helps here is a gamefaqs scraper I think for xbmc written in ...
by avalanch
Sat Jan 04, 2014 9:00 pm
Forum: BlackWidow
Topic: How to download images only from gamefaqs.com
Replies: 8
Views: 12713

How to download images only from gamefaqs.com

I wish for it to save only images @ gamefaqs(png, gif, jpg, jpeg, bmp & webp formats)... any other files, it can throw away without saving them. Can someone help me with settings to make the blackwidow do this? Thanks! This is from their sitemap page for their images. <h2>Images</h2> <ul> <li><a hre...
by avalanch
Fri Dec 27, 2013 12:43 pm
Forum: BlackWidow
Topic: Blackwidow downloading multiple copies of html pages with "bookmarklets"
Replies: 4
Views: 7646

Blackwidow downloading multiple copies of html pages with "bookmarklets"

When it encounters pages with the little hash tags, it tends to grab a copy for every hashtag element thing it finds... help?

Example here.... there's 15 of the hashtag things so it'll grab 15 copies of that page.
http://www.world-of-nintendo.com/ninten ... ault.shtml
by avalanch
Mon Jun 17, 2013 6:12 pm
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

Re: How to crawl urls with no file extension?

Yes the mime type was set correctly.... I manage several sites and do some maintenance on them now & then so having a browser set to open a htm/html/shtml/php file etc is pretty important to me but as your request I checked via regedit and here is result.
softbytemime.png
softbytemime.png (296.88 KiB) Viewed 22745 times
by avalanch
Mon Jun 17, 2013 3:31 pm
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

Re: How to crawl urls with no file extension?

And what about the threads? Those are the main one's that give trouble. Also maybe you have different rule sets than me because mine isn't saving as .php.htm instead just saving it with apparently no extension like this 2468-Ultimecia-Vs-Sephiroth-Vs-Kuja-Vs-Kefka Unless of course it does encounter ...
by avalanch
Mon Jun 17, 2013 3:06 pm
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

Re: How to crawl urls with no file extension?

even with what you suggested it still remains the same. Even further verified by opening the file in editplus... where it displays the correct file syntax colors to show opened/closed tags & other good stuff.
by avalanch
Mon Jun 17, 2013 1:04 pm
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

Re: How to crawl urls with no file extension?

Unfortunately it doesn't seem to want to put the .html for me.

Image
by avalanch
Mon Jun 17, 2013 12:01 pm
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

Re: How to crawl urls with no file extension?

I'm wanting to know, how I make your software crawl download these pages when it encounters them?
Instead of saving it and calling the end result .file can we make it save it as .html?

http://www.finalfantasyforums.net/forum.php

There is one of the sites I'd like to crawl.
by avalanch
Mon Jun 17, 2013 10:11 am
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

Re: How to crawl urls with no file extension?

Well? Is there a way to do this?
by avalanch
Fri Jun 14, 2013 4:48 pm
Forum: BlackWidow
Topic: How to crawl urls with no file extension?
Replies: 13
Views: 22785

How to crawl urls with no file extension?

For example a site that has a url like.. http://www.site.com/forums/65-Guadosalam http://www.site.com/threads/583-Happy-Birthday-Someone The two above examples generated by vbseo, a popular seo plugin for vbulletin. How to make it crawl and save it as html or something? Blackwidow v6.30 likes to jus...