Extremely large save file?

BlackWidow scans websites (it's a site ripper). It can download an entire website, or download portions of a site.
Post Reply
avalanch
Posts: 42
Joined: Fri Mar 16, 2012 11:31 am

Extremely large save file?

Post by avalanch » Thu Jan 25, 2018 1:05 am

I just got partway done with a site that has a little over 2k pages to download and I notice that the save file for this project is over 2GB?

2.22GB to be exact.

Why? How? Will it even be able to load that save file if I try to open it tomorrow?
2018-01-25.png
2018-01-25.png (96.08 KiB) Viewed 3127 times

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Extremely large save file?

Post by Support » Thu Jan 25, 2018 3:28 pm

Yes it will load.
Your support team.
http://SoftByteLabs.com

avalanch
Posts: 42
Joined: Fri Mar 16, 2012 11:31 am

Re: Extremely large save file?

Post by avalanch » Tue Feb 06, 2018 4:03 pm

Hello, after 2 long hours, blackwidow finally manages to load this thing, is there any way to cut it down to size or make it load faster?

My system is more than capable of multitasking it while doing other stuff.

I've compressed it so you can try it for yourself.
Attachments
Cheatingdome.7z
(15.23 MiB) Downloaded 149 times
speccy.jpg
speccy.jpg (74.49 KiB) Viewed 3034 times

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Extremely large save file?

Post by Support » Wed Feb 07, 2018 4:37 pm

It is saving the referring pages for every links, in fact, all the referrers for them, can be many, so it take a lot of space. There is no option to not save the referrer however!!
Your support team.
http://SoftByteLabs.com

swanisabella
Posts: 1
Joined: Wed Feb 21, 2018 3:48 am

Re: Extremely large save file?

Post by swanisabella » Wed Feb 21, 2018 3:54 am

it takes time to load.
______________________________
Best online MyAssignmentHelp services.

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Extremely large save file?

Post by Support » Wed Feb 21, 2018 5:03 pm

It can yes, especially when the site has many links pointing to a single link, it creates a lot of referring links. What I can do is create a small program that can remove all but one referrer from the .web file, it will be very small and load fast, or, you can edit the .web file (it's a text file) and run a script to remove all the referrers. What do you think?
Your support team.
http://SoftByteLabs.com

avalanch
Posts: 42
Joined: Fri Mar 16, 2012 11:31 am

Re: Extremely large save file?

Post by avalanch » Thu Mar 15, 2018 4:26 pm

Sure, I can give that a try.

avalanch
Posts: 42
Joined: Fri Mar 16, 2012 11:31 am

Re: Extremely large save file?

Post by avalanch » Sun Apr 15, 2018 9:38 pm

Support wrote:
Wed Feb 21, 2018 5:03 pm
It can yes, especially when the site has many links pointing to a single link, it creates a lot of referring links. What I can do is create a small program that can remove all but one referrer from the .web file, it will be very small and load fast, or, you can edit the .web file (it's a text file) and run a script to remove all the referrers. What do you think?
Hello @support, I have waited for 1 month and have heard no reply from you yet. Will you still do this?

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Extremely large save file?

Post by Support » Mon Apr 16, 2018 11:31 am

oh, your last reply was "Sure, I can give that a try." so I thought you will give it a try to remove the referrer yourself. ok, let me make you something.
Your support team.
http://SoftByteLabs.com

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Extremely large save file?

Post by Support » Tue May 01, 2018 11:44 am

I am sending you the file to your PM...
Your support team.
http://SoftByteLabs.com

Post Reply