Page 1 of 1

A possible problem

Posted: Tue Jul 19, 2016 4:37 pm
by pilotmike55
I am running BlackWidow v6.3 I have noticed that when I am scanning a site that randomly the program will freeze during the scan and the only thing I can do is to end the process in task manager. Because of this I routinely pause the scan, save the structure, then continue the scan. If the program freezes I end the process then load in the last saved structure for that scan.

I have also noticed that the number of remaining links to scan will be at particular number, let's say 10,000, when I say save it. If I reload that file and start the scan again, a few links will scan then the scan ends as if all of the links have been scanned and I cannot continue.

I am currently running Windows 10 and have ran BW in Windows 7 compatibility mode with the same results

Re: A possible problem

Posted: Tue Jul 19, 2016 4:41 pm
by Support
I'm not sure the nature of the freeze. BW uses your IE extension so it has to come from that. When you save the structure and reload it, make sure to resubmit the links in the Error tab because they were canceled.

Re: A possible problem

Posted: Tue Jul 19, 2016 6:43 pm
by pilotmike55
When I reload the file there is nothing under the link errors tab.

Re: A possible problem

Posted: Tue Jul 19, 2016 6:47 pm
by Support
ok, how many connections do you have it set to?

Re: A possible problem

Posted: Tue Jul 19, 2016 7:06 pm
by pilotmike55
I have it currently set to 6 connections. I have tried 12 but had the same results

Re: A possible problem

Posted: Tue Jul 19, 2016 7:17 pm
by Support
Try 4 because that's the standard setting for a web server. The freezing could be that it detect you are scanning the site and freeze the packet putting your scan in a loop. What you can do is use 12 connections but use the pause 1-1 to pause each one for 1 sec before continuing.

Re: A possible problem

Posted: Tue Jul 19, 2016 7:44 pm
by pilotmike55
I have used the pause 1-3 previously and had the same issue.

Re: A possible problem

Posted: Tue Jul 19, 2016 7:53 pm
by Support
ok, is you IE up to date? this happened when people don't want to use IE and use Firefox for example, not updating their IE.

Re: A possible problem

Posted: Wed Jul 20, 2016 8:11 am
by pilotmike55
I am currently running IE11

Re: A possible problem

Posted: Wed Jul 20, 2016 9:30 am
by Support
on win10/64?

Re: A possible problem

Posted: Wed Jul 20, 2016 10:52 am
by pilotmike55
I have IE11 on Windows 10. But there is also Microsoft Edge

Re: A possible problem

Posted: Wed Jul 20, 2016 11:44 am
by Support
I don't know what to say, it could be many things such as your service provider etc. Do you scan while downloading? or only scan and download when the scan is done? do you need a login to the site? if so, maybe it ask to login from time to time and that login window is behind the BW window freezing BW until you close the login window?

Re: A possible problem

Posted: Sat Jul 23, 2016 8:46 am
by pilotmike55
I have downloaded a couple of other programs and they do not have the freezing issue that I experience with BlackWidow. Unfortunately they do not have the programmability that BlackWidow has.

Re: A possible problem

Posted: Sat Jul 23, 2016 12:52 pm
by Support
That's because none of them use IE like BW does. The advantage on using IE is because of the customization you can have, whatever your settings are in IE, that's what BW will use. The others have limited functionalities. In your case that may be irrelevant, it depend on the site you try to scan. I haven't had this problem of freezing and no one else have complained about BW freezing so it's very hard for me to know why it does. Could be a hardware issue as well, like a bad memory, or simply it's the fault of IE directly.

How long does it scan before the freeze? did you notice a pattern in time, number of files etc?

Re: A possible problem

Posted: Sat Jul 23, 2016 1:43 pm
by pilotmike55
There is no pattern to the freezing, it could be days, hours, or minutes. I thought at first it was do to links such as statcounter.com but I modified the filter to skip that page.

Re: A possible problem

Posted: Sat Jul 23, 2016 1:54 pm
by pilotmike55
Although it seems to occur more when scan has been running for a while and the file size is large rather when first starting the scan and the file size is small

Re: A possible problem

Posted: Sat Jul 23, 2016 1:55 pm
by Support
Are you scanning the whole site or are you trying to scan particular files such as images. Is a login required? if so you might be scanning the logout link too.

Re: A possible problem

Posted: Sat Jul 23, 2016 3:07 pm
by pilotmike55
A login is not required. I am scanning various blogs in tumblr.com. The URLs are typically http://blogname.tumblr.com/archive

Re: A possible problem

Posted: Sat Jul 23, 2016 3:24 pm
by Support
ok, so you should set BW to scan nothing and add filters to only follow the URLs of interest, in your example URL, they are all http://blogname.tumblr.com/post/586793124 where the number is any, so the "What to add" RegEx mask for this is

Code: Select all

blogname\.tumblr\.com/post/\d+$
and this will make it only scan the pages of the links listed on the start page, no other links like login, logout, join etc. I assume you want to download those pages right?

Re: A possible problem

Posted: Sun Jul 24, 2016 9:16 am
by pilotmike55
That is correct

Re: A possible problem

Posted: Sun Jul 24, 2016 11:14 am
by Support
ok, have you tried the filter I suggested?

Re: A possible problem

Posted: Sun Jul 24, 2016 5:39 pm
by pilotmike55
Yes I have and it still freezes. I have built a Windows 7 virtual machine and it still freezes.

Re: A possible problem

Posted: Sun Jul 24, 2016 6:10 pm
by Support
Virtual still uses the same PC so it won't help! I'm not sure what to suggest at this point other than let me know your settings, copy them from the filters and paste them here and I'll try it myself and see where the problem is, and fix it of course.