Page 1 of 1

BW barks up all branches of the folder tree

Posted: Thu Feb 07, 2019 2:48 pm
by Latro Dektes
Goal: I would like to download pdf files similar to this one: www.boston.gov/sites/default/files/imce ... 0_2018.pdf

Try 1) I pasted that URL into BlackWidow. Unfortunately BW crashes when I press "scan"

Try 2) I shortened the URL to www.boston.gov/sites/default/files/imce ... s/2018-08/
which not surprisingly gave a 404 error.
When I try to scan that BW gives me a link error.

Try 3) I shortened the URL to www.boston.gov
BW happily scanned that URL and after a long time it eventually found the folders that interest me.

So only try 3 worked for me. QUESTION: Is there a way I can encourage BW to focus on only part of the directory tree?

Re: BW barks up all branches of the folder tree

Posted: Thu Feb 07, 2019 4:04 pm
by Support
You have to find the page where all the PDF are listed in order to get them. Like your PDF link example, what was the page you found it in?

Re: BW barks up all branches of the folder tree

Posted: Thu Feb 07, 2019 4:25 pm
by Latro Dektes
Yes, it is best to begin at the citing page. I hadn't realized that they are listed here:
www.boston.gov/departments/inspectional ... ard-appeal
So that solves today's chore.

But while I have your attention let me ask whether you are surprised by the failure of my first 2 tries.
Try 1 showed that BW crashes if you begin with the URL of a pdf file.
Try 2 shows that BW cannot proceed if your starting URL returns an error.

Re: BW barks up all branches of the folder tree

Posted: Thu Feb 07, 2019 6:06 pm
by Support
I use v6.3 and scanning just the PDF works. The directory of the PDF doesn't work not because BW isn't capable but because the server doesn't allow directory indexing.

Re: BW barks up all branches of the folder tree

Posted: Thu Feb 07, 2019 8:35 pm
by Latro Dektes
Thank you. I'm all set now.
End of line.