script help

BlackWidow scans websites (it's a site ripper). It can download an entire website, or download portions of a site.
Post Reply
dlbott
Posts: 5
Joined: Tue Oct 06, 2015 4:24 pm

script help

Post by dlbott » Wed Oct 07, 2015 11:31 am

Really having hard time with script help for blackwidow. I used to have one of the first versions and back then had no problems, being you could easily tell it you wanted pdf, mp4, avi etc. etc. now you have to be a script writer. Flaw I think, should be an easy way to just plug in the extension and it automatically put it in the script where it is needed to rip it. Also, there are no remarks inside the script to help you. example: //Remark: In this section of the script add this example to add any file extension you want to download: and then show an example to add mp3 or pdf. at least then you would know where to put it and how the syntax needs it to work.

I need a script for the following: pdf,mp4,avi,flv,doc,docx,xls,xlsx,php,js,wmv,wav,mp3,zip,rar,exe

If someone could help would appreciate it... please tell me where to add into the script or whether to just replace the whole script etc.

I have tried to rip the whole site with default and have been unable to rip any of these files in the scan, even though I know they are there. There are times when wget works but it is slow and would rather just be able to see the structure and files and pick and choose what to grab. This program used to work great and was much easier. Saddened I lost that copy of the old version when computer went down.

thanks

dlbott

dlbott
Posts: 5
Joined: Tue Oct 06, 2015 4:24 pm

Re: script help

Post by dlbott » Wed Oct 07, 2015 2:26 pm

Hmmm, was hoping for some help... something must be wrong.... I tell it to scan whole site, where I know there are files etc... I let it scan for couple hours and stop to check. And even when I watch it is only getting gifs and pictures...

I really want to see if this program works... it used to, flawlessly... hoping to find some help... something must be set wrong... even though said scan everything, when I look at structure all that is there is the index.html and png, gif, jpeg's., nothing else.....

Please help....

the bot

User avatar
Support
Site Admin
Posts: 1679
Joined: Sun Oct 02, 2011 10:49 am

Re: script help

Post by Support » Wed Oct 07, 2015 2:28 pm

The old way use to be only html, now it's all scripts like javascript and server side scripts so it's no longer that easy! What is the link you wish to scan?
Your support team.
http://SoftByteLabs.com

todd
Posts: 2
Joined: Wed Nov 04, 2015 3:21 pm

Script to 'scrape' Twitter handles from site [Re: script help]

Post by todd » Wed Nov 04, 2015 3:27 pm

I am trying to 'scrape' Twitter handles off a site, and write them into a csv file. Thought this would be simple to write, using BlackWidow to type in the URL [Browser], then have a script execute [Filters], creating a CSV file with results. However, has been a challenge to do this, and I am hoping someone has already created this script with this approach, and can post it. Searched the 'board index' and cannot find anything like this yet. Thanks!

User avatar
Support
Site Admin
Posts: 1679
Joined: Sun Oct 02, 2011 10:49 am

Re: script help

Post by Support » Wed Nov 04, 2015 3:42 pm

You can't do that with BlackWidow. It might be possible to do it with BrownRecluse, but, that's only if the site doesn't use javascript, ajax and stuff likt that to dynamically load the data you need.
Your support team.
http://SoftByteLabs.com

todd
Posts: 2
Joined: Wed Nov 04, 2015 3:21 pm

Re: script help

Post by todd » Wed Nov 04, 2015 3:44 pm

Thank you. I also have BrownRecluse. Is there a script you might be able to provide or suggest, so I can do what you are suggestion. Really appreciate the help, and a script would be great. Thanks again.

User avatar
Support
Site Admin
Posts: 1679
Joined: Sun Oct 02, 2011 10:49 am

Re: script help

Post by Support » Wed Nov 04, 2015 3:54 pm

No I do not unfortunately. I can try to help if you send me the URL you need it for.
Your support team.
http://SoftByteLabs.com

Post Reply