Java link expansion and filter

BlackWidow scans websites (it's a site ripper). It can download an entire website, or download portions of a site.
Post Reply
natsmith52
Posts: 6
Joined: Mon Jul 23, 2012 10:25 am

Java link expansion and filter

Post by natsmith52 » Mon Jul 23, 2012 10:38 am

Sirs:
I wish to extract this out of a Java script:
/view/2115587/ (/view/???????/)
Expand it to this:
http://www.furaffinity.net/view/2115587/ (http://www.furaffinity.net/view/???????/)
Keep that link and throw everything else away.
Any 'Expert Script' to do that?
Thank you for your support.
Nat

User avatar
Support
Site Admin
Posts: 1854
Joined: Sun Oct 02, 2011 10:49 am

Re: Java link expansion and filter

Post by Support » Mon Jul 23, 2012 11:48 am

Yes, but, do you want to replace a javascript call to an href call so BW can find the link? Or do you simply want to scan a javascript for links and tell BW to scan those links?
Your support team.
http://SoftByteLabs.com

natsmith52
Posts: 6
Joined: Mon Jul 23, 2012 10:25 am

Re: Java link expansion and filter

Post by natsmith52 » Mon Jul 23, 2012 5:45 pm

Only need to add/scan the generated links;
Don't care about the original Java.
Nat

User avatar
Support
Site Admin
Posts: 1854
Joined: Sun Oct 02, 2011 10:49 am

Re: Java link expansion and filter

Post by Support » Mon Jul 23, 2012 6:38 pm

Yu coud use the following...

Code: Select all

case ScannerEvent of

  BeforeParsing:
    begin
      for each matching('/view/[^/]+/') in Document as aLink do begin
        aLink.ResolveRelative(DocumentURL); // resolve links like ../foo/bar/
        Scanlink(aLink); // add the link to the scan queue.
      end;
    end;

end;
But this will find all /view/???????/ from the whole document. You have to change the mask in matching(...) to find only those in javascript. I don't have the URL you're trying to do this on, so I can't give a mark for it.
Your support team.
http://SoftByteLabs.com

natsmith52
Posts: 6
Joined: Mon Jul 23, 2012 10:25 am

Re: Java link expansion and filter

Post by natsmith52 » Wed Jul 25, 2012 4:43 am

Um....
Having a problem editing BW6 'Expert Script'; If I edit it in any .bw6 file;
it changes for every single .bw6 file!
Is this suppose to be gobal???
That's not very usefull!
Nat

User avatar
Support
Site Admin
Posts: 1854
Joined: Sun Oct 02, 2011 10:49 am

Re: Java link expansion and filter

Post by Support » Wed Jul 25, 2012 6:55 am

It only goes back to "Basic" filters. Click on "Expert" and your last script entered will still show, unless your BW6 file contain another. So I don't see the problem??
Your support team.
http://SoftByteLabs.com

natsmith52
Posts: 6
Joined: Mon Jul 23, 2012 10:25 am

Re: Java link expansion and filter

Post by natsmith52 » Thu Jul 26, 2012 5:44 am

Here's the problem:
All .bw6 files; everyone on my entire system; even ones saved months ago; show the same 'Expert Script':
-the last one edited. Which means there is just one 'Expert Script' for the entire enternet.
One 'Expert Script' for every web-site on the planet! That is not very usable.

Nat

User avatar
Support
Site Admin
Posts: 1854
Joined: Sun Oct 02, 2011 10:49 am

Re: Java link expansion and filter

Post by Support » Thu Jul 26, 2012 10:37 am

That's because none of your .bw6 file conatain a script, so that's normal that it doesn't change!

When you open a .bw6 file, the filters goes back to Basic, that means the .bw6 file does not use a script, and if you click on Expert anyway, the last script used will still be showing.
Your support team.
http://SoftByteLabs.com

natsmith52
Posts: 6
Joined: Mon Jul 23, 2012 10:25 am

Re: Java link expansion and filter

Post by natsmith52 » Fri Jul 27, 2012 7:04 am

OK!
If a .bw6 file does not contain a script; how do you get the default script back?
Nat.

User avatar
Support
Site Admin
Posts: 1854
Joined: Sun Oct 02, 2011 10:49 am

Re: Java link expansion and filter

Post by Support » Fri Jul 27, 2012 7:16 am

Delete the content in the script editor so it's empty, exit BW and go back, the default will load.
Your support team.
http://SoftByteLabs.com

Post Reply