How can I achieve this?

BlackWidow scans websites (it's a site ripper). It can download an entire website, or download portions of a site.
Post Reply
music_lover
Posts: 19
Joined: Thu May 03, 2012 1:46 pm

How can I achieve this?

Post by music_lover » Mon Apr 10, 2017 12:13 pm

Hi folks,
I'm thinking I might need BrownRecluse to do this instead of BlackWidow... if that's the case, let me know and I will purchase it.
In either case, I'd like to:
  • Visit this link: http://novalis.rightbrainmedia.com/port ... e=readings
  • And pull the text after the words "Daily Readings" to the end of the text section (which might sometimes scroll down past the bottom of the page)
  • Move to the next page (click the 'next' button**)
  • And pull the text from that page
  • see note below
  • And save it to a .txt file under the name (for example) "Wednesday_April_12.txt" (or whatever the date is shown up top)
  • And move to the next page (click the red > button, upper right corner)
  • And repeat the text grab and save as a .txt file under the new date shown up top.
  • And repeat and repeat...
**Note that sometimes it will be necessary to click the 'next' button more than once, depending on the day, in order to capture another page of text for that day. Sometimes it will be several pages, several clicks of 'next'. When the 'next' button disappears, there are no more pages - no more text - for that day.

Thanks!

User avatar
Support
Site Admin
Posts: 1581
Joined: Sun Oct 02, 2011 10:49 am

Re: How can I achieve this?

Post by Support » Mon Apr 10, 2017 2:13 pm

It won't work with BW and I can't make it work in BR either. The site uses jquery, javascript and server side scripts heavily! For example, copy the link of the Next button and paste it in your browser, you'll get an error, not the next page.
Your support team.
http://SoftByteLabs.com

music_lover
Posts: 19
Joined: Thu May 03, 2012 1:46 pm

Re: How can I achieve this?

Post by music_lover » Tue Apr 11, 2017 9:16 am

I'm sad that it can't be done but very thankful for your quick reply!
Much appreciated.

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests