I'm afraid I don't understand the filtering options in BlackWidow.
I'm writing some code to parse MIDI files and am trying to build a collection of sample files. I want to crawl a site such as http://www.free-midi.org and download any .mid files found. (Especially helpful would be if I could limit my downloads to the first few hundred found.)
How do I write a filter to do this?
Help writing filters
Re: Help writing filters
First, we have to follow each letters a to z on the page, so filter #1
Then, the authors, #2
then, each songs, #3
then, any next page links, #4
then the mid files, #5
So here are the filters. Copy the following and click on the "Paste settings" button.
Then, the authors, #2
then, each songs, #3
then, any next page links, #4
then the mid files, #5
So here are the filters. Copy the following and click on the "Paste settings" button.
Code: Select all
[BlackWidow v6.00 filters]
URL = http://www.free-midi.org/
[ ] Expert mode
[ ] Scan everything
[x] Scan whole site
Local depth: 0
[x] Scan external links
[ ] Only verify external links
External depth: 0
Default index page:
Startup referrer:
[ ] Slow down by 10:60 seconds
4 threads
[x] Follow /midi/[a-z]/$ using regular expression
[x] Follow /midi/[a-z]/[^/]+/$ using regular expression
[x] Follow /song/[^/]+\.html$ using regular expression
[x] Follow /midi/[a-z]/[^/]+/pg\d+/$ using regular expression
[x] Add \.mid$ from URL using regular expression
[end]
Your support team.
http://SoftByteLabs.com
http://SoftByteLabs.com
Re: Help writing filters
You can stop it at anytime, if you just want a few 100s.
Your support team.
http://SoftByteLabs.com
http://SoftByteLabs.com