Page 1 of 1

I need help with one website

Posted: Fri Aug 10, 2018 5:59 pm
by braciata
Hi! I used BW for some websites and it worked fine cause they were built in easy html.
Now I came across with this one http://miditune.com/
I would like to download the .mid files but I am not able.
Any help? Thank you
Fabrizio

Re: I need help with one website

Posted: Fri Aug 10, 2018 6:10 pm
by Support
When you click on the green download button in the browser, it sends a POST to the server, like a Submit button when you register, that's why BW can't see the MIDI file. Only our BrownRecluse software can do this because it's programmable and can send POST request, BW only can do GET requests from links found on the page.

Re: I need help with one website

Posted: Fri Aug 10, 2018 6:13 pm
by braciata
Thanks for the reply

Re: I need help with one website

Posted: Sat Aug 11, 2018 3:19 am
by braciata
After your answer I downloaded BrownRecluse but I am newbie and I couldn't figure out a right code. I checked also the examples (I loved the one for the images) but nothing for 'post' method. Do you have some other example for my case? Thank you

Re: I need help with one website

Posted: Sat Aug 11, 2018 8:08 pm
by Support
Here is a BrownRecluse script to download the mid files from the site. Set the download folder on line 5 and set the URL on line 4...

Code: Select all

PerlRegEx = Yes;
Output.Clear;

MidiURL = 'http://miditune.com/';
DownloadFolder = ScriptPath + 'midi download';

Link = New(URL);
fl = New(File);
rx = New(RegEx);
sk = New(Stack);

Abort = False;

DownloadFolder.Trailing('\');
Link.Get(MidiURL);
//Output(Link.Data);

rx.Data = Link.Data;
rx.Mask = 'href="(http\:\/\/miditune.com/midi-[^"]+\.html)">';
rx.Reset;

while rx.Match do
  sk.Push(rx.Value[1]);

unless sk.Empty do begin

  lnk = sk.Pop;
  //Output(lnk);
  Link.Get(lnk);
  //Output(Link.Data);

  frmID = WildGet(Link.Data, 'name="id" value=''([^'']+)''');
  frmSS = WildGet(Link.Data, 'name="session_id" value=''([^'']+)''');

  Link.Referer = Link.Location;
  Link.Location = 'http://miditune.com/download.php';
  Link.Post('id='+frmID+'&session_id='+frmSS);

  Flname = DownloadFolder + WildGet(Link.Headers, 'filename="([^"]+)"');
  //Output(Flname); Terminate;

  fl.Open(Flname);
  fl.Truncate;
  fl.Write(Link.Data);
  fl.Close;

  Output('Downloaded: '+Flname);

  if Abort then
    Break;

end;

function OnStop();
begin
  Result = @Abort;
  @Abort = True;
end;

Re: I need help with one website

Posted: Sun Aug 12, 2018 5:08 am
by braciata
Omg, thank you very much. I will try to study it

Re: I need help with one website

Posted: Mon Aug 13, 2018 1:11 pm
by Support
Here is a new script that will download all of the midi files from the entire site...

Code: Select all

PerlRegEx = Yes;
Output.Clear;

DownloadFolder = ScriptPath + 'midi download';

Link = New(URL);
fl = New(File);
rx = New(RegEx);

Abort = False;

DownloadFolder.Trailing('\');

for c = 'A' to 'Z' range('A','Z') do begin

  Link.Get('http://miditune.com/list/midi-'+c+'.html');
  //Output(Link.Data);Terminate;

    rx.Data = Link.Data;
    rx.Mask = 'href=(http[^<>]+)';
    rx.Reset;

    while rx.Match do begin

      lnk = rx.Value[1];
      lnk.Replace('\.mid$', '.html');
      //Output(lnk); Terminate;
      Link.Get(lnk);
      //Output(Link.Data);

      frmID = WildGet(Link.Data, 'name="id" value=''([^'']+)''');
      frmSS = WildGet(Link.Data, 'name="session_id" value=''([^'']+)''');

      Link.Referer = Link.Location;
      Link.Location = 'http://miditune.com/download.php';
      Link.Post('id='+frmID+'&session_id='+frmSS);

      Flname = DownloadFolder + WildGet(Link.Headers, 'filename="([^"]+)"');
      //Output(Flname); Terminate;

      fl.Open(Flname);
      fl.Truncate;
      fl.Write(Link.Data);
      fl.Close;

      Output('Downloaded: '+Flname);

      if Abort then
        Break;

    end;

end;

function OnStop();
begin
  Result = @Abort;
  @Abort = True;
end;

Re: I need help with one website

Posted: Mon Aug 13, 2018 1:26 pm
by braciata
Perfect! Thank you so much

Re: I need help with one website

Posted: Mon Aug 13, 2018 3:23 pm
by braciata
Here I am again!
When it comes ATB - 9PM Till I Come.mid BR stops.
I am not sure but I think because the url is different > http://miditune.com/midi-17680-download ... -come.html
9PM vs 9pm

Re: I need help with one website

Posted: Mon Aug 13, 2018 3:31 pm
by Support
they both work for me, how long did you leave it when it appeared stopped?

Re: I need help with one website

Posted: Mon Aug 13, 2018 3:35 pm
by braciata
After it downloaded 42 midi files from page A it found that file and it stops.

Re: I need help with one website

Posted: Mon Aug 13, 2018 3:49 pm
by Support
I guess that explains it...
2018-08-13_154812.png
2018-08-13_154812.png (43.16 KiB) Viewed 16273 times

Re: I need help with one website

Posted: Mon Aug 13, 2018 3:51 pm
by braciata
:o thats why