Yellow page script to pull Name, Street, City, State, Zip...

BeownReclise is a programmable web spider. Scan a web site and retrieve from it the information you need. You could scan a Real Estate web site and collect all of the agent addresses, phone numbers and emails, and place all this data into a tab delimited database file. Then import this data in your Excel application for example.
Post Reply
User avatar
Support
Site Admin
Posts: 1903
Joined: Sun Oct 02, 2011 10:49 am

Yellow page script to pull Name, Street, City, State, Zip...

Post by Support »

Here is a YellowPages.com (http://www.yellowpages.com/) script that will pull the Business name, street address, city, state, zip, phone and website URL from a search criteria.

Code: Select all

// Edit this section as you see fit. 

State    = 'tx'; 
Business = 'used-Computers'; 

OutputFile = ScriptPath + Business + '.txt'; 

//--------------------------------------------------------------------------------------------------------------------
 
PerlRegEx = Yes; 
Output.Clear; 

Link = New(URL); 
rx   = New(RegEx); 
sx   = New(RegEx); 

OutFile = New(File); 
OutFile.Open(OutputFile); 
OutFile.Truncate; 
OutFile.Write('BusinessName'+tab+'Street'+tab+'City'+tab+'State'+tab+'ZipCode'+tab+'Phone'+tab+'Website'+crlf);
 
Abort = No; 

CurPage  = 1; 
LastPage = 0; 

while not Abort do begin 

  lnk = 'http://www.yellowpages.com/'+State+'/'+Business+'?page='+CurPage+'&sort=alpha';
   Link.Get(lnk); 

  if LastPage = 0 then begin 
    LastPage = Val(WildGet(Link.Data, '<a href="[^"]+\?page=(\d+)[^"]*">Last</a>'));
     Progress.Maximum = LastPage; 
  end; 

  Progress.Position = CurPage; 

  CurPage = Decode(WildGet(Link.Data, '<a href="[^"]+\?page=(\d+)[^"]+">Next</a>'));
 
  sx.Data = Link.Data; 
  sx.Mask = '<div class="info">(.*?)</ul>'; 

  while (sx.Match) and (not Abort) do begin 

    rx.Data = sx.Value[1]; 

    BusName = Trim(Decode(WildGet(rx.Data, '<a\s+[^>]+>(.*?)</a>'))); 
    Street  = Trim(Decode(WildGet(rx.Data, 'class="street-address">(.*?)'))); 
    City = Trim(Decode(WildGet(rx.Data, 'class="locality">(.*?)'))); 
    st = Trim(Decode(WildGet(rx.Data, 'class="region">(.*?)'))); 
    ZipCode = Trim(Decode(WildGet(rx.Data, 'class="postal-code">(.*?)'))); 
    Phone = Trim(Decode(WildGet(rx.Data, 'phone">(.*?)'))); 
    Website = Decode(WildGet(rx.Data, '<li><a href="([^"]+)" class="track-visit-website'));
 
    DataLine = 
      BusName +tab+ 
      Street  +tab+ 
      City    +tab+ 
      st      +tab+ 
      ZipCode +tab+ 
      Phone   +tab+ 
      Website 
    ; 
    Output(DataLine); 

    OutFile.Write(DataLine+crlf); 

  end; 

  if CurPage = Nothing then Break; 

end; 

function OnStop(); 
begin 
  Result = @Abort; 
  @Abort = Yes; 
end; 

function OnTerminate(); 
begin 
  if @OutFile then @OutFile.Close; 
  Display('Data saved to...'+crlf+@OutputFile); 
end; 
Your support team.
http://SoftByteLabs.com
User avatar
Alienizer
Posts: 36
Joined: Sun Oct 02, 2011 11:41 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Alienizer »

Thanks Support for the script. I didn't know BrownRecluse could do that! I only found out today while looking at your new forum :shock:

Can we use BrownRecluse to harvest emails as well?
User avatar
Support
Site Admin
Posts: 1903
Joined: Sun Oct 02, 2011 10:49 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Support »

That's one reason why we upgraded it. The old one was too old.

Yes, you can harvest emails using BrownRecluse, even those with the at word to stop spiders, but it will not stop BrownRecluse. In fact, you can harvest anything you want from any site.
Your support team.
http://SoftByteLabs.com
User avatar
Alienizer
Posts: 36
Joined: Sun Oct 02, 2011 11:41 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Alienizer »

Cool :D

How hard is it to script this thing? I can program a little bit in javascript, but I'm good at PHP.
User avatar
Support
Site Admin
Posts: 1903
Joined: Sun Oct 02, 2011 10:49 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Support »

It's very easy if you know PHP. It's like Basic, easier. Have you look at our free scripts?...

http://softbytelabs.com/Support/scripts ... on=default

The best way to learn scripting in BrownRecluse is to look at other scripts. But of course you can ask questions here on the forum.
Your support team.
http://SoftByteLabs.com
User avatar
Alienizer
Posts: 36
Joined: Sun Oct 02, 2011 11:41 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Alienizer »

Yep, I was just looking at them. Didn't pay attention to them for the longest time!

I'm gonna go play with this now, and if I have any questions, I'll let you know.

Thanks for the quick support!
User avatar
Support
Site Admin
Posts: 1903
Joined: Sun Oct 02, 2011 10:49 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Support »

You are welcome.
Your support team.
http://SoftByteLabs.com
steve1040
Posts: 5
Joined: Wed Jul 25, 2012 2:31 pm

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by steve1040 »

How can I modify this script to work on
http://www.yellowpages.com.au

Going from page to page seems to work different.

Example
here is page 10 of a search

http://www.yellowpages.com.au/search/li ... redCount=0
User avatar
Support
Site Admin
Posts: 1903
Joined: Sun Oct 02, 2011 10:49 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Support »

It wasn't designed to work on the one. A new script would have to be made specifically for that one alone.
Your support team.
http://SoftByteLabs.com
BRuser
Posts: 1
Joined: Mon Mar 02, 2015 5:24 pm

Re: Yellow page script update

Post by BRuser »

The YP script doesn't seem to be working, are their any plans to update the script? Also the scripts pages on the website don't seem to be working.

Thanks
User avatar
Support
Site Admin
Posts: 1903
Joined: Sun Oct 02, 2011 10:49 am

Re: Yellow page script to pull Name, Street, City, State, Zip...

Post by Support »

The scripts we provide are as-is and wee functional at the time they were made, after many years, they may no longer work and we hope our users update them as it is too much work for us to handle. That's why we removed the page of scripts, most were never updated.
Your support team.
http://SoftByteLabs.com
Post Reply