Parse Website for State Specific Datat

BeownReclise is a programmable web spider. Scan a web site and retrieve from it the information you need. You could scan a Real Estate web site and collect all of the agent addresses, phone numbers and emails, and place all this data into a tab delimited database file. Then import this data in your Excel application for example.
Post Reply
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Parse Website for State Specific Datat

Post by neronet »

I already posted this under black widow.. just not sure how to get started.

I'm brand new at this and looking for general guidance on how to setup my search.

I'm trying to parse http://www.hudhomestore.com/Home/Index.aspx and download specific information. So say I am looking within Indiana, I want to parse each property listing and find the corresponding listing broker information for each property (identified as a case #). An example would be for the property with case number #151-316799, 3625 Payton Ave
Indianapolis, IN, 46226, I want to download the corresponding listing broker information.

I'd like to eventually compile this information into an excel spread sheet for each state.
User avatar
Support
Site Admin
Posts: 1896
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support »

ok, let me make a script for you. Should be ready tomorrow. Do you want me to send in a private message or here is ok?
Your support team.
http://SoftByteLabs.com
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet »

Either is fine with me
User avatar
Support
Site Admin
Posts: 1896
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support »

ok, can you list me all the column titles you need for this? For example...

Case#, Street, City, State, Zip etc.
Your support team.
http://SoftByteLabs.com
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet »

I will need case number, state, and zip. So I am going to be downloading all the Indiana listing brokers, all the Kentucky brokers, etc.
User avatar
Support
Site Admin
Posts: 1896
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support »

ok, here is the script, it output a TAB delimited text file named hudhomestore.txt in the same folder as the script and it looks like this...

151-316799 IN 46226
151-348048 IN 46403
151-432797 IN 46628
151-461729 IN 46060
151-474550 IN 46544
151-485631 IN 46544
151-496008 IN 46256
151-503058 IN 46052

Edit line 4 in the script to include all the States you need, it's set for IN and KY right now. You can import that text file in Excel, it's tab delimited.
Attachments
hudhomestore.brp
(1.6 KiB) Downloaded 1087 times
Your support team.
http://SoftByteLabs.com
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet »

Thanks!!

Is there a way to get into the property (so you would click on the case number normally) and go to the agent tab? I would like to get the Listing Broker information as well .
User avatar
Support
Site Admin
Posts: 1896
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support »

Yes, that's why ask you which columns (fields) you need in the text file, so I can fill them. Just list me the fields you need, and in the order you need them as well.
Your support team.
http://SoftByteLabs.com
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet »

Ok.. Case #, State, (this is all under the Agent Info Tab then Listing Broker) Company Name, Contact Name, Address, Phone Number, Fax Number, email.

One other question. The program doesn't seem to be going through each page of listings. For for example, there are 277 properties in Indiana, the script is only returning 59 results (or properties)
User avatar
Support
Site Admin
Posts: 1896
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support »

I was using the "Next" link thinking it brings the next page (from 1 to 2) but it doesn't, it brings the next set of pages (from 1 to 11). Here is the new script, it loaded all 277 listings for IN.
Attachments
hudhomestore.brp
(2.15 KiB) Downloaded 1004 times
Your support team.
http://SoftByteLabs.com
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet »

Thanks very much. Works like a charm.
Post Reply