Parse Website for State Specific Datat

BeownReclise is a programmable web spider. Scan a web site and retrieve from it the information you need. You could scan a Real Estate web site and collect all of the agent addresses, phone numbers and emails, and place all this data into a tab delimited database file. Then import this data in your Excel application for example.
Post Reply
neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Parse Website for State Specific Datat

Post by neronet » Fri Oct 03, 2014 7:44 pm

I already posted this under black widow.. just not sure how to get started.

I'm brand new at this and looking for general guidance on how to setup my search.

I'm trying to parse http://www.hudhomestore.com/Home/Index.aspx and download specific information. So say I am looking within Indiana, I want to parse each property listing and find the corresponding listing broker information for each property (identified as a case #). An example would be for the property with case number #151-316799, 3625 Payton Ave
Indianapolis, IN, 46226, I want to download the corresponding listing broker information.

I'd like to eventually compile this information into an excel spread sheet for each state.

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support » Fri Oct 03, 2014 9:29 pm

ok, let me make a script for you. Should be ready tomorrow. Do you want me to send in a private message or here is ok?
Your support team.
http://SoftByteLabs.com

neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet » Fri Oct 03, 2014 10:03 pm

Either is fine with me

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support » Sat Oct 04, 2014 1:20 pm

ok, can you list me all the column titles you need for this? For example...

Case#, Street, City, State, Zip etc.
Your support team.
http://SoftByteLabs.com

neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet » Sat Oct 04, 2014 2:14 pm

I will need case number, state, and zip. So I am going to be downloading all the Indiana listing brokers, all the Kentucky brokers, etc.

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support » Sat Oct 04, 2014 2:58 pm

ok, here is the script, it output a TAB delimited text file named hudhomestore.txt in the same folder as the script and it looks like this...

151-316799 IN 46226
151-348048 IN 46403
151-432797 IN 46628
151-461729 IN 46060
151-474550 IN 46544
151-485631 IN 46544
151-496008 IN 46256
151-503058 IN 46052

Edit line 4 in the script to include all the States you need, it's set for IN and KY right now. You can import that text file in Excel, it's tab delimited.
Attachments
hudhomestore.brp
(1.6 KiB) Downloaded 759 times
Your support team.
http://SoftByteLabs.com

neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet » Sat Oct 04, 2014 3:40 pm

Thanks!!

Is there a way to get into the property (so you would click on the case number normally) and go to the agent tab? I would like to get the Listing Broker information as well .

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support » Sat Oct 04, 2014 3:56 pm

Yes, that's why ask you which columns (fields) you need in the text file, so I can fill them. Just list me the fields you need, and in the order you need them as well.
Your support team.
http://SoftByteLabs.com

neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet » Sat Oct 04, 2014 6:35 pm

Ok.. Case #, State, (this is all under the Agent Info Tab then Listing Broker) Company Name, Contact Name, Address, Phone Number, Fax Number, email.

One other question. The program doesn't seem to be going through each page of listings. For for example, there are 277 properties in Indiana, the script is only returning 59 results (or properties)

User avatar
Support
Site Admin
Posts: 1720
Joined: Sun Oct 02, 2011 10:49 am

Re: Parse Website for State Specific Datat

Post by Support » Sat Oct 04, 2014 7:40 pm

I was using the "Next" link thinking it brings the next page (from 1 to 2) but it doesn't, it brings the next set of pages (from 1 to 11). Here is the new script, it loaded all 277 listings for IN.
Attachments
hudhomestore.brp
(2.15 KiB) Downloaded 646 times
Your support team.
http://SoftByteLabs.com

neronet
Posts: 8
Joined: Fri Oct 03, 2014 6:55 pm

Re: Parse Website for State Specific Datat

Post by neronet » Sun Oct 05, 2014 10:14 am

Thanks very much. Works like a charm.

Post Reply