Page 1 of 1

Parse Website for State Specific Datat

Posted: Fri Oct 03, 2014 7:44 pm
by neronet
I already posted this under black widow.. just not sure how to get started.

I'm brand new at this and looking for general guidance on how to setup my search.

I'm trying to parse http://www.hudhomestore.com/Home/Index.aspx and download specific information. So say I am looking within Indiana, I want to parse each property listing and find the corresponding listing broker information for each property (identified as a case #). An example would be for the property with case number #151-316799, 3625 Payton Ave
Indianapolis, IN, 46226, I want to download the corresponding listing broker information.

I'd like to eventually compile this information into an excel spread sheet for each state.

Re: Parse Website for State Specific Datat

Posted: Fri Oct 03, 2014 9:29 pm
by Support
ok, let me make a script for you. Should be ready tomorrow. Do you want me to send in a private message or here is ok?

Re: Parse Website for State Specific Datat

Posted: Fri Oct 03, 2014 10:03 pm
by neronet
Either is fine with me

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 1:20 pm
by Support
ok, can you list me all the column titles you need for this? For example...

Case#, Street, City, State, Zip etc.

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 2:14 pm
by neronet
I will need case number, state, and zip. So I am going to be downloading all the Indiana listing brokers, all the Kentucky brokers, etc.

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 2:58 pm
by Support
ok, here is the script, it output a TAB delimited text file named hudhomestore.txt in the same folder as the script and it looks like this...

151-316799 IN 46226
151-348048 IN 46403
151-432797 IN 46628
151-461729 IN 46060
151-474550 IN 46544
151-485631 IN 46544
151-496008 IN 46256
151-503058 IN 46052

Edit line 4 in the script to include all the States you need, it's set for IN and KY right now. You can import that text file in Excel, it's tab delimited.

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 3:40 pm
by neronet
Thanks!!

Is there a way to get into the property (so you would click on the case number normally) and go to the agent tab? I would like to get the Listing Broker information as well .

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 3:56 pm
by Support
Yes, that's why ask you which columns (fields) you need in the text file, so I can fill them. Just list me the fields you need, and in the order you need them as well.

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 6:35 pm
by neronet
Ok.. Case #, State, (this is all under the Agent Info Tab then Listing Broker) Company Name, Contact Name, Address, Phone Number, Fax Number, email.

One other question. The program doesn't seem to be going through each page of listings. For for example, there are 277 properties in Indiana, the script is only returning 59 results (or properties)

Re: Parse Website for State Specific Datat

Posted: Sat Oct 04, 2014 7:40 pm
by Support
I was using the "Next" link thinking it brings the next page (from 1 to 2) but it doesn't, it brings the next set of pages (from 1 to 11). Here is the new script, it loaded all 277 listings for IN.

Re: Parse Website for State Specific Datat

Posted: Sun Oct 05, 2014 10:14 am
by neronet
Thanks very much. Works like a charm.