Scanning a PHP site

BlackWidow scans websites (it's a site ripper). It can download an entire website, or download portions of a site.
Post Reply
Martyn
Posts: 2
Joined: Mon Oct 10, 2011 6:04 am

Scanning a PHP site

Post by Martyn »

Hi there,

First time using BW for a long time, it looks like it's had quite a few changes!

Anyway, I'm trying to build a structure of one of our PHP sites that uses a CMS and I'm struggling.

If I start a scan I get a link error on the root URL (Code: "0", Description: "ReadFile:") and the structure shows a single blank (0KB) "index.html" file in the root directory, and that's all.

I supsect it's something to do with it looking for files ending in .html, or ending in any extension? The pages on this site are simply: www.domain.com/page-title

I've had a look in filters and removed the "default index page name" field which was "index.html" but I don't know what to replace it with, if anything.

Assistance would be appreciated!

Cheers,

Martyn

User avatar
Support
Site Admin
Posts: 1879
Joined: Sun Oct 02, 2011 10:49 am

Re: Scanning a PHP site

Post by Support »

Hi,

You can leave that field blank. Here are the settings you should use (no filters). Does your site redirect links? Say, from www.domain.com to domain.com?
Attachments
bwDef.png
bwDef.png (105.47 KiB) Viewed 9708 times
Your support team.
http://SoftByteLabs.com

Martyn
Posts: 2
Joined: Mon Oct 10, 2011 6:04 am

Re: Scanning a PHP site

Post by Martyn »

Hi,

No, there's no redirection in place.

Tried it with those settings and it's still not working, same error code, same result in the structure.

The site is on a private network, only accessible via a VPN, but the BW browser screen can load it, so I'd assume the spider can too. Could that cause a problem?

Cheers,

Martyn

User avatar
Support
Site Admin
Posts: 1879
Joined: Sun Oct 02, 2011 10:49 am

Re: Scanning a PHP site

Post by Support »

It should not be a problem on a VPN, but it all depend which server is used. Does it run Linux? Is it up to date and configured correctly?
Your support team.
http://SoftByteLabs.com

BWuser
Posts: 3
Joined: Sat Oct 29, 2011 8:36 pm

Re: Scanning a PHP site

Post by BWuser »

Mine is doing the same thing. Used the setting in the pic. Does not seem to be getting past the main page.

User avatar
Support
Site Admin
Posts: 1879
Joined: Sun Oct 02, 2011 10:49 am

Re: Scanning a PHP site

Post by Support »

Maybe the links on the site are not on the same doamin. For example, if you scan a site by using its IP and the site has links pointing the domain name instead, it will not work.

I need something to know what's going on. Can you email me the main page (remove any data from the html)? And how is the address? Is it an IP from the VPN?
Your support team.
http://SoftByteLabs.com

Post Reply