Webcheck is a nice tool written in Python 2.4. Given a URL to a website it crawls the whole site and generates a nice HTML report.
I use it to…
- … get the structure of a site.
- … check if a site would index well with search engines.
- … track down broken links.
- … list links pointing to external sites.
You can download webcheck from Arthur de Jong’s website.
I started it this way from the commandline:
cd /var/_test/webcheck;
./webcheck.py -f -o /var/_test/webcheck/report http://www.my-site.com/;
You can get help and find out about further options by typing:
./webcheck.py --help
RSS