Spider a website via the command line …

Found myself needing to do this this morning as it’d just be handier to be able to grep everything from the site … it’s wget to the rescue

wget –mirror -p –convert-links -P ./temp http://website

While there’s nothing amazing in there it’ll be something I’ll need to figure out again at some stage and it’s rather handy :)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.