Toolkit

Essential tools: Wget, a command line tool to retrieve web pages

There is nothing more annoying than having a web server or site down and IE (or FireFox, for that matter) become dog slow or simply getting in the way of trouble shooting the page. Additionally, sometimes these browsers actually get in the way of troubleshooting the problem by masking the error page the server sends back– IE’s “friendly” HTTP errors messages, for example. When it comes right down to fixing the problem, sometimes you need to retrieve just the HTML code a particular web page sends simply for inspection or analysis. That is where our next essential tool comes in!

Wget is a small (~325K) command line utility that allows you to download a HTTP, HTTPS, or FTP file quickly from the command line and save it locally so you can open it with a text editor, simply have it in an alternate location, or use in a comparison to what a specific browser renders after download. Wget for windows can be downloaded here. It’s a powerful tool, and covering all the options in one posting isn’t possible, so let’s start off with a little syntax to get you rolling:

In its  simplest form you can download a specific page, including a full URL, as shown below:

wget http://www.gamersigs.net

Alternatively, you can download a site and all its linked items recursively to a specific number of levels. This is useful to archive a site or  to simply grab pages that the HTML uses, but doesn’t link to directly– Cascading Style Sheets (css) for example. The syntax below will recursively get 2 levels of www.msn.com and automatically create a directory called www.msn.com in the current directory.

wget -r –level=2 http://www.msn.com

If the page links to a HTTPS page, wget will automatically try to negotiate a SSL connection. You can optionally specify the SSL protocol to use by adding –secure-protocol=PR, where PR is either auto, SSLv2, SSLv3, or TLSv1. This is especially helpful in testing and ensuring your servers do not respond to the weaker SSLv2 SSL protocol.

If you deal with websites as a part of your Systems Administration duties– or if you’re just interested in it as a side project at the office, I’m sure you’ll add this tool to your essential tools.

Know of a tool that you think is essential? Post a comment here and if I don’t already have it in my tool belt, I’ll add it and give it a shot. If it makes the grade– I’ll add it to the list of tools to review. The only criteria are:

1. The tool must be free, or inexpensive with a “Per User” or “site” type license. (No pay per installation licenses, please)
2. The tool (or it’s installation file) must be small enough to fit on a 256Mb flash drive for portability.
3. Command line run time options are beneficial, but not required.
4. If it has ads… it needs be truly INVALUABLE.
5. It should make the user’s job easier by gathering information or preforming a task that a typical Network or Systems Administrator would preform.

Enjoy!

Essential tools: Tools every systems administrator needs to know about and use

If you read this blog regularly, you know that much of what I write about is VBScript. But that’s not completely the goal of this blog. The main goal is to make your life as a systems admin easier! (Scripting just happens to be a big way to accomplish this.) So, on occasion I do write about other topics.

One topic I plan on starting is one for useful tools I find to help make your job easier. (Please don’t applaud, just keep coming back and reading the blog.) To help identify these postings, I will place “Essential Tools:” at the front of the title and tag them “essential tools“.

This will be a virtual corner of the blog that you will be able to find free tools and applications that will either make your life easier or provide those sysadmins with less resources (read budget) than a fortune 500 company gives to the systems administration team to accomplish things.

Stay tuned, the first tool is on the way!