Varnish, the Open Source Web Accelerator
Varnish is an open source web accelerator created by Varnish Software (Norway) first released in 2006. Vanish accelerates the website by sitting in front of the web server, caching the content. Unlike Squid (a caching proxy server that with could be turned into a fairly functional http accelerator), Varnish is designed from the ground up to be an http accelerator, a caching reverse proxy.
Varnish keeps the most recently requested pages in memory and serves them from memory, producing high performance responses with reduced latency. Instead of logging directly to log files Varnish logs everything to a segment in shared memory and that means it doesn’t get slowed down waiting for disks. Logging to disk can then be done by a separate program.
Varnish can be used for application not designed to work with a cache and in such cases it behaves very conservatively. For example, Varnish will not cache any page when the request that started it included a cookie.
Varnish commands can be executed using telnet or the “varnishdm” tool and the critical information can be passed as command line arguments.
Varnish has an extremely flexible configuration language called Varnish Configuration language (VCL) that is translated to C, compiled, loaded and executed. Varnish can be used for purposes other than just caching. The policies written in VCL are simple and flexible letting you control how specific pages or requests should be handled. For example, Varnish can be used to avoid hotlinking to your web pages.
When an application is designed with using Varnish in mind, it can be used much more effectively. VCL is very powerful and allows for evaluation and modification of nearly every aspect of an HTTP request and response. It allows you to examine headers, do regular expression comparisons and substitutions.
VCL allows you to embed “C” into the configuration, and have been used by user to log to syslog, to locate server content based on the IP’s origin, to manipulate HTTP headers to a large extent.
A Varnish server can be used as both a cache and a load balancer, serving traffic to multiple backend servers. VCL allows you to use complex logic to decide how the traffic is routed, do basis web-server health checking etc.
Companies such as creativecommons.org, Twitter and LinkedIn use Varnish in production. Varnish Software offers commercial support and an enterprise offering aimed at enterprise customers.