Varnish: the secret weapon for boosting page speed

Before We Begin

Request processing (especially dynamic) with web servers such as Apache can be a heavy strain on system resources.

That’s why more and more people started using lightweight web servers like nginx or lighttpd; they can work as a frontend for static content paired with Apache or independently. But there is another way to improve your site: using a web server accelerator such as Varnish Cache – a solution from Varnish Software.

There are a few ways to learn all the specifics of Varnish Cache processing: reading the Varnish Book, the Official documentation and a lot of practice and experimentation. We highly recommend starting with the first two. This article is based on all three, materials found at stackoverflow and other professional sites, and personal experience.


What is Varnish?

VARNISH is an open source caching reverse HTTP proxy, sometimes referred to as a web application accelerator. A reverse proxy is a proxy server that appears to clients as an ordinary server. It stores (caches) files or fragments of files in memory that are used to reduce the response time and network bandwidth consumption.

Varnish is available under a two-clause BSD licence. Commercial support is available from Varnish Software. Version 1.0 of Varnish was released in 2006, Varnish 2.0 in 2008, Varnish 3.0 in 2011 and Varnish 4.0 in 2014.

Varnish is flexible because you can configure it by writing your own caching policies in its Varnish Configuration Language (VCL). VCL is a domain specific language based on C. VCL is then translated to C code and compiled.


How does Varnish work?

Let’s imagine that the client sends a request to your server. Varnish receives it and tries to find an appropriate response in the cache storage. If it finds one and its content is still valid, then we have a response we can deliver to the client. Otherwise Varnish will forward the request to the backend, fetch the response, store it in the cache and send the response to the client. Next time the same request is made, Varnish will have the response in cache storage.

It’s easy to represent this with a simple diagram:

varnish cache flowchart


Installation and basic configuration of Varnish


You can easily find Varnish binary packages or compile it for yourself from source code. Use the official site for grabbing a current version. Installation can be done with the system package manager, but please note that this might not be the latest version of Varnish. For example under Debian 8 (“jessie”) do the following as root:


apt-get install apt-transport-https
curl | apt-key add -
echo "deb jessie varnish-4.1" >> /etc/apt/sources.list.d/varnish-cache.list
apt-get update
apt-get install varnish


If compiling from source, make sure that you already have dependencies installed. Please check the official documentation for a list.

The next step is configuration. First, let’s look at the configuration file
/etc/default/varnish (Debian)

/etc/sysconfig/varnish (Red Hat)


This holds a lot of useful and important settings for the Varnish daemon, such as locked shared memory, default TTL, numbers of threads, etc.

Here is a simple example:


             -f ${VARNISH_VCL_CONF} \
             -t ${VARNISH_TTL} \
             -p esi_syntax=0x2 \
             -p cli_buffer=16384 \
             -p http_resp_hdr_len=131072 \
             -p thread_pools=4 \
             -p thread_pool_add_delay=2 \
             -p thread_pool_min=15 \
             -p thread_pool_max=50 \
             -p thread_pool_timeout=3600 \
             -u varnish -g varnish \
             -S ${VARNISH_SECRET_FILE} \
             -s ${VARNISH_STORAGE}"


In VARNISH_VCL_CONF we define a path to a second file, which describes request handling and the caching process in the Varnish Configuration Language (VCL). VCL is translated to C and compiled into binary code which is then executed when requests arrive. This feature makes Varnish a powerful HTTP processor not only for caching, since VCL has inherited a lot from C and it reads much like a simple C or Perl script. We will look on it closer in next chapter.


A look inside

Let’s look inside request processing in Varnish v.4!

To start, here’s a simplified diagram of the Varnish Finite State Machine and a brief description of each of its processes:


varnish processes



The start of all processes, called just after the request has been received and parsed. It determines the next steps for incoming requests – will they be modified, cached or not, should anything be forwarded for processing by the backend or simply return a response from cache, etc.


Called after vcl_recv to create a hash value for the request that will be used as a lookup key for the object in Varnish. Passes control to the functions, depended on vcl_recv result.


Called when a cache lookup is successful. Its result determines how to handle the rest of the request.


Called after a cache lookup if the requested document was not found in cache or if vcl_hit returned fetch.


Called upon entering pass mode. In this mode, the request is passed on to the backend, and the backend’s response is passed on to the client, but is not entered into the cache.


Called upon entering pipe mode. In this mode, the request is passed on to the backend, and any further data from both the client and backend is passed on unaltered until either end closes the connection.


Backend servers

In the descriptions above, we referred to the backend several times. In Varnish terminology “backend” refers to any server(s) that provide cacheable content for Varnish, the one(s) that Varnish accelerates.

To tell Varnish where it can find a backend, define it in the VCL file like this:


backend default {
        .host = "";
        .port = "8080";



In real life we could encounter server crashes or other problems in the backend. To ensure continuity of work we will define a set of parameters called “probe” for checking the backend’s state.


backend default{
        .host = "";
        .port = "8000";
        .probe = {
                .url = "/alive/";
                .timeout = 2s;
                .interval = 3s;
                .window = 10;
                .threshold = 9;



Now Varnish will send a simple GET request to “/alive/” url every 3 seconds, with a timeout limit of 2 seconds. Since we set “window” to 10 and “threshold” to 9, the backend will be considered healthy if at least 9 of the last 10 requests succeeded. Varnish will not send traffic to hosts that are marked as unhealthy.

Great, now when Varnish needs content it will know where to get it by using a connection to a default backend. But what if you have multiple backend servers or maybe you have some special policy for some requests? We could define a new backend called ‘’magic’’ and add a new rule for processing requests:



backend magic {
        .host = "";
        .port = "8000";
sub vcl_recv {
        if (req.url ~ "^/magic/") {
                set req.backend_hint = magic;
        } else {
                set req.backend_hint = default;



You are free to define your own method of specifying a backend for a particular request, based on rules, parameters or randomly. You can also group several backend servers together into a “director”. This requires you to load a VMOD, a Varnish module, and then call certain actions in vcl_init.



import directors;    # load the directors
backend default {
        .host = "";
        .port = "8080";
backend magic {
        .host = "";
        .port = "8000";
sub vcl_init {
        new foobar = directors. round_robin();
sub vcl_recv {
        set req.backend_hint = foobar.backend();


This is a good way to use Varnish for load balancing. Here we are using a round-robin director, but there is also a random director which distributes requests randomly.


Let’s talk about Edge Side Includes (ESI)

Sometimes we need to display unique information to each user, like a user’s name, contents of their cart or a list of past orders etc. Caching such data can be a huge problem, that’s why one of the best things implemented in Varnish are ESI blocks.

Using ESI we can break our page up into smaller pieces and decide which of them should be cached or not, then assembling them into a response. These fragments can have individual cache policies, for example if you have a list of the 5 most popular products in your online store, this list can probably be cached for a time and included in other pages, while other fragments might never be cached and must always be loaded dynamically.


varnish esi diagram


With this strategy you can increase your hit rate and reduce the load on your servers.

ESI only implements a small subset of the full Varnish functionality, but it is enough. You can use three ESI statements:

  1. esi:include
  2. esi:remove
  3. <!–esi …–>

Varnish will not process ESI instructions in HTML comments.

Let’s look at some examples.



We will make a simple example. This is user.php file:

// Including some libraries
$user = $this->getUser();
echo $user->getName();



Now some HTML file that has an ESI include statement. This is test.html:

        Hello <esi:include src="user.php"/>!


And the last step before ESI will work is activation ESI processing in VCL.


sub vcl_backend_response {
        if (bereq.url == "/test.html") {
               set beresp.do_esi = true;
               set beresp.ttl = 24 h;   
        } elseif (bereq.url == "/user.php") {
               set beresp.ttl = 1m;




ESI:remove and <!–esi … –>


But what should we do if ESI is not available? For that there are the <esi:remove> and <!–esi … –> constructs. You can use it to present content whether or not ESI is available, for example you can include content when ESI is available or link to it when it is not.

The ESI processors will remove the starting “<!–esi” and the ending “-→” tags when the page is processed, while still processing the contents. If the page is not processed, it will remain intact, becoming a HTML/XML comment tag. ESI processors will remove <esi:remove> tags and all content contained in them, allowing you to only render the content when the page is not being ESI-processed. For example:

  <a href="">The license</a>
<p>The full text of the license:</p>
<esi:include src="" />


Who uses Varnish Cache

Based on information from Wikipedia, Varnish is used in around a tenth of the TOP 10K sites – online newspapers, social media and media content sites; here’s short a list of the most famous: Wikipedia, Facebook, Twitter, Vimeo, The New York Times, The Guardian, The Hindu, and Tumblr.


Use Varnish in Magento


Magento is the top ecommerce platform in the world. It’s known for being flexible, but not so much for being fast. If you want to use Varnish with Magento 1, you should find the ready solution for it or if you have a lot of time: Do-It-Yourself. You just need to learn how to use Magento, understand HTTP Headers and get a hold of reverse proxy caching and Varnish Cache specifically.

"There are only two hard things in Computer Science: cache invalidation and naming things."Phil Karlton

In case of Magento 2 – everything is not so bad. Magento 2 officially supports Varnish versions 3.0.5 or later or any Varnish 4.x version. Just follow the tutorial in the documentation, but we’ll summarize it for the sake of completeness:

  • Install Varnish and test it by accessing any Magento page to see if you’re getting an HTTP response header that indicates Varnish is working.
  • Install the Magento software and use the Magento Admin to create a Varnish configuration file.
  • Replace your existing Varnish configuration file with the one generated by the Admin.
  • Test everything again and if there is nothing in your /var/page_cache directory, you’ve successfully configured Varnish with Magento!

Finally, remember that your (or your customers’) ecommerce site is the last possible place on the web where you should be testing random VCL snippets and unknown PHP code.



111 replies
  1. Esperanza Iacuzio says:

    Aw, this was a really nice post. In idea I wish to put in writing like this moreover – taking time and actual effort to make a very good article… but what can I say… I procrastinate alot and certainly not seem to get something done.

  2. Methadone Clinics In My Area says:

    Alcohol Rehabilitation Center Drug Rehab Near Me Luxury Drug Rehab Centers

  3. bitcoin crash chart says:

    Whats up are using WordPress for your site platform? I’m new to the blog world but I’m trying to get started and set up my own. Do you need any coding expertise to make your own blog? Any help would be greatly appreciated!

  4. Mechanical stock trading says:

    With the whole thing that seems to be developing inside this specific subject matter, many of your points of view are rather exciting. Nonetheless, I beg your pardon, because I can not subscribe to your whole theory, all be it refreshing none the less. It seems to everybody that your commentary are not totally validated and in fact you are yourself not totally certain of your point. In any event I did appreciate examining it.

  5. low cost health insurance says:

    Excellent read, I just passed this onto a friend who was doing a little research on that. And he just bought me lunch since I found it for him smile Thus let me rephrase that: Thank you for lunch! “Feeling passionate about something is like getting a peak at your soul smiling back at you.” by Amanda Medinger.

  6. multicultural children's books says:

    I’m extremely impressed with your writing skills as well as with the layout on your weblog. Is this a paid theme or did you customize it yourself? Anyway keep up the excellent quality writing, it is rare to see a great blog like this one these days..

  7. bludv says:

    I want to show my love for your kindness for men who really want help on the area of interest. Your real dedication to passing the message all over ended up being extraordinarily productive and has regularly made women much like me to arrive at their endeavors. Your informative help signifies so much a person like me and especially to my office workers. Thanks a ton; from each one of us.

  8. erjilo pterin says:

    I’m still learning from you, but I’m making my way to the top as well. I definitely love reading all that is written on your blog.Keep the stories coming. I loved it!

  9. Login Joker says:

    I have not checked in here for some time as I thought it was getting boring, but the last few posts are great quality so I guess I will add you back to my everyday bloglist. You deserve it my friend :)

  10. Slot Game says:

    Excellent weblog right here! Also your website rather a lot up fast! What host are you the use of? Can I am getting your associate hyperlink to your host? I wish my site loaded up as fast as yours lol

  11. GLOCK G-19 GEN4 says:

    It’s the best time to make a few plans for the longer term and it is time to be happy. I have read this put up and if I may I desire to suggest you few interesting things or advice. Maybe you can write subsequent articles regarding this article. I wish to read more things approximately it!

  12. hospedagem de sites says:

    Hello there! This is kind of off topic but I need some help from an established blog. Is it hard to set up your own blog? I’m not very techincal but I can figure things out pretty quick. I’m thinking about making my own but I’m not sure where to begin. Do you have any points or suggestions? Many thanks

  13. hospedagem de site says:

    I keep listening to the newscast lecture about getting free online grant applications so I have been looking around for the best site to get one. Could you advise me please, where could i find some?

  14. casas de madeira says:

    Hello there, simply changed into alert to your blog through Google, and found that it is really informative. I am gonna be careful for brussels. I will be grateful if you happen to proceed this in future. Numerous other people can be benefited out of your writing. Cheers!

  15. Instagram TV Views says:

    Hello there, I found your website by way of Google at the same time as searching for a comparable subject, your site came up, it appears to be like good. I have bookmarked it in my google bookmarks.

  16. Instagram id says:

    obviously like your web site but you have to check the spelling on quite a few of your posts. Many of them are rife with spelling issues and I in finding it very bothersome to inform the reality on the other hand I will certainly come back again.

  17. dinosaur 500 teeth says:

    I have been absent for a while, but now I remember why I used to love this website. Thank you, I¦ll try and check back more frequently. How frequently you update your website?

  18. medio padrao says:

    I have been exploring for a little for any high quality articles or blog posts on this kind of area . Exploring in Yahoo I at last stumbled upon this web site. Reading this info So i am happy to convey that I have an incredibly good uncanny feeling I discovered exactly what I needed. I most certainly will make sure to don’t forget this web site and give it a look regularly.

  19. dinosaur with 500 teeth says:

    Hi there! This post couldn’t be written any better! Reading through this post reminds me of my previous room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thank you for sharing!

  20. passeio laguna humantay says:

    I conceive this site has some rattling superb info for everyone :D. “Time–our youth–it never really goes, does it It is all held in our minds.” by Helen Hoover Santmyer.

  21. College says:

    My brother suggested I may like this web site. He used to be entirely right. This publish actually made my day. You can not believe simply how much time I had spent for this information! Thanks!

  22. marketing 1on1 agency says:

    Thanks for sharing excellent informations. Your site is so cool. I’m impressed by the details that you have on this site. It reveals how nicely you understand this subject. Bookmarked this website page, will come back for extra articles. You, my friend, ROCK! I found simply the info I already searched everywhere and simply could not come across. What an ideal site.

  23. cheap wordpress hosting says:

    Wonderful beat ! I wish to apprentice while you amend your website, how can i subscribe for a blog site? The account helped me a acceptable deal. I have been a little bit acquainted of this your broadcast offered vivid clear concept

  24. Canada says:

    you are really a good webmaster. The web site loading speed is amazing. It seems that you are doing any unique trick. Furthermore, The contents are masterwork. you’ve done a fantastic job on this topic!

  25. Joker388.Net says:

    The very crux of your writing while sounding reasonable originally, did not really settle very well with me after some time. Someplace throughout the paragraphs you managed to make me a believer unfortunately only for a very short while. I nevertheless have got a problem with your jumps in logic and one might do nicely to fill in those gaps. If you can accomplish that, I could definitely end up being impressed.

  26. Joker388.Net says:

    Heya i’m for the primary time here. I found this board and I in finding It truly helpful & it helped me out much. I’m hoping to offer one thing again and aid others like you aided me.

  27. Daftar Joker388 says:

    Hi, just required you to know I he added your site to my Google bookmarks due to your layout. But seriously, I believe your internet site has 1 in the freshest theme I??ve came across. It extremely helps make reading your blog significantly easier.

  28. Slot Joker388 says:

    Very efficiently written article. It will be helpful to anybody who utilizes it, including myself. Keep doing what you are doing – looking forward to more posts.

  29. Download Joker388 says:

    It is appropriate time to make some plans for the longer term and it’s time to be happy. I have read this post and if I could I wish to recommend you few fascinating things or suggestions. Perhaps you can write subsequent articles relating to this article. I want to learn even more things approximately it!

  30. Joker388 Gamming says:

    It¦s actually a great and helpful piece of information. I¦m glad that you just shared this helpful info with us. Please keep us informed like this. Thank you for sharing.

  31. aduslot says:

    Hey, you used to write magnificent, but the last few posts have been kinda boring… I miss your tremendous writings. Past several posts are just a little bit out of track! come on!

  32. hospedagem de site barata says:

    Hello, I think your website might be having browser compatibility issues. When I look at your blog in Opera, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, very good blog!

  33. Camp Half Blood says:

    Wow, amazing blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your web site is great, let alone the content!

  34. kevin says:

    I like what you guys are up also. Such smart work and reporting! Keep up the excellent works guys I have incorporated you guys to my blogroll. I think it’ll improve the value of my website :)

  35. kevin david contact says:

    F*ckin’ remarkable things here. I’m very glad to see your article. Thanks a lot and i am looking forward to contact you. Will you kindly drop me a e-mail?

  36. click here says:

    I’m extremely impressed with your writing skills as well as with the layout on your weblog. Is this a paid theme or did you customize it yourself? Anyway keep up the nice quality writing, it is rare to see a nice blog like this one these days..

  37. aumentar o pênis naturalmente says:

    Very good site you have here but I was curious about if you knew of any user discussion forums that cover the same topics discussed in this article? I’d really like to be a part of online community where I can get opinions from other experienced people that share the same interest. If you have any recommendations, please let me know. Appreciate it!

  38. FafaSlot Gamming says:

    fantastic points altogether, you simply received a brand new reader. What might you recommend about your submit that you made some days in the past? Any certain?

  39. Daftar VivoSlot says:

    Excellent weblog here! Also your website loads up fast! What host are you the use of? Can I am getting your associate link in your host? I desire my web site loaded up as quickly as yours lol

  40. Joker123 Gamming says:

    You have remarked very interesting details! ps decent internet site. “Enemies, as well as lovers, come to resemble each other over a period of time.” by Sydney Harris.

  41. Slot Joker123 says:

    There are some interesting cut-off dates in this article but I don’t know if I see all of them heart to heart. There’s some validity however I’ll take hold opinion until I look into it further. Good article , thanks and we would like extra! Added to FeedBurner as nicely

  42. Login VivoSlot says:

    F*ckin’ remarkable things here. I am very glad to see your post. Thanks a lot and i’m looking forward to contact you. Will you please drop me a e-mail?

  43. Leah Satchell says:

    I simply want to say I’m new to blogs and absolutely enjoyed this blog site. Likely I’m going to bookmark your blog post . You certainly come with great stories. Kudos for sharing with us your blog site.

  44. Daftar Joker123 says:

    I liked as much as you will obtain performed proper here. The sketch is attractive, your authored subject matter stylish. nevertheless, you command get got an shakiness over that you want be delivering the following. ill indisputably come further previously again as exactly the same nearly very often inside case you defend this increase.

  45. VivoSlot Gamming says:

    Good day! I know this is kinda off topic however I’d figured I’d ask. Would you be interested in exchanging links or maybe guest writing a blog article or vice-versa? My site covers a lot of the same topics as yours and I feel we could greatly benefit from each other. If you happen to be interested feel free to send me an email. I look forward to hearing from you! Awesome blog by the way!

  46. Joker388 says:

    you are in point of fact a excellent webmaster. The site loading speed is incredible. It sort of feels that you are doing any unique trick. In addition, The contents are masterpiece. you’ve done a magnificent job in this matter!

  47. Daftar Joker388 says:

    Just want to say your article is as amazing. The clearness in your post is just cool and i can assume you’re an expert on this subject. Fine with your permission let me to grab your feed to keep updated with forthcoming post. Thanks a million and please keep up the enjoyable work.

  48. Joker123 says:

    Thank you for sharing superb informations. Your website is very cool. I’m impressed by the details that you have on this website. It reveals how nicely you perceive this subject. Bookmarked this website page, will come back for extra articles. You, my pal, ROCK! I found simply the information I already searched everywhere and just could not come across. What an ideal web-site.

  49. Joker388 Online says:

    Excellent web site. Lots of useful info here. I am sending it to several pals ans also sharing in delicious. And naturally, thank you in your effort!

  50. Login FafaSlot says:

    Youre so cool! I dont suppose Ive learn anything like this before. So good to find any person with some unique ideas on this subject. realy thanks for beginning this up. this web site is something that is wanted on the web, someone with slightly originality. helpful job for bringing one thing new to the internet!

  51. สล็อต says:

    This is very interesting, You are a very skilled blogger. I have joined your rss feed and look forward to seeking more of your magnificent post. Also, I’ve shared your web site in my social networks!

  52. conheça a kaidi says:

    you’re really a good webmaster. The website loading speed is incredible. It seems that you are doing any unique trick. In addition, The contents are masterpiece. you’ve done a magnificent job on this topic!

  53. says:

    An impressive share, I just given this onto a colleague who was doing a little analysis on this. And he in fact bought me breakfast because I found it for him.. smile. So let me reword that: Thnx for the treat! But yeah Thnkx for spending the time to discuss this, I feel strongly about it and love reading more on this topic. If possible, as you become expertise, would you mind updating your blog with more details? It is highly helpful for me. Big thumb up for this blog post!

  54. pop over to these guys says:

    Does your blog have a contact page? I’m having trouble locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing. Either way, great blog and I look forward to seeing it expand over time.

  55. Game podcasts says:

    The next time I read a blog, I hope that it doesnt disappoint me as much as this one. I mean, I know it was my choice to read, but I actually thought youd have something interesting to say. All I hear is a bunch of whining about something that you could fix if you werent too busy looking for attention.

  56. criminal lawyer description says:

    I got what you mean , appreciate it for posting.Woh I am delighted to find this website through google. “If one does not know to which port one is sailing, no wind is favorable.” by Seneca.

  57. website says:

    Does your blog have a contact page? I’m having problems locating it but, I’d like to send you an e-mail. I’ve got some suggestions for your blog you might be interested in hearing. Either way, great site and I look forward to seeing it expand over time.

  58. ssl with green bar says:

    Hello there, simply was aware of your blog thru Google, and located that it’s truly informative. I am going to be careful for brussels. I’ll be grateful if you continue this in future. Lots of folks might be benefited from your writing. Cheers!

Trackbacks & Pingbacks

  1. generic ventolin inhalers for sale

    Varnish: the secret weapon for boosting page speed –

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published.