If you’re configuring cookbook s3 URL TTL in your `chef-server.rb` configuration file (`opscode_erchef[‘s3_url_ttl’]`), then you’ve been creating cookbook URLs that expire that many (28800 by default) seconds from “now” (i.e. the time of the request), which is great for Chef Client runs, but it’s terrible for caching!
Each signed URL has a query string with an expiration time, which means that every time a signed URL is generated, it’s unique. If you want to cache URLs, we need URLs that don’t change as frequently.
Today in Chef Server 12.0.4, we’re introducing a new setting: `opscode_erchef[‘s3_url_expiry_window_size’]`. Don’t need it? Just set it to `:off` and close this browser tab (you have too many open, tbqh). Actually, it’s `:off` by default, so you don’t have to do anything.
If you want less uniqueness in URLs, you can set `s3_url_expiry_window_size` to be the length of time for which a URL should be unique. For example, let’s set it to `”15m”`. Now, every URL generated in a 15 minute window will be the same. The price we pay is that the link will actually take a little longer to expire than you’ve configured in `s3_url_ttl`, but no more than 15 minutes longer.
You can also set `s3_url_expiry_window_size` to be a percentage of the `s3_url_ttl`. The default of `28800` (or 8 hours), and a `s3_url_expiry_window_size` value of `”10%”` would mean a window size of 48 minutes.
Here’s a walkthrough of the `”15m”` setting:
Each letter represents a unique URL, with the capital letter being the first time that URL is seen. Look at the URL `A` generated at 1:03. Until 1:15 they’ll all be set to expire at 2:15 indicated by the lower case `a`s.
Once it’s 1:15, we’re in a new interval, but we don’t try getting a URL until 1:25. We set that one to expire at 2:30 (1hr + the remaining time in the interval).
Nobody asks for a URL until after 1:30, so the URL `B` is only used once and is never asked for again. Oh well. We played the odds and lost this time. It’s not the end of the world.
At 1:33, the URL `C` is generated and this interval gets used alot, so it’s good that we have this feature.
You get the idea. Over the course of the day we will only ever generate 96 unique expiration times, as opposed to a new expiration time for every URL requested.
Now, 15m may not be the optimal window size. If we went with `”60m”` then we’d only generate 24 unique URLs per day. That’s why we’ve made it configurable.
If you’ve got an F5 load balancer or even if you want nginx to serve up cached cookbook content instead of hitting s3 or Bookshelf, well, now you can!
After you’ve enabled the `s3_url_expiry_window_size`, you have another choice to make. If you’re using nginx to cache cookbooks:
“`
opscode_erchef[‘nginx_bookshelf_caching’] = :on
“`
Then nginx will serve up the cached content instead of forwarding the request to s3 or Bookshelf. If you’re using an F5 or other load balancer, turn that setting off like this:
“`
opscode_erchef[‘nginx_bookshelf_caching’] = :off
“`
and your load balancer will take care of serving up cached content.
