Wednesday, July 20, 2016

Cache Misc



http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ReplacingObjectsSameName.html
Although you can update existing objects in a CloudFront distribution and use the same object names, we don't recommend it. CloudFront distributes objects to edge locations only when the objects are requested, not when you put new or updated objects in your origin. If you update an existing object in your origin with a newer version that has the same name, an edge location won't get that new version from your origin until both of the following occur:
If you use the same names when you replace objects, you can't control when CloudFront starts to serve the new files. By default, CloudFront caches objects in edge locations for 24 hours. (For more information, see Specifying How Long Objects Stay in a CloudFront Edge Cache (Expiration).) For example, if you're replacing all of the objects on an entire website:
  • Objects for the less popular pages may not be in any edge locations. The new versions of these objects will start being served on the next request.
  • Objects for some pages may be in some edge locations and not in others, so your end users will see different versions depending on which edge location they're served from.
  • New versions of the objects for the most popular pages might not be served for up to 24 hours because CloudFront might have retrieved the objects for those pages just before you replaced the objects with new versions.
http://stackoverflow.com/questions/8606571/what-is-a-multi-tier-cache
https://scratchpad101.com/2014/07/25/caching-architecture-adobe-aem-part-1/

https://www.varnish-cache.org/intro/index.html#intro
Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture
https://www.revolvy.com/main/index.php?s=Distributed%20cache&item_type=topic

Web Scalability for Startup Engineers
The ability to cache web service responses is, in fact, one of the key scalability benefits of REST-ful services.
You can always put an HTTP cache between your web service and your clients and leverage the same caching mechanisms that you use for your web pages.

Cache-Control
private
no-store
no-transform
must-revalidate
the object becomes stale can still be returned to clients if they explicitly allow stale responses - use must-revalidate
Vary
tell caches that you may need to generate multiple variations of the response based on some HTTP request headers
you may return responses encoded in different ways depending on the Accept-Encoding header
Age, Last-Modified, If-Modified-Since, and Etag

Static content files should be considered immutable, and whenever you need to make a change to the contents of such a file, you should publish it under a new URL
For example, when you deploy a new version of your web application, you can bundle and minify all of your CSS files and include a timestamp or a hash of the contents of the file in the URL
Use a private cache to ensure that users see their own personalized pages

avoid htmp cache metatags, as they do not work for intermediate cache.

How to specify http cache header for html, css, js file?
Types of HTTP Cache Technologies
browser cache, caching proxies(not work with ssl), reverse proxies, and CDNs

ehcache
https://www.mkyong.com/ehcache/ehcache-hello-world-example/
 <cache name="cache1"
  maxEntriesLocalHeap="10000"
  maxEntriesLocalDisk="1000"
  eternal="false"
  diskSpoolBufferSizeMB="20"
  timeToIdleSeconds="300" timeToLiveSeconds="600"
  memoryStoreEvictionPolicy="LFU"
  transactionalMode="off">
  <persistence strategy="localTempSwap" />
 </cache>
https://community.akamai.com/community/web-performance/blog/2015/12/23/akamai-api-capabilities
Frequently, the content of a JSON API response is a mix of cacheable and non-cacheable content. This is largely based on early design decisions made by developers who may not have considered API responses as cacheable.

API responses can be cached and Akamai will hold this data at the Edge (even for mere seconds) to position the content closer to the consumer requesting it. This minimizes data transfer time to consumers of APIs such as mobile apps, where latency is crucial.

The Content Control Utility (CCU) is Akamai's technology for purging Edge content by request. The Luna Control Center provides a graphical interface to CCU for content administrators. The CCU REST API provides a programmatic interface for developers.
https://www.maxcdn.com/blog/accept-encoding-its-vary-important/
“Many HTTP caches decide that Vary: User-Agent is effectively Vary: * since the number of user-agents in the wild is so large. By asking to Vary on User-Agent you are asking your CDN to store many copies of your resource which is not very efficient for them, hence their turning off caching in this case.”

The HTTP Vary header is used by servers to indicate that the object being served will vary (the content will be different) based on some attribute of the incoming request, such as the requesting client's specified user-agent or language. The Akamai servers cannot cache different versions of the content based on the values of the Vary header. As a result, objects received with a Vary header that contains any value(s) other than "Accept-Encoding" will not be cached. To do so might result in some users receiving the incorrect version of the content (wrong language, etc.)

“Vary: User-Agent is broken for the Internet in general.  ...the basic problem is that the user-agents vary so wildly that they are almost unique for every individual (not quite that bad but IE made it a mess by including the version numbers of .Net that are installed on users machines as part of the string). If you Vary on User-Agent then intermediate caches will pretty much end up never caching resources (like Akamai).”

Imagine two clients: an old browser without compression, and a modern one with it. If they both request the same page, then depending on who sent the request first, the compressed or uncompressed version would be stored in the CDN. Now the problems start: the old browser could ask for a regular “index.html” and get the cached, compressed version (random junk data), or the new browser could get the cached, uncompressed version and try to “unzip” it. Bad news, either way.
The fix is for the origin server to send back Vary: Accept-Encoding. Now the intermediate CDNs will keep separate cache entries (one for Accept-encoding: gzip, another if you didn’t send the header). These days you’re unlikely to have clients without compression, but why risk cache mixups?
https://www.keycdn.com/support/vary-header/
One of the most important HTTP headers is the Vary header. Depending on your setup, this must be configured to ensure the right content is delivered to the right browser. When requesting content, browsers include HTTP headers to let the server decide what to send back 
https://community.akamai.com/community/web-performance/blog/2015/09/16/is-caching-on-akamai-impacted-by-vary-header
Akamai will cache the content only if Vary header has value "Accept-Encoding".
Vary: Accept-Encoding
Any other values won't be cached. Examples:
Vary: Accept-Encoding,Referer
Vary: Accept-Encoding,User-Agent
Vary: User-Agent

http://my.globaldots.com/knowledgebase.php?action=displayarticle&id=32
The HTTP Vary header is used by servers to indicate that the object being served will vary (the content will be different) based on some attribute of the incoming request, such as the requesting client's specified user-agent or language. The Akamai servers cannot cache different versions of the content based on the values of the Vary header. As a result, objects received with a Vary header that contains any value(s) other than "Accept-Encoding" will not be cached. To do so might result in some users receiving the incorrect version of the content (wrong language, etc.)


An "Accept-Encoding" value to the Vary header is the exception to this rule only when it relates to serving the object compressed. Since compression does not change the content of the object (only its size for transfer), an object that varies only by its compression can be safely cached.

To summarize, Akamai servers will cache the object, based on the configuration settings, in either of the following cases:
  • If "Vary: Accept-Encoding" is received and the content is served compressed ("Content-Encoding: gzip").
  • If no Vary header at all is received.
http://mark.koli.ch/understanding-the-http-vary-header-and-caching-proxies-squid-etc
To fix this, I located the source of the annoying User-Agent addition to my Vary response header, which happened to come from Apache’s very own mod_deflate module. The recommended mod_deflate configuration involves appending User-Agent to the Vary response header on any response that is not compressed by mod_deflate. I don’t really see why this is necessary, but the Apache folks seemed to think this was important. Here’s the relevant lines from the Apache suggested mod_deflateconfiguration:
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|ico)$ no-gzip dont-vary
Header append Vary User-Agent env=!dont-vary
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding
The Accept-Encoding request HTTP header advertises which content encoding, usually a compression algorithm, the client is able to understand. Using content negotiation, the server selects one of the proposals, uses it and informs the client of its choice with the Content-Encoding response header.
Even if both the client and the server supports the same compression algorithms, the server may choose not to compress the body of a response, if the identity value is also acceptable. Two common cases lead to this:
  • The data to be sent is already compressed and a second compression won't lead to smaller data to be transmitted. This may the case with some image formats;
  • The server is overloaded and cannot afford the computational overhead induced by the compression requirement. Typically, Microsoft recommends not to compress if a server use more than 80 % of its computational power.
http://orcaman.blogspot.com/2013/08/cdn-caching-problems-vary-user-agent.html
If you accidentally send the header and you are using a CDN, you can expect to experience the following nasty problems:
1. Your server load will increase dramatically, since requests will hit the origin instead of getting the content from the CDN.
2. Your users will experience slower response times because your origin will have to recomptue the file's content on every request, and the Geo-based content delivery will not be activated (the request will have to reach your origin server anyway).
3. Your CDN bill might explode (the CDN's usually charge by bandwidth, and they will be serving a lot more content if they are not able to cache it).

The solution for this problem can be:
1. If you don't really need the vary-by: user-agent header, don't send it! (.NET guys - if you use Microsoft BundleConfigto bundle and minify your CSS/JS, you will get this header automatically. You will need to get the System.Web.Optimization source code, remove the header from there, and recompile the DLL).
2. You can try to tell the CDN to ignore the vary-by: user agent header (in Akamai it is possible by configuration).


https://www.fastly.com/blog/best-practices-for-using-the-vary-header
So now there's an object in the cache that has a little flag on it that says "only to be used for requests that have no Accept-Encoding in the request."

Vary: *

Don't use this, period.
The HTTP RFC says that if a Vary header contains the special header name *, each request for said URL is supposed to be treated as a unique (and uncacheable) request.
This is much better indicated by using Cache-Control: private, which is clearer to anyone reading the response headers. It also signifies that the object shouldn't ever be stored, which is much more secure.

Vary: Cookie

Cookie is probably one of the most unique request headers, and is therefore very bad. Cookies often carry authentication details, in which case you're better off not trying to cache pages, but just passing them through. If you're interested in caching with tracking cookies, read more here.
However, sometimes cookies are used for A/B testing purposes, in which case it's a good idea to Vary on a custom header and leave the Cookie header intact. This avoids a lot of additional logic to make sure the Cookie header is left for URLs that need it (and are probably not cacheable).

http://forums.terracotta.org/forums/posts/list/3537.page
disable ehcache
http://ehcache.org/apidocs/net/sf/ehcache/Cache.html#setDisabled(boolean
http://cleancode.se/2012/03/31/refreshing-caches-in-ehcache.html
Ehcache has several ways to populate its cache. The sandardway is When a request is made and the value isnt in the cache. The cache forwards the request to the method beeing cahed. If you for example have a database call cached the call to the database will be made. This is whats commonly known as a read through cache. There is one thing to note: The method call is not blocking. If a second call to the same method is done before the first call populates the cache. The second call will also forward to the method.
Ehcache has a way to change this behaviour called decorators. And ehcache spring annotationsprovide some with annotations.
The first one solves the problem mentioned above. The only difference to the standard cache is that it is blocking. So if 10 calls comes at the same time, it will still just render one call to the database. To use the blocking cache annotate the method as follows:
@Cacheable(cacheName="propertyCache"
 ,decoratedCacheType= DecoratedCacheType.SELF_POPULATING_CACHE)
    public List<String> getProperties(int id) {
 //call database
 }
You can also add selfPopulatingTimeout=1234 to set the time to wait before giving up and forwarding to the method.
I would say this solution is good enough. But, the second annotation provided is a self refreshing cache. all you need to do is to change the above to:
    @Cacheable(cacheName="propertyCache"
    ,decoratedCacheType= DecoratedCacheType.REFRESHING_SELF_POPULATING_CACHE)
    public List<String> getProperties(int id) {
 //call database
 }
you can also specify the interval for the refresh by adding “refreshInterval=1000” to the annotation. I added “refreshInterval=[value i had for timeToLive in ehcache.xml]”, to keep the caches to be updated in the same interval as before the change.
This will cause ehcache to save the metod calls used when getting a value, and in the background run it again every [refreshInterval] ms. This means your cache will be updated without any http request needing to wait for the slow database call.
You probably also want to open up your ehcache.xml and change timeToLiveSeconds to timeToIdleSeconds. Since the cache will be updated anyway, It is probably not a good idea to expire the cache based on when it was first used (the refresh thread wont reset the timeToLive timer). Since if you do,tThe items will still expire and cause them to be removed and that will cause an empty cache hit. Have in mind, that timeToIdle is probably going to be reset alot more then timeToLive. So if you keep it at the same value as you had timeToLive, you will probably end up with more items in the cache, taking up more memory. But that is dependent on how your application is used.
After a Cache has been started, its configuration is not generally changeable. However, since Ehcache 2.0, certain cache configuration parameters can be modified dynamically at runtime. In the current version of Ehcache, this includes the following:
  • timeToLive
    The maximum number of seconds an element can exist in the cache regardless of use. The element expires at this limit and will no longer be returned from the cache. The default value is 0, which means no timeToLive (TTL) eviction takes place (infinite lifetime).
  • timeToIdle
    The maximum number of seconds an element can exist in the cache without being accessed. The element expires at this limit and will no longer be returned from the cache. The default value is 0, which means no timeToIdle (TTI) eviction takes place (infinite lifetime).
  • Local sizing attributes maxEntriesLocalHeap, maxBytesLocalHeap, maxEntriesLocalDisk, maxBytesLocalDisk.
  • CacheEventListeners can be added and removed dynamically
Note that the eternal attribute, when set to “true”, overrides timeToLive and timeToIdle so that no expiration can take place. This example shows how to dynamically modify the cache configuration of a running cache:
Cache cache = manager.getCache("sampleCache");
CacheConfiguration config = cache.getCacheConfiguration();
config.setTimeToIdleSeconds(60);
config.setTimeToLiveSeconds(120);
config.setmaxEntriesLocalHeap(10000);
config.setmaxEntriesLocalDisk(1000000);
Dynamic cache configurations can also be frozen to prevent future changes:
Cache cache = manager.getCache("sampleCache");
cache.disableDynamicFeatures();
In ehcache.xml, you can disable dynamic configuration by setting the <ehcache> element’s dynamicConfig attribute to “false”.
https://balamaci.wordpress.com/2009/12/15/cachingehcachepart2/
CacheManager cacheManager = CacheManager.getInstance();
Cache customCache = (Cache) cacheManager.getCache("customCache");
SelfPopulatingCache selfPopulatingCache = new SelfPopulatingCache(customCache, new CacheEntryFactory() {
public Object createEntry(Object key) throws Exception {
if("players".equals((String)key)) {
     List<Player> lstPlayers = remoteService.getAllPlayers();
     return lstPlayers;
}
return null;
}
cacheManager.replaceCacheWithDecoratedCache(customCache,
selfPopulatingCache); //this method does what the name implies and from now on any call to cacheManager
.getEhCache("customCache") will return the SelfPopulatingCache. Be carefull to not call cacheManager.getCache("customCache")
since this method will not return null, and not the decorated cache.
List players = customCache.get("players"); //This first call invokes the createEntry method and the cache is populated
customCache.get("players"); //When this gets called the data is pulled from the cache and the createEntry
method do not gets called
//After 60 seconds - the value of the timeToLiveSeconds passes
customCache.get("players"); //this call will block until createEntry(Object key)
which gets called, will finish repopulating the cache

http://www.ehcache.org/documentation/2.8/apis/cache-event-listeners.html
  • an Element expires, either because timeToLive or timeToIdle have been reached.
cache.getCacheEventNotificationService().registerListener(myListener);

Example:
https://gist.github.com/bivas/1196809


An expired element remains in the cache until either:
  • Ehcache#evictExpiredElements() is called
  • Or, a get or remove operation is called on the cache that passes in the same key as the expired element
If either of the above actions is performed, Ehcache immediately nullifies the expired element in the cache. The application would then be responsible for fetching the data for a new replacement element and repopulating the cache with this element.
SelfPopulatingCache acts as a wrapper (or decorator) around another instance of EhCache. When you ask the SelfPopulatingCache for a cached value, and that value is not in the underlying cache, the SelfPopulatingCache will create the value for you. It does this using the CacheEntryFactorythat you also provide.
So to create a SelfPopulatingCache, you need:
  • An instance of EhCache, which you fetch from the ChacheManager
  • An instance of CacheEntryFactory, which you write yourself.
Pass them both to the constructor of SelfPopulatingCache, and there you are.
SelfPopulatingCache cacheStatus = new SelfPopulatingCache(ehcache, new CacheEntryFactory() {
        @Override
        public Object createEntry(Object key) throws Exception {
            if (key.toString().equals(FLAG1)) {
                 return true;
            } else if (key.toString().equals(FLAG2)) {
                return false;
            }
            return null;
        }
     });

@Cacheable(cacheName = "fpodcache",refreshInterval=60000, decoratedCacheType= DecoratedCacheType.REFRESHING_SELF_POPULATING_CACHE)
http://stackoverflow.com/questions/11462662/getting-an-ehcache-instance-with-spring-intelligently

@Value("#{cacheManager.getCache('myCacheName')}")
private Cache myCache;
    @Resource(name="myEhCacheManager")
    private CacheManager cacheManager;
@Autowired
private net.sf.ehcache.Cache cache;

https://spring.io/blog/2016/03/04/core-container-refinements-in-spring-framework-4-3
    @Cacheable(cacheNames = "foos", sync = true)

  • Wider applicability of SpEL expressions (e.g. on @Scheduled)
https://dzone.com/articles/process-caching-vs-distributed
ConsiderationsIn-Process CacheDistributed CacheComments
ConsistencyWhile using an in-process cache, your cache elements are local to a single instance of your application. Many medium-to-large applications, however, will not have a single application instance as they will most likely be load-balanced. In such a setting, you will end up with as many caches as your application instances, each having a different state resulting in inconsistency. State may however be eventually consistent as cached items time-out or are evicted from all cache instances.Distributed caches, although deployed on a cluster of multiple nodes, offer a single logical view (and state) of the cache. In most cases, an object stored in a distributed cache cluster will reside on a single node in a distributed cache cluster. By means of a hashing algorithm, the cache engine can always determine on which node a particular key-value resides. Since there is always a single state of the cache cluster, it is never inconsistent.If you are caching immutable objects, consistency ceases to be an issue. In such a case, an in-process cache is a better choice as many overheads typically associated with external distributed caches are simply not there. If your application is deployed on multiple nodes, you cache mutable objects and you want your reads to  always be consistent rather than eventually consistent, a distributed cache is the way to go.
OverheadsThis dated but very descriptive article describes how an in-process cache can negatively effect performance of an application with an embedded cache primarily due to garbage collection overheads. Your results however are heavily dependent on factors such as the size of the cache and how quickly objects are being evicted and timed-out.A distributed cache will have two major overheads that will make it slower than an in-process cache (but better than not caching at all): network latency and object serializationAs described earlier, if you are looking for an always-consistent global cache state in a multi-node deployment, a distributed cache is what you are looking for (at the cost of performance that you may get from a local in-process cache).
ReliabilityAn in-process cache makes use of the same heap space as your program so one has to be careful when determining the upper limits of memory usage for the cache. If your program runs out of memory there is no easy way to recover from it.A distributed cache runs as an independent processes across multiple nodes and therefore failure of a single node does not result in a complete failure of the cache. As a result of a node failure, items that are no longer cached will make their way into surviving nodes on the next cache miss. Also in the case of distributed caches, the worst consequence of a complete cache failure should be degraded performance of the application as opposed to complete system failure.An in-process cache seems like a better option for a small and predictable number of frequently accessed, preferably immutable objects. For large, unpredictable volumes, you are better off with a distributed cache.
https://github.com/ben-manes/caffeine
https://info.varnish-software.com/blog/multi-tiered-caching-made-easier
The Akamai Connector for Varnish was developed to manage your cache settings using the language you are already familiar with - VCL. Through its seamless sync between Akamai and Varnish, the Connector will dramatically boost your day-to-day productivity and efficiency, saving you both time and effort.


EhCache
https://www.mkyong.com/ehcache/ehcache-hello-world-example/
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:noNamespaceSchemaLocation="ehcache.xsd" updateCheck="true"
 monitoring="autodetect" dynamicConfig="true">

 <!-- By default, Ehcache stored the cached files in temp folder. -->
 <!-- <diskStore path="java.io.tmpdir" /> -->

 <!-- Ask Ehcache to store cache in this path -->
 <diskStore path="c:\\cache" />

 <!-- Sample cache named cache1
    This cache contains a maximum in memory of 10000 elements, and will expire
    an element if it is idle for more than 5 minutes and lives for more than
    10 minutes.

    If there are more than 10000 elements it will overflow to the
    disk cache, which in this configuration will go to wherever java.io.tmp is
    defined on your system. On a standard Linux system this will be /tmp" -->
 <cache name="cache1"
  maxEntriesLocalHeap="10000"
  maxEntriesLocalDisk="1000"
  eternal="false"
  diskSpoolBufferSizeMB="20"
  timeToIdleSeconds="300" timeToLiveSeconds="600"
  memoryStoreEvictionPolicy="LFU"
  transactionalMode="off">
  <persistence strategy="localTempSwap" />
 </cache>

</ehcache>
http://www.ehcache.org/ehcache.xml

http://www.ehcache.org/documentation/2.8/configuration/configuration.html
The update checker is used to see if you have the latest version of Ehcache. It is also used to get non-identifying feedback on the operating system architectures using Ehcache. To disable the check, do one of the following:

  • timeToLive
    The maximum number of seconds an element can exist in the cache regardless of use. The element expires at this limit and will no longer be returned from the cache. The default value is 0, which means no timeToLive (TTL) eviction takes place (infinite lifetime).
  • timeToIdle
    The maximum number of seconds an element can exist in the cache without being accessed. The element expires at this limit and will no longer be returned from the cache. The default value is 0, which means no timeToIdle (TTI) eviction takes place (infinite lifetime).
  • Local sizing attributes maxEntriesLocalHeap, maxBytesLocalHeap, maxEntriesLocalDisk, maxBytesLocalDisk.

Labels

Review (572) System Design (334) System Design - Review (198) Java (189) Coding (75) Interview-System Design (65) Interview (63) Book Notes (59) Coding - Review (59) to-do (45) Linux (43) Knowledge (39) Interview-Java (35) Knowledge - Review (32) Database (31) Design Patterns (31) Big Data (29) Product Architecture (28) MultiThread (27) Soft Skills (27) Concurrency (26) Cracking Code Interview (26) Miscs (25) Distributed (24) OOD Design (24) Google (23) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Interview Q&A (20) System Design - Practice (20) Tips (19) Algorithm (17) Company - Facebook (17) Security (17) How to Ace Interview (16) Brain Teaser (14) Linux - Shell (14) Redis (14) Testing (14) Tools (14) Code Quality (13) Search (13) Spark (13) Spring (13) Company - LinkedIn (12) How to (12) Interview-Database (12) Interview-Operating System (12) Solr (12) Architecture Principles (11) Resource (10) Amazon (9) Cache (9) Git (9) Interview - MultiThread (9) Scalability (9) Trouble Shooting (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Cassandra (8) Company - Uber (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Design (7) Interview Corner (7) JVM (7) Java Basics (7) Kafka (7) Mac (7) Machine Learning (7) NoSQL (7) C++ (6) Chrome (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Python (5)

Popular Posts