https://www.jianshu.com/p/93767dac6b56
上面的代码，是一个典型的写法：当查询的时候，先从Redis集群中取，如果没有，那么再从DB中查询并设置到Redis集群中。
注意，在实际开发中，我们一般在缓存中，存储的数据结构是JSON。（JDK提供的序列化方式效率稍微比JSON序列化低一些；而且JDK序列化非常严格，字段的增减，就很可能导致反序列失败，而JSON这方面兼容性较好）
假设从DB中查询需要2S，那么显然这段时间内过来的请求，在上述的代码下，会全部走DB查询，相当于缓存被直接穿透，这样的现象就称之为“缓存击穿”！
synchronized+双重检查机制
双重检查
通过synchronized+双重检查机制：在同步块中，继续判断检查，保证不存在，才去查DB。
一个模板：
Template
Spring不是有很多Template类么？我们也可以通过这种思想对代码进行一个抽象，让外界来决定具体的业务实现，而把模板步骤写好。（有点类似AOP的概念）
http://venkateshcm.com/2014/05/Caching-To-Scale-Web-Applications/从这里可以看出，我们并不关心缓存的数据从哪里加载，而是交给具体的使用方，而且使用方在使用时再也不必关注缓存击穿的问题，因为我们都给抽象了。
Caching is the most common way to improve performance or scalability of an web application. In fact, effectiveness of caching is arguably main reason for developer confidence to ignore premature optimisation or to postpone fixing probable performance issue until detecting an issue. The underlying assumption for this confidence is (a) the performance optimisation is may not be required (b) if required, performance issue could be fixed by caching.
Cache could be populated lazily after the executing the operation very first time or by pre-populating cache at the start of application or by another process/background job. Lazy cache population is most common usage pattern but cache population by another background job can be effective when possible.
As shown in the above diagram, caching can be done right from browser to database layer of architecture. Let us walk through each layer of caching touching how and what could be cached at that layer and understand pros and cons of caching at each layer.
Few general points to note in the above diagram
- Caching at left most layer is better for latency.
- Caching at right most layers gives better control over granularity of caching and ways to clear or refreshing cache.
Browser loads a web page by makes several requests to server for both dynamic resource and static resources like images, style sheets, java scripts etc. Since web application is used by user several times, most of the resources are repeatedly requested from server. Browser can store some of the resources in browser cache and subsequent requests load locally cached resource instead of making server request, reducing the load on web server.
Cache granularity
- Static file caching like images , style sheets, java scripts etc
- Browsers also provide local storage (HTML5) and cookies which can be used for caching dynamic data.
How to populate cache
- Browser caching works by setting HTTP header parameters like cache control headers of resource to be cached with time to live (TTL).
Regular Cache Refresh
- Browser makes server requests after TTL time period and refreshes browser cache.
Forced Cache Refresh
- Cache refresh can be forced by changing cache invalidation parameter in the URL of the resource. Cache invalidation can be
- a query parameter Eg. http://server.com/js/library.js?timestamp
- a version number in resource path URL Eg. http://server.com/v2/js/library.js
In Single Page Applications (SPA), the initial page is always loaded from server but all other dependents like images, stylesheets, java scripts etc. are loaded based on (query parameter or version number in URL). When a new resource or application is deployed to production, cache invalidation parameter is changed to refresh cache.
Pros
- User will experience best response time with no latency.
- Can cache both static files and dynamic data.
Cons
- Load on server is marginally reduced since repeated requests for the same resource from other machine still hit the server.
- Dynamic data stored in HTML5 local storage could be security risk depending on application even through the html6 local storage is accessible to application domain only.
Content Delivery Network (CDN) providers usually work with Internet Service Providers (ISP) and Telecom companies to add data centres at the last mile of internet connection. Browser request latency is reduced since servers handling requests to cached resource are close to browser client. Browsers usually hold limited number of connection to each host, if the number of requests to a host is more than browser connection limit, requests are queued. CDN caching provides another benefit of distributing resources over several named domains, hence handling more browser requests parallely provides better performance or response time to user. Only static resources are cached in CDN since TTL of resources is usually in the order of days and not seconds.
Cache granularity
- Static File caching like images , style sheets, java scripts etc
How to populate cache
- Static resources such as images, stylesheets, java scripts can be stored in CDN using CDN provided tool or API. CDN provides an url for each resource which can be used in web application.
Regular Cache Refresh
- CDN tool or API can be used to configure time to live and when to replace static resource on CDN network servers, but replication of CDN resource over distributed network takes time, hence can not be used for very short TTL dynamic resource.
Forced Cache Refresh
- Similar to browser cache refresh above, forced cache refresh can be achieved by introducing versioning in url to invalidate cache.
Pros
- User will experience good response time with very little latency since resource is returned from nearest server.
- Load on server is greatly reduced since all requests for the resource will be handle by CDN.
Cons
- Cost of using CDN.
- Cache refresh time takes time as it has to refresh on highly distributed network of servers all over the world.
- Cache static resources only, since cache refresh is not easy.
Reverse Proxy Cache
A reverse proxy server is similar to normal proxy server, both act as intermediary between browser client and web server. The main difference is normal proxy server is placed closer to client and reverse proxy server is placed closer to web server. Since requests and responses go through reverse proxy, reverse proxy can cache response to a url and use it to respond to subsequent requests without hitting the web application server. Dynamic resource caching is significant benefit of caching at reverse proxy level with very low TTLs (few seconds). Reverse proxy can cache data in memory or in external cache management tool like memcache, will discuss more on it later.
There are several reverse proxy servers to choose from. Varnish, Squid and Ngnix are some of the options.
Cache granularity
- Static file caching like images, style sheets, java scripts etc
- Dynamic page response of http requests.
How to populate cache
- Other then reverse proxy server configurations to cache certain resources, proxy servers caching works using HTTP header parameters like cache control, expires headers of resource to be cached with time to leave (TTL).
Regular Cache Refresh
- Reverse proxy servers provide ways to invalidate and refresh cache and they make server requests after TTL time period and to refresh cache.
Forced Cache Refresh
- Reverse proxy servers can invalidate existing cache on demand.
Pros
- Web server load is reduced significantly as multi machine requests can use cache compared to browser cache where only requests for single machine are cached
- Reverse proxy servers can cache both static and dynamic resources. Eg. Home page of a site which is same for all clients can be cached for few seconds at reverse proxy.
- Most reverse proxy servers act as static file servers as-well, so web application never gets request for static resources.
Cons
- User will experience latency since request has to hit distant reverse proxy to get response.
- Reverse proxy processing and caching is required, which means added hardware costs compared to CDN costs. But most reverse proxies can be used for multiple purposes -as load balancer, as static file servers as-well as caching layer, added cost of additional box should not be a problem.
Caching within web application can be done at different scope and granularity.
Some well know (common) ways of caching in web application is based on scope of cached values
- Session Cache :– Cached value is stored per key per user and cached values are used for a single user. For example :– User home country.
- Application Cache :– Cached value is stored per key and cached values are used for multiple users. For example :– List of countries.
Session cache’s Time To Live (TTL) is usually in minutes while Application cache’s TTL varies widely and usually cached forever until changed with application events or business triggers.
Application cache is memory efficient compared to Session cache, as same cached values is used by multiple users in application cache. But if the cached value is personalized data for a specific user, it should stay in Session Cache.
While Session Cache has been used a lot (sometime abused), Application cache is not widely used.
Cache Policies
- Write-through : Cache is updated along with backing datastore synchronously. Since both datastore and cache is always kept in sync, Write-through provides high data integrity and consistency at the cost of performance. Write-through caching makes sense with read heavy applications with very few writes.
- Write-back : Cache is updated synchronously and backing datastore is updated asynchronously. Since only cache is updated synchronously Write-back provides better performance but at the cost of inconsistency or data loss in an event of crash. Write-back caching policy makes sense when there is large number of writes and lossing latest data does not effect application.
As you can see detecting and handling cached value modification is very important. In fact, it is good practice to make all cached values immutable.
Cache strategies
Below are different cache storage options available
- In-memory cache : cached values are stored in RAM memory.
- (a) in-process : caching with-in application process
- (b) out-of-process : caching in another process
- Persistent cache : caching in persistent systems like files or database.
- In-memory and in-process caching is 500 times faster than In-memory and out-of-process cache fetch time
- In-memory and out-of-process cache fetch time is 20 times faster than Persistent caching
large scale web application should be able to
- Horizontal scale out We should be able to add identical nodes in each layer to scale web applications.
- No single point of failure Is large cluster of nodes, failure of single node can happen and it should not bring down application.
Due to above two characteristics we end-up with cluster of nodes in any large scale applications.
Getting back to caching, if we go with in-memory in-process caching, each node will have to cache required data. Below are few issues with in-process caching
- Redundant caching consumes lot of memory. In cluster with N web application processes same cached value has to be stored N times.
- Each node will have to face cache miss and perform resource intensive operation before caching. i.e. In N Node cluster resource intensive operation is performed N times. This problem become noticeable if TTL of cached value is low. For Example :– A web application cluster with 100 nodes, and TTL of 1 min will perform 100 resource intensive operations every minute.
- Cache Refresh is another major problem with in-memory in process in cluster of web application process. Refreshing cache across the cluster is not easy and if cache is refreshed based on time. Due to machine time synchronization issue stale cached value will be used in some nodes, depending on application it could cause application in-consistent results based on which node handles request.
Out-of-process caching gets around the above issues by storing cached values in distributed caching systems like memcache. Due to central cache handling in out-of-process caching
- If one of the web application node caches a value, it is available to all other nodes reducing cache misses.
- Cache can be refreshed or invalided easily by updating central cache system.
Persistent caches is used when it is important to recover from crash with cached values intact. It is achieved by re-loading cached data from disk.
http://carlosfu.iteye.com/blog/2249316
一、什么是缓存雪崩
1. 由于Cache层承载着大量请求，有效的保护了Storage层(通常认为此层抗压能力稍弱)，所以Storage的调用量实际很低，所以它很爽。
2. 但是，如果Cache层由于某些原因(宕机、cache服务挂了或者不响应了)整体crash掉了，也就意味着所有的请求都会达到Storage层，所有Storage的调用量会暴增，所以它有点扛不住了，甚至也会挂掉
雪崩的危害显而易见，通常来讲可能很久以前storage已经扛不住大量请求了，于是加了cache层，所以雪崩会使得storage压力山大，甚至是挂掉。
三、如何预防缓存雪崩
1. 保证Cache服务高可用性：
和飞机都有多个引擎一样，如果我们的cache也是高可用的，即使个别实例挂掉了，影响不会很大（主从切换或者可能会有部分流量到了后端），实现自动化运维。例如：
memcache的一致性hash：
redis的sentinel和cluster机制：
2. 依赖隔离组件为后端限流：
其实无论是cache或者是mysql, hbase, 甚至别人的API，都会出现问题，我们可以将这些视同为资源，作为并发量较大的系统，假如有一个资源不可访问了，即使设置了超时时间，依然会hang住所有线程，造成其他资源和接口也不可以访问。
降级在高并发系统中是非常正常的：比如推荐服务中，很多都是个性化的需求，假如个性化需求不能提供服务了，可以降级补充热点数据，不至于造成前端页面是个大空白（开了天窗了）
在实际项目中，我们对重要的资源都进行隔离，比如hbase, elasticsearch, zookeeper, redis，别人的api(可能是http, rpc)，让每种资源都单独运行在自己的线程池中，即使资源出现了问题，对其他服务没有影响。
但是线程池如何管理，比如如何关闭资源池，开启资源池，资源池阀值管理，这些做起来还是相当麻烦的，幸好netfilx公司提供了一个很牛逼的工具：hystrix，可以做各种资源的线程池隔离。
Replicated Topology
Goal: Extreme Performance. Solution : Cache Data is Replicated to allmembers of the cluster.
Zero Latency Access :
Since the data is replicated to each cluster member, it is available for use without any waiting. This provides the highest possible speed for data access. Each member accesses the data from its own memory.
Limitations: Cost Per Update : Updating a replicated cache requires pushing the new version of the data to all other cluster members, which will limit scalability if there are a high frequency of updates per member.
Cost Per Entry :
The data is replicated to every cluster member, so Java heap space is used on each member, which will impact performance for large caches.
Partitioned Topology
Goal : Extreme Scalability.
Solution : Transparently partition the Cache Data to distribute the load across all cluster members.
Linear Scalability : By partitioning the data evenly, the per-port throughput (the amount of work being performed by each server) remains constant.
Benefits
Partitioned : The size of the cache and the processing power available growlinearly with the size of the cluster.
Load-Balanced : The responsibility for managing the data is automatically load- balanced across the cluster.
Ownership : Exactly one node in the cluster is responsible for each piece of data in the cache.
Point-To-Point : The communication for the distributed cache is all point-to-point, enabling linear scalability.
Failover : All cache services must provide lossless failover and failback
Configurable level of redundancy
Data is explicitly backed up on different physical servers (mesh architecture)
There is never a moment when the cluster is not ready for any server to die: No data vulnerabilities, no SPOFs
Near Cache Topology
Goal: Extreme Performance. Extreme Scalability. Solution: Local L1 In- Memory cache in front of a
Clustered L2 Partitioned Cache.
Cache-Aside Architecture
Cache-Aside refers to an architecture in which the application developer manages the caching of data from a data source
Adding cache-aside to an existing application:
Check the cache before reading from the data source
Put data into the cache after reading from the data source
Evict or update the cache when updating the data source
Cache-Through: Architecture
Cache-Through places the cache between the client of
the data source and the data source itself, requiring
access to the data source to go through the cache. A Cache Loader represents access to a data source. When a cache is asked for data, if it is a cache miss,
then any data that it cannot provide it will attempt to load
by delegating to the Cache Loader. A Cache Store is an extension to Cache Loader that
adds the set of operations generally referred to as
Create, Read, Update and Delete (CRUD)
Write-Behind: Description
Write-Behind accepts cache modifications directly into the cache The modifications are then asynchronously written through the Cache Store, optionally after a specified delay
The write-behind data is clustered, making it resilient to server failure
Goal: Extreme Performance. Solution : Cache Data is Replicated to allmembers of the cluster.
Zero Latency Access :
Since the data is replicated to each cluster member, it is available for use without any waiting. This provides the highest possible speed for data access. Each member accesses the data from its own memory.
Limitations: Cost Per Update : Updating a replicated cache requires pushing the new version of the data to all other cluster members, which will limit scalability if there are a high frequency of updates per member.
Cost Per Entry :
The data is replicated to every cluster member, so Java heap space is used on each member, which will impact performance for large caches.
Partitioned Topology
Goal : Extreme Scalability.
Solution : Transparently partition the Cache Data to distribute the load across all cluster members.
Linear Scalability : By partitioning the data evenly, the per-port throughput (the amount of work being performed by each server) remains constant.
Benefits
Partitioned : The size of the cache and the processing power available growlinearly with the size of the cluster.
Load-Balanced : The responsibility for managing the data is automatically load- balanced across the cluster.
Ownership : Exactly one node in the cluster is responsible for each piece of data in the cache.
Point-To-Point : The communication for the distributed cache is all point-to-point, enabling linear scalability.
Failover : All cache services must provide lossless failover and failback
Configurable level of redundancy
Data is explicitly backed up on different physical servers (mesh architecture)
There is never a moment when the cluster is not ready for any server to die: No data vulnerabilities, no SPOFs
Near Cache Topology
Goal: Extreme Performance. Extreme Scalability. Solution: Local L1 In- Memory cache in front of a
Clustered L2 Partitioned Cache.
Cache-Aside Architecture
Cache-Aside refers to an architecture in which the application developer manages the caching of data from a data source
Adding cache-aside to an existing application:
Check the cache before reading from the data source
Put data into the cache after reading from the data source
Evict or update the cache when updating the data source
Cache-Through: Architecture
Cache-Through places the cache between the client of
the data source and the data source itself, requiring
access to the data source to go through the cache. A Cache Loader represents access to a data source. When a cache is asked for data, if it is a cache miss,
then any data that it cannot provide it will attempt to load
by delegating to the Cache Loader. A Cache Store is an extension to Cache Loader that
adds the set of operations generally referred to as
Create, Read, Update and Delete (CRUD)
Write-Behind: Description
Write-Behind accepts cache modifications directly into the cache The modifications are then asynchronously written through the Cache Store, optionally after a specified delay
The write-behind data is clustered, making it resilient to server failure
