Friday, January 20, 2017

Spring Cache


java.lang.IllegalArgumentException: Null key returned for cache operation (maybe you are using named params on classes without debug info?)

- Not use right param name in @CacheXXX annotation
- Not param names are available - javac compiler
http://stackoverflow.com/questions/2237803/can-i-obtain-method-parameter-name-using-java-reflection
  • getting parameter names is possible if debug information is included during compilation. See this answer for more details
  • otherwise getting parameter names is not possible
  • getting parameter type is possible, using method.getParameterTypes()
For the sake of writing autocomplete functionality for an editor (as you stated in one of the comments) there are a few options:
  • use arg0arg1arg2 etc.
  • use intParamstringParamobjectTypeParam, etc.
  • use a combination of the above - the former for non-primitive types, and the latter for primitive types.
  • don't show argument names at all - just the types.
http://stackoverflow.com/questions/6759880/getting-the-name-of-a-method-parameter/6759953#6759953
Parameter names are available if you have told the compiler to include them (compile with debug information). Spring has ParameterNameDiscoverer which can help you obtain the names. The default implementation uses asm ClassReader to do so.
With javac you should include the -g argument to include debug information. With eclipse I think it is there by default.
Some frameworks use this. For example spring-mvc has @RequestParam which defaults to the param name, if resolvable. It also supports explicit naming - @RequestParam("foo") in case no debug information is provided.
http://stackoverflow.com/questions/11462662/getting-an-ehcache-instance-with-spring-intelligently
@Value("#{cacheManager.getCache('myCacheName')}")
private Cache myCache;
See also examples how to use Spring EL inside the @Value()http://www.mkyong.com/spring3/spring-el-method-invocation-example/ if you are interested.
<bean id="cache" class="net.sf.ehcache.Cache" factory-bean="cacheManager" factory-method="getCache">
    <constructor-arg value="CacheNameHere" />          
</bean>
And my java class
@Autowired
private net.sf.ehcache.Cache cache;

http://ankushs92.github.io/libraries/2016/09/06/spring-redis-bypass-cache-layer.html
https://gitlab.com/ankushs92/sample-boot-redis-errorHandler/blob/master/src/main/groovy/com/example/config/CacheConfig.java
All you need to do is to extend CachingConfigurerSupport and override errorHandler(). Now, whenever you are performing a GET operation ,like our findById(Integer id) , and the same error occurs,then Spring would bypass the Cache layer and execute the method.

@Cacheable maps to handleCacheGetError(RuntimeException ex, Cache cache, Object key) .
@CacheEvict maps to handleCacheEvictError(RuntimeException ex, Cache cache, Object key) .
@CachePut maps to handleCachePutError(RuntimeException ex, Cache cache, Object arg1, Object arg2)
You can use a similar strategy with other Cache backends that Spring provides.

http://stackoverflow.com/questions/27707857/spring-cache-with-redis-how-to-gracefully-handle-or-even-skip-caching-in-case
As from Spring Framework 4.1, there is a CacheErrorHandler that you can implement to handle such exceptions. Refer to the javadoc for more details.
http://stackoverflow.com/questions/26021991/spring-redis-error-handle
The above code presumes that Redis was completely down, and unpersisted, and therefore would be empty when it becomes available. In the event that Redis was only temporarily unreachable, or if Redis went down but the caches were persisted, updates to the database would still have gone through and the cache entries could be stale. Perhaps it should "remember" cache keys received while Redis was unavailable, and evict them when Redis becomes available

http://stackoverflow.com/questions/32994544/spring-cacheable-not-caching
There is a nasty trick you can use to override this called inject yourself:
public class YourClass {
    @Autowired
    private YourClass instance;

    @Cacheable
    public String method1() {
          // now you go through the cache again
          return instance.method2();
    }

    @Cacheable
    public String method2() {
          return "2";
    }
}
http://stackoverflow.com/questions/16899604/spring-cache-cacheable-not-working-while-calling-from-another-method-of-the-s
Here is what I do for small projects with only marginal usage of method calls within the same class. In-code documentation is strongly advidsed, as it may look strage to colleagues. But its easy to test, simple, quick to achieve and spares me the full blown AspectJ instrumentation. However, for more heavy usage I'd advice the AspectJ solution.
@Service
@Scope(proxyMode = ScopedProxyMode.TARGET_CLASS)
class AService {

    private final AService _aService;

    @Autowired
    public AService(AService aService) {
        _aService = aService;
    }

    @Cacheable("employeeData")
    public List<EmployeeData> getEmployeeData(Date date){
        ..println("Cache is not being used");
        ...
    }

    public List<EmployeeEnrichedData> getEmployeeEnrichedData(Date date){
        List<EmployeeData> employeeData = _aService.getEmployeeData(date);
        ...
    }
}

@Cacheable("settingsCache")
public String findValue(String name) {
    Setting setting = settingRepository.findOne(name);
    if(setting == null){
        return null;
    }
    return setting.getValue();
}

@Override
public Boolean findBoolean(String name) {
    String value = getSpringProxy().findValue(name);
    if (value == null) {
        return null;
    }
    return Boolean.valueOf(value);
}

/**
 * Use proxy to hit cache 
 */
private SettingService getSpringProxy() {
    return applicationContext.getBean(SettingService.class);
}

Another simple and good solution: move @Cacheable annotation to lower (DAO) level. Problem remain at inside DAO class, but solved for Service.

Redis
Atomic counters
Operations that require that the key and the value are given every time an operation is performed. These operations are handy when we have to execute a single operation by using a key and a value.
Operations that are bound to a specific key that is given only once. We should use this approach when we have to perform multiple operations by using the same key.
@Bean
public RedisAtomicLong redisAtomicLong() {
  return new RedisAtomicLong("contact", redisConnectionFactory());
}

https://github.com/akihyro/try-spring-boot-with-redis/blob/master/src/main/java/akihyro/tryspringbootwithredis/RedisConfiguration.java
http://stackoverflow.com/questions/21672245/spring-redistemplate-serialise-multiple-model-classes-into-json-need-to-use-mu
This will add @Class property to the JSON to understand the type, which helps Jackson to deserialize, so no need to explicitly map the model on the configuration class.
"{\"@class\":\"com.prnv.model.WhitePaper\",\"title\":\"Hey\",\"author\":{\"@class\":\"com.prnv.model.Author\",\"name\":\"Hello\"},\"description\":\"Description\"}"
https://blog.pranavek.com/integrating-redis-with-spring-application/
  public RedisCacheManager cacheManager() {
    RedisCacheManager redisCacheManager = new RedisCacheManager(redisTemplate());
    redisCacheManager.setTransactionAware(true);
    redisCacheManager.setLoadRemoteCachesOnStartup(true);
    redisCacheManager.setUsePrefix(true);
    return redisCacheManager;
  }

http://m.blog.csdn.net/article/details?id=52251148
如果不进行设置的话,默认使用JdkSerializationRedisSerializer进行数据序列化。
(把任何数据保存到redis中时,都需要进行序列化)

所有的key和value还有hashkey和hashvalue的原始字符前,都加了一串字符。查了一下,这是JdkSerializationRedisSerializer进行序列化时,加上去的。
原以为只会在value或hashvalue上加,没想到在key和hashkey上也加了,这样的话,用原来的key就取不到我们保存的数据了。
StringRedisSerializer:对String数据进行序列化。序列化后,保存到Redis中的数据,不会有像上面的“\xAC\xED\x00\x05t\x00\x09”多余字符。就是"frequency".
Jackson2JsonRedisSerializer:用Jackson2,将对象序列化成Json。这个Serializer功能很强大,但在现实中,是否需要这样使用,要多考虑。一旦这样使用后,要修改对象的一个属性值时,就需要把整个对象都读取出来,再保存回去。
JdkSerializationRedisSerializer:使用Java序列化。结果就像最上面的样子。
GenericToStringSerializer:使用Spring转换服务进行序列化。在网上没有找到什么例子,使用方法和StringRedisSerializer相比,StringRedisSerializer只能直接对String类型的数据进行操作,如果要被序列化的数据不是String类型的,需要转换成String类型,例如:String.valueOf()。但使用GenericToStringSerializer的话,不需要进行转换,直接由String帮我们进行转换。但这样的话,也就定死了序列化前和序列化后的数据类型,例如:template.setValueSerializer(new GenericToStringSerializer<Long>(Long.class));
我们只能用对Long型进行序列化和反序列化。(但基础类型也不多,定义8个可能也没什么)
OxmSerializer:使用SpringO/X映射的编排器和解排器实现序列化,用于XML序列化。


http://yetanotherdevblog.com/introduction_to_spring_data_redis

http://www.jiagoushuo.com/article/1000380.html
//创建List条目,key是cart
BoundListOperations<String, Object>cart=redisTemplate.boundListOps("cart");
//删除最后的一条数据
cart.rightPop();
//在最后,添加一条数据
cart.rightPush("我笑了");

@Cacheable(value="workDetailV150",key="'workDetailV150'.concat(#param.workId.toString()).concat(#param.userId.toString())")

http://m.2cto.com/kf/201606/518712.html
对于使用 @Cacheable 注解的方法,每个缓存的 key 生成策略默认使用的是参数名+参数值,比如以下方法:
@Cacheable(value="commonCache")// 使用了一个缓存名叫 accountCache
public Account getAccountByName(String userName) {
// 方法内部实现不考虑缓存逻辑,直接实现业务
System.out.println("real query account."+userName);
return getFromDB(userName);
}
username取值为zhenghuasheng时,key为userName-zhenghuasheng,一般情况下没啥问题,二般情况如方法 key 取值相等然后参数名也一样的时候就出问题了,如:
@Cacheable(value="commonCache")
public LoginData getLoginData(String userName){

}
这个方法的缓存也将保存于 key 为 users~keys 的缓存下。对于 userName 取值为 “zheghuasheng” 的缓存,key 也为 “userName-zhenghuasheng”,将另外一个方法的缓存覆盖掉。
解决办法是使用自定义缓存策略,对于同一业务(同一业务逻辑处理的方法,哪怕是集群/分布式系统),生成的 key 始终一致,对于不同业务则不一致
http://www.jianshu.com/p/2513f625fcae
@Cacheable(value="users", key="#userid.toString() + #username.toString()")  
public User findByUsername(String username, String userid)  

@Cacheable(value="books", key="#isbn.rawNumber")
public Book findBook(ISBN isbn, boolean checkWarehouse, boolean includeUsed)

@Cacheable(value="users", key="#p0")
public User find(Integer id) {
    returnnull;
}

@Cacheable(value="users", key="#p0.id")
public User find(User user) {
    returnnull;
}

http://www.codeceo.com/article/redis-spring-cache.html
(1)Redis数据库完全在内存中,使用磁盘仅用于持久性。
(2)相比许多键值数据存储,Redis拥有一套较为丰富的数据类型。
(3)Redis可以将数据复制到任意数量的从服务器。
Redis 优势?
(1)异常快速:Redis的速度非常快,每秒能执行约11万集合,每秒约81000+条记录。
(2)支持丰富的数据类型:Redis支持最大多数开发人员已经知道像列表,集合,有序集合,散列数据类型。这使得它非常容易解决各种各样的问题,因为我们知道哪些问题是可以处理通过它的数据类型更好。
(3)操作都是原子性:所有Redis操作是原子的,这保证了如果两个客户端同时访问的Redis服务器将获得更新后的值。
(4)多功能实用工具:Redis是一个多实用的工具,可以在多个用例如缓存,消息,队列使用(Redis原生支持发布/订阅),任何短暂的数据,应用程序,如Web应用程序会话,网页命中计数等。
 public Object get(final String key) {
  Object result = null;
  ValueOperations<Serializable, Object> operations = redisTemplate
    .opsForValue();
  result = operations.get(key);
  return result;
 }

 /**
  * 写入缓存
  * 
  * @param key
  * @param value
  * @return
  */
 public boolean set(final String key, Object value) {
  boolean result = false;
  try {
   ValueOperations<Serializable, Object> operations = redisTemplate
     .opsForValue();
   operations.set(key, value);
   result = true;
  } catch (Exception e) {
   e.printStackTrace();
  }
  return result;
 }

http://www.cnblogs.com/softidea/p/5801499.html

  1.     public RedisTemplate<String, String> redisTemplate(  
  2.             RedisConnectionFactory factory) {  
  3.         StringRedisTemplate template = new StringRedisTemplate(factory);  
  4.         Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);  
  5.         ObjectMapper om = new ObjectMapper();  
  6.         om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);  
  7.         om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);  
  8.         jackson2JsonRedisSerializer.setObjectMapper(om);  
  9.         template.setValueSerializer(jackson2JsonRedisSerializer);  
  10.         template.afterPropertiesSet();  
  11.         return template;  
  12.     }  
  1. 要缓存的 Java 对象必须实现 Serializable 接口,因为 Spring 会将对象先序列化再存入 Redis,比如本文中的 com.defonds.bdp.city.bean.City 类,如果不实现 Serializable 的话将会遇到类似这种错误:nested exception is java.lang.IllegalArgumentException: DefaultSerializer requires a Serializable payload but received an object of type [com.defonds.bdp.city.bean.City]]。
  2. 缓存的生命周期我们可以配置,然后托管 Spring CacheManager,不要试图通过 redis-cli 命令行去管理缓存。比如 provinceCities 方法的缓存,某个省份的查询结果会被以 key-value 的形式存放在 Redis,key 就是我们刚才自定义生成的 key,value 是序列化后的对象,这个 key 会被放在 key 名为 provinceCities~keys key-value 存储中,参考下图"provinceCities 方法在 Redis 中的缓存情况"。可以通过 redis-cli 使用 del 命令将 provinceCities~keys 删除,但每个省份的缓存却不会被清除。
  3. CacheManager 必须设置缓存过期时间,否则缓存对象将永不过期,这样做的原因如上,避免一些野数据“永久保存”。此外,设置缓存过期时间也有助于资源利用最大化,因为缓存里保留的永远是热点数据。
  4. 缓存适用于读多写少的场合,查询时缓存命中率很低、写操作很频繁等场景不适宜用缓存。


provinceCities方法在Redis中的存储.png

Labels

Review (551) System Design (281) System Design - Review (188) Java (177) Coding (75) Interview-System Design (65) Book Notes (59) Coding - Review (59) Interview (58) to-do (45) Knowledge (39) Linux (38) Interview-Java (35) Knowledge - Review (32) Database (29) Design Patterns (29) Product Architecture (28) Big Data (26) Miscs (25) Concurrency (24) Cracking Code Interview (24) MultiThread (24) Soft Skills (23) Career (22) Interview - Review (21) Java - Code (21) Operating System (21) Distributed (20) Interview Q&A (20) OOD Design (20) System Design - Practice (19) How to Ace Interview (15) Security (15) Brain Teaser (14) Algorithm (13) Linux - Shell (13) Spark (13) Code Quality (12) How to (12) Interview-Database (12) Interview-Operating System (12) Tools (12) Architecture Principles (11) Company - LinkedIn (11) Google (11) Redis (11) Resource (10) Testing (10) Amazon (9) Search (9) Spring (9) Web Dev (9) Architecture Model (8) Better Programmer (8) Cache (8) Company - Uber (8) Interview - MultiThread (8) Java67 (8) Math (8) OO Design principles (8) SOLID (8) Scalability (8) Solr (8) Git (7) Interview Corner (7) JVM (7) Java Basics (7) Machine Learning (7) NoSQL (7) C++ (6) File System (6) Highscalability (6) How to Better (6) Network (6) Restful (6) CareerCup (5) Code Review (5) Company - Facebook (5) Design (5) Hash (5) How to Interview (5) JDK Source Code (5) JavaScript (5) Leetcode (5) Must Known (5) Trouble Shooting (5) Be Architect (4) Big Fata (4) C (4) Cassandra (4) Company Product Architecture (4) Design Principles (4) Facebook (4) GeeksforGeeks (4) Generics (4) Google Interview (4) Hardware (4) JDK8 (4) Kafka (4) Optimization (4) Product + Framework (4) Shopping System (4) Source Code (4) Web Service (4) node.js (4) Back-of-Envelope (3) Company - Pinterest (3) Company - Twiiter (3) Company - Twitter (3) Consistent Hash (3) Data structures (3) GOF (3) Game Design (3) GeoHash (3) Growth (3) Guava (3) Interview-Big Data (3) Interview-Linux (3) Interview-Network (3) Java EE Patterns (3) Javarevisited (3) Map Reduce (3) Math - Probabilities (3) Performance (3) Puzzles (3) Python (3) Resource-System Desgin (3) Scala (3) UML (3) geeksquiz (3) AI (2) API Design (2) AngularJS (2) Behavior Question (2) Bugs (2) Coding Interview (2) Company - Netflix (2) Crawler (2) Cross Data Center (2) Data Structure Design (2) Database-Shard (2) Debugging (2) Elasticsearch (2) Garbage Collection (2) Go (2) Hadoop (2) Html (2) Interview - Soft Skills (2) Interview-Miscs (2) Interview-Web (2) JDK (2) Logging (2) POI (2) Papers (2) Programming (2) Project Practice (2) Random (2) Software Desgin (2) System Design - Feed (2) Thread Synchronization (2) Video (2) ZooKeeper (2) reddit (2) Ads (1) Advanced data structures (1) Algorithm - Review (1) Android (1) Approximate Algorithms (1) Base X (1) Bash (1) Books (1) C# (1) CSS (1) Chrome (1) Client-Side (1) Cloud (1) CodingHorror (1) Company - Yelp (1) Counter (1) DSL (1) Dead Lock (1) Difficult Puzzles (1) Distributed ALgorithm (1) Docker (1) Eclipse (1) Facebook Interview (1) Function Design (1) Functional (1) GoLang (1) How to Solve Problems (1) ID Generation (1) IO (1) Important (1) Internals (1) Interview - Dropbox (1) Interview - Project Experience (1) Interview Tips (1) Interview-Brain Teaser (1) Interview-How (1) Interview-Mics (1) Interview-Process (1) Jeff Dean (1) Joda (1) LeetCode - Review (1) Library (1) LinkedIn (1) Mac (1) Micro-Services (1) Mini System (1) MySQL (1) Nigix (1) NonBlock (1) Process (1) Productivity (1) Program Output (1) Programcreek (1) Quora (1) RPC (1) Raft (1) RateLimiter (1) Reactive (1) Reading (1) Reading Code (1) Resource-Java (1) Resource-System Design (1) Resume (1) SQL (1) Sampling (1) Shuffle (1) Slide Window (1) Spotify (1) Stability (1) Storm (1) Summary (1) System Design - TODO (1) Tic Tac Toe (1) Time Management (1) Web Tools (1) algolist (1) corejavainterviewquestions (1) martin fowler (1) mitbbs (1)

Popular Posts