编程开源技术交流，分享技术与知识

网站首页 > 开源技术 正文

Plumecache | 一个常见分布式缓存封装工具类库

wxchong 2024-08-08 00:55:59 开源技术 27 ℃ 0 评论

为什么有plumecache?

日常项目开发中使用缓存的时候，无论选择redis、memcached,或是其它类型缓存，我们总要在做一些封装，才能很好的使用。plumecache是我个人缓存使用经验的一些总结，旨在于解决日常使用缓存时的一些问题和痛点，写出来和大家分享，也欢迎大家的star和issue Gitee地址

简介

Plumecache 是一个常见分布式缓存封装工具类库，降低相关client api学习成本，提高工作效率，当前支持reids（standalone,sentinel,cluster）、memcached

Plumecache解决什么问题？

  • 1.用于替换项目中CacheService、CacheHelper、CacheHandler、CacheUtils等缓存工具类或胶水代码
  • 2.支持多缓存实例管理，解决既要还要的问题
  • 3.支持缓存key前缀、缓存版本的配置
  • 4.支持缓存序列化统一处理，不需要在每个调用的地方写序列化，支持自定义
  • 5.支持大缓存压缩处理，支持自定义
  • 6.支持自定义拦截器的AOP扩展，可以处理参数、返回值、异常

快速使用

通用接入

  • 1.maven(因域名问题，暂没有上传中央仓库)，可以下载源码install到本地或者上传到私服，也可以先下载jar
<dependency>
    <groupId>org.plume</groupId>
    <artifactId>plumecache-core</artifactId>
    <version>1.0.0</version>
</dependency>
复制代码
  • 2.配置
plumecache:
  instances:
    - type: redis
      endpoint: 127.0.0.1:6379
      prefix: order
复制代码
  • 3.使用
@Test
public void test(){
        CacheService cacheService=CacheServiceFactory.getInstance();
        cacheService.set("name","anson.yin");
        System.out.println(cacheService.get("name"));
}
复制代码

spring boot接入

  • 1.maven(因域名问题，暂没有上传中央仓库)，可以下载源码install到本地或者上传到私服，也可以先下载jar

<dependency>
    <groupId>org.plume</groupId>
    <artifactId>plumecache-spring-boot-starter</artifactId>
    <version>1.0.0</version>
</dependency>
复制代码
  • 2.配置
plumecache:
  instances:
    - type: redis
      endpoint: 127.0.0.1:6379
      prefix: order
复制代码
  • 3.使用
@Autowired
private CacheService cacheService;

@RequestMapping(value = "/hello")
public String hello(){
        cacheService.set("name","anson.yin");
        return"hello,".concat(cacheService.get("name"));
}
复制代码

功能说明

配置

plumecache:
  instances:
    - type: redis #必填
      name: redis #非必填，缺省为和type一样
      endpoint: 127.0.0.1:6379  #必填
      prefix: order #非必填，like "prefix@key"
      serializer: org.plumecache.samples.FastjsonCacheSerializer  #非必填，指定序列化类，缺省使用Gson序列化
      compressor: org.plumecache.samples.NoneCacheCompressor  #非必填，指定压缩类，缺省使用Gzip压缩
      exclude-interceptors: SlowCacheInterceptor,LogCacheInterceptor #非必填，排除不需要的拦截器
复制代码

多实例

配置

plumecache:
  instances:
    - type: redis #必填
      name: redis #非必填，缺省为和type一样
      endpoint: 127.0.0.1:6379  #必填
      prefix: order #非必填，like "prefix@key"
      serializer: org.plumecache.samples.FastjsonCacheSerializer  #非必填，指定序列化类，缺省使用Gson序列化
      compressor: org.plumecache.samples.NoneCacheCompressor  #非必填，指定压缩类，缺省使用Gzip压缩
      exclude-interceptors: SlowCacheInterceptor,LogCacheInterceptor #非必填，排除不需要的拦截器
    - type: memcached
      endpoint: 127.0.0.1:11211
    - type: memcached
      name: memcached2
      endpoint: 127.0.0.1:11222
    - type: rediscluster
      endpoint: 127.0.0.1:6379,127.0.0.1:6389,127.0.0.1:6399
复制代码

说明

  • 0.上面的配置了四个缓存实例
  • 1.默认的缓存实例是第一个配置
  • 2.同一种类型多个实例，需要配置不同name来区分

代码示例

@Test
public void testMultipleInstances() {
        //和CacheService redis =CacheServiceFactory.getInstance("redis")效果一样
        CacheService redis = CacheServiceFactory.getInstance();
        redis.set("instance", "redis");
        System.out.println(redis.get("instance"));

        CacheService memcached = CacheServiceFactory.getInstance("memcached");
        memcached.set("instance", "memcached");
        System.out.println(memcached.get("instance"));
}
复制代码

序列化

  • 1.默认使用Gson作为序列化
  • 2.自定义序列化，需要实现CacheSerializer接口，并且在配置配置序列化类

下面是FastjsonCacheSerializer的实现示例

public class FastjsonCacheSerializer implements CacheSerializer {

    @Override
    public <T> String serialize(T value) {
        return JSON.toJSONString(value);
    }

    @Override
    public <T> T deserialize(String value, Class<T> clazz) {
        return JSON.parseObject(value, clazz);
    }
}
复制代码

缓存压缩

  • 1.实体类添加 @CacheCompress 注解
@Data
@CacheCompress
public class User {
    private String name;
    private Integer age;
    private String address;
}
复制代码
  • 2.调用方式不变
    @Test
    public void testCompress() {
        User user = new User();
        user.setAge(100);
        user.setName("zhangsanfeng");
        user.setAddress("zhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfengzhangsanfeng");

        cacheService.set("user", user);
        User cacheUser = cacheService.get("user", User.class);
        
        System.out.println(cacheUser);
    }
复制代码
127.0.0.1:6379> get order@user
"\x04>]H4sIAAAAAAAAAKtWSkxJKUotLlayUqrKSMxLL07MS0vNSx8otpKOUmJ6qpKVoYGBjlJeYm4qmruU\nagHJDU9isgAAAA=="
复制代码
  • 3.自定义压缩器

默认使用gzip压缩，也可以自定义实现,且添加配置
参考GzipCacheCompressor.java

缓存版本

  • 1.实体类添加 @CacheVersion 注解
@Data
@CacheVersion("2.0")
public class User {
    private String name;
    private Integer age;
    private String address;
}
复制代码
  • 2.调用方式不变
    @Test
    public void testVersion() {
        User user = new User();
        user.setAge(100);
        user.setName("zhangsanfeng");
        user.setAddress("address");

        cacheService.set("user", user);
        User cacheUser = cacheService.get("user", User.class);

        System.out.println(cacheUser);
    }
复制代码
127.0.0.1:6379> get order@user@2.0
"\x04>5{\"name\":\"zhangsanfeng\",\"age\":100,\"address\":\"address\"}"
复制代码

拦截器

拦截器是对缓存操作的切面处理，自定义拦截器操作如下

  • 1.实现CacheInterceptor，或者继承BaseCacheInterceptor,举例如下
@Slf4j
public class SlowCacheInterceptor extends BaseCacheInterceptor {

    @Override
    public boolean preHandle(CacheService target, Method method, Object[] args, Map<String, Object> context) {
        Instant begin = Instant.now();
        context.put("SlowCacheInstantBegin", begin);
        return true;
    }

    @Override
    public void postHandle(CacheService target, Method method, Object[] args, Map<String, Object> context, Object result) {
        Instant begin = (Instant) context.get("SlowCacheInstantBegin");

        if (Duration.between(begin, Instant.now()).toMillis() > 500) {
            log.warn("[SlowCacheInterceptor]slow cache, method:{},args:{},result:{},cost:{}"
                    , method.getDeclaringClass().getName() + "." + method.getName()
                    , JSON.toJSONString(args)
                    , JSON.toJSONString(result)
                    , Duration.between(begin, Instant.now()).toMillis());

            //do something others
        }
    }
}
复制代码
  • 2.添加SPI的配置

path：resources/services/com.plumecache.core.interceptor.CacheInterceptor

org.plumecache.samples.SlowCacheInterceptor
复制代码

或者参考core包实现

  • LogCacheInterceptor.java
  • VersionCacheInterceptor.java
  • PrefixCacheInterceptor.java

管理接口

在spring boot stater且为web项目时生效

  • 1.list 显示实例列表
curl -X GET localhost:8080/plumecache/list
复制代码
[{"properties":{"name":"memcached","type":"memcached","endpoint":"127.0.0.1:11211","prefix":null,"serializer":null,"compressor":null,"excludeInterceptors":null},"statistics":{"cmd_touch":"0","moves_to_cold":"5","incr_hits":"1","get_flushed":"0","evictions":"0","touch_hits":"0","expired_unfetched":"0","pid":"1","time_in_listen_disabled_us":"0","response_obj_bytes":"65536","cas_badval":"0","cmd_flush":"0","total_items":"6","read_buf_oom":"0","round_robin_fallback":"0","slab_reassign_rescues":"0","log_watcher_skipped":"0","cas_hits":"0","accepting_conns":"1","auth_errors":"0","slab_reassign_evictions_nomem":"0","log_watcher_sent":"0","reserved_fds":"20","slab_reassign_running":"0","response_obj_oom":"0","crawler_items_checked":"12","direct_reclaims":"0","conn_yields":"0","slab_reassign_busy_deletes":"0","version":"1.6.10","read_buf_count":"8","listen_disabled_num":"0","slab_global_page_pool":"0","get_misses":"0","hash_is_expanding":"0","touch_misses":"0","get_expired":"0","auth_cmds":"0","cas_misses":"0","delete_misses":"0","cmd_meta":"0","get_hits":"4","slab_reassign_inline_reclaim":"0","malloc_fails":"0","delete_hits":"0","log_worker_written":"0","read_buf_bytes_free":"49152","lru_bumps_dropped":"0","curr_connections":"2","bytes_written":"4650","slab_reassign_busy_items":"0","hash_bytes":"524288","libevent":"2.1.8-stable","read_buf_bytes":"131072","lrutail_reflocked":"0","crawler_reclaimed":"0","decr_hits":"0","limit_maxbytes":"67108864","max_connections":"1024","decr_misses":"0","lru_crawler_running":"0","reclaimed":"0","rejected_connections":"0","cmd_get":"4","hash_power_level":"16","curr_items":"3","threads":"4","cmd_set":"5","bytes_read":"360","slab_reassign_chunk_rescues":"0","lru_crawler_starts":"27","uptime":"22253","log_worker_dropped":"0","unexpected_napi_ids":"0","total_connections":"9","evicted_active":"0","incr_misses":"1","connection_structures":"4","bytes":"221","lru_maintainer_juggles":"30247","evicted_unfetched":"0","rusage_system":"12.600134","time":"1638171110","slabs_moved":"0","moves_within_lru":"0","pointer_size":"64","moves_to_warm":"0","rusage_user":"8.090528","response_obj_count":"1"}},{"properties":{"name":"redis","type":"redis","endpoint":"127.0.0.1:6379","prefix":"order","serializer":"org.plumecache.samples.FastjsonCacheSerializer","compressor":"com.plumecache.core.compressor.NoneCacheCompressor","excludeInterceptors":["SlowCacheInterceptor","LogCacheInterceptor"]},"statistics":{"io_threaded_reads_processed":"0","tracking_clients":"0","uptime_in_seconds":"22224","cluster_connections":"0","current_cow_size":"0","maxmemory_human":"0B","aof_last_cow_size":"0","master_replid2":"0000000000000000000000000000000000000000","mem_replication_backlog":"0","aof_rewrite_scheduled":"0","total_net_input_bytes":"19507","rss_overhead_ratio":"1.58","hz":"10","current_cow_size_age":"0","redis_build_id":"69ab6eec4665acbc","aof_last_bgrewrite_status":"ok","multiplexing_api":"epoll","client_recent_max_output_buffer":"0","allocator_resident":"5181440","mem_fragmentation_bytes":"6549960","repl_backlog_first_byte_offset":"0","tracking_total_prefixes":"0","redis_mode":"standalone","cmdstat_get":"calls=10,usec=342,usec_per_call=34.20,rejected_calls=0,failed_calls=0","redis_git_dirty":"0","allocator_rss_bytes":"3031040","repl_backlog_histlen":"0","io_threads_active":"0","rss_overhead_bytes":"2990080","total_system_memory":"2082197504","loading":"0","evicted_keys":"0","maxclients":"10000","cmdstat_set":"calls=10,usec=1941,usec_per_call=194.10,rejected_calls=0,failed_calls=0","cluster_enabled":"0","redis_version":"6.2.5","repl_backlog_active":"0","mem_aof_buffer":"0","allocator_frag_bytes":"447160","io_threaded_writes_processed":"0","instantaneous_ops_per_sec":"0","used_memory_human":"1.59M","cmdstat_incr":"calls=1,usec=68,usec_per_call=68.00,rejected_calls=0,failed_calls=0","total_error_replies":"0","role":"master","maxmemory":"0","used_memory_lua":"37888","rdb_current_bgsave_time_sec":"-1","used_memory_startup":"809880","used_cpu_sys_main_thread":"146.683765","lazyfree_pending_objects":"0","used_memory_dataset_perc":"38.70%","allocator_frag_ratio":"1.26","arch_bits":"64","used_cpu_user_main_thread":"38.482678","mem_clients_normal":"512400","expired_time_cap_reached_count":"0","unexpected_error_replies":"0","mem_fragmentation_ratio":"5.04","aof_last_rewrite_time_sec":"-1","master_replid":"9c1aa9fc501b889e35727910956569cac1a2fda1","aof_rewrite_in_progress":"0","lru_clock":"10781132","maxmemory_policy":"noeviction","run_id":"d05aa7e4384c6ccd09ee83e919c9e9edcd2d8450","latest_fork_usec":"439","tracking_total_items":"0","total_commands_processed":"1349","expired_keys":"0","used_memory":"1664360","module_fork_in_progress":"0","dump_payload_sanitizations":"0","mem_clients_slaves":"0","keyspace_misses":"3","server_time_usec":"1638171084987972","executable":"/data/redis-server","lazyfreed_objects":"0","db0":"keys=232,expires=0,avg_ttl=0","used_memory_peak_human":"4.53M","keyspace_hits":"7","rdb_last_cow_size":"520192","used_memory_overhead":"1333640","active_defrag_hits":"0","tcp_port":"6379","uptime_in_days":"0","used_memory_peak_perc":"35.03%","current_save_keys_processed":"0","blocked_clients":"0","total_reads_processed":"1900","expire_cycle_cpu_milliseconds":"7066","sync_partial_err":"0","used_memory_scripts_human":"0B","aof_current_rewrite_time_sec":"-1","aof_enabled":"0","process_supervised":"no","cmdstat_info":"calls=2,usec=559,usec_per_call=279.50,rejected_calls=0,failed_calls=0","master_repl_offset":"0","used_memory_dataset":"330720","used_cpu_user":"38.497928","rdb_last_bgsave_status":"ok","tracking_total_keys":"0","cmdstat_ping":"calls=1325,usec=19580,usec_per_call=14.78,rejected_calls=0,failed_calls=0","atomicvar_api":"c11-builtin","allocator_rss_ratio":"2.41","client_recent_max_input_buffer":"16","clients_in_timeout_table":"0","aof_last_write_status":"ok","mem_allocator":"jemalloc-5.1.0","cmdstat_incrby":"calls=1,usec=21,usec_per_call=21.00,rejected_calls=0,failed_calls=0","used_memory_scripts":"0","used_memory_peak":"4751720","process_id":"1","master_failover_state":"no-failover","used_cpu_sys":"146.736539","repl_backlog_size":"1048576","connected_slaves":"0","current_save_keys_total":"0","gcc_version":"8.3.0","total_system_memory_human":"1.94G","sync_full":"0","connected_clients":"25","module_fork_last_cow_size":"0","total_writes_processed":"1349","allocator_active":"2150400","total_net_output_bytes":"18709","pubsub_channels":"0","current_fork_perc":"0.00","active_defrag_key_hits":"0","rdb_changes_since_last_save":"0","instantaneous_input_kbps":"0.00","configured_hz":"10","used_memory_rss_human":"7.79M","expired_stale_perc":"0.00","active_defrag_misses":"0","used_cpu_sys_children":"0.029570","number_of_cached_scripts":"0","sync_partial_ok":"0","used_memory_lua_human":"37.00K","rdb_last_save_time":"1638169845","pubsub_patterns":"0","slave_expires_tracked_keys":"0","redis_git_sha1":"00000000","used_memory_rss":"8171520","rdb_last_bgsave_time_sec":"0","os":"Linux 5.10.47-linuxkit x86_64","mem_not_counted_for_evict":"0","active_defrag_running":"0","rejected_connections":"0","total_forks":"3","active_defrag_key_misses":"0","allocator_allocated":"1703240","instantaneous_output_kbps":"0.00","second_repl_offset":"-1","rdb_bgsave_in_progress":"0","used_cpu_user_children":"0.012939","total_connections_received":"575","migrate_cached_sockets":"0"}}]
复制代码
  • 2.execute 执行缓存操作
curl localhost:8080/plumecache/execute \
-X POST \
-H "Content-Type: application/json" \
-d '{"command":"get","key":"name"}'
复制代码
anson.yin
复制代码

接口说明

参考CacheService.java

RoadMap

version 1.0.0

  • 1.支持spring boot 项目接入, 基于spring boot starter和 enable 以及 conditional注解完成初始化 --done
  • 2.支持spring项目介入 提供初始化工具类 --done
  • 3.支持key前缀功能 --done
  • 4.支持多个cache实例 --done
  • 5.支持get、set、delete、exists、expire、ttl命令 --done
  • 6.支持dashboard api（实例列表，实例参数，命令执行）
  • 7.支持自定义拦截器 --done
  • 8.支持自定义序列化 --done
  • 9.返回具体实例（redissonclient、MemcachedClient）对象 --done
  • 10.实现hget,hset,hgetall,lock 命令 --done
  • 11.支持缓存压缩功能（大value压缩） --done
  • 12.支持版本功能(version 注解) --done

version 2.0.0

  • 0.缓存参数配置（timeout,poolsize,idlecount,attempts）
  • 1.支持线程内缓存（基于threadlocal）
  • 2.支持字段序列化（基于ObjectOutputStream）

项目结构

作者：广陵笑笑生
链接：https://juejin.cn/post/7038850628965105672

Tags：

猜你喜欢

本文暂时没有评论，来添加一个吧(●'◡'●)

欢迎 你 发表评论:

最近发表
标签列表
赣ICP备2023011894号-2