ITKeyword,专注技术干货聚合推荐

注册 | 登录

(十二) Thunder分布式RPC框架

相关推荐:(一) Thunder分布式RPC框架

Thunder(QQ 群 471164539)发布在淘宝代码基地 http://code.taobao.org/p/Thunder/1. 概要1.1 Thunder是基于Netty + Hessian + Kafka + ActiveMQ + Ti

Thunder(QQ 群 471164539)发布在淘宝代码基地 http://code.taobao.org/p/Thunder/ 1. 本地配置如果本地配置启动,需要在XML配置config="local"<thunder:registry id="registry" type="zookeeper" address="localhost:2181" config="local"/> 新建业务工程时候,在相关路径下新建thunder-ext.properties。配置取值的优先级顺序是thunder-ext.properties->thunder.properties(在thunder.jar里)。如果配置名相同,取优先级高的值。它是一个集成式的配置,对于通信中间件,应用哪个就配置哪个,内容如下:# 该项只用在hessian上,即访问Hessian Servlet的Web module

# Web module path, if that is provided as a web servicepath = /thunder# 在配置XML的时候,调用模式缺省下的默认值

# Spring XML里不配置async,即采用默认同步的方式# Method invoke with async false, that is a default valueasync = false# 在配置XML的时候,广播模式缺省下的默认值

# Spring XML里不配置broadcast,即采用默认禁止广播方式# Method invoke with broadcast false, that is a default valuebroadcast = false# 在配置XML的时候,同步超时时间缺省下的默认值

# Spring XML里不配置同步方法的timeout,即采用默认同步超时为30秒,如果超过30秒,同步方法没返回,就抛出超时异常# Sync method timeout, that is a default valuesyncTimeout = 30 * 1000# 在配置XML的时候,异步超时时间缺省下的默认值

# Spring XML里不配置异步方法的timeout,即采用默认异步超时为60秒,如果超过5分钟,异步结果没回调回来,就清除异步缓存,并抛出异常# Async method timeout, aysnc cache will be remove when timeout, that is a default valueasyncTimeout = 5 * 60 * 1000# 异步守护线程对异步超时结果的扫描,默认10秒扫描一次,该值不能在XML里定义# Async method scan interval, thread will start to clear timeout cache every scan intervalasyncScan = 10 * 1000# 是否采用压缩,采用压缩其实是用少量CPU的开销,来换取网络传输的开销# 一般来说,2KB-20KB的消息体,通过压缩传递,吞吐量TPS可以提高20%-30%;但是大数据量会增加CPU开销# 如果采用Netty协议,它内部会有一个JdkZlib等压缩算法,可以再次压缩# Compress transport data, maybe it works well for 2KB-20KBcompress = false# 因网络原因(例如所有服务器挂掉)下,调用失败后,Thunder将自动采取重复调用,以保证请求不丢失。当侦测到网络原因调用失败后,会负载均衡去调用其他的服务器。

# 该项机制,只支持Netty和Hessian。MQ出现网络原因而服务调用,会自动采取重连而阻塞进程,所以没必要实现这种机制。

# Load Balance Configuration (used for Netty and Hessian)# 是否需要重试,如果true,客户端将阻塞线程,等到服务端至少一台恢复;如果false,将直接抛出异常loadBalanceRetry = true# 重试次数loadBalanceRetryTimes = 10000# 每次重试的间隔时间loadBalanceRetryDelay = 5 * 1000# 是否需要异步异常事件发布通知# If event notification is true, all produce and consume failure will be thrown out via an async event, so that will make the invocation do a retry eventNotification = true# 调用过程中,出现异常用邮件通知# If event notification and smtp mail notification are all true, all produce and consume failure will be sent as a mail# 关闭邮件通知smtpNotification = false# 是否负载均衡信息的打印(负载均衡信息有助于判断哪台服务器被选中),开启后,可能会造成日志量大增# Load balance log will help to indicate which server will be selected (used for Netty and Hessian)loadBalanceLogPrint = true# 是否传输信息的打印(传输信息有助于判断请求发往哪台服务器,从哪台服务器响应)# Transport log will help to indicate the transport informationtransportLogPrint = true# 是否心跳信息的打印(心跳信息有助于判断Netty数据通道是否还存活),开启后,可能会造成日志量大增# Heart beat log will help to indicate the data channel is alive or not (used for Netty)heartBeatLogPrint = true# 是否序列化后字节数组压缩和未压缩等长度信息的打印(长度信息有助于判断业务对象的大小,供业务部门选择是否压缩)# Serializer log will help to indicate the byte array lengthserializerLogPrint = false# 异常通知的白名单,一旦指定,白名单里的异常将不会通过邮件发送,用逗号分隔# Failure Notification exclusion, use "," to separatesmtpNotificationExclusion = com.nepxion.thunder.security.SecurityException# 是否是Ssl模式smtpSsl = true# 邮件服务器的地址smtpHost = smtp.qq.com# 发件人的账户smtpUser = *******@qq.com# 发件人的账户密码smtpPassword = # 发件人的邮件地址 smtpMailFrom = *******@qq.com# 收件人的邮件地址列表,用逗号分隔smtpMailTo = *******@qq.com,*******@qq.com# 抄送的邮件地址列表,用逗号分隔smtpMailCC = # 暗送的邮件地址列表,用逗号分隔smtpMailBCC = # 线程池

# Thread Pool Configuration# 线程隔离,一旦值为true,每个接口将对应一个线程池# Multi thread poolthreadPoolMultiMode = false# CPU unitthreadPoolServerCorePoolSize = 4# CPU unitthreadPoolServerMaximumPoolSize = 8threadPoolServerKeepAliveTime = 15 * 60 * 1000threadPoolServerAllowCoreThreadTimeout = false# CPU unitthreadPoolClientCorePoolSize = 4# CPU unitthreadPoolClientMaximumPoolSize = 8threadPoolClientKeepAliveTime = 15 * 60 * 1000threadPoolClientAllowCoreThreadTimeout = false# LinkedBlockingQueue, ArrayBlockingQueue, SynchronousQueuethreadPoolQueue = LinkedBlockingQueue# CPU unit (Used for LinkedBlockingQueue or ArrayBlockingQueue)threadPoolQueueCapacity = 1024# BlockingPolicyWithReport, CallerRunsPolicyWithReport, AbortPolicyWithReport, RejectedPolicyWithReport, DiscardedPolicyWithReport# BlockingPolicyWithReport :使用阻塞生产者的饱和策略,不抛弃任务,也不抛出异常,当队列满时改为调用BlockingQueue.put来实现生产者的阻塞# CallerRunsPolicyWithReport :使用Caller-Runs(调用者执行)饱和策略,不抛弃任务,也不抛出异常,而是将当前任务回退到发起这个调用者执行的线程所在的上级线程去执行# AbortPolicyWithReport :任务饱和时, 抛弃任务,抛出异常# RejectedPolicyWithReport :如果该任务实现了RejectedRunnable接口,那么交给用户去实现拒绝服务的逻辑,否则以FIFO的方式抛弃队列中一部分现有任务,再添加新任务# DiscardedPolicyWithReport :任务饱和时以FIFO的方式抛弃队列中一部分现有任务,再添加新任务threadPoolRejectedPolicy = BlockingPolicyWithReport# 对象池,主要是用于FST快速序列化实现

# FST Object Pool Configuration# CPU unitfstObjectPoolMaxTotal = 64# CPU unitfstObjectPoolMaxIdle = 32# CPU unitfstObjectPoolMinIdle = 16fstObjectPoolMaxWaitMillis = 3000fstObjectPoolTimeBetweenEvictionRunsMillis = 1000 * 60 * 15fstObjectPoolMinEvictableIdleTimeMillis = 1000 * 60 * 30fstObjectPoolSoftMinEvictableIdleTimeMillis = 1000 * 60 * 30fstObjectPoolBlockWhenExhausted = truefstObjectPoolLifo = truefstObjectPoolFairness = false# Netty调优

# Netty ConfigurationnettySoBacklog = 128nettySoSendBuffer = 64 * 1024nettySoReceiveBuffer = 64 * 1024nettyWriteBufferHighWaterMark = 32 * 1024nettyWriteBufferLowWaterMark = 8 * 1024nettyMaxMessageSize = 1 * 1024 * 1024nettyWriteIdleTime = 90 * 1000nettyReadIdleTime = 60 * 1000nettyAllIdleTime = 45 * 1000nettyWriteTimeout = 5 * 1000nettyReadTimeout = 5 * 1000nettyConnectTimeout = 10 * 1000nettyReconnectDelay = 1000# Hessian调优

# Hessian ConfigurationhessianReadTimeout = 30 * 1000hessianConnectTimeout = 10 * 1000# Redis调优

# Redis Configuration# CPU unitredisOb

相关推荐:(九) Thunder分布式RPC框架

Thunder(QQ 群 471164539)发布在淘宝代码基地 http://code.taobao.org/p/Thunder/1. 介绍治理中心是基于Nepxion Swing Repository组件,Java Desktop版

jectPoolMaxTotal = 30# CPU unitredisObjectPoolMaxIdle = 10# CPU unitredisObjectPoolMinIdle = 10redisObjectPoolMaxWaitMillis = 1000 * 3redisObjectPoolTimeBetweenEvictionRunsMillis = 1000 * 30redisObjectPoolMinEvictableIdleTimeMillis = 1000 * 60redisObjectPoolSoftMinEvictableIdleTimeMillis = 1000 * 60redisObjectPoolBlockWhenExhausted = trueredisObjectPoolLifo = trueredisObjectPoolFairness = falseredisSoTimeout = 5 * 1000redisConnectionTimeout = 5 * 1000redisDataExpiration = 15 * 24 * 60 * 1000# Setinel parametersredisReconnectionWait = 5redisDatabase = 0# Cluster parametersredisMaxRedirections = 5# Splunk调优# Splunk ConfigurationsplunkMaximumTime = 60# 消息队列调优

# Kafka Configurationkafka.producer.acks = allkafka.producer.linger.ms = 0kafka.producer.retries = 0kafka.producer.buffer.memory = 33554432kafka.producer.send.buffer.bytes = 131072kafka.producer.receive.buffer.bytes = 32768kafka.producer.max.request.size = 1048576kafka.producer.batch.size = 16384kafka.producer.timeout.ms = 30000kafka.producer.request.timeout.ms = 30000kafka.producer.connections.max.idle.ms = 540000kafka.producer.reconnect.backoff.ms = 50kafka.producer.retry.backoff.ms = 100kafka.producer.max.block.ms = 60000kafka.producer.max.in.flight.requests.per.connection = 5kafka.producer.metadata.max.age.ms = 300000kafka.producer.metadata.fetch.timeout.ms = 60000kafka.consumer.server.poll.timeout.ms = 1000kafka.consumer.client.poll.timeout.ms = 1000kafka.consumer.enable.auto.commit = truekafka.consumer.auto.offset.reset = latestkafka.consumer.send.buffer.bytes = 131072kafka.consumer.receive.buffer.bytes = 32768kafka.consumer.max.partition.fetch.bytes = 1048576kafka.consumer.fetch.min.bytes = 1024kafka.consumer.fetch.max.wait.ms = 500kafka.consumer.auto.commit.interval.ms = 10000kafka.consumer.heartbeat.interval.ms = 3000kafka.consumer.session.timeout.ms = 30000kafka.consumer.request.timeout.ms = 40000kafka.consumer.connections.max.idle.ms = 540000kafka.consumer.reconnect.backoff.ms = 50kafka.consumer.retry.backoff.ms = 100kafka.consumer.metadata.max.age.ms = 300000# MQ Common Configuration (used for ActiveMQ and Tibco)mqRetryNotificationDelay = 5 * 1000mqReconnectOnException = true# CacheConnectionFactory Configuration (used for ActiveMQ and Tibco)mqSessionCacheSize = 20mqCacheConsumers = truemqCacheProducers = true# PoolConnectionFactory Configuration (used for ActiveMQ and Tibco)mqMaxConnections = 20mqMaximumActiveSessionPerConnection = 100mqIdleTimeout = 30 * 1000mqExpiryTimeout = 0mqBlockIfSessionPoolIsFull = truemqBlockIfSessionPoolIsFullTimeout = -1mqTimeBetweenExpirationCheckMillis = -1mqCreateConnectionOnStartup = true# DefaultMessageListenerContainer Configuration (used for ActiveMQ and Tibco)mqConcurrentConsumers = 10mqMaxConcurrentConsumers = 1mqReceiveTimeout = 1000mqRecoveryInterval = 5000mqIdleConsumerLimit = 1mqIdleTaskExecutionLimit = 1mqCacheLevel = 4mqAcceptMessagesWhileStopping = false# Apache HttpComponent调优,作为辅助组件,支持同步和异步两种方式,可被外部调用# Apache ConfigurationapacheBacklogSize = 128apacheSendBufferSize = 64 * 1024apacheReceiveBufferSize = 64 * 1024apacheSoTimeout = 5 * 1000apacheConnectTimeout = 5 * 1000# CPU unitapacheMaxTotal = 32# Zookeeper调优# Zookeeper ConfigurationzookeeperSessionTimeout = 15 * 1000zookeeperConnectTimeout = 15 * 1000zookeeperConnectWaitTime = 1000# Zookeeper的连接配置,效果等同于Spring XML的<thunder:registry ... address="localhost:2181".../># Zookeeper Connection ConfigurationzookeeperAddress = localhost:2181# Redis的哨兵连接配置,用于Netty和Hessian的发布/订阅功能,以及日志缓存上# Redis Sentinel Connection ConfigurationredisSentinel = 192.168.126.151:16379;192.168.126.151:26379;192.168.126.151:36379redisMasterName = mymasterredisClientName =

redisPassword = # Redis的集群连接配置,用于日志缓存上,哨兵和集群配置最好两者选其一,推荐用哨兵配置# Redis Cluster Connection Configuration# redisCluster = 192.168.126.131:7001;192.168.126.131:7002;192.168.126.131:7003;192.168.126.131:7004;192.168.126.131:7005;192.168.126.131:7006# Splunk日志服务器配置# Splunk Connection ConfigurationsplunkHost = localhostsplunkPort = 8089splunkUserName = adminsplunkPassword = admin# Kafka的连接配置# Kafka Connection Configurationkafka-1.kafka.producer.bootstrap.servers = localhost:9092kafka-1.kafka.consumer.bootstrap.servers = localhost:9092# ActiveMQ的连接配置# ActiveMQ Connection ConfigurationactiveMQ-1.mqUrl = failover://(tcp://localhost:61616)?initialReconnectDelay=1000&jms.prefetchPolicy.all=5# activeMQ-1.mqUrl = failover://(tcp://broker1:61616,tcp://broker2:61616)?randomize=true# SingleConnectionFactory, CachingConnectionFactory, PooledConnectionFactoryactiveMQ-1.mqConnectionFactoryType = CachingConnectionFactoryactiveMQ-1.mqUserName = adminactiveMQ-1.mqPassword = admin# activeMQ-1.mqJndiName = ConnectionFactory# Tibco的连接配置# Tibco Connection Configurationtibco-1.mqUrl = tcp://localhost:7222# SingleConnectionFactory, CachingConnectionFactory, PooledConnectionFactorytibco-1.mqConnectionFactoryType = CachingConnectionFactorytibco-1.mqUserName = admintibco-1.mqPassword =tibco-1.mqJndiName = seashellConnectionFactorytibco-2.mqUrl = tcp://localhost:7222# SingleConnectionFactory, CachingConnectionFactory, PooledConnectionFactorytibco-2.mqConnectionFactoryType = CachingConnectionFactorytibco-2.mqUserName = admintibco-2.mqPassword =tibco-2.mqJndiName = FTQueueConnectionFactory 2. 远程配置如果本地配置启动,需要在XML配置config="remote"<thunder:registry id="registry" type="zookeeper" address="localhost:2181" config="remote"/>配置取值的优先级顺序是远程配置->thunder-ext.properties->thunder.properties(在thunder.jar里)。如何进行远程配置,参照(九) Thunder分布式RPC框架 - 治理中心

相关推荐:(十五) Thunder分布式RPC框架

Thunder(QQ 群 471164539)发布在淘宝代码基地 http://code.taobao.org/p/Thunder/框架支持两种方式的序列化,Java对象和字节数组的序列化和反序列化,Java

Thunder(QQ 群 471164539)发布在淘宝代码基地 http://code.taobao.org/p/Thunder/?1. 本地配置如果本地配置启动,需要在XML配置config="local"<thunder:registry id="registry" type=...

相关阅读排行


用户评论

游客

相关内容推荐

最新文章

×

×

请激活账号

为了能正常使用评论、编辑功能及以后陆续为用户提供的其他产品,请激活账号。

您的注册邮箱: 修改

重新发送激活邮件 进入我的邮箱

如果您没有收到激活邮件,请注意检查垃圾箱。