Tomcat梳理

1.问题

当Tomcat线程被打满时，可以按照以下步骤进行排查：

1. 查看Tomcat的日志文件，特别是错误日志。在日志中搜索关键字，比如"OutOfMemoryError"，以确定是否存在内存溢出异常。
2. 检查Tomcat的JVM参数设置，尤其是内存相关的参数，如-Xmx和-Xms。确保这些参数适当设置，以满足应用的需求。如果需要增加内存，可以调整这些参数的值，并重新启动Tomcat。
3. 使用工具（如jmap、jstack等）获取Tomcat的内存快照和线程堆栈信息，以了解线程的状态和运行情况。这些信息可以帮助确定是否存在线程死锁或长时间运行的线程，从而引起了线程池被打满的问题。
4. 检查Tomcat的配置文件，特别是连接器（Connector）的配置。确保连接器的最大线程数（maxThreads）合理设置，可以根据实际需求适当增加或减少线程数。
5. 检查应用程序的代码，特别是与并发相关的代码块。确保没有出现死锁、线程安全问题或无限循环等导致线程被阻塞的情况。可以使用调试器或性能分析工具来定位问题代码段。
6. 监控服务器的系统资源使用情况，如CPU、内存、磁盘等。如果某个资源被耗尽，可能导致Tomcat线程被打满。可以使用监控工具，如Nagios、Zabbix等，来实时监测服务器资源的使用情况，并进行预警和调整。
7. 在Tomcat集群环境中，检查负载均衡器的配置和运行情况，确保请求能够均匀分发到各个Tomcat节点，避免某一节点负载过高。
8. 如果以上步骤仍然无法解决问题，可以尝试升级Tomcat版本或寻求Tomcat官方或社区的支持。他们可能能提供更具体的调试和排查建议。

总之，当Tomcat线程被打满时，需要综合考虑内存、线程、代码和系统资源等多个因素，并通过逐步排查来定位和解决问题。

2.监控tomcat线程池

springboot1.x

@Autowired
ServletWebServerApplicationContext applicationContext;

TomcatServletWebServerFactory tomcatServletWebServerFactory=applicationContext.getBean(TomcatServletWebServerFactory.class);
String tomcatLog = (((TomcatWebServer)(tomcatServletWebServerFactory.getWebServer())).getTomcat().getConnector()).getProtocolHandler().getExecutor().toString();

打印结果

org.apache.tomcat.util.threads.ThreadPoolExecutor@2a3ffb6f[Running, pool size = 10, active threads = 0, queued tasks = 0, completed tasks = 0]

springboot2.x

	@Autowired
	ServletWebServerApplicationContext applicationContext;
    
	String tomcatLog = ((TomcatWebServer) applicationContext.getWebServer()).getTomcat().getConnector().getProtocolHandler().getExecutor().toString();

打印结果

org.apache.tomcat.util.threads.ThreadPoolExecutor@3a236dad[Running, pool size = 10, active threads = 1, queued tasks = 0, completed tasks = 0]

转为json格式打印输出

@RestController
public class DemoController {
    @Autowired
    ServletWebServerApplicationContext applicationContext;

    @GetMapping("/hs")
    public String test(){
        TomcatServletWebServerFactory tomcatServletWebServerFactory=applicationContext.getBean(TomcatServletWebServerFactory.class);
        String tomcatLog = (((TomcatWebServer)(tomcatServletWebServerFactory.getWebServer())).getTomcat().getConnector()).getProtocolHandler().getExecutor().toString();
        return tomcatLog;
    }


    @GetMapping("/hs2")
    public String test2(){
        String tomcatLog = ((TomcatWebServer) applicationContext.getWebServer()).getTomcat().getConnector().getProtocolHandler().getExecutor().toString();
        return tomcatLog;
    }

    @GetMapping("/hs3")
    public String test3(){
        Executor executor = ((TomcatWebServer) applicationContext.getWebServer()).getTomcat().getConnector().getProtocolHandler().getExecutor();
        return JSON.toJSONString(executor, SerializerFeature.WriteMapNullValue);
    }
}

打印结果

{
	"activeCount": 1,
	"completedTaskCount": 0,
	"corePoolSize": 10,
	"largestPoolSize": 10,
	"maximumPoolSize": 200,
	"poolSize": 10,
	"queue": [],
	"rejectedExecutionHandler": {},
	"shutdown": false,
	"submittedCount": 1,
	"taskCount": 1,
	"terminated": false,
	"terminating": false,
	"threadFactory": {},
	"threadRenewalDelay": 1000
}

3.SpringBoot内置线程解析测试

SpringBoot版本

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.3</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

controller

package com.kang.mongodb.controller;

import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.serializer.SerializerFeature;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.web.embedded.tomcat.TomcatWebServer;
import org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.concurrent.Executor;

/**
 * @Author Emperor Kang
 * @ClassName DemoController
 * @Description TODO
 * @Date 2023/8/21 19:59
 * @Version 1.0
 * @Motto 让营地比你来时更干净
 */
@Slf4j
@RestController
@RequestMapping("/tomcat")
public class TomcatController {
    @Autowired
    ServletWebServerApplicationContext applicationContext;

    /**
     * localhost:10093/tomcat/getString
     * @return
     */
    @RequestMapping("/getString")
    public String getString() {
        try {
            Executor executor = ((TomcatWebServer) applicationContext.getWebServer()).getTomcat().getConnector().getProtocolHandler().getExecutor();
            log.info("{}-{}",Thread.currentThread().getName(), JSON.toJSONString(executor, SerializerFeature.WriteMapNullValue));
            // 将线程挂起3秒，方便观察结果
            Thread.sleep(3000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return "success";
    }

}

yaml配置

server:
  port: 10093
  tomcat:
    accept-count: 5        # 等待队列长度
    threads:
      min-spare: 10        # 最小工作空闲线程数
      max: 20              # 最大线程数
    max-connections: 30    # 最大连接数,一个线程可以处理多个连接

SpringBoot内置tomcat默认配置

SpringBoot内置Tomcat的配置存放在 spring-configuration-metadata.json 文件中，配置以 server.tomcat 开头

最小工作线程数

   {
      "name": "server.tomcat.threads.min-spare",
      "type": "java.lang.Integer",
      "description": "Minimum amount of worker threads:最小工作线程数。",
      "sourceType": "org.springframework.boot.autoconfigure.web.ServerProperties$Tomcat$Threads",
      "defaultValue": 10
    }

• min-spare:最小备用

# 最小工作空闲线程数，默认10
server.tomcat.threads.min-spare=10

最大工作线程数

    {
      "name": "server.tomcat.threads.max",
      "type": "java.lang.Integer",
      "description": "Maximum amount of worker threads:最大工作线程数",
      "sourceType": "org.springframework.boot.autoconfigure.web.ServerProperties$Tomcat$Threads",
      "defaultValue": 200
    }

# 最大工作线程数，默认200
server.tomcat.threads.max=200

最大连接数

    {
      "name": "server.tomcat.max-connections",
      "type": "java.lang.Integer",
      "description": "Maximum number of connections that the server accepts and processes at any given time. Once the limit has been reached, the operating system may still accept connections based on the \"acceptCount\" property.",
      "sourceType": "org.springframework.boot.autoconfigure.web.ServerProperties$Tomcat",
      "defaultValue": 8192
    }

服务器在任何给定时间内接受和处理的最大连接数。一旦达到该限制，操作系统仍可根据 “acceptCount” 属性接受连接。

# 最大连接数默认是8192
server.tomcat.max-connections=8192

队列长度

    {
      "name": "server.tomcat.accept-count",
      "type": "java.lang.Integer",
      "description": "Maximum queue length for incoming connection requests when all possible request processing threads are in use.",
      "sourceType": "org.springframework.boot.autoconfigure.web.ServerProperties$Tomcat",
      "defaultValue": 100
    }

使用所有可能的请求处理线程时，接收到的连接请求的最大队列长度

# 等待队列长度，默认100
server.tomcat.accept-count=100

总结

参数	设置方式	默认值	含义
acceptCount	server.tomcat.accept-count	100	接收队列,实质上是在操作系统已经完成三次握手等待accept的socket队列
maxConnections	server.tomcat.max-connections	8192	最大连接数,实质上是指已经accept的socket数量
maxThreads	server.tomcat.threads.max	200	用于处理已经accept的socket的线程最大数量
connectionTimeout	server.tomcat.connection-timeout	60000ms	是指socket连接后或读取报文字节过程中的超时时间
keepAliveTimeout	实现WebServerFactoryCustomizer	默认取connectionTimeout	在同一个socket连接,处理完一次http请求后,等待下一次http请求的超时时间
MaxKeepAliveRequests	实现WebServerFactoryCustomizer	100	在同一个socket连接,最大处理http请求数量,达到数量后关闭socket

线程的生命周期

可知ThreadPoolExecutor有如下五种线程池状态。

• RUNNING，线程池接受新任务，会执行任务阻塞队列中的任务，ctl前三位表示为111；
• SHUTDOWN，线程池拒绝新任务，会执行任务阻塞队列中的任务，ctl前三位表示为000；
• STOP，线程池拒绝新任务，不会执行任务阻塞队列中的任务，尝试中断正在执行的任务，ctl前三位表示为001；
• TIDYING，所有任务被关闭，Worker数量为0，ctl前三位表示为010；
• TERMINATED，terminated() 执行完毕，ctl前三位表示为011。

线程池 的状态	说明
RUNNING	允许提交并处理任务
SHUTDOWN	不允许提交新的任务，但是会处理完已提交的任务
STOP	不允许提交新的任务，也不会处理阻塞队列中未执行的任务，
并设置正在执行的线程的中断标志位
TIDYING	所有任务执行完毕，池中工作的线程数为0，等待执行terminated()勾子方法
TERMINATED	terminated()勾子方法执行完毕

线程池监控指标

//获取webServer线程池
ThreadPoolExecutor executor = (ThreadPoolExecutor) ((TomcatWebServer) webServerApplicationContext.getWebServer())
        .getTomcat()
        .getConnector()
        .getProtocolHandler()
        .getExecutor();
Map<String, String> returnMap = new LinkedHashMap<>();
returnMap.put("核心线程数", String.valueOf(executor.getCorePoolSize()));
returnMap.put("最大线程数", String.valueOf(executor.getMaximumPoolSize()));
returnMap.put("活跃线程数", String.valueOf(executor.getActiveCount()));
returnMap.put("池中当前线程数", String.valueOf(executor.getPoolSize()));
returnMap.put("历史最大线程数", String.valueOf(executor.getLargestPoolSize()));
returnMap.put("线程允许空闲时间/s", String.valueOf(executor.getKeepAliveTime(TimeUnit.SECONDS)));
returnMap.put("核心线程数是否允许被回收", String.valueOf(executor.allowsCoreThreadTimeOut()));
returnMap.put("提交任务总数", String.valueOf(executor.getSubmittedCount()));
returnMap.put("历史执行任务的总数(近似值)", String.valueOf(executor.getTaskCount()));
returnMap.put("历史完成任务的总数(近似值)", String.valueOf(executor.getCompletedTaskCount()));
returnMap.put("工作队列任务数量", String.valueOf(executor.getQueue().size()));
returnMap.put("拒绝策略", executor.getRejectedExecutionHandler().getClass().getSimpleName());

并发测试

并发请求数 <= Tomcat最大线程数20

当请求量小于等于tomcat最大线程数时

• 全部请求都正常，没有失败
• 从结果可以看到越是靠后的线程，连接时间和处理时间都增长了，说明随着线程数增多性能下降了

tomcat线程池监控日志打印

http-nio-10093-exec-5-{"activeCount":6,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":10,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":6,"taskCount":6,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-6-{"activeCount":6,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":10,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":6,"taskCount":6,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-1-{"activeCount":6,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":10,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":6,"taskCount":6,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-4-{"activeCount":6,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":10,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":6,"taskCount":6,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-2-{"activeCount":6,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":10,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":6,"taskCount":6,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-3-{"activeCount":6,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":10,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":6,"taskCount":6,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-8-{"activeCount":15,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":17,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":18,"taskCount":18,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-10-{"activeCount":16,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":16,"maximumPoolSize":20,"poolSize":16,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":16,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-7-{"activeCount":15,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":16,"maximumPoolSize":20,"poolSize":16,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":16,"taskCount":16,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-9-{"activeCount":16,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":16,"maximumPoolSize":20,"poolSize":16,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":17,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-14-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-13-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-11-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-12-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-15-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-17-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-16-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-18-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-19-{"activeCount":19,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":19,"maximumPoolSize":20,"poolSize":19,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":19,"taskCount":19,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-20-{"activeCount":20,"completedTaskCount":0,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":20,"taskCount":20,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}

并发请求数 > Tomcat最大线程数20

当请求量大于tomcat最大线程数时

• 可以看到前20个请求都在3s的时间左右完成了，但是后五个因为没有线程资源去处理只能等到这20个线程资源被释放后才能执行这几个线程，所以后面五个都是6s多才执行完成
• 从聚合数据分析来看，我们如果不修改tomcat的配置，随着请求量的增多，系统的吞吐量是下降的

tomcat线程池监控日志打印

http-nio-10093-exec-4-{"activeCount":1,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":8,"taskCount":40,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-16-{"activeCount":4,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":14,"taskCount":46,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-17-{"activeCount":4,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":47,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-21-{"activeCount":12,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":15,"queue":[{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":47,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-18-{"activeCount":11,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":14,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":47,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-1-{"activeCount":16,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":16,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":16,"taskCount":49,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-24-{"activeCount":16,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":16,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":49,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-2-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-20-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-19-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-12-{"activeCount":4,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-13-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-22-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-26-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-25-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-27-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-23-{"activeCount":17,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":17,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":17,"taskCount":50,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-30-{"activeCount":20,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":24,"taskCount":57,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-28-{"activeCount":20,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":24,"taskCount":57,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-29-{"activeCount":20,"completedTaskCount":33,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":24,"taskCount":57,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-21-{"activeCount":16,"completedTaskCount":44,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":60,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-20-{"activeCount":16,"completedTaskCount":44,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":60,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-4-{"activeCount":16,"completedTaskCount":45,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":13,"taskCount":60,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-23-{"activeCount":17,"completedTaskCount":45,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":11,"taskCount":60,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-1-{"activeCount":17,"completedTaskCount":44,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":11,"taskCount":60,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}

挑几个典型的说说

• poolSize：池中当前线程10，15，都已经达到最小空闲线程数的设置了，那么后面队列中排队的大小，就是加入队列等待处理的请求
• submittedCount：提交的任务数，可以看到队列中有排队的，提交的任务数14，15，15，所以后面的五个必须要等到有线程资源的时候才可以去执行

并发请求数大于最大连接数：max-connections

并发请求数大于最大连接数：max-connections

• 所有的请求都执行成功了，后面的15个请求都需要等待线程资源释放后才会去处理，所以从上面可以看到我们的处理时长后面的都是6S多
• 有意思的是我们的吞吐量反而增加了，因为大量的线程被同时释放，所以大量排队的请求被处理了，从而增加了我们的吞吐量
• 注意:连接是由线程去处理的，所以连接在等待线程，而不是线程等待连接(并发请求量大的时候)

tomcat线程池监控日志打印

http-nio-10093-exec-25-{"activeCount":3,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":8,"taskCount":91,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-26-{"activeCount":5,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":8,"taskCount":91,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-21-{"activeCount":5,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":10,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":8,"taskCount":91,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-16-{"activeCount":8,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":13,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":13,"taskCount":96,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-13-{"activeCount":8,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":13,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":14,"taskCount":97,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-1-{"activeCount":19,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":22,"taskCount":105,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-20-{"activeCount":19,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":22,"taskCount":105,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-4-{"activeCount":19,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":22,"taskCount":105,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-23-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-2-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-38-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-31-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-36-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-39-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-34-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-32-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-37-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-33-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-35-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-40-{"activeCount":20,"completedTaskCount":83,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":26,"taskCount":109,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-13-{"activeCount":20,"completedTaskCount":88,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":25,"taskCount":113,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-16-{"activeCount":20,"completedTaskCount":88,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":25,"taskCount":113,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-21-{"activeCount":20,"completedTaskCount":88,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":25,"taskCount":113,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-25-{"activeCount":20,"completedTaskCount":88,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":25,"taskCount":113,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-26-{"activeCount":20,"completedTaskCount":88,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":25,"taskCount":113,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-36-{"activeCount":18,"completedTaskCount":105,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":20,"taskCount":126,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-37-{"activeCount":20,"completedTaskCount":100,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":20,"taskCount":127,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-39-{"activeCount":20,"completedTaskCount":100,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":20,"taskCount":126,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-20-{"activeCount":20,"completedTaskCount":99,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{},{},{},{},{},{},{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":23,"taskCount":126,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-38-{"activeCount":16,"completedTaskCount":114,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[{}],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":14,"taskCount":133,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-33-{"activeCount":15,"completedTaskCount":122,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":137,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-4-{"activeCount":15,"completedTaskCount":122,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":137,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-31-{"activeCount":15,"completedTaskCount":122,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":137,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-34-{"activeCount":15,"completedTaskCount":122,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":137,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}
http-nio-10093-exec-32-{"activeCount":15,"completedTaskCount":122,"corePoolSize":10,"largestPoolSize":20,"maximumPoolSize":20,"poolSize":20,"queue":[],"rejectedExecutionHandler":{},"shutdown":false,"submittedCount":15,"taskCount":137,"terminated":false,"terminating":false,"threadFactory":{},"threadRenewalDelay":1000}

并发请求数大于最大连接数+等待队列大小

• 30+5=35

并发请求数大于最大连接数+等待队列大小

• 我们请求量是45，但是我们tomcat能接收的最大请求为最大连接数+等待队列大小：30+5=35，所以我们发现有10个请求被拒绝了
• Response code:Non HTTP response code: org.apache.http.conn.HttpHostConnectException
• Response message:Non HTTP response message: Connect to 127.0.0.1:10093 [/127.0.0.1] failed: Connection refused: connect
• 但是这种情况并不绝对，因为连接是由线程去处理的，因为每个请求过来肯定是尝试连接，一直尝试，我们可以看到，只有尝试时间很长的才失败了，但是很多尝试连接成功的，就被释放的线程资源去处理了，比如我把请求数设置40，结果如下

• 我们发现发起了40个请求，理论上应该是35个成功，5个失败，但是我们看到，他有这个尝试机制，只要连接成功了，并且有线程资源处理他就会执行返回

总结

• 最大连接数：每个Http请求占用一个连接
• 等待队列长度：Http请求数超过最大连接数时，多出的请求会放入等待队列，如果队列也满了请求会被丢弃。所以Tomcat能接收的请求数为最大连接数+等待队列长度。
• 最大线程数：Tomcat最多能同时处理的请求数，比如有30个请求，但是最大线程数为20，那么同时有20个请求可以被处理
• 但是这种情况并不绝对，因为连接是由线程去处理的，因为每个请求过来肯定是尝试连接，一直尝试，我们可以看到，只有尝试时间很长的才失败了，但是很多尝试连接成功的，就被释放的线程资源去处理了，比如我把请求数设置40，结果如下

• 我们发现发起了40个请求，理论上应该是35个成功，5个失败，但是我们看到，他有这个尝试机制，只要连接成功了，并且有线程资源处理他就会执行返回

总结

• 最大连接数：每个Http请求占用一个连接
• 等待队列长度：Http请求数超过最大连接数时，多出的请求会放入等待队列，如果队列也满了请求会被丢弃。所以Tomcat能接收的请求数为最大连接数+等待队列长度。
• 最大线程数：Tomcat最多能同时处理的请求数，比如有30个请求，但是最大线程数为20，那么同时有20个请求可以被处理