⭐️ 前言
开发的小伙伴们对于Netty并不陌生,本文就Netty粘包拆包问题及其解决方案做一个介绍,希望能对大家有所帮助。
⭐️ 什么是粘包拆包问题
我们知道,传统的IO是面向流的,而Netty(它的底层是Java NIO)是面向Buffer的。所以当发送很多体量比较小的消息时,消息会堆积在Buffer(例如send buffer)中,触发Flush后才会真正在网路上传输数据。这种情况下,接收端会接收到很多连在一起的体量较小的消息,这就产生了粘包;另外,当需要发送的消息体量较大,大到超出Buffer的最大容量时,只能先发送消息的一部分,剩下的部分会在稍晚一些发送,这样,一个大体量的消息被拆开了,于是就产生了拆包问题。
下文会演示粘包现象,并探讨粘包拆包的解决方案。
⭐️ netty handler 执行顺序
在演示之前,我们先来看看netty handler 的执行顺序,handler可以通过ChannelPipeline的addLast方法按顺序添加。总体来说,接收消息会沿着handler链路从前往后寻找InboundHandler依次处理;发送消息时,会沿着handler链路从后往前寻找OutboundHandler依次处理,若在handler链路中间的某个InboundHandler发送数据,则分两种情况:
1、调用ctx.writeAndFlush,从当前handler沿链路向前寻找OutboundHandler依次处理
2、调用ctx.channel().writeAndFlush,从handler链路的tail向前寻找OutboundHandler依次处理
⭐️ 日志设置
日志很重要,在java项目中,日志需要两个组件,日志门面和日志实现,日志门面这里采用slf4j,具体日志实现选择log4j,依赖如下,这里顺便给出netty的依赖。
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.59.Final</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
</dependencies>
log4j.properties对log进行相关设置
log4j.rootLogger=DEBUG,stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy/MM/dd HH:mm:ss,SSS} [%p]%C{1}-%m%n
⭐️ 粘包演示
这里客户端发送了三条消息,在最后一条消息发送时进行flush操作。
server端代码
public class Server {
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.SO_KEEPALIVE, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new StringDecoder())
.addLast(new ServerTestHandler());
}
});
System.out.println("server ready");
ChannelFuture sync = bootstrap.bind(8888).sync();
sync.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
客户端代码
public class Client {
public static void main(String[] args) {
EventLoopGroup workGroup = new NioEventLoopGroup();
Bootstrap bootstrap = new Bootstrap();
try {
bootstrap.group(workGroup)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new StringEncoder(CharsetUtil.UTF_8));
}
});
System.out.println("client ok");
ChannelFuture localhost = bootstrap.connect("localhost", 8888).sync();
// 发送消息
String msg1 = "hello world";
localhost.channel().write(msg1);
String msg2 = "my name is eryx";
localhost.channel().write(msg2);
String msg3 = "i am robot";
localhost.channel().writeAndFlush(msg3);
localhost.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
workGroup.shutdownGracefully();
}
}
}
这里定义一个ServerTestHandler,用以显示客户端发给服务端的数据
public class ServerTestHandler extends ChannelInboundHandlerAdapter {
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println("来自client的消息:" + String.valueOf(msg));
}
}
客户端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 |hello world |
+--------+-------------------------------------------------+----------------+
2023/11/17 12:04:15,780 [DEBUG]AbstractInternalLogger-[id: 0x07a76a5a, L:/127.0.0.1:54667 - R:localhost/127.0.0.1:8888] WRITE: 15B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 6d 79 20 6e 61 6d 65 20 69 73 20 65 72 79 78 |my name is eryx |
+--------+-------------------------------------------------+----------------+
2023/11/17 12:04:15,780 [DEBUG]AbstractInternalLogger-[id: 0x07a76a5a, L:/127.0.0.1:54667 - R:localhost/127.0.0.1:8888] WRITE: 10B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 69 20 61 6d 20 72 6f 62 6f 74 |i am robot |
+--------+-------------------------------------------------+----------------+
2023/11/17 12:04:15,780 [DEBUG]AbstractInternalLogger-[id: 0x07a76a5a, L:/127.0.0.1:54667 - R:localhost/127.0.0.1:8888] FLUSH
服务端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 6d 79 20 6e 61 |hello worldmy na|
|00000010| 6d 65 20 69 73 20 65 72 79 78 69 20 61 6d 20 72 |me is eryxi am r|
|00000020| 6f 62 6f 74 |obot |
+--------+-------------------------------------------------+----------------+
来自client的消息:hello worldmy name is eryxi am robot
可以看到,在客户端分3次写入的消息,服务端接收到时,三条消息连接在了一起,发生了粘包问题。
⭐️ 如何解决粘包拆包
为解决粘拆包问题,netty提供了一些解码器,这里介绍两个:
1、FixedLengthFrameDecoder 固定长度帧解码器
2、LengthFieldBasedFrameDecoder 以长度字段为基础的解码器
FixedLengthFrameDecoder
FixedLengthFrameDecoder对于固定长度的消息很方便,我们对server做一些调整,这里客户端发送的虽然不是固定长度消息,但可以更清洗的理解FixedLengthFrameDecoder的效果。
server端代码
public class Server {
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.SO_KEEPALIVE, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
// 以5为帧的固定长度
.addLast(new FixedLengthFrameDecoder(5))
.addLast(new StringDecoder())
.addLast(new ServerTestHandler());
}
});
System.out.println("server ready");
ChannelFuture sync = bootstrap.bind(8888).sync();
sync.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
客户端的代码和日志均与上一小节相同,而server端的日志输出如下:
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 6d 79 20 6e 61 |hello worldmy na|
|00000010| 6d 65 20 69 73 20 65 72 79 78 69 20 61 6d 20 72 |me is eryxi am r|
|00000020| 6f 62 6f 74 |obot |
+--------+-------------------------------------------------+----------------+
来自client的消息:hello
来自client的消息: worl
来自client的消息:dmy n
来自client的消息:ame i
来自client的消息:s ery
来自client的消息:xi am
来自client的消息: robo
可见,server端解析的消息,每一条都有5个字符。
LengthFieldBasedFrameDecoder
LengthFieldBasedFrameDecoder包含一些参数,说明如下:
(1) maxFrameLength:发送的数据包最大长度
(2) lengthFieldOffset:长度域偏移量,指的是长度域位于整个数据包字节数组中的下标
(3) lengthFieldLength:长度域的字节数长度
(4) lengthAdjustment:长度域的偏移量矫正
(5) initialBytesToStrip:丢弃的起始字节数
在只关注消息本身和其长度的情况下,LengthFieldBasedFrameDecoder可以和LengthFieldPrepender配合使用,LengthFieldPrepender可以指定一个参数,即长度域的字节数长度。
server端代码
public class Server {
public static void main(String[] args) {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.SO_KEEPALIVE, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new LengthFieldBasedFrameDecoder(1024,0, 4, 0, 4))
.addLast(new StringDecoder())
.addLast(new ServerTestHandler());
}
});
System.out.println("server ready");
ChannelFuture sync = bootstrap.bind(8888).sync();
sync.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
client端代码
public class Client {
public static void main(String[] args) {
EventLoopGroup workGroup = new NioEventLoopGroup();
Bootstrap bootstrap = new Bootstrap();
try {
bootstrap.group(workGroup)
.channel(NioSocketChannel.class)
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast(new LoggingHandler())
.addLast(new LengthFieldPrepender(4))
.addLast(new StringEncoder(CharsetUtil.UTF_8));
}
});
System.out.println("client ok");
ChannelFuture localhost = bootstrap.connect("localhost", 8888).sync();
// 发送消息
String msg1 = "hello world";
localhost.channel().write(msg1);
String msg2 = "my name is eryx";
localhost.channel().write(msg2);
String msg3 = "i am robot";
localhost.channel().writeAndFlush(msg3);
localhost.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
workGroup.shutdownGracefully();
}
}
}
客户端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0b |.... |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,338 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 11B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 77 6f 72 6c 64 |hello world |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,341 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 4B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0f |.... |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 15B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 6d 79 20 6e 61 6d 65 20 69 73 20 65 72 79 78 |my name is eryx |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 4B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0a |.... |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] WRITE: 10B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 69 20 61 6d 20 72 6f 62 6f 74 |i am robot |
+--------+-------------------------------------------------+----------------+
2023/11/17 15:16:19,342 [DEBUG]AbstractInternalLogger-[id: 0x5598fbcb, L:/127.0.0.1:50147 - R:localhost/127.0.0.1:8888] FLUSH
服务端日志
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 0b 68 65 6c 6c 6f 20 77 6f 72 6c 64 00 |....hello world.|
|00000010| 00 00 0f 6d 79 20 6e 61 6d 65 20 69 73 20 65 72 |...my name is er|
|00000020| 79 78 00 00 00 0a 69 20 61 6d 20 72 6f 62 6f 74 |yx....i am robot|
+--------+-------------------------------------------------+----------------+
来自client的消息:hello world
来自client的消息:my name is eryx
来自client的消息:i am robot
2023/11/17 15:16:19,383 [DEBUG]AbstractInternalLogger-[id: 0xb4d43375, L:/127.0.0.1:8888 - R:/127.0.0.1:50147] READ COMPLETE
可见,每一天消息的都被正确的解析了,消息的界限明确了,粘包问题解决了!!!
笔者水平有限,若有不对的地方欢迎评论指正!