SpringBoot：使用HTTP2+protobuf实现高性能微服务调用（一）服务器端实现

在当前的企业IT系统建设中，基于HTTP/1.1 + JSON的微服务架构因其实现简单、部署灵活和维护方便等特性而广受青睐，逐渐成为主流选择。然而，随着业务需求的增长，特别是高并发场景的出现，这种传统架构的局限性也日益凸显：
（1）报文转换效率较低：由于JSON格式的解析和序列化过程相对复杂，导致了较高的CPU和内存消耗。
（2）传输数据量较大：JSON文本格式较为冗长，增加了网络传输的数据量，尤其是在处理大批量数据时，这一问题更为显著。
（3）HTTP连接资源占用多：HTTP/1.1协议创建的连接一次只能处理一个请求或响应，若需要同时处理多个请求则需要创建多个连接，导致占用较多的内存和线程资源，容易造成性能瓶颈。
在这些问题共同影响下，往往会引起计算机资源消耗过高、调用处理时间延长、单机TPS（每秒事务处理数）上限被拉低等一系列性能挑战，影响了微服务应用的整体响应速度和服务质量，难以充分满足对性能有较高要求的应用场景的需求。

基于HTTP/2.0 + protobuf的微服务调用框架，将很好的解决上述问题，下面将介绍一下如何基于SpringBoot实现HTTP/2.0 + protobuf。

1、服务器端配置

从SpringBoot2.x.x开始都已默认支持HTTP2.0功能，只需要配置中开启即可。

若要支持HTTP2.0+SSL，则使用如下配置

server.http2.enabled=true

若只想支持HTTP2.0，不想使用SSL，则配置如下

server.http2.enabled=false

且要添加代码，下面以tomcat9举例：

@Bean
public TomcatConnectorCustomizer connectorCustomizer() {
    return (connector) -> connector.addUpgradeProtocol(new Http2Protocol());
}

其它服务器的配置方式，可在SpringBoot的如下文档中找到：

【“How-to” Guides】>【3. Embedded Web Servers】>【3.8. Configure HTTP/2】>【3.8.5. HTTP/2 Cleartext with supported servers】

配置完后，启动应用，会在日志中查看到如下记录

09:21:00.898 [main] INFO org.apache.coyote.http11.Http11NioProtocol - The ["http-nio-8080"] connector has been configured to support HTTP upgrade to [h2c]

2、服务器端代码

先给出一个采用JSON报文的常用微服务调用的例子。需要说明的是，对于已有的大量Controller层代码，几乎不用做任何修改，就可以支持HTTP/2.0 + protobuf。

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import com.test.model.ReqtObj;
import com.test.model.RespObj;

@RestController
public class ObjectController {

	@PostMapping(value = "/object/doTest")
	public RespObj doTest(@RequestBody ReqtObj reqtObj) {

		RespObj resp = new RespObj();
		resp.setNo(101);
		resp.setCode("ERR");
		resp.setMsg("something is wrong");

		return resp;
	}
}

public class ReqtObj {

	private String name;

	private String sex;

	private int age;

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public String getSex() {
		return sex;
	}

	public void setSex(String sex) {
		this.sex = sex;
	}

	public int getAge() {
		return age;
	}

	public void setAge(int age) {
		this.age = age;
	}

	@Override
	public String toString() {
		return "ReqtObj [name=" + name + ", sex=" + sex + ", age=" + age + "]";
	}

}

public class RespObj {

	private int no;

	private String code;

	private String msg;

	public int getNo() {
		return no;
	}

	public void setNo(int no) {
		this.no = no;
	}

	public String getCode() {
		return code;
	}

	public void setCode(String code) {
		this.code = code;
	}

	public String getMsg() {
		return msg;
	}

	public void setMsg(String msg) {
		this.msg = msg;
	}

	@Override
	public String toString() {
		return "RespObj [no=" + no + ", code=" + code + ", msg=" + msg + "]";
	}

}

通过执行如下命令并查看运行结果：

$ curl -H 'Content-Type: application/json' 127.0.0.1:8080/object/doTest -d '{"name": "xxxx", "sex": "male", "age": 123}' -s
{"no":101,"code":"ERR","msg":"something is wrong"}

若要支持protobuf，需要在pom.xml中添加如下内容。其中protostuff库的最大特点是不用生成.proto文件，protostuff会自动根据对象生成.proto文件对应的内存结构，从而免去了维护.proto文件的工作。

pom.xml

		<dependency>
			<groupId>com.google.protobuf</groupId>
			<artifactId>protobuf-java</artifactId>
			<version>4.29.3</version>
		</dependency>
		<dependency>
			<groupId>com.dyuproject.protostuff</groupId>
			<artifactId>protostuff-runtime</artifactId>
			<version>1.3.1</version>
		</dependency>
		<dependency>
			<groupId>com.dyuproject.protostuff</groupId>
			<artifactId>protostuff-core</artifactId>
			<version>1.3.1</version>
		</dependency>

java代码

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;

import com.dyuproject.protostuff.LinkedBuffer;
import com.dyuproject.protostuff.ProtostuffIOUtil;
import com.dyuproject.protostuff.Schema;
import com.dyuproject.protostuff.runtime.RuntimeSchema;

public class ProtoBufTools {

	public static <T> byte[] serialize(T obj) {
		@SuppressWarnings("unchecked")
		Schema<T> schema = (Schema<T>) RuntimeSchema.getSchema(obj.getClass());
		return ProtostuffIOUtil.toByteArray(obj, schema, LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE));
	}

	public static <T> T deserialize(byte[] data, Class<T> cls) throws Exception {
		T message = cls.getDeclaredConstructor().newInstance();
		Schema<T> schema = RuntimeSchema.getSchema(cls);
		ProtostuffIOUtil.mergeFrom(data, message, schema);
		return message;
	}

	public static byte[] getBytes(InputStream in) throws IOException {
		ByteArrayOutputStream bos = new ByteArrayOutputStream();

		byte[] buf = new byte[1024];
		int len = 0;
		while ((len = in.read(buf)) != -1) {
			bos.write(buf, 0, len);
		}

		return bos.toByteArray();
	}
}

import java.io.IOException;
import java.io.InputStream;

import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.converter.AbstractHttpMessageConverter;
import org.springframework.http.converter.HttpMessageNotReadableException;
import org.springframework.http.converter.HttpMessageNotWritableException;

public class ProtoBufHttpMessageConverter extends AbstractHttpMessageConverter<Object> {

	public ProtoBufHttpMessageConverter() {
		this.setSupportedMediaTypes(Collections.singletonList(new MediaType("application", "protobuf")));
	}

	@Override
	protected boolean supports(Class<?> clazz) {
		return true;
	}

	@Override
	protected void writeInternal(Object t, HttpOutputMessage outputMessage)
			throws IOException, HttpMessageNotWritableException {
		byte[] buf = ProtoBufTools.serialize(t);
		outputMessage.getBody().write(buf);
	}

	@Override
	protected Object readInternal(Class<? extends Object> clazz, HttpInputMessage inputMessage)
			throws IOException, HttpMessageNotReadableException {
		InputStream in = inputMessage.getBody();
		try {
			byte[] buf = ProtoBufTools.getBytes(in);
			return ProtoBufTools.deserialize(buf, clazz);
		} catch (Throwable e) {
			e.printStackTrace();
		}
		return null;
	}
}

import java.util.Collections;
import java.util.List;

import org.springframework.context.annotation.Configuration;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;

@Configuration
public class WebConfig implements WebMvcConfigurer {

	@Override
	public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
		ProtoBufHttpMessageConverter converter = new ProtoBufHttpMessageConverter();

		converters.add(converter);
	}
}

创建ProtoBufHttpMessageConverter，处理请求时，将protobuf报文转换成对象，处理响应时，将对象转换成protobuf报文。

3、客户器端测试代码

3.1、使用apacheHttpClient4

pom.xml

		<dependency>
			<groupId>org.apache.httpcomponents</groupId>
			<artifactId>httpclient</artifactId>
			<version>4.5.14</version>
		</dependency>

代码：

import java.io.InputStream;
import java.nio.charset.Charset;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ByteArrayEntity;
import org.apache.http.entity.ContentType;
import org.apache.http.impl.client.HttpClientBuilder;

import com.test.model.ReqtObj;
import com.test.model.RespObj;

public class ProtoBufTester {

	private static void doTest() {
		try {
			HttpClient httpClient = HttpClientBuilder.create().build();

			String url = "http://127.0.0.1:8080/object/doTest";
			HttpPost httpPost = new HttpPost(url);
			httpPost.setHeader("Content-Type", "application/protobuf");
			httpPost.setHeader("Accept", "application/protobuf");  // #1

			ReqtObj reqt = new ReqtObj();
			reqt.setName("PPPPPP");
			reqt.setSex("female");
			reqt.setAge(13);

			byte[] data = ProtoBufTools.serialize(reqt);

			System.out.println("======================" + reqt.toString());
			System.out.println("reqtLen=" + data.length);

			httpPost.setEntity(
					new ByteArrayEntity(data, ContentType.create("application/protobuf", Charset.forName("UTF-8"))));

			HttpResponse httpResponse = httpClient.execute(httpPost);
			InputStream in = httpResponse.getEntity().getContent();

			byte[] buf = ProtoBufTools.getBytes(in);

			System.out.println("respLen=" + buf.length);

			RespObj resp = ProtoBufTools.deserialize(buf, RespObj.class);
			System.out.println("======================" + resp.toString());
		} catch (Throwable e) {
			e.printStackTrace();
		}
	}

	public static void main(String[] args) {
		doTest();
	}
}

#1，这行代码非常关键，如果没有这行代码，服务器端生成的响应报文还会是JSON的

apacheHttpClient4只支持HTTP/1.1，不支持HTTP/2.0，因此若要使用HTTP/2.0，则看下面的代码

3.2、使用reactorHttpClient

import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import reactor.core.publisher.Flux;
import reactor.netty.http.HttpProtocol;
import reactor.netty.http.client.HttpClient;

public class ReactorHttpClientTester {

	public static void main(String[] args) {

		ReqtObj reqt = new ReqtObj();
		reqt.setName("PPPPPP");
		reqt.setSex("female");
		reqt.setAge(13);

		try {
			byte[] data = ProtoBufTools.serialize(reqt);

			HttpClient client = HttpClient.create().protocol(HttpProtocol.H2C).headers(headers -> {
				headers.add("Content-Type", "application/protobuf");
				headers.add("Accept", "application/protobuf");
			});

			Flux<ByteBuf> bufFlux = Flux.just(data).map(Unpooled::wrappedBuffer);

			byte[] retData = client.post().uri("http://127.0.0.1:8080/object/doTest").send(bufFlux)
					.responseSingle((resp, bytes) -> {
						System.out.println(resp.status());
						return bytes.asByteArray();
					}).block();

			RespObj resp = ProtoBufTools.deserialize(retData, RespObj.class);
			System.out.println("======================" + resp.toString());
		} catch (Throwable e) {
			e.printStackTrace();
		}
	}
}

这个例子中HTTP2和protobuf就都支持了。但美中不足的是这个客户端没有跟SpringBoot中的RestTemplate或WebClient相结合，也没有自动做对象到protobuf的相互转换，后续会有文章专门解决这个问题。

参考文档
“How-to” Guides