前言
最近做项目，需要支持kafka多数据源，实际上我们也可以通过代码固定写死多套kafka集群逻辑，但是如果需要不修改代码扩展呢，因为kafka本身不处理额外逻辑，只是起到削峰，和数据的传递，那么就需要对架构做一定的设计了。
准备test
kafka本身非常容易上手，如果我们需要单元测试，引入jar依赖，JDK使用1.8，当然也可以使用JDK17
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>2.7.17</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<version>2.7.17</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
<version>2.9.13</version>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<version>2.9.13</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.testcontainers/kafka -->
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>kafka</artifactId>
<version>1.20.1</version>
<scope>test</scope>
</dependency>
</dependencies>修改发送者和接收者
@Component
public class KafkaProducer {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaProducer.class);
@Autowired
private KafkaTemplate<String, String> kafkaTemplate;
public void send(String topic, String payload) {
LOGGER.info("sending payload='{}' to topic='{}'", payload, topic);
kafkaTemplate.send(topic, payload);
}
}
@Component
public class KafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaConsumer.class);
private String payload;
@KafkaListener(topics = "${test.topic}")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("----------------received payload='{}'", consumerRecord.toString());
payload = consumerRecord.toString();
}
public String getPayload() {
return payload;
}
public void setPayload(String payload) {
this.payload = payload;
}
}然后写main方法，随意写一个即可，配置入戏
spring:
kafka:
consumer:
auto-offset-reset: earliest
group-id: mytest
test:
topic: embedded-test-topic写一个单元测试
@SpringBootTest
@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9092", "port=9092" })
class DemoMainTest {
@Autowired
private KafkaConsumer consumer;
@Autowired
private KafkaProducer producer;
@Value("${test.topic}")
private String topic;
@Test
void embedKafka() throws InterruptedException {
String data = "Sending with our own simple KafkaProducer";
producer.send(topic, data);
Thread.sleep(3000);
assertThat(consumer.getPayload(), containsString(data));
Thread.sleep(10000);
}
}通过
@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9092", "port=9092" })
直接模拟一个kafka，里面有一些注解参数，可以设置broker的 数量端口，zk的端口，topic和partition数量等
实际上是通过embed zk和kafka来mock了一个kafka server
单元测试运行成功
思路
有了kafka单元测试后，根据springboot map可以接收多套配置的方式不就实现了kafka的多数据源的能力，貌似非常简单；但是如果需要不用修改代码，消费端怎么办，发送者可以手动创建，消费端是注解方式，topic等信息在注解参数中，注解参数值却是常量，代码写死的，那么我们就需要：
- 不让Springboot自动扫描，根据配置手动扫描注册bean
- 字节码生成bean，就可以根据参数
这里没考虑把消费端和发送者的额外处理逻辑写在这里的做法，统一处理kafka，类似kafka网关，因为kafka一般不会仅一套，且不会仅有一个topic，需要分发处理，比如slb，feign等。
kafka消费者的原理
其实kafka发送者和消费者也是类似逻辑，但是spring-kafka通过注解方式实现消费者，如果我们使用原生kafka的kafkaconsumer，那么只需要通过Map接收参数，然后自己实现消费逻辑就行，但是spring-kafka毕竟做了很多公共没必要的逻辑，拉取消费的一系列参数，线程池管理等处理措施。看看Spring-kafka的消费者初始化原理，
BeanPostProcessor的kafka实现
org.springframework.kafka.annotation.KafkaListenerAnnotationBeanPostProcessor
看前置处理
什么都没做，所以，所有逻辑都在后置处理
public Object postProcessAfterInitialization(final Object bean, final String beanName) throws BeansException {
if (!this.nonAnnotatedClasses.contains(bean.getClass())) {
Class<?> targetClass = AopUtils.getTargetClass(bean);
//找到注解，消费注解KafkaListener打在类上，一般不用这种方式
Collection<KafkaListener> classLevelListeners = findListenerAnnotations(targetClass);
//类上KafkaListener注解的标志
final boolean hasClassLevelListeners = classLevelListeners.size() > 0;
final List<Method> multiMethods = new ArrayList<>();
//找到消费方法，去每个方法上找KafkaListener注解
Map<Method, Set<KafkaListener>> annotatedMethods = MethodIntrospector.selectMethods(targetClass,
(MethodIntrospector.MetadataLookup<Set<KafkaListener>>) method -> {
Set<KafkaListener> listenerMethods = findListenerAnnotations(method);
return (!listenerMethods.isEmpty() ? listenerMethods : null);
});
if (hasClassLevelListeners) {
//类上KafkaListener注解的时候，通过另外的注解KafkaHandler的方式，找到消费方法
Set<Method> methodsWithHandler = MethodIntrospector.selectMethods(targetClass,
(ReflectionUtils.MethodFilter) method ->
AnnotationUtils.findAnnotation(method, KafkaHandler.class) != null);
multiMethods.addAll(methodsWithHandler);
}
//实际上大部分类是没有kafka消费注解的，效率并不高，但是因为日志是trace，所以日志一般默认看不见
//注解KafkaListener打在方法上的时候
if (annotatedMethods.isEmpty() && !hasClassLevelListeners) {
this.nonAnnotatedClasses.add(bean.getClass());
this.logger.trace(() -> "No @KafkaListener annotations found on bean type: " + bean.getClass());
}
else {
// Non-empty set of methods
for (Map.Entry<Method, Set<KafkaListener>> entry : annotatedMethods.entrySet()) {
Method method = entry.getKey();
for (KafkaListener listener : entry.getValue()) {
//核心逻辑
processKafkaListener(listener, method, bean, beanName);
}
}
this.logger.debug(() -> annotatedMethods.size() + " @KafkaListener methods processed on bean '"
+ beanName + "': " + annotatedMethods);
}
//注解KafkaListener打在类上，实际上处理逻辑跟KafkaListener打在方法上差不多
if (hasClassLevelListeners) {
processMultiMethodListeners(classLevelListeners, multiMethods, bean, beanName);
}
}
return bean;
}如果是注解打在类上，如下
本文中的示例的@KafkaListener打在方法上，所以分析
processKafkaListener
其实原理都一样，spring-kafka不会写2份一样逻辑，只是读取处理的参数略有不同
protected synchronized void processKafkaListener(KafkaListener kafkaListener, Method method, Object bean,
String beanName) {
//检查代理
Method methodToUse = checkProxy(method, bean);
//终端设计思想，Spring很多地方都这样设计，尤其是swagger
MethodKafkaListenerEndpoint<K, V> endpoint = new MethodKafkaListenerEndpoint<>();
endpoint.setMethod(methodToUse);
//bean的名称，这里需要定制全局唯一，否则多个listener会冲突
String beanRef = kafkaListener.beanRef();
this.listenerScope.addListener(beanRef, bean);
String[] topics = resolveTopics(kafkaListener);
TopicPartitionOffset[] tps = resolveTopicPartitions(kafkaListener);
if (!processMainAndRetryListeners(kafkaListener, bean, beanName, methodToUse, endpoint, topics, tps)) {
//核心逻辑
processListener(endpoint, kafkaListener, bean, beanName, topics, tps);
}
this.listenerScope.removeListener(beanRef);
}继续
processListener
protected void processListener(MethodKafkaListenerEndpoint<?, ?> endpoint, KafkaListener kafkaListener,
Object bean, String beanName, String[] topics, TopicPartitionOffset[] tps) {
//MethodKafkaListenerEndpoint赋值了，这个很关键
processKafkaListenerAnnotation(endpoint, kafkaListener, bean, topics, tps);
//容器工厂
String containerFactory = resolve(kafkaListener.containerFactory());
KafkaListenerContainerFactory<?> listenerContainerFactory = resolveContainerFactory(kafkaListener,
containerFactory, beanName);
//注册终端，最终生效
this.registrar.registerEndpoint(endpoint, listenerContainerFactory);
}processKafkaListenerAnnotation
private void processKafkaListenerAnnotation(MethodKafkaListenerEndpoint<?, ?> endpoint,
KafkaListener kafkaListener, Object bean, String[] topics, TopicPartitionOffset[] tps) {
endpoint.setBean(bean);
endpoint.setMessageHandlerMethodFactory(this.messageHandlerMethodFactory);
endpoint.setId(getEndpointId(kafkaListener));
endpoint.setGroupId(getEndpointGroupId(kafkaListener, endpoint.getId()));
endpoint.setTopicPartitions(tps);
endpoint.setTopics(topics);
endpoint.setTopicPattern(resolvePattern(kafkaListener));
endpoint.setClientIdPrefix(resolveExpressionAsString(kafkaListener.clientIdPrefix(), "clientIdPrefix"));
endpoint.setListenerInfo(resolveExpressionAsBytes(kafkaListener.info(), "info"));
String group = kafkaListener.containerGroup();
if (StringUtils.hasText(group)) {
Object resolvedGroup = resolveExpression(group);
if (resolvedGroup instanceof String) {
endpoint.setGroup((String) resolvedGroup);
}
}
String concurrency = kafkaListener.concurrency();
if (StringUtils.hasText(concurrency)) {
endpoint.setConcurrency(resolveExpressionAsInteger(concurrency, "concurrency"));
}
String autoStartup = kafkaListener.autoStartup();
if (StringUtils.hasText(autoStartup)) {
endpoint.setAutoStartup(resolveExpressionAsBoolean(autoStartup, "autoStartup"));
}
resolveKafkaProperties(endpoint, kafkaListener.properties());
endpoint.setSplitIterables(kafkaListener.splitIterables());
if (StringUtils.hasText(kafkaListener.batch())) {
endpoint.setBatchListener(Boolean.parseBoolean(kafkaListener.batch()));
}
endpoint.setBeanFactory(this.beanFactory);
resolveErrorHandler(endpoint, kafkaListener);
resolveContentTypeConverter(endpoint, kafkaListener);
resolveFilter(endpoint, kafkaListener);
}各种参数注册，尤其是其中的ID和handler是必须的，不注册不行；笔者试着自己设置endpoint，发现其中的各种handler注册。
解决方式
先写一个工具类，用于创建一些关键类的bean，定义了发送者创建，消费者工厂类，消费者的创建由注解扫描实现，引用工具类的消费者容器工厂bean。
public class KafkaConfigUtil {
private DefaultKafkaProducerFactory<String, String> initProducerFactory(KafkaProperties kafkaProperties) {
return new DefaultKafkaProducerFactory<>(kafkaProperties.buildProducerProperties());
}
public KafkaTemplate<String, String> initKafkaTemplate(KafkaProperties kafkaProperties) {
return new KafkaTemplate<>(initProducerFactory(kafkaProperties));
}
private ConsumerFactory<? super Integer, ? super String> initConsumerFactory(KafkaProperties kafkaProperties) {
return new DefaultKafkaConsumerFactory<>(kafkaProperties.buildConsumerProperties());
}
public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>>
initKafkaListenerContainerFactory(KafkaProperties kafkaProperties) {
ConcurrentKafkaListenerContainerFactory<Integer, String> factory =
new ConcurrentKafkaListenerContainerFactory<>();
factory.setConsumerFactory(initConsumerFactory(kafkaProperties));
return factory;
}
}
1、通过Map接收多数据源
定义一个配置接收器，仿造zuul的模式
@ConfigurationProperties(prefix = "spring.kafka")
public class KafkaMultiProperties {
private Map<String, KafkaProperties> routes;
public Map<String, KafkaProperties> getRoutes() {
return routes;
}
public void setRoutes(Map<String, KafkaProperties> routes) {
this.routes = routes;
}
}每一个route其实就说一套kafka，再写一个Configuration，注入配置文件
@Configuration
@EnableConfigurationProperties(KafkaMultiProperties.class)
public class KafkaConfiguration {
}这样就可以注入配置了，从此可以根据配置的不同初始化不同的kafka集群逻辑。 这样就可以把自定义的Properties注入Springboot的placeholder中。
2、通过自定义扫描支持消费者
如果消费者或者发送者逻辑需要写在当前kafka网关应用，那么只能通过自定义扫描方式支持配置不同，所有配置的生成者和消费者必须代码实现逻辑，通过配置加载方式，自定义扫描注入bean即可。以消费者为例，生产者不涉及注解发送方式相对简单。
public class KafkaConfigInit {
private KafkaMultiProperties kafkaMultiProperties;
private ConfigurableApplicationContext applicationContext;
public KafkaConfigInit(KafkaMultiProperties kafkaMultiProperties,
ConfigurableApplicationContext applicationContext) {
this.kafkaMultiProperties = kafkaMultiProperties;
this.applicationContext = applicationContext;
}
@PostConstruct
public void initConfig() {
if (kafkaMultiProperties == null || kafkaMultiProperties.getRoutes() == null) return;
kafkaMultiProperties.getRoutes().forEach((k, v) -> {
//register producer by config
ConfigurableListableBeanFactory beanFactory = applicationContext.getBeanFactory();
beanFactory.registerSingleton(k + "_producer", KafkaConfigUtil.initKafkaTemplate(v));
//register consumer container factory
KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>> kafkaListenerContainerFactory = KafkaConfigUtil.initKafkaListenerContainerFactory(v);
beanFactory.registerSingleton(k + "_consumerFactory", kafkaListenerContainerFactory);
});
}
}写了一个初始化的bean，用于通过配置加载bean。但是有2个问题：
- 消费者是注解方式扫描，bean需要根据配置加载，不能写在代码里面
- 这里仅仅是注册bean，并不会被beanpostprocessor处理
关于第1点
因为需要按照配置加载，不能代码写bean的加载逻辑，只能自己扫描按照配置加载，那么需要自定义扫描注解和扫描包名（减少扫描范围，提高效率）
关于第2点
需要手动执行beanpostprocessor的逻辑即可
show me the code
完善刚刚写的部分代码：
写一个注解
@Target({ ElementType.TYPE, ElementType.ANNOTATION_TYPE })
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface KafkaConfigConsumer {
String beanId() default "";
}通过beanId区分，配置文件的key+"_consumer"可以作为唯一标识，定义一种标准
可以使用Spring的
PathMatchingResourcePatternResolver
自己解析resources信息，来拿到写的自定义注解的类，然后生成对象，注入Spring
public class KafkaConfigInit {
private KafkaMultiProperties kafkaMultiProperties;
private ConfigurableApplicationContext applicationContext;
private KafkaListenerAnnotationBeanPostProcessor<?,?> kafkaListenerAnnotationBeanPostProcessor;
private static final Map<String, Object> consumerMap = new ConcurrentHashMap<>();
public KafkaConfigInit(KafkaMultiProperties kafkaMultiProperties, ConfigurableApplicationContext applicationContext, KafkaListenerAnnotationBeanPostProcessor<?, ?> kafkaListenerAnnotationBeanPostProcessor) {
this.kafkaMultiProperties = kafkaMultiProperties;
this.applicationContext = applicationContext;
this.kafkaListenerAnnotationBeanPostProcessor = kafkaListenerAnnotationBeanPostProcessor;
}
@PostConstruct
public void initConfig() throws IOException {
scanConsumer();
if (kafkaMultiProperties == null || kafkaMultiProperties.getRoutes() == null) return;
kafkaMultiProperties.getRoutes().forEach((k, v) -> {
//register producer by config
ConfigurableListableBeanFactory beanFactory = applicationContext.getBeanFactory();
beanFactory.registerSingleton(k + "_producer", KafkaConfigUtil.initKafkaTemplate(v));
//register consumer container factory
KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>> kafkaListenerContainerFactory = KafkaConfigUtil.initKafkaListenerContainerFactory(v);
beanFactory.registerSingleton(k + "_containerFactory", kafkaListenerContainerFactory);
beanFactory.registerSingleton(k+"_consumer", consumerMap.get(k+"_consumer"));
kafkaListenerAnnotationBeanPostProcessor.postProcessAfterInitialization(consumerMap.get(k+"_consumer"), k+"_consumer");
});
}
private void scanConsumer() throws IOException {
SimpleMetadataReaderFactory register = new SimpleMetadataReaderFactory();
PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver();
Resource[] resources = resolver.getResources(ResourcePatternResolver.CLASSPATH_ALL_URL_PREFIX + "com/feng/kafka/demo/init/*");
Arrays.stream(resources).forEach((resource)->{
try {
MetadataReader metadataReader = register.getMetadataReader(resource);
if (metadataReader.getAnnotationMetadata().hasAnnotatedMethods("org.springframework.kafka.annotation.KafkaListener")){
String className = metadataReader.getClassMetadata().getClassName();
Class<?> clazz = Class.forName(className);
KafkaConfigConsumer kafkaConfigConsumer = clazz.getDeclaredAnnotation(KafkaConfigConsumer.class);
Object obj = clazz.newInstance();
consumerMap.put(kafkaConfigConsumer.beanId(), obj);
}
} catch (IOException | ClassNotFoundException | InstantiationException | IllegalAccessException e) {
throw new RuntimeException(e);
}
});
}
}同时，需要手动执行
kafkaListenerAnnotationBeanPostProcessor
的逻辑，上面有源码分析，而且因为要支持多数据源，所以需要修改消费者的注解参数
//@KafkaListener(topics = "${test.topic}")
//@Component
@KafkaConfigConsumer(beanId = "xxx_consumer")
public class KafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaConsumer.class);
private String payload;
// @KafkaHandler
@KafkaListener(topics = "${test.topic}", beanRef = "xxx_listener", containerFactory = "xxx_containerFactory")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("----------------received payload='{}'", consumerRecord.toString());
payload = consumerRecord.toString();
}
// other getters
public String getPayload() {
return payload;
}
public void setPayload(String payload) {
this.payload = payload;
}
}增加beanRef属性外加我们自己写的注解，然后通过@Configuration注入
@Configuration
@EnableConfigurationProperties(KafkaMultiProperties.class)
public class KafkaConfiguration {
@Bean
public KafkaConfigInit initKafka(KafkaMultiProperties kafkaMultiProperties,
ConfigurableApplicationContext applicationContext,
KafkaListenerAnnotationBeanPostProcessor<?, ?> kafkaListenerAnnotationBeanPostProcessor){
return new KafkaConfigInit(kafkaMultiProperties, applicationContext, kafkaListenerAnnotationBeanPostProcessor);
}
}然后修改配置文件和单元测试类
spring:
kafka:
routes:
xxx:
producer:
batchSize: 1
consumer:
auto-offset-reset: earliest
group-id: xxx然后修改单元测试代码
@SpringBootTest
@EmbeddedKafka(partitions = 1, brokerProperties = { "listeners=PLAINTEXT://localhost:9092", "port=9092" })
class DemoMainTest {
@Lazy
@Autowired
private KafkaConsumer consumer;
@Autowired
private ApplicationContext applicationContext;
@Value("${test.topic}")
private String topic;
@Test
void embedKafka() throws InterruptedException {
String data = "Sending with our own simple KafkaProducer";
applicationContext.getBean("xxx_producer", KafkaTemplate.class).send(topic, data);
Thread.sleep(3000);
assertThat(consumer.getPayload(), containsString(data));
Thread.sleep(10000);
}
}执行单元测试成功
数据正确发送消费，断言正常
3、通过字节码生成支持消费者
上面的方式觉得还是不方便，一般而言处理消息和消费消息是异步的，即使是同步也不会在消费线程直接处理，一般是发送到其他地方接口处理，所以为啥还要写消费者代码呢，默认一个不就好了，但是注解参数确是常量，那么字节码生成一个唯一的类即可。
如果生成者和消费者处理逻辑不用网关应用处理，那么仅仅是无脑转发，类似zuul，可以通过字节码生成方式实现统一逻辑，主要是消费者，毕竟有注解，生产者不存在注解可以直接new出来注入bean。
以javassist为例，简单些，当然asm也可以
show me the code
其实就说把扫描的消费者类，变成固定某个类消费
//@KafkaListener(topics = "${test.topic}")
//@Component
//@KafkaConfigConsumer(beanId = "xxx_consumer")
public class KafkaConsumer {
private static final Logger LOGGER = LoggerFactory.getLogger(KafkaConsumer.class);
private String payload;
// @KafkaHandler
// @KafkaListener(topics = "${test.topic}", beanRef = "xxx_listener", containerFactory = "xxx_containerFactory")
public void receive(ConsumerRecord<?, ?> consumerRecord) {
LOGGER.info("----------------received payload='{}'", consumerRecord.toString());
payload = consumerRecord.toString();
}去掉注解，因为注解需要我们动态加上去，下一步修改bean创建流程
@PostConstruct
public void initConfig() throws IOException {
// scanConsumer();
if (kafkaMultiProperties == null || kafkaMultiProperties.getRoutes() == null) return;
kafkaMultiProperties.getRoutes().forEach((k, v) -> {
//register producer by config
ConfigurableListableBeanFactory beanFactory = applicationContext.getBeanFactory();
beanFactory.registerSingleton(k + "_producer", KafkaConfigUtil.initKafkaTemplate(v));
//register consumer container factory
KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<Integer, String>> kafkaListenerContainerFactory = KafkaConfigUtil.initKafkaListenerContainerFactory(v);
beanFactory.registerSingleton(k + "_containerFactory", kafkaListenerContainerFactory);
// beanFactory.registerSingleton(k + "_consumer", consumerMap.get(k + "_consumer"));
Object obj = initConsumerBean(k);
beanFactory.registerSingleton(k + "_consumer", obj);
kafkaListenerAnnotationBeanPostProcessor.postProcessAfterInitialization(obj, k + "_consumer");
});
}
private Object initConsumerBean(String key) {
try {
ClassPool pool = ClassPool.getDefault();
CtClass ct = pool.getCtClass("com.feng.kafka.demo.init.KafkaConsumer");
//修改类名，避免重复
ct.setName("com.feng.kafka.demo.init.KafkaConsumer"+key);
//获取类中的方法
CtMethod ctMethod = ct.getDeclaredMethod("receive");
MethodInfo methodInfo = ctMethod.getMethodInfo();
ConstPool cp = methodInfo.getConstPool();
//获取注解属性
AnnotationsAttribute attribute = new AnnotationsAttribute(cp, AnnotationsAttribute.visibleTag);
Annotation annotation = new Annotation("org.springframework.kafka.annotation.KafkaListener", cp);
ArrayMemberValue arrayMemberValue = new ArrayMemberValue(cp);
arrayMemberValue.setValue(new MemberValue[]{new StringMemberValue("embedded-test-topic", cp)});
annotation.addMemberValue("topics", arrayMemberValue);
annotation.addMemberValue("beanRef", new StringMemberValue(key+"_listener", cp));
annotation.addMemberValue("containerFactory", new StringMemberValue(key+"_containerFactory", cp));
attribute.addAnnotation(annotation);
methodInfo.addAttribute(attribute);
byte[] bytes = ct.toBytecode();
Class<?> clazz = ReflectUtils.defineClass("com.feng.kafka.demo.init.KafkaConsumer" + key, bytes, Thread.currentThread().getContextClassLoader());
return clazz.newInstance();
} catch (Exception e) {
throw new RuntimeException(e);
}
}通过字节码生成和动态加载class方式，生成唯一的对象，实现通过配置方式支持多数据源，不需要写一句消费代码。
单元测试
去掉了断言，因为类是动态变化的了。
总结
实际上spring-kafka已经非常完善了，spring-kafka插件的支持也很完善，不需要关注kafka的消费过程，只需要配置即可，但是也为灵活性埋下了隐患，当然一般而言我们基本上用不到多kafka的情况，也不会做一个kafka网关应用，不过当业务需要的时候，可以设计一套kafka网关应用，分发kafka的消息，起到一个流量网关的能力，解耦业务的应用，实现架构的松耦合。
![[uni-app]小兔鲜-08云开发](https://img-blog.csdnimg.cn/img_convert/b4ff268fe3ba3342a876cd8f4faf1043.png)
![[含文档+PPT+源码等]精品大数据项目-基于python爬虫实现的大数据岗位的挖掘与分析](https://img-blog.csdnimg.cn/img_convert/bcd835087bdf6be5c5115e4af597adaa.png)
