Orange 开源项目 - 集成百度智能云-千帆大模型

1 集成百度智能云-千帆大模型

百度智能云-千帆ModelBuilder百度智能云千帆大模型服务与开发平台ModelBuilder（以下简称千帆ModelBuilder）是面向企业开发者的一站式大模型开发及服务运行平台。千帆ModelBuilder不仅提供了包括文心一言底层模型和第三方开源大模型，还提供了各种AI开发工具和整套开发环境，方便客户轻松使用和开发大模型应用。

2 集成百度智能云-千帆大模型

2.1 获取API-KEY

获取百度智能云-千帆AK/SK,请参考百度智能云千帆官网「获取应用的AK和SK」

2.2 引入Maven依赖

<dependency>
     <groupId>org.springframework.ai</groupId>
     <artifactId>spring-ai-qianfan-spring-boot-starter</artifactId>
     <version>1.0.0-M5</version>
 </dependency>

2.3 集成对话模型(Chat Model)

 public class QianFanChatModelServiceImpl implements ChatModelService {

    private final QianFanChatModel chatModel;

    @Override
    public PlatformEnum getPlatform() {
        return PlatformEnum.QIAN_FAN;
    }

    @Override
    public Flux<Result<ConversationReplyVO>> conversationStream(ConversationParam param) {
        return conversationStream(param, List.of());
    }

    @Override
    public Flux<Result<ConversationReplyVO>> conversationStream(ConversationParam param, List<ChatSessionRecordVO> contextMessageList) {
        List<Message> messages = new ArrayList<>(MessageConverter.toMessageList(contextMessageList));
        messages.add(new UserMessage(param.getPrompt()));

        Prompt prompt = new Prompt(messages, QianFanChatOptions.builder()
                .model(param.getModelCode())
                .build());
        Flux<ChatResponse> stream = chatModel.stream(prompt);
        return stream
                .takeWhile(chatResponse -> Objects.nonNull(chatResponse) && Objects.nonNull(chatResponse.getResult())
                        && Objects.nonNull(chatResponse.getResult().getOutput()))
                .map(chatResponse -> {
                    if (log.isDebugEnabled()) {
                        log.debug("chatResponse: {}", chatResponse);
                    }
                    Usage usage = chatResponse.getMetadata().getUsage();
                    String content = chatResponse.getResult().getOutput().getContent();
                    ConversationReplyVO replyVO = ConversationReplyVO.builder()
                            .content(content)
                            .tokenUsage(TokenUsageVO.builder()
                                    .promptTokens(usage.getPromptTokens())
                                    .generationTokens(usage.getGenerationTokens())
                                    .totalTokens(usage.getTotalTokens())
                                    .build())
                            .build();
                    return ResultWrapper.ok(replyVO);
                });
    }

}