大模型专栏--Spring Ai Alibaba介绍和功能演示

Spring AI Alibaba 介绍和功能演示

背景

Spring AI Alibaba 开源项目基于 Spring AI 构建，是阿里云通义系列模型及服务在 Java AI 应用开发领域的最佳实践，提供高层次的 AI API 抽象与云原生基础设施集成方案，帮助开发者快速构建 AI 应用。

Spring AI Alibaba 生态图如下：

演示

在此节中，将演示如何使用 Spring AI Alibaba 提供的接口功能完成和 LLMs 的交互。

框架搭建

pom.xml

<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
			<version>3.3.4</version>
		</dependency>
    	
    	<!-- 最新版本 Spring AI Alibaba -->
		<dependency>
			<groupId>com.alibaba.cloud.ai</groupId>
			<artifactId>spring-ai-alibaba-starter</artifactId>
			<version>1.0.0-M3.1</version>
		</dependency>
	</dependencies>

	<!-- 添加仓库配置，否则报错，如果添加之后仍然报错，刷新 mvn 或者清楚 IDEA 缓存 -->
	<repositories>
		<repository>
			<id>spring-milestones</id>
			<name>Spring Milestones</name>
			<url>https://repo.spring.io/milestone</url>
			<snapshots>
				<enabled>false</enabled>
			</snapshots>
		</repository>
	</repositories>

	<!-- 解决报错： Name for argument of type [java.lang.String] not specified, and parameter name information not available via reflection. Ensure that the compiler uses the '-parameters' flag -->
	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<version>3.11.0</version>
				<configuration>
					<parameters>true</parameters>
				</configuration>
			</plugin>
		</plugins>
	</build>

</project>

application.yml

spring:
  ai:
    dashscope:
        api-key: ${AI_DASHSCOPE_API_KEY}

启动类

@SpringBootApplication
public class AIApplication {

	public static void main(String[] args) {

		SpringApplication.run(AIApplication.class, args);
	}

}

到此为止，我们已经搭建好了一个基本的 AI 应用雏形，现在开始和大模型交互 🎉🎉

只演示朴素请求，流式 API 不演示！

Chat 功能

AIChatController.java

@RestController
@RequestMapping("/ai")
public class AIChatController {

    // 使用高级 Client API，也可以使用低级 ChatModel API
	private final ChatClient chatClient;

	public AIChatController(ChatClient.Builder builder) {

		this.chatClient = builder.build();
	}

	@GetMapping("/chat/{prompt}")
	public String chatWithChatMemory(@PathVariable String prompt) {

		return chatClient.prompt()
				.user(prompt)
				.call()
				.chatResponse()
				.getResult()
				.getOutput()
				.getContent();
	}

}

如果一切顺利，请求 http://localhost:8080/ai/chat/你好接口，将得到以下输出：

你好！有什么我可以帮助你的吗？

从代码中可以看到，使用 Spring AI Alibaba 之后，和模型交互变得非常简单容易。

但是大模型是无状态的，怎么能让他变得有记忆？Spring AI 提供了 ChatMemory 的接口，只需要调用接口即可（源码将在后续文章中分析）

@RestController
@RequestMapping("/ai")
public class AIChatController {

	private final ChatClient chatClient;

	public AIChatController(ChatModel chatModel) {

		this.chatClient = ChatClient.builder(chatModel)
				.defaultAdvisors(
						new MessageChatMemoryAdvisor(new InMemoryChatMemory())
				).build();
	}

	@GetMapping("/chat/{chatId}/{prompt}")
	public String chatWithChatMemory(
			@PathVariable String chatId,
			@PathVariable String prompt
	) {

		return chatClient.prompt()
				.user(prompt)
				.advisors(
						a -> a
								.param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId)
								.param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 100)
				).call().chatResponse().getResult().getOutput().getContent();
	}

}

我们像这样请求接口：

# 1
input: http://localhost:8080/ai/chat/10001/你好，我是牧生
output：你好，牧生！很高兴认识你。你可以叫我Qwen，我是阿里云推出的一种超大规模语言模型。我有强大的语言生成和理解能力，可以进行自然流畅的对话，还能写故事、写公文、写邮件、写剧本等等，也能表达观点，玩游戏等。有什么我可以帮助你的吗？

# 2
input：http://localhost:8080/ai/chat/10001/我是谁
output：你刚才提到你的名字是牧生。如果你有任何问题或需要进一步的帮助，随时告诉我，我很乐意为你服务！

# 当切换 chatId 时
input：http://localhost:8080/ai/chat/10000/我叫什么名字
output：您还没有告诉我您的名字呢。如果您愿意，可以告诉我您希望被称为什么，或者您想如何介绍自己。

能看到借助 Spring AI 的 ChatMemory 接口，已经使大模型变得聪明了起来。 😀

PS：Spring AI Alibaba 的 Chat Memory 功能已经在规划中了！

Image 功能

本节我们将展示如何使用 Spring AI Alibaba 提供的 Image API 完成和大模型的图像交互功能，包含文生图，多模态等功能。

AIImageController.java

@RestController
@RequestMapping("/ai")
public class AIImageController {

	private final ImageModel imageModel;

	public AIImageController(ImageModel imageModel) {
		this.imageModel = imageModel;
	}

	@GetMapping("/image/{input}")
	public void image(@PathVariable("input") String input, HttpServletResponse response) {
		
		String imageUrl = imageModel
				.call(new ImagePrompt(input))
				.getResult()
				.getOutput()
				.getUrl();

		try {
			URL url = URI.create(imageUrl).toURL();
			InputStream in = url.openStream();

			response.setHeader("Content-Type", MediaType.IMAGE_PNG_VALUE);
			response.getOutputStream().write(in.readAllBytes());
			response.getOutputStream().flush();
		} catch (IOException e) {
			response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
		}
	}
	
}

请求接口 http://localhost:8080/ai/image/给我一张AI图片，将会得到以下输出：

多模态

MultiModelController.java

多模态还支持视频解析和流式 API。

@RestController
@RequestMapping("/ai")
public class AIImageController {

	private final ImageModel imageModel;

    // 加入 chatClient
	private final ChatClient client;

	public AIImageController(ImageModel imageModel, ChatClient.Builder builder) {
		this.imageModel = imageModel;
		this.client = builder.build();
	}

	@GetMapping("/image/{input}")
	public void image(@PathVariable("input") String input, HttpServletResponse response) {

		ImageOptions options = ImageOptionsBuilder.builder()
				.withModel("wanx-v1")
				.build();

		String imageUrl = imageModel
				.call(new ImagePrompt(input, options))
				.getResult()
				.getOutput()
				.getUrl();

		try {
			URL url = URI.create(imageUrl).toURL();
			InputStream in = url.openStream();

			response.setHeader("Content-Type", MediaType.IMAGE_PNG_VALUE);
			response.getOutputStream().write(in.readAllBytes());
			response.getOutputStream().flush();
		} catch (IOException e) {
			response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
		}
	}

	@GetMapping("/image")
	public String image(
			@RequestParam(value = "prompt", required = false, defaultValue = "图片里是什么")
			String prompt
	) throws Exception {

        // 图片资源 同时支持读取本地文件作为输入
		List<Media> mediaList = List.of(
				new Media(
						MimeTypeUtils.IMAGE_PNG,
						new URI("https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg").toURL()
				)
		);

		UserMessage message = new UserMessage(prompt, mediaList);
		message.getMetadata().put(DashScopeChatModel.MESSAGE_FORMAT, MessageFormat.IMAGE);

		ChatResponse response = client.prompt(
				new Prompt(
						message,
						DashScopeChatOptions.builder()
								.withModel("qwen-vl-max-latest")
								.withMultiModel(true)
								.build()
				)
		).call().chatResponse();

		return response.getResult().getOutput().getContent();
	}

}

请求 http://localhost:8080/ai/image 接口，将得到以下输出：

这张图片展示了一位女士和一只狗在海滩上互动的温馨场景。女士坐在沙滩上，面带微笑，与狗握手。狗戴着项圈，显得非常温顺和友好。背景是广阔的海洋和天空，阳光洒在沙滩上，营造出一种温暖和谐的氛围。

Audio 功能

本节我们将展示如何使用 Spring AI Alibaba 提供的 Audio API 完成和大模型的音频交互功能，包含文生语音，语音转文字等功能。

截止文章发布期间，Spring AI Alibaba 的语音转录接口还没有发版，有关 stt 和 tts 的更多使用，参考官方 example。

https://github.com/alibaba/spring-ai-alibaba/tree/main/spring-ai-alibaba-examples/audio-example/src/main/java/com/alibaba/cloud/ai/example/audio

AIAudioController.java

@RestController
@RequestMapping("/ai")
public class AIAudioController implements ApplicationRunner {

	private final SpeechSynthesisModel speechSynthesisModel;

	private static final String TEXT = "白日依山尽，黄河入海流。";

	private static final String FILE_PATH = "src/main/resources/gen/tts/";

	private AIAudioController(
			SpeechSynthesisModel speechSynthesisModel
	) {
		this.speechSynthesisModel = speechSynthesisModel;
	}

	@GetMapping("/tts")
	public void tts() throws IOException {

		SpeechSynthesisResponse response = speechSynthesisModel.call(
				new SpeechSynthesisPrompt(TEXT)
		);

		File file = new File(FILE_PATH + "output.mp3");
		try (FileOutputStream fos = new FileOutputStream(file)) {
			ByteBuffer byteBuffer = response.getResult().getOutput().getAudio();
			fos.write(byteBuffer.array());
		}
		catch (IOException e) {
			throw new IOException(e.getMessage());
		}
	}

	@Override
	public void run(ApplicationArguments args) {

		File file = new File(FILE_PATH);
		if (!file.exists()) {
			file.mkdirs();
		}
	}

	@PreDestroy
	public void destroy() throws IOException {

		FileUtils.deleteDirectory(new File(FILE_PATH));
	}

}

请求接口，将得到一个语音文件的输出。

函数调用

函数调用是为了弥补大模型的训练数据落后的问题，用外部的 API 来补充 LLMs 的知识，给用户最合理的回答。

天气函数注册 MockWeatherService.java

public class MockWeatherService implements Function<MockWeatherService.Request, Response> {

	@Override
	public Response apply(Request request) {
		if (request.city().contains("杭州")) {
			return new Response(String.format("%s%s晴转多云, 气温32摄氏度。", request.date(), request.city()));
		}
		else if (request.city().contains("上海")) {
			return new Response(String.format("%s%s多云转阴, 气温31摄氏度。", request.date(), request.city()));
		}
		else {
			return new Response(String.format("暂时无法查询%s的天气状况。", request.city()));
		}
	}

	@JsonInclude(JsonInclude.Include.NON_NULL)
	@JsonClassDescription("根据日期和城市查询天气")
	public record Request(
			@JsonProperty(required = true, value = "city") @JsonPropertyDescription("城市, 比如杭州") String city,
			@JsonProperty(required = true, value = "date") @JsonPropertyDescription("日期, 比如2024-08-22") String date) {
	}

}

紧接着，我们在 AIChatController 中加入函数调用的代码：

@RestController
@RequestMapping("/ai")
public class AIChatController {

	private final ChatClient chatClient;

	public AIChatController(ChatModel chatModel) {

		this.chatClient = ChatClient.builder(chatModel)
				.defaultAdvisors(
						new MessageChatMemoryAdvisor(new InMemoryChatMemory())
				).build();
	}

	@GetMapping("/chat/{chatId}/{prompt}")
	public String chatWithChatMemory(
			@PathVariable String chatId,
			@PathVariable String prompt
	) {

		return chatClient.prompt()
				.user(prompt)
				.advisors(
						a -> a
								.param(CHAT_MEMORY_CONVERSATION_ID_KEY, chatId)
								.param(CHAT_MEMORY_RETRIEVE_SIZE_KEY, 100)
				).call().chatResponse().getResult().getOutput().getContent();
	}

    // 函数调用
	@GetMapping("/weather-service/{city}")
	public String weatherService(@PathVariable String city) {
		return chatClient.prompt()
				.function("getWeather", "根据城市查询天气", new MockWeatherService())
				.user(city)
				.call()
				.content();
	}

}

请求 http://localhost:8080/ai/weather-service/杭州2024年11月29日天气怎么样 接口，将得到如下响应：

2024年11月29日，杭州的天气预报为晴转多云，气温为32摄氏度。请根据天气情况做好相应的准备。如果您有其他问题，欢迎随时询问！

RAG

本节中我们将使用 ES 作为 RAG 的实现，演示 Spring AI Alibaba RAG 的实现。

在 resource 目录下准备一个 system-qa.st

Context information is below.
---------------------
{question_answer_context}
---------------------
Given the context and provided history information and not prior knowledge,
reply to the user comment. If the answer is not in the context, inform
the user that you can't answer the question.

之后准备一个 df 文件，点击这里下载：https://github.com/alibaba/spring-ai-alibaba/blob/main/spring-ai-alibaba-examples/rag-example/src/main/resources/data/spring_ai_alibaba_quickstart.pdf

使用 docker compose up -d 启动一个 es：

准备配置文件：

config/es.yaml

cluster.name: docker-es
node.name: es-node-1
network.host: 0.0.0.0
network.publish_host: 0.0.0.0
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: true

# 关闭认证授权 es 8.x 默认开启
# 如果不关闭，spring boot 连接会 connection closed
xpack.security.enabled: false

docker-compose.yaml

version: '3.3'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.16.1
    container_name: elasticsearch
    privileged: true
    environment:
      - "cluster.name=elasticsearch"
      - "discovery.type=single-node"
      - "ES_JAVA_OPTS=-Xms512m -Xmx1096m"
      - bootstrap.memory_lock=true
    volumes:
      - ./config/es.yaml:/usr/share/elasticsearch/config/elasticsearch.yml
    ports:
      - "9200:9200"
      - "9300:9300"
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: 1000M
        reservations:
          memory: 200M

application.yml 中加入 rag 相关配置：

server:
  port: 9097

spring:
  ai:
    dashscope:
        api-key: ${AI_DASHSCOPE_API_KEY}

    vectorstore:
      elasticsearch:
        index-name: spring-ai-alibaba-index
        similarity: cosine
        dimensions: 1536
        initialize-schema: true

pom.xml 中加入如下配置：

<dependency>
	<groupId>org.springframework.ai</groupId>
	<artifactId>spring-ai-pdf-document-reader</artifactId>
	<version>1.0.0-M3</version>
</dependency>

<dependency>
	<groupId>org.springframework.ai</groupId>
	<artifactId>spring-ai-elasticsearch-store-spring-boot-starter</artifactId>
	<version>1.0.0-M3</version>
</dependency>

AIRagController.java

package indi.yuluo.controller;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch._types.mapping.DenseVectorProperty;
import co.elastic.clients.elasticsearch._types.mapping.KeywordProperty;
import co.elastic.clients.elasticsearch._types.mapping.ObjectProperty;
import co.elastic.clients.elasticsearch._types.mapping.Property;
import co.elastic.clients.elasticsearch._types.mapping.TextProperty;
import co.elastic.clients.elasticsearch._types.mapping.TypeMapping;
import co.elastic.clients.elasticsearch.indices.CreateIndexResponse;
import co.elastic.clients.elasticsearch.indices.IndexSettings;
import com.alibaba.cloud.ai.advisor.RetrievalRerankAdvisor;
import com.alibaba.cloud.ai.model.RerankModel;
import jakarta.servlet.http.HttpServletResponse;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import reactor.core.publisher.Flux;

import org.springframework.ai.autoconfigure.vectorstore.elasticsearch.ElasticsearchVectorStoreProperties;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.document.Document;
import org.springframework.ai.document.DocumentReader;
import org.springframework.ai.reader.pdf.PagePdfDocumentReader;
import org.springframework.ai.transformer.splitter.TokenTextSplitter;
import org.springframework.ai.vectorstore.SearchRequest;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.ai.vectorstore.filter.FilterExpressionBuilder;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.core.io.Resource;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

/**
 * @author yuluo
 * @author <a href="mailto:[email protected]">yuluo</a>
 */

@RestController
@RequestMapping("/ai")
public class AIRagController implements ApplicationRunner {

	private static final Logger logger = LoggerFactory.getLogger(AIRagController.class);

	@Value("classpath:/data/spring_ai_alibaba_quickstart.pdf")
	private Resource PdfResource;

	@Value("classpath:/prompts/system-qa.st")
	private Resource systemResource;

	private static final String textField = "content";

	private static final String vectorField = "embedding";

	private final ChatModel chatModel;

	private final VectorStore vectorStore;

	private final RerankModel rerankModel;

	private final ElasticsearchClient elasticsearchClient;

	private final ElasticsearchVectorStoreProperties options;

	public AIRagController(
			ChatModel chatModel,
			VectorStore vectorStore,
			RerankModel rerankModel,
			ElasticsearchClient elasticsearchClient,
			ElasticsearchVectorStoreProperties options
	) {

		this.chatModel = chatModel;
		this.vectorStore = vectorStore;
		this.rerankModel = rerankModel;
		this.elasticsearchClient = elasticsearchClient;
		this.options = options;
	}

	@GetMapping("/rag")
	public Flux<String> generate(
			@RequestParam(value = "message", defaultValue = "how to get start with spring ai alibaba?")
			String message,
			HttpServletResponse response
	) throws IOException {

        // 不设置返回值会乱码
		response.setCharacterEncoding(StandardCharsets.UTF_8.name());
		return this.retrieve(message).map(x -> x.getResult().getOutput().getContent());
	}

	private Flux<ChatResponse> retrieve(String message) throws IOException {
		// Enable hybrid search, both embedding and full text search
		SearchRequest searchRequest = SearchRequest.defaults()
				.withFilterExpression(new FilterExpressionBuilder().eq(textField, message).build());

		// Step3 - Retrieve and llm generate
		String promptTemplate = systemResource.getContentAsString(StandardCharsets.UTF_8);;
		ChatClient chatClient = ChatClient
				.builder(chatModel)
				.defaultAdvisors(
						new RetrievalRerankAdvisor(
								vectorStore,
								rerankModel,
								searchRequest,
								promptTemplate,
								0.1)
				).build();

		return chatClient.prompt().user(message).stream().chatResponse();
	}

	@Override
	public void run(ApplicationArguments args) throws Exception {

		// 1. parse document
		DocumentReader reader = new PagePdfDocumentReader(PdfResource);
		List<Document> documents = reader.get();
		logger.info("{} documents loaded", documents.size());

		// 2. split trunks
		List<Document> splitDocuments = new TokenTextSplitter().apply(documents);
		logger.info("{} documents split", splitDocuments.size());

		// 3. create embedding and store to vector store
		logger.info("create embedding and save to vector store");
		createIndexIfNotExists();
		vectorStore.add(splitDocuments);
	}

	private void createIndexIfNotExists() {

		try {
			String indexName = options.getIndexName();
			Integer dimsLength = options.getDimensions();

			if (Objects.isNull(indexName) || indexName.isEmpty()) {
				throw new IllegalArgumentException("Elastic search index name must be provided");
			}

			boolean exists = elasticsearchClient.indices().exists(idx -> idx.index(indexName)).value();
			if (exists) {
				logger.debug("Index {} already exists. Skipping creation.", indexName);
				return;
			}

			String similarityAlgo = options.getSimilarity().name();
			IndexSettings indexSettings = IndexSettings.of(
					settings -> settings.numberOfShards(
							String.valueOf(1)
					).numberOfReplicas(String.valueOf(1)));

			Map<String, Property> properties = new HashMap<>();
			properties.put(vectorField, Property.of(
					property -> property.denseVector(
							DenseVectorProperty.of(
									dense -> dense.index(true)
											.dims(dimsLength)
											.similarity(similarityAlgo)
								)
						)
				)
			);
			properties.put(textField, Property.of(property -> property.text(TextProperty.of(t -> t))));

			Map<String, Property> metadata = new HashMap<>();
			metadata.put("ref_doc_id", Property.of(property -> property.keyword(KeywordProperty.of(k -> k))));

			properties.put("metadata",
					Property.of(property -> property.object(ObjectProperty.of(op -> op.properties(metadata)))));

			CreateIndexResponse indexResponse = elasticsearchClient.indices()
					.create(createIndexBuilder -> createIndexBuilder.index(indexName)
							.settings(indexSettings)
							.mappings(TypeMapping.of(mappings -> mappings.properties(properties))));

			if (!indexResponse.acknowledged()) {
				throw new RuntimeException("failed to create index");
			}

			logger.info("create elasticsearch index {} successfully", indexName);
		}
		catch (IOException e) {
			logger.error("failed to create index", e);
			throw new RuntimeException(e);
		}
	}

}

之后，请求 http://localhost:8080/ai/rag 接口，将得到如下响应：

根据提供的上下文信息，以下是开始使用 Spring AI Alibaba 的步骤： ### 概述 Spring AI Alibaba 实现了与阿里云通义模型的完整适配。下面将介绍如何使用 Spring AI Alibaba 开发一个基于通义模型服务的智能聊天应用。 ### 快速体验示例 #### 注意事项 - **JDK 版本**：因为 Spring AI Alibaba 基于 Spring Boot 3.x 开发，所以本地 JDK 版本要求为 17 及以上。 #### 步骤 1. **下载项目** - 运行以下命令下载源码，并进入 `helloworld` 示例目录： ```sh git clone --depth=1 https://github.com/alibaba/spring-ai-alibaba.git cd spring-ai-alibaba/spring-ai-alibaba-examples/helloworld-example ```2. **运行项目** - 首先，需要获取一个合法的 API-KEY 并设置 `AI_DASHSCOPE_API_KEY` 环境变量。你可以跳转到 [阿里云百炼平台](https://sca.aliyun.com/) 了解如何获取 API-KEY。 ```sh export AI_DASHSCOPE_API_KEY=${REPLACE-WITH-VALID-API-KEY} ```- 启动示例应用： ```sh ./mvnw compile exec:java -Dexec.mainClass="com.alibaba.cloud.ai.example.helloworld.HelloWorldExample" ```3. **访问应用** - 打开浏览器，访问 `http://localhost:8080/ai/chat?input=给我讲一个笑话吧`，向通义模型提问并得到回答。 希望这些步骤能帮助你快速上手 Spring AI Alibaba！如果有任何问题，可以随时提问。

总结

Spring AI Alibaba 基于 Spring AI 开发，并在上层提供更多高级的抽象 API。帮助开发者构建 Java LLMs 应用。

在这里插入图片描述