企业官网建设流程全解析

如何用 FastAPI 与 Outlines 的异步 vLLM 模型部署结构化客服 API？

【免费下载链接】outlinesStructured Outputs项目地址: https://gitcode.com/GitHub_Trending/ou/outlines

这篇文章解决的问题是：用 Outlines 的models.from_vllm异步集成，在 FastAPI 中构建一个客服 API，它对工单做两类结构化操作——分析工单（输出类别、优先级、摘要、情绪、关键问题、是否需人工介入），以及生成结构化客服回复。前提是已有一个运行中的 vLLM 服务器（本地或远程），且 Outlines 结构化输出要求 vLLM 服务器版本不低于 0.12。

参考文档：docs/guide/fastapi_vllm_deployment.md。

准备条件

在本地或远程启动好 vLLM 服务器，然后安装应用所需依赖：

pip install fastapi uvicorn outlines openai pydantic

按文档给出的目录组织应用代码：

models.py # Pydantic 数据模型 main.py # FastAPI 应用 prompts/ categorize.txt # 工单分析提示词模板 respond.txt # 生成回复的提示词模板

定义 Pydantic 数据模型

models.py定义两个枚举和两个模型。TicketAnalysis是分析接口的输出结构，SupportResponse是回复接口的输出结构：

# models.py from enum import Enum from typing import List from pydantic import BaseModel, Field class TicketCategory(str, Enum): BILLING = "billing" TECHNICAL = "technical" ACCOUNT = "account" PRODUCT = "product" OTHER = "other" class TicketPriority(str, Enum): LOW = "low" MEDIUM = "medium" HIGH = "high" URGENT = "urgent" class TicketAnalysis(BaseModel): category: TicketCategory priority: TicketPriority summary: str = Field(description="Brief summary of the issue") customer_sentiment: str = Field(description="Customer emotional state") key_issues: List[str] = Field(description="List of main problems") requires_human: bool = Field(description="Whether this needs human intervention") class SupportResponse(BaseModel): greeting: str acknowledgment: str = Field(description="Acknowledge the customer's issue") solution_steps: List[str] = Field(description="Steps to resolve the issue") closing: str

这两个模型既用于 FastAPI 的response_model响应校验，也直接作为 Outlines 的结构化输出类型传入async_model(...)。

准备 Jinja 提示词模板

Outlines 的Template使用 Jinja 2 语法。把提示词单独放在prompts/目录下，便于修改和版本管理。文档将提示词与应用实现分离，两个模板如下：

{# prompts/categorize.txt #} Analyze this customer support ticket: Customer ID: {{ customer_id }} Message: {{ message }} Extract the category, priority, and other relevant information.

{# prompts/respond.txt #} Generate a professional customer support response. Customer Message: {{ message }} Category: {{ category }} Priority: {{ priority }} Customer Sentiment: {{ customer_sentiment }} Create a helpful, empathetic response that addresses their concerns.

注意Template.from_file的 Jinja 环境使用StrictUndefined（见 src/outlines/templates.py），模板变量必须全部提供，缺少变量会直接报错而不是渲染为空串。

编写 FastAPI 应用

main.py的关键点：用lifespan函数在应用启动时初始化异步 vLLM 模型，客户端用openai.AsyncOpenAI指向 vLLM 服务器，base_url按实际服务器地址调整：

# main.py import asyncio from contextlib import asynccontextmanager from typing import Optional import openai from outlines import models, Template from fastapi import FastAPI, HTTPException from pydantic import BaseModel from models import TicketAnalysis, SupportResponse # Request model class TicketRequest(BaseModel): customer_id: str message: str # Global model instance async_model = None # The lifespan function is a FastAPI construct # used to define startup and shutdown logic for the API. @asynccontextmanager async def lifespan(app: FastAPI): """Initialize the async vLLM model on startup.""" global async_model client = openai.AsyncOpenAI( base_url="http://localhost:8000/v1", # Adjust to your vLLM server URL api_key="dummy" # vLLM doesn't require a real API key ) async_model = models.from_vllm(client, "Qwen/Qwen2.5-VL-7B-Instruct") yield async_model = None # Cleanup # Create FastAPI app app = FastAPI( title="Customer Support Assistant API", description="AI-powered customer support with structured outputs", version="1.0.0", lifespan=lifespan ) @app.post("/analyze-ticket", response_model=TicketAnalysis) async def analyze_ticket(request: TicketRequest): """Analyze a customer support ticket and extract structured information.""" if async_model is None: raise HTTPException(status_code=503, detail="Model not initialized") template = Template.from_file("prompts/categorize.txt") prompt = template( customer_id=request.customer_id, message=request.message ) try: # Generate and parse a structured response result = await async_model(prompt, TicketAnalysis, max_tokens=5000) analysis = TicketAnalysis.model_validate_json(result) return analysis except Exception as e: raise HTTPException(status_code=500, detail=f"Analysis failed: {str(e)}") @app.post("/generate-response", response_model=SupportResponse) async def generate_response( request: TicketRequest, analysis: TicketAnalysis ): """Generate a structured support response based on ticket analysis.""" if async_model is None: raise HTTPException(status_code=503, detail="Model not initialized") template = Template.from_file("prompts/respond.txt") prompt = template( message=request.message, category=analysis.category, priority=analysis.priority, customer_sentiment=analysis.customer_sentiment ) try: # Generate and parse a structured response result = await async_model(prompt, SupportResponse, max_tokens=5000) response = SupportResponse.model_validate_json(result) return response except Exception as e: raise HTTPException(status_code=500, detail=f"Response generation failed: {str(e)}")

代码中两处需要你按实际环境调整的值：

• base_url：文档默认http://localhost:8000/v1，改成你的 vLLM 服务器地址；
• 模型名：文档示例为Qwen/Qwen2.5-VL-7B-Instruct，必须与 vLLM 服务器加载的模型一致。

async_model(prompt, TicketAnalysis, max_tokens=5000)的第二个参数就是结构化输出类型，Outlines 会把它转换成对 vLLM 的约束请求；返回结果是 JSON 字符串，再用model_validate_json解析回 Pydantic 模型。

启动 vLLM 服务器与 FastAPI 应用

vllm serve Qwen/Qwen2.5-VL-7B-Instruct

vLLM 服务器起来后，在应用目录启动 FastAPI：

uvicorn main:app --reload --host 0.0.0.0 --port 8080

main.py里Template.from_file("prompts/categorize.txt")是相对路径，所以要在prompts/目录所在的目录下运行 uvicorn。

验证接口

用文档给出的请求测试/analyze-ticket：

curl -X POST "http://localhost:8080/analyze-ticket" \ -H "Content-Type: application/json" \ -d '{ "customer_id": "CUST123", "message": "I have been charged twice for my subscription this month. This is unacceptable and I want a refund immediately!" }'

文档示例的返回如下（这是文档示例输出，实际字段值会随模型响应变化）：

{ "category": "billing", "priority": "high", "summary": "Customer charged twice for subscription, requesting refund", "customer_sentiment": "angry", "key_issues": ["duplicate charge", "subscription billing", "refund request"], "requires_human": false }

第二个接口/generate-response需要把工单和上一步的分析结果一起放进请求体，文档给出的调用方式：

# First, get the analysis ANALYSIS=$(curl -s -X POST "http://localhost:8080/analyze-ticket" \ -H "Content-Type: application/json" \ -d '{ "customer_id": "CUST456", "message": "My app keeps crashing when I try to upload photos." }') # Then generate a response curl -X POST "http://localhost:8080/generate-response" \ -H "Content-Type: application/json" \ -d "{ \"request\": { \"customer_id\": \"CUST456\", \"message\": \"My app keeps crashing when I try to upload photos.\" }, \"analysis\": $ANALYSIS }"

注意/generate-response的请求体结构与/analyze-ticket不同：前者是{"request": {...}, "analysis": {...}}（两个 body 参数request和analysis），后者的 body 直接是customer_id和message字段。

接口异常时的判断依据来自代码本身：模型未初始化时返回 503Model not initialized；生成或解析失败时返回 500，detail 中带Analysis failed: ...或Response generation failed: ...。

版本限制与排错要点

• vLLM 版本：Outlines 通过structured_outputs请求字段下发约束，该字段要求 vLLM 服务器 >= 0.12。旧版本服务器会静默忽略该字段并返回无约束输出，不报错——如果返回值不符合 Pydantic 结构，先检查 vLLM 服务器版本。依据见 from_vllm 的 docstring。
• 模型名不匹配：from_vllm(client, "Qwen/Qwen2.5-VL-7B-Instruct")中的模型名会作为请求的model参数发给 vLLM 服务器，需与vllm serve加载的模型一致。
• 端口约定：文档默认 vLLM 在 8000 端口（http://localhost:8000/v1），FastAPI 应用监听 8080 端口；base_url和--port按实际部署调整。

可选：切换到 SGLang 或 TGI 后端

文档同时给出两个可选分支：只修改lifespan中的模型初始化，FastAPI 端点、错误处理和业务逻辑保持不变。

用 SGLang 时，客户端改为指向 SGLang 服务器：

@asynccontextmanager async def lifespan(app: FastAPI): """Initialize the async SGLang model on startup.""" global async_model client = openai.AsyncOpenAI( base_url="http://localhost:30000/v1", # SGLang server URL api_key="dummy" ) async_model = models.from_sglang(client) yield async_model = None

对应启动命令：

python -m sglang.launch_server \ --model-path meta-llama/Llama-2-7b-chat-hf \ --port 30000

用 TGI 时改用 Hugging Face 客户端：

import huggingface_hub @asynccontextmanager async def lifespan(app: FastAPI): """Initialize the async TGI model on startup.""" global async_model client = huggingface_hub.AsyncInferenceClient( "http://localhost:8080" # TGI server URL ) async_model = models.from_tgi(client) yield async_model = None

对应启动命令：

docker run --gpus all -p 8080:80 \ ghcr.io/huggingface/text-generation-inference:latest \ --model-id meta-llama/Llama-2-7b-chat-hf

注意 TGI 示例把容器端口映射到宿主机 8080，与主路径中 FastAPI 应用的默认端口相同；实际部署时两处端口需要自行错开，AsyncInferenceClient的地址也要指向 TGI 实际监听端口。

下一步

完成上述步骤后，/analyze-ticket与/generate-response两个接口即按TicketAnalysis、SupportResponse的结构返回校验过的 JSON。若要把提示词、输出结构换成其他客服流程，只需替换models.py中的 Pydantic 模型与prompts/下的 Jinja 模板，main.py的端点逻辑与 Outlines 集成方式不变。

【免费下载链接】outlinesStructured Outputs项目地址: https://gitcode.com/GitHub_Trending/ou/outlines