Spring AI in Action

Prompt Qualitydesign lever

The degree to which a prompt submitted to an LLM is well-structured, contains appropriate instructions, assigns correct roles (system vs. user), uses templates with filled parameters, and provides sufficient grounding context to elicit accurate and relevant responses.

RAG Configuration Qualitydesign lever

The effectiveness of the retrieval-augmented generation setup, encompassing document chunking strategy, embedding model choice, vector store selection, similarity search parameters (Top-K, similarity threshold, metadata filters), and query transformation techniques such as rewriting and expansion.

Conversational Memory Configurationdesign lever

The design choices governing how chat history is stored, retrieved, and injected into prompts, including the choice of chat memory advisor (MessageChatMemoryAdvisor, PromptChatMemoryAdvisor, or VectorStoreChatMemoryAdvisor), the persistence backend (in-memory, JDBC, Cassandra, Neo4j, vector store), the message window size, and conversation ID scoping.

Tool Integration Completenessdesign lever

The extent to which an application exposes relevant, well-described, and correctly scoped @Tool-annotated methods or Function implementations to the LLM, enabling it to retrieve real-time data or trigger application actions that it could not accomplish from training alone.

AI Security Controlsdesign lever

The breadth and depth of security mechanisms applied to the generative AI layer, including Spring Security authorization on tool methods via @PreAuthorize, metadata filters on vector store similarity searches to enforce document access controls, SafeGuardAdvisor for sensitive term blocking, CanaryWordAdvisor for prompt-leak detection, and ModerationModel usage for content policy enforcement.

Observability Instrumentation Leveldesign lever

The degree to which the Spring AI application is instrumented for operational visibility, including enabling Spring Boot Actuator metrics, configuring Micrometer Prometheus export, building Grafana dashboards for token usage and operation latency, and enabling distributed tracing with Jaeger or Zipkin to trace requests through the full AI pipeline.

Agentic Workflow Compositiondesign lever

The sophistication of the agentic architecture employed, ranging from single-prompt interactions through explicit workflow patterns (chaining, routing, parallelization) to fully autonomous planning via Embabel's Goal-Oriented Action Planning, where the agent derives its own execution path from action pre- and postconditions.

Model and Provider Selectiondesign lever

The choice of AI provider (OpenAI, Anthropic, Azure OpenAI, Ollama, Mistral, Google, etc.) and specific model (gpt-4o-mini, gpt-4.1, gemma, llama, etc.), along with tuning of generation parameters such as temperature, Top-P, and Top-K, which determine the capability ceiling, cost, context window size, and response variability of the application.

Context Relevance in Promptpsychological state

The degree to which the documents, tool results, and conversation history injected into a prompt as context are genuinely relevant to the current user query, as determined by vector similarity search quality, metadata filtering accuracy, and query transformation effectiveness.

LLM Response Groundingpsychological state

The extent to which the LLM's generated response is anchored in the provided prompt context (documents, tool outputs, memory) rather than relying solely on parametric training knowledge, thereby reducing hallucination and improving factual accuracy.

Token Efficiencybehavioral pattern

The ratio of useful information delivered to total tokens consumed per interaction, reflecting how well prompt construction, RAG chunking, and memory window sizing avoid wasting tokens on irrelevant or redundant content, directly affecting API cost and context window utilization.

Developer Productivitybehavioral pattern

The speed and ease with which a Java developer can implement, iterate on, test, and maintain generative AI features within a Spring Boot application, encompassing the reduction of boilerplate through Spring AI autoconfiguration, the portability of code across providers, and the availability of testing infrastructure.

Response Qualityoutcome metric

The overall quality of the LLM-generated response as experienced by the end user, encompassing factual accuracy (FactCheckingEvaluator), relevance to the question (RelevancyEvaluator), coherence across multi-turn conversation, appropriate tone and language, and absence of hallucinated content.

Operational Safetyoutcome metric

The degree to which the application prevents unauthorized data access through the AI layer, resists adversarial and manipulative prompting, blocks content policy violations, and ensures that tool invocations respect authorization boundaries, measured as the absence of security incidents and moderation violations.

System Observabilityoutcome metric

The operational visibility into the generative AI application's behavior, including token usage metrics, operation latency distributions, active operation counts, vector store query performance, and end-to-end distributed traces linking user requests to LLM calls, tool invocations, and vector store operations.

API Costoutcome metric

The monetary cost incurred for LLM API usage, determined by the total number of prompt tokens and completion tokens consumed across chat, embedding, and image generation operations, multiplied by the per-token pricing of the selected model and provider.

Feature Capability Breadthoutcome metric

The range of generative AI capabilities exposed by the application, including text Q&A, RAG over documents, conversational memory, real-time tool invocation, MCP server/client integration, voice transcription and synthesis, vision understanding, image generation, summarization, translation, sentiment analysis, and agentic workflows.