Sap business ai joule

SAP Business AI & Joule Development

Related Skills

• sap-hana-cloud — Vector engine for embeddings, HANA Cloud as knowledge store
• sap-rap-comprehensive — RAP-based data access for grounding AI with SAP data
• sap-cap-advanced — CAP MCP plugin for AI-assisted development
• sap-build-apps — AI-powered low-code app generation
• sap-integration-suite-advanced — AI-assisted mapping in Integration Advisor

Quick Start

Choose your AI scenario:

Scenario	Service	Entry Point
Custom ML model training/serving	AI Core	AI Launchpad → ML Operations
LLM orchestration (chat, completion)	Generative AI Hub	AI Core API / orchestration
Embed AI in SAP standard apps	Joule	Extension Center / Joule Studio
RAG with SAP data	GenAI Hub + HANA Vector	Orchestration service
Document extraction	Document Information Extraction	BTP service instance

Minimal GenAI Hub call (Python):

from gen_ai_hub.proxy.core.proxy_clients import get_proxy_client
from gen_ai_hub.proxy.langchain import ChatOpenAI

proxy_client = get_proxy_client('gen-ai-hub')

llm = ChatOpenAI(
    proxy_model_name='gpt-4o',
    proxy_client=proxy_client,
    temperature=0.0
)

response = llm.invoke("Summarize SAP S/4HANA extensibility options")
print(response.content)

Core Concepts

SAP AI Core Architecture

• Resource groups: Isolated execution environments (multi-tenant)
• Configurations: Define which model/pipeline + parameters to use
• Deployments: Running model inference endpoints
• Executions: One-time training or batch jobs
• Artifacts: Models, datasets registered in AI Core

Generative AI Hub

• Proxy access: Unified API for multiple LLM providers (OpenAI, Azure OpenAI, Anthropic, Google, AWS Bedrock)
• Orchestration service: Chain LLM calls with grounding, content filtering, templating
• Prompt registry: Version-controlled prompt templates
• Content filtering: Input/output moderation (hate, self-harm, sexual, violence)

Joule Architecture

• Joule Foundations: Core capabilities (NLU, context management, response generation)
• Joule Skills: Discrete capabilities mapped to SAP business actions
• Extension Center: Register custom skills for Joule
• Guided answers: Structured multi-turn flows for complex tasks

Vector Engine (HANA Cloud)

• Native REAL_VECTOR data type (up to 5000 dimensions)
• Distance functions: COSINE_SIMILARITY, L2DISTANCE, INNER_PRODUCT
• HNSW index for approximate nearest neighbor (ANN)
• Integrated with SAP GenAI Hub embedding models

Common Patterns

Pattern 1: Orchestration Service — Templating + Grounding + Filtering

from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage
from gen_ai_hub.orchestration.models.template import Template, TemplateValue
from gen_ai_hub.orchestration.models.llm import LLM
from gen_ai_hub.orchestration import OrchestrationClient

llm = LLM(name="gpt-4o", version="latest", parameters={"max_tokens": 1000, "temperature": 0.2})

template = Template(
    messages=[
        SystemMessage("You are an SAP expert assistant. Answer based on the provided context only."),
        UserMessage("Context: {{?context}}\n\nQuestion: {{?question}}")
    ],
    defaults=[TemplateValue(name="context", value="No context provided")]
)

client = OrchestrationClient(llm=llm, template=template)

response = client.run(
    template_values=[
        TemplateValue(name="context", value="S/4HANA supports tier-1 (key user) and tier-2 (developer) extensibility..."),
        TemplateValue(name="question", value="What extensibility tiers does S/4HANA support?")
    ]
)
print(response.content)

Pattern 2: RAG with HANA Cloud Vector Engine

from gen_ai_hub.proxy.core.proxy_clients import get_proxy_client
from gen_ai_hub.proxy.langchain import OpenAIEmbeddings, ChatOpenAI
from hdbcli import dbapi

proxy_client = get_proxy_client('gen-ai-hub')
embeddings = OpenAIEmbeddings(proxy_model_name='text-embedding-ada-002', proxy_client=proxy_client)

# 1. Embed the query
query = "How do I create a custom CDS view extension?"
query_vector = embeddings.embed_query(query)

# 2. Search HANA Cloud vector store
conn = dbapi.connect(address='<host>', port=443, user='<user>', password='<pwd>', encrypt=True)
cursor = conn.cursor()
cursor.execute("""
    SELECT TOP 5 "CONTENT",
           COSINE_SIMILARITY("EMBEDDING", TO_REAL_VECTOR(?)) AS score
    FROM "KNOWLEDGE_BASE"
    ORDER BY score DESC
""", [str(query_vector)])
chunks = [row[0] for row in cursor.fetchall()]

# 3. Generate answer with context
llm = ChatOpenAI(proxy_model_name='gpt-4o', proxy_client=proxy_client, temperature=0.0)
context = "\n---\n".join(chunks)
response = llm.invoke(f"Context:\n{context}\n\nQuestion: {query}\n\nAnswer based on context only:")
print(response.content)

Pattern 3: Content Filtering Configuration

from gen_ai_hub.orchestration.models.content_filter import ContentFilter, AzureFilterThreshold

input_filter = ContentFilter(
    provider="azure",
    hate=AzureFilterThreshold.ALLOW_SAFE,
    self_harm=AzureFilterThreshold.ALLOW_SAFE,
    sexual=AzureFilterThreshold.ALLOW_SAFE,
    violence=AzureFilterThreshold.ALLOW_SAFE
)

output_filter = ContentFilter(
    provider="azure",
    hate=AzureFilterThreshold.ALLOW_SAFE,
    self_harm=AzureFilterThreshold.ALLOW_SAFE,
    sexual=AzureFilterThreshold.ALLOW_SAFE,
    violence=AzureFilterThreshold.ALLOW_SAFE_LOW
)

client = OrchestrationClient(
    llm=llm,
    template=template,
    input_filter=input_filter,
    output_filter=output_filter
)

Pattern 4: CAP Plugin for AI (Node.js)

// package.json — add cap-llm-plugin
// "dependencies": { "@cap-js/hana": "^1", "cap-llm-plugin": "^1" }

// srv/ai-service.js
const cds = require('@sap/cds');

module.exports = class AIService extends cds.ApplicationService {
  async init() {
    this.on('askQuestion', async (req) => {
      const { question } = req.data;
      const vectorPlugin = await cds.connect.to('cap-llm-plugin');

      // RAG: retrieve + generate
      const response = await vectorPlugin.getRagResponse(
        question,
        'KNOWLEDGE_BASE',  // HANA table with embeddings
        'EMBEDDING',        // vector column
        'CONTENT',          // text column
        'text-embedding-ada-002',
        'gpt-4o',
        5  // top-k
      );

      return { answer: response };
    });
    await super.init();
  }
};

Pattern 5: Joule Custom Skill Definition

{
  "name": "lookup-material",
  "description": "Look up material master data by material number or description",
  "parameters": {
    "type": "object",
    "properties": {
      "materialNumber": {
        "type": "string",
        "description": "SAP material number (e.g., MAT-001)"
      },
      "searchTerm": {
        "type": "string",
        "description": "Free text search term for material description"
      }
    }
  },
  "endpoint": {
    "url": "https://<app>.cfapps.<region>.hana.ondemand.com/api/materials/search",
    "method": "POST",
    "authentication": "OAuth2ClientCredentials"
  }
}

Error Catalog

Error	Message	Root Cause	Fix
401 Unauthorized	JWT token validation failed	AI Core service key expired or wrong	Regenerate service key in BTP Cockpit
429 Too Many Requests	Rate limit exceeded	Too many LLM calls per minute	Implement retry with exponential backoff; check quota
404 Deployment not found	No running deployment	Model not deployed or deployment scaled to 0	Check AI Launchpad → Deployments; redeploy
VECTOR_DIM_MISMATCH	Dimension mismatch	Query vector dimensions ≠ stored vector dimensions	Ensure same embedding model for indexing and querying
Content filtered	Output blocked by content filter	Response triggered moderation	Adjust filter thresholds or rephrase prompt
RESOURCE_EXHAUSTED	Resource group quota exceeded	Too many concurrent deployments	Delete unused deployments; request quota increase

Performance Tips

1. Batch embeddings — Embed documents in batches of 100-500; single calls are 10-50x slower
2. Cache embeddings — Store in HANA REAL_VECTOR column; never re-embed unchanged content
3. HNSW index — Create for vector columns with >10K rows: CREATE HNSW VECTOR INDEX ON "TABLE"("COL")
4. Chunk size — 512-1024 tokens per chunk for RAG; too small loses context, too large dilutes relevance
5. Streaming — Use streaming responses for chat UIs to reduce perceived latency
6. Model selection — Use smaller models (GPT-4o-mini, Claude Haiku) for classification/extraction; larger for reasoning
7. Prompt caching — Orchestration service caches prompt templates; reuse templates with variable substitution
8. Connection pooling — Reuse AI Core proxy client instances; don't create per request

Gotchas

• Resource group isolation: Models deployed in one resource group are NOT accessible from another
• Token limits: Orchestration service has max token limits per model; check max_tokens in deployment config
• Embedding model consistency: If you change embedding model, you MUST re-embed all existing documents
• GenAI Hub model availability: Not all models available in all regions; check SAP Discovery Center
• HANA vector index: HNSW index build is CPU-intensive; schedule during low-usage periods
• Joule skill registration: Custom skills require SAP Extension Center access and admin approval