Table of Contents

Class AIService

Namespace
Mythosia.AI.Services.Base
Assembly
Mythosia.AI.dll
public abstract class AIService : IAIService, IAIRunService, IAIRequestFeatureService, IFunctionRegisterable, IAIProcessingInfoService
Inheritance
AIService
Implements
Derived
Inherited Members
Extension Methods

Constructors

AIService(string?, string, HttpClient) 

protected AIService(string? apiKey, string baseUrl, HttpClient httpClient)

Parameters

apiKey string
baseUrl string
httpClient HttpClient

Fields

ApiKey 

protected readonly string ApiKey

Field Value

string

HttpClient 

protected readonly HttpClient HttpClient

Field Value

HttpClient

_chatRequests 

protected List<ChatBlock> _chatRequests

Field Value

List<ChatBlock>

_structuredOutputSchemaJson

JSON schema string for structured output mode. Null when not in structured output mode. Legacy provider default; request execution uses RequestStructuredOutputSchemaJson.

protected string? _structuredOutputSchemaJson

Field Value

string

Properties

ActivateChat

The currently active chat block containing conversation history.

public ChatBlock ActivateChat { get; protected set; }

Property Value

ChatBlock

ChatRequests 

public IReadOnlyCollection<ChatBlock> ChatRequests { get; }

Property Value

IReadOnlyCollection<ChatBlock>

ContextRecoveryMaxRetries

How many times a rejected-for-context-length request may be compacted and re-sent. Default 1. Set to 0 to disable reactive recovery entirely — the rejection then propagates to the caller unchanged, which is the pre-6.8 behaviour.

The budget is per attempt unit, and the two paths count differently. Non-streaming counts whole turns: one provider call covers all of its function-calling rounds, so a turn issues at most 1 + ContextRecoveryMaxRetries calls. Streaming counts rounds, replaying only the round that overflowed, so a turn can compact up to MaxRounds × ContextRecoveryMaxRetries times.

Recovery only ever runs when the server itself reports the overflow. It does not require that nothing has reached the caller yet — a streaming round that already emitted chunks is left alone, but earlier rounds in the same turn may well have streamed. 

public int ContextRecoveryMaxRetries { get; set; }

Property Value

int

ConversationPolicy

When set, automatically summarizes old messages when conversation exceeds the configured threshold. The summary is injected as a system message prefix. Set to null to disable (default).

public SummaryConversationPolicy? ConversationPolicy { get; set; }

Property Value

SummaryConversationPolicy

CurrentFeatureRequestMessage

The user message anchoring the current logical request in conversation history.

protected Message? CurrentFeatureRequestMessage { get; }

Property Value

Message

CurrentPolicy 

protected FunctionCallingPolicy? CurrentPolicy { get; set; }

Property Value

FunctionCallingPolicy

CurrentProviderRequestOptions

Provider-specific options captured for this logical request, or null for internal work.

protected object? CurrentProviderRequestOptions { get; }

Property Value

object

CurrentRequestContext

The effective context of the active request, including resolved dynamic system instructions.

protected AIRequestContext? CurrentRequestContext { get; }

Property Value

AIRequestContext

CurrentRequestFeatures

Captured options for the current logical request.

protected AIRequestFeatures CurrentRequestFeatures { get; }

Property Value

AIRequestFeatures

DefaultPolicy 

public FunctionCallingPolicy DefaultPolicy { get; set; }

Property Value

FunctionCallingPolicy

EnableFunctions 

public bool EnableFunctions { get; set; }

Property Value

bool

ForceFunctionName 

public string? ForceFunctionName { get; set; }

Property Value

string

FrequencyPenalty 

public float FrequencyPenalty { get; set; }

Property Value

float

FunctionCallMode 

public FunctionCallMode FunctionCallMode { get; set; }

Property Value

FunctionCallMode

FunctionCancellationToken

The current tool execution token, also available to existing single-call overrides.

protected CancellationToken FunctionCancellationToken { get; }

Property Value

CancellationToken

FunctionResultsRequireContinuation

Whether observed tool results still require a separate provider round.

protected virtual bool FunctionResultsRequireContinuation { get; }

Property Value

bool

Functions 

public List<FunctionDefinition> Functions { get; set; }

Property Value

List<FunctionDefinition>

FunctionsDisabled

Quick toggle for function calling (like StatelessMode)

public bool FunctionsDisabled { get; set; }

Property Value

bool

HasPendingAsyncFunctions 

protected bool HasPendingAsyncFunctions { get; }

Property Value

bool

HasPendingRunContinuation

Whether the transport will continue this run without a new user request.

protected virtual bool HasPendingRunContinuation { get; }

Property Value

bool

LastCitations

A snapshot of source references collected by the most recent request.

public IReadOnlyList<AICitation> LastCitations { get; }

Property Value

IReadOnlyList<AICitation>

LastProcessing

Processing observations from the latest logical request, including its tools and repairs. Internal summaries and query rewriting do not replace or append to these observations.

public IReadOnlyList<AIProcessingInfo> LastProcessing { get; }

Property Value

IReadOnlyList<AIProcessingInfo>

MaxTokens 

public uint MaxTokens { get; set; }

Property Value

uint

Model

The AI model identifier currently in use.

public string Model { get; protected set; }

Property Value

string

PresencePenalty 

public float PresencePenalty { get; set; }

Property Value

float

Provider

The AI provider for this service

public abstract string Provider { get; }

Property Value

string

RequestCancellationToken

The caller's cancellation token for the current logical request, including summaries and retries.

protected CancellationToken RequestCancellationToken { get; }

Property Value

CancellationToken

Remarks

Custom providers should pass this token to their transport and tools, or use CreateRequestTimeoutCts to combine it with the request policy timeout. Cancellation does not certify that the remote server stopped generating or charging for the request.

RequestEnableFunctions 

protected bool RequestEnableFunctions { get; }

Property Value

bool

RequestForceFunctionName 

protected string? RequestForceFunctionName { get; }

Property Value

string

RequestFrequencyPenalty 

protected float RequestFrequencyPenalty { get; }

Property Value

float

RequestFunctionCallMode 

protected FunctionCallMode RequestFunctionCallMode { get; }

Property Value

FunctionCallMode

RequestFunctions 

protected List<FunctionDefinition> RequestFunctions { get; }

Property Value

List<FunctionDefinition>

RequestFunctionsDisabled 

protected bool RequestFunctionsDisabled { get; }

Property Value

bool

RequestMaxTokens 

protected uint RequestMaxTokens { get; }

Property Value

uint

RequestModel 

protected string RequestModel { get; }

Property Value

string

RequestPresencePenalty 

protected float RequestPresencePenalty { get; }

Property Value

float

RequestSpeed 

protected InferenceSpeed RequestSpeed { get; }

Property Value

InferenceSpeed

RequestStatelessMode 

protected bool RequestStatelessMode { get; }

Property Value

bool

RequestStream 

protected bool RequestStream { get; }

Property Value

bool

RequestStructuredOutputSchemaJson 

protected string? RequestStructuredOutputSchemaJson { get; }

Property Value

string

RequestSystemMessage 

protected string RequestSystemMessage { get; }

Property Value

string

RequestTemperature 

protected float RequestTemperature { get; }

Property Value

float

RequestTopP 

protected float RequestTopP { get; }

Property Value

float

ShouldUseFunctions 

public bool ShouldUseFunctions { get; }

Property Value

bool

StatelessMode

When true, each request is processed independently without maintaining conversation history

public bool StatelessMode { get; set; }

Property Value

bool

Stream 

public bool Stream { get; set; }

Property Value

bool

StructuredOutputMaxRetries

Maximum number of auto-correction retries when the LLM produces invalid JSON for structured output. Default is 2. Values below zero disable repairs; Int32.MaxValue is rejected before provider work because the initial attempt must also fit in the reported attempt count. This is NOT a network/rate-limit retry — it is an "output quality/format correction" retry that sends a correction prompt asking the model to fix its JSON output.

public int StructuredOutputMaxRetries { get; set; }

Property Value

int

SupportsAsyncFunctionCalls

Whether this provider and model accept native asynchronous function calls.

protected virtual bool SupportsAsyncFunctionCalls { get; }

Property Value

bool

SystemMessage

Convenience property for ActivateChat.SystemMessage

public string SystemMessage { get; set; }

Property Value

string

SystemMessageProvider

Optional async provider that supplies a baseline AIRequestContext for each caller request (excluding internally generated summaries). Invoked automatically right before each call to GetCompletionAsync(Message, AIRequestProfile?, AIRequestContext?, CancellationToken) or StreamAsync(Message, StreamOptions, AIRequestContext?, CancellationToken) (including agent-path calls) so callers no longer need to build and pass an AIRequestContext at every entry point.

The property is set through the fluent helper WithSystemMessageProvider(AIService, Func<AIRequestContext?>) (sync) or WithSystemMessageProvider(AIService, Func<CancellationToken, ValueTask<AIRequestContext?>>) (async). Both overloads normalize to the async delegate stored here, so the runtime only deals with a single invocation shape.

If a request also passes an explicit AIRequestContext, the two are merged field-by-field: the explicit context wins on SystemMessagePrefix, SystemMessageSuffix, and RequestMessageOverride when non-null; for AdditionalMessages, the provider's list comes first and the explicit list is appended. 

public Func<CancellationToken, ValueTask<AIRequestContext?>>? SystemMessageProvider { get; }

Property Value

Func<CancellationToken, ValueTask<AIRequestContext>>

Temperature 

public float Temperature { get; set; }

Property Value

float

TopP 

public float TopP { get; set; }

Property Value

float

UsedAsyncFunctions 

protected bool UsedAsyncFunctions { get; }

Property Value

bool

Methods

AddFunctionCallBatchToHistory(string, FunctionCallBatch, Dictionary<string, object>?) 

protected Message AddFunctionCallBatchToHistory(string content, FunctionCallBatch functionCalls, Dictionary<string, object>? metadata = null)

Parameters

content string
functionCalls FunctionCallBatch
metadata Dictionary<string, object>

Returns

Message

AddFunctionResultBatchToHistory(FunctionCallResultBatch, Dictionary<string, object>?) 

protected Message AddFunctionResultBatchToHistory(FunctionCallResultBatch functionResults, Dictionary<string, object>? metadata = null)

Parameters

functionResults FunctionCallResultBatch
metadata Dictionary<string, object>

Returns

Message

AddNewChat() 

public void AddNewChat()

AddNewChat(ChatBlock) 

public void AddNewChat(ChatBlock newChat)

Parameters

newChat ChatBlock

ApplyCapabilityRequestProfile(AIRequestProfile)

Applies only execution-local profile settings needed to describe capabilities.

protected virtual void ApplyCapabilityRequestProfile(AIRequestProfile profile)

Parameters

profile AIRequestProfile

Remarks

Common builder overrides are already captured. Custom providers may override this hook with SetExecutionSetting calls for native mode flags. Do not prepare execution, reserve token budgets, invoke callbacks, validate requests, or modify service/caller-owned state. Execution profile hooks are intentionally not called during inspection.

ApplyProviderSpecificRequestProfile(AIRequestProfile) 

protected virtual Action ApplyProviderSpecificRequestProfile(AIRequestProfile profile)

Parameters

profile AIRequestProfile

Returns

Action

ApplyRequestContext(AIRequestContext) 

protected virtual Action ApplyRequestContext(AIRequestContext context)

Parameters

context AIRequestContext

Returns

Action

ApplyRequestProfile(AIRequestProfile) 

protected virtual Action ApplyRequestProfile(AIRequestProfile profile)

Parameters

profile AIRequestProfile

Returns

Action

ApplySummaryPolicyIfNeededAsync(CancellationToken)

Checks whether the conversation should be summarized based on the current ConversationPolicy, and if so, performs the summarization using StatelessMode. Called automatically at the beginning of GetCompletionAsync(string). For streaming scenarios, call this explicitly before StreamAsync().

public Task ApplySummaryPolicyIfNeededAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Returns

Task

BeginAsyncFunctionScope(bool)

Owns deferred handlers for one request. Cleanup completes started work before the request releases its chat, including on failure, cancellation, or stream disposal.

protected Func<Task> BeginAsyncFunctionScope(bool useFunctions = true)

Parameters

useFunctions bool

Returns

Func<Task>

BeginIndependentRequestScope()

Starts an independent helper request inside a provider's request adapter.

protected IDisposable BeginIndependentRequestScope()

Returns

IDisposable

Remarks

A framework invocation of a virtual adapter and its first matching base call belong to the same request, even when the adapter replaces the input. If an override calls that same base entry for unrelated work before forwarding the outer input, wrap the helper and its await (or full stream enumeration) in this scope. Its settings come from service defaults; disposing restores the outer dispatch. This does not isolate conversation history or permit concurrent use of a service. Use a stateless profile for helper history.

BeginProcessingObservation()

Creates one observation for a provider inference attempt, before sending a request or when a server continuation begins. Update the same object as headers and stream events arrive.

protected AIService.ProcessingObservation BeginProcessingObservation()

Returns

AIService.ProcessingObservation

BeginRequestFeaturesScope(Message)

Starts a provider execution, or accepts a framework handoff for the current request.

protected IDisposable BeginRequestFeaturesScope(Message message)

Parameters

message Message

Returns

IDisposable

BeginRequestSettingsScope()

Supplies initial settings until request preparation establishes an independent execution or accepts a framework handoff.

protected IDisposable BeginRequestSettingsScope()

Returns

IDisposable

BeginStream(string)

Begin a streaming structured output run. Returns a StreamBuilder for fluent configuration.

Example: 

var run = service.BeginStream(prompt)
    .WithStructuredOutput(new StructuredOutputPolicy { MaxRepairAttempts = 2 })
    .As<MyDto>();

await foreach (var chunk in run.Stream(ct))
Console.Write(chunk);


MyDto dto = await run.Result;
public StreamBuilder BeginStream(string prompt)

Parameters

prompt string

The user prompt to send to the LLM.

Returns

StreamBuilder

A StreamBuilder for fluent configuration.

CaptureProviderRequestOptions(Message)

Captures and consumes provider-specific per-request options before execution starts.

protected virtual object? CaptureProviderRequestOptions(Message message)

Parameters

message Message

Returns

object

Remarks

Framework delegation and tool rounds reuse the returned snapshot; new public calls capture independently. Internal summary and rewrite work do not call this hook and do not inherit these options.

CaptureRequestSettings(IDictionary<string, object?>)

Captures provider and common defaults. Overrides copy native mutable option values.

protected virtual void CaptureRequestSettings(IDictionary<string, object?> settings)

Parameters

settings IDictionary<string, object>

ChangeModel(string) 

public void ChangeModel(string model)

Parameters

model string

CloneProviderRequestOptions(object?)

Creates fresh provider execution state when a prepared request is executed again.

protected virtual object? CloneProviderRequestOptions(object? options)

Parameters

options object

Returns

object

CollectAsyncFunctionResultsAsync(bool, CancellationToken)

Delivers completed results, optionally waiting until at least one is ready.

protected Task<IReadOnlyList<FunctionCallResultBatch>> CollectAsyncFunctionResultsAsync(bool waitForResult, CancellationToken cancellationToken)

Parameters

waitForResult bool
cancellationToken CancellationToken

Returns

Task<IReadOnlyList<FunctionCallResultBatch>>

ConfigureRequestFeatures(AIRequestFeatures)

Merges and copies non-null settings for the next logical request.

public void ConfigureRequestFeatures(AIRequestFeatures features)

Parameters

features AIRequestFeatures

CopyFrom(AIService)

Copies the source service's state into this instance — conversation, function registrations, sampling parameters, conversation policy, and service-level callbacks (SystemMessageProvider, streaming diagnostics).

Service-level delegates are propagated by reference, not deep-copied (deep copy of a delegate is not meaningful — its captured target objects, e.g. an ILogger, are external infrastructure that the library cannot clone). The typical case is callbacks wrapping a shared logger/metrics/telemetry sink, where reference sharing is the desired behavior.

Caveat: if a callback closure captures the source service itself (e.g. line => Log(sourceService.Provider, line)), the copy will still log under the original provider's identity. Prefer capturing only stable external resources inside callbacks.

public AIService CopyFrom(AIService sourceService)

Parameters

sourceService AIService

Returns

AIService

CopyTokenUsage(TokenUsage) 

protected static TokenUsage CopyTokenUsage(TokenUsage usage)

Parameters

usage TokenUsage

Returns

TokenUsage

CreateFunctionMessageRequest()

Creates HTTP request with function definitions

protected abstract HttpRequestMessage CreateFunctionMessageRequest()

Returns

HttpRequestMessage

CreateMessageRequest()

Creates the HTTP request message for the AI service

protected abstract HttpRequestMessage CreateMessageRequest()

Returns

HttpRequestMessage

CreateRequest(Message)

Captures a message and service defaults for an independent request configuration.

public AIRequestBuilder CreateRequest(Message message)

Parameters

message Message

Returns

AIRequestBuilder

CreateRequest(string)

Captures the service defaults and input without starting a request.

public AIRequestBuilder CreateRequest(string prompt)

Parameters

prompt string

Returns

AIRequestBuilder

Remarks

Each With method returns a new builder. Conversation history remains owned by the service and is selected at execution. Legacy next-call feature/policy settings are consumed into this builder. Custom content implementations and delegate targets must remain immutable or externally synchronized.

CreateRequestTimeoutCts(FunctionCallingPolicy, CancellationToken)

Creates a CancellationTokenSource that fires after the resolved request timeout, linked to an optional external cancellation token. This is the one place that turns a policy into an effective request timeout.

protected CancellationTokenSource CreateRequestTimeoutCts(FunctionCallingPolicy policy, CancellationToken external = default)

Parameters

policy FunctionCallingPolicy
external CancellationToken

Returns

CancellationTokenSource

CreateRoundUsageContent(int, bool, TokenUsage) 

protected static StreamingContent CreateRoundUsageContent(int roundIndex, bool isFinalRound, TokenUsage usage)

Parameters

roundIndex int
isFinalRound bool
usage TokenUsage

Returns

StreamingContent

CreateRunSessionAsync(Message, StreamOptions, AIRequestContext?, CancellationToken)

Creates a provider session for a run without changing legacy request transports.

protected virtual Task<AIService.RunSession> CreateRunSessionAsync(Message message, StreamOptions executionOptions, AIRequestContext? context, CancellationToken cancellationToken)

Parameters

message Message
executionOptions StreamOptions
context AIRequestContext
cancellationToken CancellationToken

Returns

Task<AIService.RunSession>

EnsureUserFirstMessage(List<Message>)

Ensures the message list starts with a User message. Some APIs (Gemini, Claude) require conversations to begin with a user turn. If the first message is not from a user, a synthetic context message is prepended.

protected static void EnsureUserFirstMessage(List<Message> messages)

Parameters

messages List<Message>

ExtractFunctionCalls(string)

Extracts every function call from one API response.

protected abstract (string content, FunctionCallBatch functionCalls) ExtractFunctionCalls(string response)

Parameters

response string

Returns

(string content, FunctionCallBatch functionCalls)

ExtractResponseContent(string)

Extracts the response content from the API response

protected abstract string ExtractResponseContent(string responseContent)

Parameters

responseContent string

Returns

string

GetCapabilities()

Inspects current service defaults and pending feature settings without starting or consuming a request.

public AIModelCapabilities GetCapabilities()

Returns

AIModelCapabilities

Remarks

No HTTP call, context callback, validation, history change or pending-option capture occurs. Use a builder's GetCapabilities to inspect its independent settings snapshot instead.

GetCompletionAsync(Message) 

public abstract Task<string> GetCompletionAsync(Message message)

Parameters

message Message

Returns

Task<string>

GetCompletionAsync(Message, AIRequestProfile?, AIRequestContext?, CancellationToken)

Returns the final text. Cancellation stops transport and cooperative local work, with cleanup.

public virtual Task<string> GetCompletionAsync(Message message, AIRequestProfile? profile = null, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

message Message
profile AIRequestProfile
context AIRequestContext
cancellationToken CancellationToken

Returns

Task<string>

Remarks

Custom provider overrides use RequestCancellationToken in their existing Message execution hook.

GetCompletionAsync(string, AIRequestProfile?, AIRequestContext?, CancellationToken)

Returns the final text. Cancellation stops transport and cooperative local work, with cleanup.

public virtual Task<string> GetCompletionAsync(string prompt, AIRequestProfile? profile = null, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

prompt string
profile AIRequestProfile
context AIRequestContext
cancellationToken CancellationToken

Returns

Task<string>

Remarks

Server generation and billing cancellation depend on the provider.

GetCompletionAsync<T>(string, CancellationToken)

Sends a prompt and deserializes the LLM response to the specified type. Internally generates a JSON schema from T, instructs the LLM to respond in that format, and deserializes the JSON response. If the LLM produces invalid JSON, sends an auto-correction prompt and retries up to StructuredOutputMaxRetries times.

public Task<T> GetCompletionAsync<T>(string prompt, CancellationToken cancellationToken = default) where T : class

Parameters

prompt string

The user prompt.

cancellationToken CancellationToken

Cancels this call, including summaries and JSON repair requests.

Returns

Task<T>

The deserialized response object.

Type Parameters

T

The type to deserialize the response to. Must have public properties.

Exceptions

StructuredOutputException

Thrown when deserialization fails after all retry attempts.

GetCompletionWithImageAsync(string, string, CancellationToken) 

public virtual Task<string> GetCompletionWithImageAsync(string prompt, string imagePath, CancellationToken cancellationToken = default)

Parameters

prompt string
imagePath string
cancellationToken CancellationToken

Returns

Task<string>

GetCompletionWithImageUrlAsync(string, string, CancellationToken) 

public virtual Task<string> GetCompletionWithImageUrlAsync(string prompt, string imageUrl, CancellationToken cancellationToken = default)

Parameters

prompt string
imageUrl string
cancellationToken CancellationToken

Returns

Task<string>

GetConversationCompactionBlockReason()

Returns a reason when protocol state requires retaining the original conversation prefix.

protected virtual string? GetConversationCompactionBlockReason()

Returns

string

GetEffectiveMaxTokens()

Returns the effective max tokens, capped by the current model's limit. Use this instead of MaxTokens when building request bodies.

protected uint GetEffectiveMaxTokens()

Returns

uint

GetEffectiveSystemMessageWithRequestContext() 

protected string GetEffectiveSystemMessageWithRequestContext()

Returns

string

GetExecutionPolicy() 

protected FunctionCallingPolicy GetExecutionPolicy()

Returns

FunctionCallingPolicy

GetImageCapabilities(string?)

Inspects an image generation model independently of the selected chat model, without HTTP.

public virtual ImageModelCapabilities GetImageCapabilities(string? model = null)

Parameters

model string

Returns

ImageModelCapabilities

Remarks

A null model uses the provider's image default. Unknown does not mean unsupported.

GetInputTokenCountAsync()

Gets the token count for the current conversation

public abstract Task<uint> GetInputTokenCountAsync()

Returns

Task<uint>

GetInputTokenCountAsync(string)

Gets the token count for a specific prompt

public abstract Task<uint> GetInputTokenCountAsync(string prompt)

Parameters

prompt string

Returns

Task<uint>

GetLatestMessages()

Gets the active conversation messages for an outgoing request. Conversation trimming is handled exclusively by ConversationPolicy.

protected IEnumerable<Message> GetLatestMessages()

Returns

IEnumerable<Message>

GetLatestMessagesWithFunctionFallback()

Gets messages for non-function path, converting function-related messages to plain text. Original messages in ChatBlock are never modified.

protected IEnumerable<Message> GetLatestMessagesWithFunctionFallback()

Returns

IEnumerable<Message>

GetModelMaxOutputTokens()

Returns the maximum output tokens allowed for the current model. Override in each service to provide model-specific limits.

protected virtual uint GetModelMaxOutputTokens()

Returns

uint

GetRequestMessageOverrideTargetId()

Identifies the user input replaced by request context while tool rounds append messages.

protected virtual string? GetRequestMessageOverrideTargetId()

Returns

string

GetRunRequestedModel()

Resolves the explicit model ID sent for the captured run request.

protected virtual string? GetRunRequestedModel()

Returns

string

Remarks

Override when provider options replace the configured model or translate it to a wire ID. Return null when the request delegates model selection without a single model ID. Called within the captured request settings and provider-feature scope.

GetStructuredOutputInstruction()

Returns the structured output instruction to append to system messages. Returns null if not in structured output mode.

protected string? GetStructuredOutputInstruction()

Returns

string

MapFinishReason(string?)

Normalizes a reported provider reason without guessing when none was supplied.

protected static AIFinishReason MapFinishReason(string? reason)

Parameters

reason string

Returns

AIFinishReason

ProcessFunctionBatchForRoundAsync(string, FunctionCallBatch, Dictionary<string, object>?, FunctionCallingPolicy, CancellationToken)

Records a validated call batch and runs its required calls. Eligible asynchronous calls may finish in later rounds; every result is recorded exactly once.

protected Task<IReadOnlyList<FunctionCallResultBatch>> ProcessFunctionBatchForRoundAsync(string content, FunctionCallBatch calls, Dictionary<string, object>? metadata, FunctionCallingPolicy policy, CancellationToken cancellationToken)

Parameters

content string
calls FunctionCallBatch
metadata Dictionary<string, object>
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<IReadOnlyList<FunctionCallResultBatch>>

ProcessFunctionCallAsync(FunctionCall)

Process function call

protected virtual Task<FunctionCallResult> ProcessFunctionCallAsync(FunctionCall functionCall)

Parameters

functionCall FunctionCall

Returns

Task<FunctionCallResult>

ProcessFunctionCallsAsync(FunctionCallBatch, FunctionCallingPolicy, CancellationToken)

Executes one validated provider batch using the configured execution mode.

protected virtual Task<FunctionCallResultBatch> ProcessFunctionCallsAsync(FunctionCallBatch functionCalls, FunctionCallingPolicy policy, CancellationToken cancellationToken = default)

Parameters

functionCalls FunctionCallBatch
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<FunctionCallResultBatch>

ProcessFunctionCallsInParallelAsync(FunctionCallBatch, FunctionCallingPolicy, CancellationToken)

Validates the complete provider batch, then executes calls concurrently while preserving provider order in the returned result batch.

protected virtual Task<FunctionCallResultBatch> ProcessFunctionCallsInParallelAsync(FunctionCallBatch functionCalls, FunctionCallingPolicy policy, CancellationToken cancellationToken = default)

Parameters

functionCalls FunctionCallBatch
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<FunctionCallResultBatch>

ProcessFunctionCallsSequentiallyAsync(FunctionCallBatch, FunctionCallingPolicy, CancellationToken)

Validates the complete provider batch, then executes each call in provider order. Parallel execution is intentionally not part of this contract.

protected virtual Task<FunctionCallResultBatch> ProcessFunctionCallsSequentiallyAsync(FunctionCallBatch functionCalls, FunctionCallingPolicy policy, CancellationToken cancellationToken = default)

Parameters

functionCalls FunctionCallBatch
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<FunctionCallResultBatch>

QuickAskAsync(string, string, string, CancellationToken) 

public static Task<string> QuickAskAsync(string apiKey, string prompt, string model = "gpt-4o-mini", CancellationToken cancellationToken = default)

Parameters

apiKey string
prompt string
model string
cancellationToken CancellationToken

Returns

Task<string>

QuickAskWithImageAsync(string, string, string, string, CancellationToken) 

public static Task<string> QuickAskWithImageAsync(string apiKey, string prompt, string imagePath, string model = "gpt-4.1", CancellationToken cancellationToken = default)

Parameters

apiKey string
prompt string
imagePath string
model string
cancellationToken CancellationToken

Returns

Task<string>

ReadCompletionResponseBodyAsync(HttpResponseMessage, CancellationToken)

Reads and decodes a response body with cancellation on .NET Standard 2.1.

protected static Task<string> ReadCompletionResponseBodyAsync(HttpResponseMessage response, CancellationToken cancellationToken)

Parameters

response HttpResponseMessage
cancellationToken CancellationToken

Returns

Task<string>

Remarks

Use with ResponseHeadersRead. Disposing the response interrupts a pending network read; the token is also passed to stream copying. Charset and BOM handling match HttpContent.

ReadSseLinesAsync(HttpResponseMessage, StreamDiagnostics, CancellationToken)

Reads an SSE response body line-by-line with diagnostics, async stream disposal, and structured exception wrapping. Yields raw SSE lines without any provider-specific parsing — callers handle "data:", "[DONE]", JSON, etc.

Behavior:

  • Disposes the underlying response stream via DisposeAsync() in the iterator's finally block. Avoids the NotSupportedException that some HttpContent transports throw on synchronous Dispose.
  • Wraps any read-side exception in StreamReadException with a StreamDiagnostics snapshot taken at the moment of failure.
  • Invokes the service-level OnRawLine callback (set via WithStreamDiagnostics) for every line. Callback exceptions are swallowed so a faulty logger cannot break the stream.
  • Invokes the service-level OnComplete callback exactly once on iterator exit, regardless of how the iteration ended.
  • Honors cancellationToken while a read is pending as well as between reads. A losing pending read is observed and the iterator's finally block disposes the reader and response stream.

The caller owns the diagnostics object and may mutate fields like DataLinesProcessed or AccumulatedTextLength while consuming lines.

protected IAsyncEnumerable<string> ReadSseLinesAsync(HttpResponseMessage response, StreamDiagnostics diagnostics, CancellationToken cancellationToken)

Parameters

response HttpResponseMessage
diagnostics StreamDiagnostics
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

RecordCitation(AICitation)

Retains a provider source even if callers do not consume stream events.

protected void RecordCitation(AICitation citation)

Parameters

citation AICitation

RequestSetting<T>(string, T)

Reads the executing request's setting, falling back to a service default outside execution.

protected T RequestSetting<T>(string name, T defaultValue)

Parameters

name string
defaultValue T

Returns

T

Type Parameters

T

ResolveRequestCapabilities()

Resolves adapter support using RequestSetting and CurrentRequestFeatures without side effects.

protected virtual AIModelCapabilities ResolveRequestCapabilities()

Returns

AIModelCapabilities

Remarks

Custom providers may override this hook. The default preserves compatibility by returning Unknown. Overrides must not send requests, consume options, change history or invoke user callbacks.

ResolveRequestMessage(Message)

Returns the current request's owned input snapshot without changing caller-owned messages.

protected Message ResolveRequestMessage(Message message)

Parameters

message Message

Returns

Message

Remarks

Provider implementations may call this after BeginRequestFeaturesScope and retain the returned input. Existing providers which retain their original input remain supported. Repair messages remain distinct.

ResolveRequestTimeoutSeconds(FunctionCallingPolicy)

Single source of truth for per-request timeouts. Returns the timeout in seconds for the given policy, or null for no timeout. Providers override this to adjust the timeout per model (e.g. slow "pro" reasoning models that routinely exceed the default). All request paths must obtain their timeout from here via CreateRequestTimeoutCts(FunctionCallingPolicy, CancellationToken).

protected virtual int? ResolveRequestTimeoutSeconds(FunctionCallingPolicy policy)

Parameters

policy FunctionCallingPolicy

Returns

int?

ResolveSpeedSupport(InferenceSpeed)

Locally resolves processing-mode support without checking account entitlement or live capacity.

protected virtual CapabilitySupport ResolveSpeedSupport(InferenceSpeed speed)

Parameters

speed InferenceSpeed

Returns

CapabilitySupport

RunAgentAsync(string, int, AIRequestContext?, CancellationToken)

Runs a ReAct (Reasoning + Acting) agent loop that repeatedly calls the LLM and executes function calls until the goal is achieved or maxSteps is exceeded.

Reuses existing function calling infrastructure registered via WithFunction. The loop terminates when the LLM returns a text response without any function calls, or when maxSteps is exceeded. 

[Obsolete("Use StartRunAsync with the desired function-calling round policy and read (await run.Result).Text. This compatibility method remains supported.", false)]
public virtual Task<string> RunAgentAsync(string goal, int maxSteps = 10, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

goal string

The goal or task for the agent to accomplish

maxSteps int

Maximum number of agent steps (LLM round-trips) to prevent infinite loops. Default is 10.

context AIRequestContext

Optional per-call request context (e.g. dynamic system message prefix/suffix).

cancellationToken CancellationToken

Cancels the completion, local tools and subsequent rounds.

Returns

Task<string>

The final text response from the LLM after completing the goal

Exceptions

AgentMaxStepsExceededException

Thrown when maxSteps is exceeded without a final answer. The exception contains a PartialResponse property with the last assistant message, if any.

RunAgentStreamAsync(string, int, StreamOptions?, AIRequestContext?, CancellationToken)

Runs the ReAct agent loop using the streaming pipeline.

This is the streaming counterpart to RunAgentAsync(string, int, AIRequestContext?, CancellationToken). Function calling is forced on for this request so the agent can act, and TextOnly is disabled so the stream can emit a final Completion event. 

[Obsolete("Use StartRunAsync with the desired function-calling round policy and observe run.StreamAsync(). This compatibility method remains supported.", false)]
public virtual IAsyncEnumerable<StreamingContent> RunAgentStreamAsync(string goal, int maxSteps = 10, StreamOptions? options = null, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

goal string

The goal or task for the agent to accomplish.

maxSteps int

Maximum number of agent steps (LLM round-trips). Default is 10.

options StreamOptions

Optional streaming options. Function calling will be enabled automatically.

context AIRequestContext

Optional per-call request context (e.g. dynamic system message prefix/suffix).

cancellationToken CancellationToken

Cancellation token for the streaming operation.

Returns

IAsyncEnumerable<StreamingContent>

A stream of agent events including text, function calls, and function results.

Exceptions

AgentMaxStepsExceededException

Thrown when maxSteps is exceeded without a final completion event. The exception contains a PartialResponse property with the last assistant message, if any.

SetActivateChat(string) 

public void SetActivateChat(string chatBlockId)

Parameters

chatBlockId string

SetExecutionSetting<T>(string, T)

Changes only an execution-local setting (for internal profiles or protocol preparation).

protected void SetExecutionSetting<T>(string name, T value)

Parameters

name string
value T

Type Parameters

T

StartRunAsync(Message, Action<string>?, StreamOptions?, AIRequestContext?, CancellationToken)

Starts a message request; output selection does not disable tool execution.

public Task<AIRun> StartRunAsync(Message message, Action<string>? onText = null, StreamOptions? options = null, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

message Message
onText Action<string>
options StreamOptions
context AIRequestContext
cancellationToken CancellationToken

Returns

Task<AIRun>

StartRunAsync(string, Action<string>?, StreamOptions?, AIRequestContext?, CancellationToken)

Starts one request with optional text observation and a separately awaitable result.

public Task<AIRun> StartRunAsync(string prompt, Action<string>? onText = null, StreamOptions? options = null, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

prompt string
onText Action<string>
options StreamOptions
context AIRequestContext
cancellationToken CancellationToken

Returns

Task<AIRun>

Remarks

Only one StartRunAsync request may be active on a service instance. Do not mix a running request with legacy request methods or mutate the service's conversation/configuration. The text callback is registered before production; it must not depend on the returned run variable already being assigned. A callback exception fails and cancels the run. Built-in text, image, and audio payloads are copied at startup. Custom MessageContent implementations are preserved by reference and must remain immutable until cleanup.

StreamAsync(Message, AIRequestContext?, CancellationToken)

Simple text streaming with Message input

public IAsyncEnumerable<string> StreamAsync(Message message, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

message Message
context AIRequestContext
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

Remarks

Planned for withdrawal from the public API in the next major release, once the replacement is available. This API remains supported during the minor transition. Use StartRunAsync and the returned run's output stream for new integrations. Execution logic will be preserved behind non-public implementation hooks.

StreamAsync(Message, StreamOptions, AIRequestContext?, CancellationToken)

Core streaming implementation using Template Method pattern. Manages the round loop, StatelessMode, and conversation summary policy. Providers override StreamRoundAsync(StreamOptions, bool, FunctionCallingPolicy, CancellationToken) to handle a single round. Providers that do not support function calling rounds (e.g., Sonar) may override this method directly.

public virtual IAsyncEnumerable<StreamingContent> StreamAsync(Message message, StreamOptions options, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

message Message
options StreamOptions
context AIRequestContext
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

Remarks

Planned for withdrawal from the public API in the next major release, once the replacement is available. This API remains supported during the minor transition. Use StartRunAsync and the returned run's output stream for new integrations. Execution logic will be preserved behind non-public implementation hooks.

StreamAsync(string, StreamOptions, CancellationToken)

Advanced streaming with options

public IAsyncEnumerable<StreamingContent> StreamAsync(string prompt, StreamOptions options, CancellationToken cancellationToken = default)

Parameters

prompt string
options StreamOptions
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

Remarks

Planned for withdrawal from the public API in the next major release, once the replacement is available. This API remains supported during the minor transition. Use StartRunAsync and the returned run's output stream for new integrations. Execution logic will be preserved behind non-public implementation hooks.

StreamAsync(string, CancellationToken)

Simple text streaming (most common use case)

public IAsyncEnumerable<string> StreamAsync(string prompt, CancellationToken cancellationToken = default)

Parameters

prompt string
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

Remarks

Planned for withdrawal from the public API in the next major release, once the replacement is available. This API remains supported during the minor transition. Use StartRunAsync and the returned run's output stream for new integrations. Execution logic will be preserved behind non-public implementation hooks.

StreamCompletionAsync(Message, Func<string, Task>) 

public abstract Task StreamCompletionAsync(Message message, Func<string, Task> messageReceivedAsync)

Parameters

message Message
messageReceivedAsync Func<string, Task>

Returns

Task

StreamCompletionAsync(string, Action<string>) 

public virtual Task StreamCompletionAsync(string prompt, Action<string> messageReceived)

Parameters

prompt string
messageReceived Action<string>

Returns

Task

StreamCompletionAsync(string, Func<string, Task>) 

public virtual Task StreamCompletionAsync(string prompt, Func<string, Task> messageReceivedAsync)

Parameters

prompt string
messageReceivedAsync Func<string, Task>

Returns

Task

StreamCoreAsync(Message, StreamOptions, CancellationToken)

Core streaming loop. Override this method to replace the full streaming pipeline (round loop, StatelessMode, summary policy). Most providers should override StreamRoundAsync(StreamOptions, bool, FunctionCallingPolicy, CancellationToken) instead.

protected virtual IAsyncEnumerable<StreamingContent> StreamCoreAsync(Message message, StreamOptions options, CancellationToken cancellationToken = default)

Parameters

message Message
options StreamOptions
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

StreamOnceAsync(Message, CancellationToken)

Streams as one-off query without affecting conversation history

public IAsyncEnumerable<string> StreamOnceAsync(Message message, CancellationToken cancellationToken = default)

Parameters

message Message
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

StreamOnceAsync(string, CancellationToken)

Streams as one-off query without affecting conversation history

public IAsyncEnumerable<string> StreamOnceAsync(string prompt, CancellationToken cancellationToken = default)

Parameters

prompt string
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

StreamParseJson(string)

Parses streaming JSON data

protected abstract string StreamParseJson(string jsonData)

Parameters

jsonData string

Returns

string

StreamRoundAsync(StreamOptions, bool, FunctionCallingPolicy, CancellationToken)

Executes a single streaming round: sends an HTTP request, reads the SSE stream, yields chunks, and handles function execution if detected. Yield a FunctionResult to signal the template to continue to the next round. Pending native asynchronous calls also keep the round loop active until their results have been delivered and the model finishes.

protected virtual IAsyncEnumerable<StreamingContent> StreamRoundAsync(StreamOptions options, bool useFunctions, FunctionCallingPolicy policy, CancellationToken cancellationToken)

Parameters

options StreamOptions
useFunctions bool
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

ValidatePendingRequestFeatures()

Validates pending settings without consuming them or changing conversation history.

public void ValidatePendingRequestFeatures()

ValidateProviderRequestOptions(object?, Message)

Validates captured native options against the completed request before history or HTTP changes.

protected virtual void ValidateProviderRequestOptions(object? options, Message message)

Parameters

options object
message Message

ValidateRequestFeatures(AIRequestFeatures)

Reject unsupported features before any network request or history mutation.

protected virtual void ValidateRequestFeatures(AIRequestFeatures features)

Parameters

features AIRequestFeatures
Edit this page