Table of Contents

Class AIService

Namespace
Mythosia.AI.Services.Base
Assembly
Mythosia.AI.dll
public abstract class AIService : IAIService, IFunctionRegisterable
Inheritance
AIService
Implements
Derived
Inherited Members
Extension Methods

Constructors

AIService(string?, string, HttpClient) 

protected AIService(string? apiKey, string baseUrl, HttpClient httpClient)

Parameters

apiKey string
baseUrl string
httpClient HttpClient

Fields

ApiKey 

protected readonly string ApiKey

Field Value

string

HttpClient 

protected readonly HttpClient HttpClient

Field Value

HttpClient

_chatRequests 

protected List<ChatBlock> _chatRequests

Field Value

List<ChatBlock>

_structuredOutputSchemaJson

JSON schema string for structured output mode. Null when not in structured output mode. Set temporarily during GetCompletionAsync<T>() and cleared in finally block.

protected string? _structuredOutputSchemaJson

Field Value

string

Properties

ActivateChat

The currently active chat block containing conversation history.

public ChatBlock ActivateChat { get; protected set; }

Property Value

ChatBlock

ChatRequests 

public IReadOnlyCollection<ChatBlock> ChatRequests { get; }

Property Value

IReadOnlyCollection<ChatBlock>

ContextRecoveryMaxRetries

How many times a rejected-for-context-length request may be compacted and re-sent. Default 1. Set to 0 to disable reactive recovery entirely — the rejection then propagates to the caller unchanged, which is the pre-6.8 behaviour.

The budget is per attempt unit, and the two paths count differently. Non-streaming counts whole turns: one provider call covers all of its function-calling rounds, so a turn issues at most 1 + ContextRecoveryMaxRetries calls. Streaming counts rounds, replaying only the round that overflowed, so a turn can compact up to MaxRounds × ContextRecoveryMaxRetries times.

Recovery only ever runs when the server itself reports the overflow. It does not require that nothing has reached the caller yet — a streaming round that already emitted chunks is left alone, but earlier rounds in the same turn may well have streamed. 

public int ContextRecoveryMaxRetries { get; set; }

Property Value

int

ConversationPolicy

When set, automatically summarizes old messages when conversation exceeds the configured threshold. The summary is injected as a system message prefix. Set to null to disable (default).

public SummaryConversationPolicy? ConversationPolicy { get; set; }

Property Value

SummaryConversationPolicy

CurrentPolicy 

protected FunctionCallingPolicy? CurrentPolicy { get; set; }

Property Value

FunctionCallingPolicy

DefaultPolicy 

public FunctionCallingPolicy DefaultPolicy { get; set; }

Property Value

FunctionCallingPolicy

EnableFunctions 

public bool EnableFunctions { get; set; }

Property Value

bool

ForceFunctionName 

public string? ForceFunctionName { get; set; }

Property Value

string

FrequencyPenalty 

public float FrequencyPenalty { get; set; }

Property Value

float

FunctionCallMode 

public FunctionCallMode FunctionCallMode { get; set; }

Property Value

FunctionCallMode

Functions 

public List<FunctionDefinition> Functions { get; set; }

Property Value

List<FunctionDefinition>

FunctionsDisabled

Quick toggle for function calling (like StatelessMode)

public bool FunctionsDisabled { get; set; }

Property Value

bool

MaxTokens 

public uint MaxTokens { get; set; }

Property Value

uint

Model

The AI model identifier currently in use.

public string Model { get; protected set; }

Property Value

string

PresencePenalty 

public float PresencePenalty { get; set; }

Property Value

float

Provider

The AI provider for this service

public abstract string Provider { get; }

Property Value

string

ShouldUseFunctions 

public bool ShouldUseFunctions { get; }

Property Value

bool

StatelessMode

When true, each request is processed independently without maintaining conversation history

public bool StatelessMode { get; set; }

Property Value

bool

Stream 

public bool Stream { get; set; }

Property Value

bool

StructuredOutputMaxRetries

Maximum number of auto-correction retries when the LLM produces invalid JSON for structured output. Default is 2. This is NOT a network/rate-limit retry — it is an "output quality/format correction" retry that sends a correction prompt asking the model to fix its JSON output.

public int StructuredOutputMaxRetries { get; set; }

Property Value

int

SystemMessage

Convenience property for ActivateChat.SystemMessage

public string SystemMessage { get; set; }

Property Value

string

SystemMessageProvider

Optional async provider that supplies a baseline AIRequestContext for every outbound request. Invoked automatically right before each call to GetCompletionAsync(Message, AIRequestProfile?, AIRequestContext?) or StreamAsync(Message, StreamOptions, AIRequestContext?, CancellationToken) (including agent-path calls) so callers no longer need to build and pass an AIRequestContext at every entry point.

The property is set through the fluent helper WithSystemMessageProvider(AIService, Func<AIRequestContext?>) (sync) or WithSystemMessageProvider(AIService, Func<CancellationToken, ValueTask<AIRequestContext?>>) (async). Both overloads normalize to the async delegate stored here, so the runtime only deals with a single invocation shape.

If a request also passes an explicit AIRequestContext, the two are merged field-by-field: the explicit context wins on SystemMessagePrefix, SystemMessageSuffix, and RequestMessageOverride when non-null; for AdditionalMessages, the provider's list comes first and the explicit list is appended. 

public Func<CancellationToken, ValueTask<AIRequestContext?>>? SystemMessageProvider { get; }

Property Value

Func<CancellationToken, ValueTask<AIRequestContext>>

Temperature 

public float Temperature { get; set; }

Property Value

float

TopP 

public float TopP { get; set; }

Property Value

float

Methods

AddFunctionCallBatchToHistory(string, FunctionCallBatch, Dictionary<string, object>?) 

protected Message AddFunctionCallBatchToHistory(string content, FunctionCallBatch functionCalls, Dictionary<string, object>? metadata = null)

Parameters

content string
functionCalls FunctionCallBatch
metadata Dictionary<string, object>

Returns

Message

AddFunctionResultBatchToHistory(FunctionCallResultBatch, Dictionary<string, object>?) 

protected Message AddFunctionResultBatchToHistory(FunctionCallResultBatch functionResults, Dictionary<string, object>? metadata = null)

Parameters

functionResults FunctionCallResultBatch
metadata Dictionary<string, object>

Returns

Message

AddNewChat() 

public void AddNewChat()

AddNewChat(ChatBlock) 

public void AddNewChat(ChatBlock newChat)

Parameters

newChat ChatBlock

ApplyProviderSpecificRequestProfile(AIRequestProfile) 

protected virtual Action ApplyProviderSpecificRequestProfile(AIRequestProfile profile)

Parameters

profile AIRequestProfile

Returns

Action

ApplyRequestContext(AIRequestContext) 

protected virtual Action ApplyRequestContext(AIRequestContext context)

Parameters

context AIRequestContext

Returns

Action

ApplyRequestProfile(AIRequestProfile) 

protected virtual Action ApplyRequestProfile(AIRequestProfile profile)

Parameters

profile AIRequestProfile

Returns

Action

ApplySummaryPolicyIfNeededAsync()

Checks whether the conversation should be summarized based on the current ConversationPolicy, and if so, performs the summarization using StatelessMode. Called automatically at the beginning of GetCompletionAsync(string). For streaming scenarios, call this explicitly before StreamAsync().

public Task ApplySummaryPolicyIfNeededAsync()

Returns

Task

BeginStream(string)

Begin a streaming structured output run. Returns a StreamBuilder for fluent configuration.

Example: 

var run = service.BeginStream(prompt)
    .WithStructuredOutput(new StructuredOutputPolicy { MaxRepairAttempts = 2 })
    .As<MyDto>();

await foreach (var chunk in run.Stream(ct))
Console.Write(chunk);


MyDto dto = await run.Result;
public StreamBuilder BeginStream(string prompt)

Parameters

prompt string

The user prompt to send to the LLM.

Returns

StreamBuilder

A StreamBuilder for fluent configuration.

ChangeModel(string) 

public void ChangeModel(string model)

Parameters

model string

CopyFrom(AIService)

Copies the source service's state into this instance — conversation, function registrations, sampling parameters, conversation policy, and service-level callbacks (SystemMessageProvider, streaming diagnostics).

Service-level delegates are propagated by reference, not deep-copied (deep copy of a delegate is not meaningful — its captured target objects, e.g. an ILogger, are external infrastructure that the library cannot clone). The typical case is callbacks wrapping a shared logger/metrics/telemetry sink, where reference sharing is the desired behavior.

Caveat: if a callback closure captures the source service itself (e.g. line => Log(sourceService.Provider, line)), the copy will still log under the original provider's identity. Prefer capturing only stable external resources inside callbacks.

public AIService CopyFrom(AIService sourceService)

Parameters

sourceService AIService

Returns

AIService

CopyTokenUsage(TokenUsage) 

protected static TokenUsage CopyTokenUsage(TokenUsage usage)

Parameters

usage TokenUsage

Returns

TokenUsage

CreateFunctionMessageRequest()

Creates HTTP request with function definitions

protected abstract HttpRequestMessage CreateFunctionMessageRequest()

Returns

HttpRequestMessage

CreateMessageRequest()

Creates the HTTP request message for the AI service

protected abstract HttpRequestMessage CreateMessageRequest()

Returns

HttpRequestMessage

CreateRequestTimeoutCts(FunctionCallingPolicy, CancellationToken)

Creates a CancellationTokenSource that fires after the resolved request timeout, linked to an optional external cancellation token. This is the one place that turns a policy into an effective request timeout.

protected CancellationTokenSource CreateRequestTimeoutCts(FunctionCallingPolicy policy, CancellationToken external = default)

Parameters

policy FunctionCallingPolicy
external CancellationToken

Returns

CancellationTokenSource

CreateRoundUsageContent(int, bool, TokenUsage) 

protected static StreamingContent CreateRoundUsageContent(int roundIndex, bool isFinalRound, TokenUsage usage)

Parameters

roundIndex int
isFinalRound bool
usage TokenUsage

Returns

StreamingContent

EnsureUserFirstMessage(List<Message>)

Ensures the message list starts with a User message. Some APIs (Gemini, Claude) require conversations to begin with a user turn. If the first message is not from a user, a synthetic context message is prepended.

protected static void EnsureUserFirstMessage(List<Message> messages)

Parameters

messages List<Message>

ExtractFunctionCalls(string)

Extracts every function call from one API response.

protected abstract (string content, FunctionCallBatch functionCalls) ExtractFunctionCalls(string response)

Parameters

response string

Returns

(string content, FunctionCallBatch functionCalls)

ExtractResponseContent(string)

Extracts the response content from the API response

protected abstract string ExtractResponseContent(string responseContent)

Parameters

responseContent string

Returns

string

GetCompletionAsync(Message) 

public abstract Task<string> GetCompletionAsync(Message message)

Parameters

message Message

Returns

Task<string>

GetCompletionAsync(Message, AIRequestProfile?, AIRequestContext?) 

public virtual Task<string> GetCompletionAsync(Message message, AIRequestProfile? profile = null, AIRequestContext? context = null)

Parameters

message Message
profile AIRequestProfile
context AIRequestContext

Returns

Task<string>

GetCompletionAsync(string, AIRequestProfile?, AIRequestContext?) 

public virtual Task<string> GetCompletionAsync(string prompt, AIRequestProfile? profile = null, AIRequestContext? context = null)

Parameters

prompt string
profile AIRequestProfile
context AIRequestContext

Returns

Task<string>

GetCompletionAsync<T>(string)

Sends a prompt and deserializes the LLM response to the specified type. Internally generates a JSON schema from T, instructs the LLM to respond in that format, and deserializes the JSON response. If the LLM produces invalid JSON, sends an auto-correction prompt and retries up to StructuredOutputMaxRetries times.

public Task<T> GetCompletionAsync<T>(string prompt) where T : class

Parameters

prompt string

The user prompt.

Returns

Task<T>

The deserialized response object.

Type Parameters

T

The type to deserialize the response to. Must have public properties.

Exceptions

StructuredOutputException

Thrown when deserialization fails after all retry attempts.

GetCompletionWithImageAsync(string, string) 

public virtual Task<string> GetCompletionWithImageAsync(string prompt, string imagePath)

Parameters

prompt string
imagePath string

Returns

Task<string>

GetCompletionWithImageUrlAsync(string, string) 

public virtual Task<string> GetCompletionWithImageUrlAsync(string prompt, string imageUrl)

Parameters

prompt string
imageUrl string

Returns

Task<string>

GetEffectiveMaxTokens()

Returns the effective max tokens, capped by the current model's limit. Use this instead of MaxTokens when building request bodies.

protected uint GetEffectiveMaxTokens()

Returns

uint

GetEffectiveSystemMessageWithRequestContext() 

protected string GetEffectiveSystemMessageWithRequestContext()

Returns

string

GetInputTokenCountAsync()

Gets the token count for the current conversation

public abstract Task<uint> GetInputTokenCountAsync()

Returns

Task<uint>

GetInputTokenCountAsync(string)

Gets the token count for a specific prompt

public abstract Task<uint> GetInputTokenCountAsync(string prompt)

Parameters

prompt string

Returns

Task<uint>

GetLatestMessages()

Gets the active conversation messages for an outgoing request. Conversation trimming is handled exclusively by ConversationPolicy.

protected IEnumerable<Message> GetLatestMessages()

Returns

IEnumerable<Message>

GetLatestMessagesWithFunctionFallback()

Gets messages for non-function path, converting function-related messages to plain text. Original messages in ChatBlock are never modified.

protected IEnumerable<Message> GetLatestMessagesWithFunctionFallback()

Returns

IEnumerable<Message>

GetModelMaxOutputTokens()

Returns the maximum output tokens allowed for the current model. Override in each service to provide model-specific limits.

protected virtual uint GetModelMaxOutputTokens()

Returns

uint

GetStructuredOutputInstruction()

Returns the structured output instruction to append to system messages. Returns null if not in structured output mode.

protected string? GetStructuredOutputInstruction()

Returns

string

ProcessFunctionCallAsync(FunctionCall)

Process function call

protected virtual Task<FunctionCallResult> ProcessFunctionCallAsync(FunctionCall functionCall)

Parameters

functionCall FunctionCall

Returns

Task<FunctionCallResult>

ProcessFunctionCallsAsync(FunctionCallBatch, FunctionCallingPolicy, CancellationToken)

Executes one validated provider batch using the configured execution mode.

protected virtual Task<FunctionCallResultBatch> ProcessFunctionCallsAsync(FunctionCallBatch functionCalls, FunctionCallingPolicy policy, CancellationToken cancellationToken = default)

Parameters

functionCalls FunctionCallBatch
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<FunctionCallResultBatch>

ProcessFunctionCallsInParallelAsync(FunctionCallBatch, FunctionCallingPolicy, CancellationToken)

Validates the complete provider batch, then executes calls concurrently while preserving provider order in the returned result batch.

protected virtual Task<FunctionCallResultBatch> ProcessFunctionCallsInParallelAsync(FunctionCallBatch functionCalls, FunctionCallingPolicy policy, CancellationToken cancellationToken = default)

Parameters

functionCalls FunctionCallBatch
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<FunctionCallResultBatch>

ProcessFunctionCallsSequentiallyAsync(FunctionCallBatch, FunctionCallingPolicy, CancellationToken)

Validates the complete provider batch, then executes each call in provider order. Parallel execution is intentionally not part of this contract.

protected virtual Task<FunctionCallResultBatch> ProcessFunctionCallsSequentiallyAsync(FunctionCallBatch functionCalls, FunctionCallingPolicy policy, CancellationToken cancellationToken = default)

Parameters

functionCalls FunctionCallBatch
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

Task<FunctionCallResultBatch>

QuickAskAsync(string, string, string) 

public static Task<string> QuickAskAsync(string apiKey, string prompt, string model = "gpt-4o-mini")

Parameters

apiKey string
prompt string
model string

Returns

Task<string>

QuickAskWithImageAsync(string, string, string, string) 

public static Task<string> QuickAskWithImageAsync(string apiKey, string prompt, string imagePath, string model = "gpt-4.1")

Parameters

apiKey string
prompt string
imagePath string
model string

Returns

Task<string>

ReadSseLinesAsync(HttpResponseMessage, StreamDiagnostics, CancellationToken)

Reads an SSE response body line-by-line with diagnostics, async stream disposal, and structured exception wrapping. Yields raw SSE lines without any provider-specific parsing — callers handle "data:", "[DONE]", JSON, etc.

Behavior:

  • Disposes the underlying response stream via DisposeAsync() in the iterator's finally block. Avoids the NotSupportedException that some HttpContent transports throw on synchronous Dispose.
  • Wraps any read-side exception in StreamReadException with a StreamDiagnostics snapshot taken at the moment of failure.
  • Invokes the service-level OnRawLine callback (set via WithStreamDiagnostics) for every line. Callback exceptions are swallowed so a faulty logger cannot break the stream.
  • Invokes the service-level OnComplete callback exactly once on iterator exit, regardless of how the iteration ended.
  • Honors cancellationToken while a read is pending as well as between reads. A losing pending read is observed and the iterator's finally block disposes the reader and response stream.

The caller owns the diagnostics object and may mutate fields like DataLinesProcessed or AccumulatedTextLength while consuming lines.

protected IAsyncEnumerable<string> ReadSseLinesAsync(HttpResponseMessage response, StreamDiagnostics diagnostics, CancellationToken cancellationToken)

Parameters

response HttpResponseMessage
diagnostics StreamDiagnostics
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

ResolveRequestTimeoutSeconds(FunctionCallingPolicy)

Single source of truth for per-request timeouts. Returns the timeout in seconds for the given policy, or null for no timeout. Providers override this to adjust the timeout per model (e.g. slow "pro" reasoning models that routinely exceed the default). All request paths must obtain their timeout from here via CreateRequestTimeoutCts(FunctionCallingPolicy, CancellationToken).

protected virtual int? ResolveRequestTimeoutSeconds(FunctionCallingPolicy policy)

Parameters

policy FunctionCallingPolicy

Returns

int?

RunAgentAsync(string, int, AIRequestContext?)

Runs a ReAct (Reasoning + Acting) agent loop that repeatedly calls the LLM and executes function calls until the goal is achieved or maxSteps is exceeded.

Reuses existing function calling infrastructure registered via WithFunction. The loop terminates when the LLM returns a text response without any function calls, or when maxSteps is exceeded. 

public virtual Task<string> RunAgentAsync(string goal, int maxSteps = 10, AIRequestContext? context = null)

Parameters

goal string

The goal or task for the agent to accomplish

maxSteps int

Maximum number of agent steps (LLM round-trips) to prevent infinite loops. Default is 10.

context AIRequestContext

Optional per-call request context (e.g. dynamic system message prefix/suffix).

Returns

Task<string>

The final text response from the LLM after completing the goal

Exceptions

AgentMaxStepsExceededException

Thrown when maxSteps is exceeded without a final answer. The exception contains a PartialResponse property with the last assistant message, if any.

RunAgentStreamAsync(string, int, StreamOptions?, AIRequestContext?, CancellationToken)

Runs the ReAct agent loop using the streaming pipeline.

This is the streaming counterpart to RunAgentAsync(string, int, AIRequestContext?). Function calling is forced on for this request so the agent can act, and TextOnly is disabled so the stream can emit a final Completion event. 

public virtual IAsyncEnumerable<StreamingContent> RunAgentStreamAsync(string goal, int maxSteps = 10, StreamOptions? options = null, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

goal string

The goal or task for the agent to accomplish.

maxSteps int

Maximum number of agent steps (LLM round-trips). Default is 10.

options StreamOptions

Optional streaming options. Function calling will be enabled automatically.

context AIRequestContext

Optional per-call request context (e.g. dynamic system message prefix/suffix).

cancellationToken CancellationToken

Cancellation token for the streaming operation.

Returns

IAsyncEnumerable<StreamingContent>

A stream of agent events including text, function calls, and function results.

Exceptions

AgentMaxStepsExceededException

Thrown when maxSteps is exceeded without a final completion event. The exception contains a PartialResponse property with the last assistant message, if any.

SetActivateChat(string) 

public void SetActivateChat(string chatBlockId)

Parameters

chatBlockId string

StreamAsync(Message, AIRequestContext?, CancellationToken)

Simple text streaming with Message input

public IAsyncEnumerable<string> StreamAsync(Message message, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

message Message
context AIRequestContext
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

StreamAsync(Message, StreamOptions, AIRequestContext?, CancellationToken)

Core streaming implementation using Template Method pattern. Manages the round loop, StatelessMode, and conversation summary policy. Providers override StreamRoundAsync(StreamOptions, bool, FunctionCallingPolicy, CancellationToken) to handle a single round. Providers that do not support function calling rounds (e.g., DeepSeek, Sonar) may override this method directly.

public virtual IAsyncEnumerable<StreamingContent> StreamAsync(Message message, StreamOptions options, AIRequestContext? context = null, CancellationToken cancellationToken = default)

Parameters

message Message
options StreamOptions
context AIRequestContext
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

StreamAsync(string, StreamOptions, CancellationToken)

Advanced streaming with options

public IAsyncEnumerable<StreamingContent> StreamAsync(string prompt, StreamOptions options, CancellationToken cancellationToken = default)

Parameters

prompt string
options StreamOptions
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

StreamAsync(string, CancellationToken)

Simple text streaming (most common use case)

public IAsyncEnumerable<string> StreamAsync(string prompt, CancellationToken cancellationToken = default)

Parameters

prompt string
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

StreamCompletionAsync(Message, Func<string, Task>) 

public abstract Task StreamCompletionAsync(Message message, Func<string, Task> messageReceivedAsync)

Parameters

message Message
messageReceivedAsync Func<string, Task>

Returns

Task

StreamCompletionAsync(string, Action<string>) 

public virtual Task StreamCompletionAsync(string prompt, Action<string> messageReceived)

Parameters

prompt string
messageReceived Action<string>

Returns

Task

StreamCompletionAsync(string, Func<string, Task>) 

public virtual Task StreamCompletionAsync(string prompt, Func<string, Task> messageReceivedAsync)

Parameters

prompt string
messageReceivedAsync Func<string, Task>

Returns

Task

StreamCoreAsync(Message, StreamOptions, CancellationToken)

Core streaming loop. Override this method to replace the full streaming pipeline (round loop, StatelessMode, summary policy). Most providers should override StreamRoundAsync(StreamOptions, bool, FunctionCallingPolicy, CancellationToken) instead.

protected virtual IAsyncEnumerable<StreamingContent> StreamCoreAsync(Message message, StreamOptions options, CancellationToken cancellationToken = default)

Parameters

message Message
options StreamOptions
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>

StreamOnceAsync(Message, CancellationToken)

Streams as one-off query without affecting conversation history

public IAsyncEnumerable<string> StreamOnceAsync(Message message, CancellationToken cancellationToken = default)

Parameters

message Message
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

StreamOnceAsync(string, CancellationToken)

Streams as one-off query without affecting conversation history

public IAsyncEnumerable<string> StreamOnceAsync(string prompt, CancellationToken cancellationToken = default)

Parameters

prompt string
cancellationToken CancellationToken

Returns

IAsyncEnumerable<string>

StreamParseJson(string)

Parses streaming JSON data

protected abstract string StreamParseJson(string jsonData)

Parameters

jsonData string

Returns

string

StreamRoundAsync(StreamOptions, bool, FunctionCallingPolicy, CancellationToken)

Executes a single streaming round: sends an HTTP request, reads the SSE stream, yields chunks, and handles function execution if detected. Yield a FunctionResult to signal the template to continue to the next round; otherwise the stream ends.

protected virtual IAsyncEnumerable<StreamingContent> StreamRoundAsync(StreamOptions options, bool useFunctions, FunctionCallingPolicy policy, CancellationToken cancellationToken)

Parameters

options StreamOptions
useFunctions bool
policy FunctionCallingPolicy
cancellationToken CancellationToken

Returns

IAsyncEnumerable<StreamingContent>
Edit this page