Skip to main content

Class: VLLM

Unified language model interface

Extends​

Constructors​

new VLLM()​

new VLLM(params): VLLM

Parameters​

• params: VLLMParams

Returns​

VLLM

Overrides​

OpenAI.constructor

Defined in​

packages/providers/vllm/dist/index.d.ts:16

Properties​

additionalChatOptions?​

optional additionalChatOptions: OpenAIAdditionalChatOptions

Inherited from​

OpenAI.additionalChatOptions

Defined in​

packages/providers/openai/dist/index.d.ts:240

additionalSessionOptions?​

optional additionalSessionOptions: Omit<Partial<ClientOptions>, "apiKey" | "maxRetries" | "timeout">

Inherited from​

OpenAI.additionalSessionOptions

Defined in​

packages/providers/openai/dist/index.d.ts:244

apiKey?​

optional apiKey: string

Inherited from​

OpenAI.apiKey

Defined in​

packages/providers/openai/dist/index.d.ts:241

lazySession()​

lazySession: () => Promise<LLMInstance$1>

Returns​

Promise<LLMInstance$1>

Inherited from​

OpenAI.lazySession

Defined in​

packages/providers/openai/dist/index.d.ts:245

maxRetries​

maxRetries: number

Inherited from​

OpenAI.maxRetries

Defined in​

packages/providers/openai/dist/index.d.ts:242

maxTokens?​

optional maxTokens: number

Inherited from​

OpenAI.maxTokens

Defined in​

packages/providers/openai/dist/index.d.ts:239

model​

model: string & object | ChatModel

Inherited from​

OpenAI.model

Defined in​

packages/providers/openai/dist/index.d.ts:236

temperature​

temperature: number

Inherited from​

OpenAI.temperature

Defined in​

packages/providers/openai/dist/index.d.ts:237

timeout?​

optional timeout: number

Inherited from​

OpenAI.timeout

Defined in​

packages/providers/openai/dist/index.d.ts:243

topP​

topP: number

Inherited from​

OpenAI.topP

Defined in​

packages/providers/openai/dist/index.d.ts:238

Accessors​

metadata​

Get Signature​

get metadata(): LLMMetadata & object

Returns​

LLMMetadata & object

Inherited from​

OpenAI.metadata

Defined in​

packages/providers/openai/dist/index.d.ts:252

session​

Get Signature​

get session(): Promise<LLMInstance$1>

Returns​

Promise<LLMInstance$1>

Inherited from​

OpenAI.session

Defined in​

packages/providers/openai/dist/index.d.ts:246

supportToolCall​

Get Signature​

get supportToolCall(): boolean

Returns​

boolean

Inherited from​

OpenAI.supportToolCall

Defined in​

packages/providers/openai/dist/index.d.ts:251

Methods​

chat()​

chat(params)​

chat(params): Promise<AsyncIterable<ChatResponseChunk<ToolCallLLMMessageOptions>, any, any>>

Get a chat response from the LLM

Parameters​

• params: LLMChatParamsStreaming<OpenAIAdditionalChatOptions, ToolCallLLMMessageOptions>

Returns​

Promise<AsyncIterable<ChatResponseChunk<ToolCallLLMMessageOptions>, any, any>>

Inherited from​

OpenAI.chat

Defined in​

packages/providers/openai/dist/index.d.ts:255

chat(params)​

chat(params): Promise<ChatResponse<ToolCallLLMMessageOptions>>

Parameters​

• params: LLMChatParamsNonStreaming<OpenAIAdditionalChatOptions, ToolCallLLMMessageOptions>

Returns​

Promise<ChatResponse<ToolCallLLMMessageOptions>>

Inherited from​

OpenAI.chat

Defined in​

packages/providers/openai/dist/index.d.ts:256

complete()​

complete(params)​

complete(params): Promise<AsyncIterable<CompletionResponse, any, any>>

Get a prompt completion from the LLM

Parameters​

• params: LLMCompletionParamsStreaming

Returns​

Promise<AsyncIterable<CompletionResponse, any, any>>

Inherited from​

OpenAI.complete

Defined in​

packages/core/llms/dist/index.d.ts:163

complete(params)​

complete(params): Promise<CompletionResponse>

Parameters​

• params: LLMCompletionParamsNonStreaming

Returns​

Promise<CompletionResponse>

Inherited from​

OpenAI.complete

Defined in​

packages/core/llms/dist/index.d.ts:164

streamChat()​

protected streamChat(baseRequestParams): AsyncIterable<ChatResponseChunk<ToolCallLLMMessageOptions>, any, any>

Parameters​

• baseRequestParams: ChatCompletionCreateParams

Returns​

AsyncIterable<ChatResponseChunk<ToolCallLLMMessageOptions>, any, any>

Inherited from​

OpenAI.streamChat

Defined in​

packages/providers/openai/dist/index.d.ts:257

toOpenAIMessage()​

static toOpenAIMessage(messages): ChatCompletionMessageParam[]

Parameters​

• messages: ChatMessage<ToolCallLLMMessageOptions>[]

Returns​

ChatCompletionMessageParam[]

Inherited from​

OpenAI.toOpenAIMessage

Defined in​

packages/providers/openai/dist/index.d.ts:254

toOpenAIRole()​

static toOpenAIRole(messageType): ChatCompletionRole

Parameters​

• messageType: MessageType

Returns​

ChatCompletionRole

Inherited from​

OpenAI.toOpenAIRole

Defined in​

packages/providers/openai/dist/index.d.ts:253

toTool()​

static toTool(tool): ChatCompletionTool

Parameters​

• tool: BaseTool<any>

Returns​

ChatCompletionTool

Inherited from​

OpenAI.toTool

Defined in​

packages/providers/openai/dist/index.d.ts:258