Middlewares

A middleware is a function or handler that processes and manipulates chat messages and context as they flow through the chat engine. Middlewares are used to add various functionalities, such as transforming messages, interacting with external APIs, managing state, and handling errors. They are a core part of the chat engine's architecture, allowing for modular and extensible processing of chat interactions.

Middlewares enable the separation of concerns, making it easier to add, remove, or modify functionalities without affecting the core logic of the chat engine. Each middleware performs a specific task and can be composed together to form a processing pipeline, ensuring that chat messages are handled efficiently and effectively.

How Middleware Works

The chat engine processes messages by passing them through a series of middleware functions. Each middleware function receives the current context and a next function. The middleware can perform its task and then call next to pass control to the next middleware in the pipeline. If a middleware does not call next, it effectively short-circuits the pipeline, preventing subsequent middlewares from executing.

Middlewares can be conditionally enabled or disabled based on the context, allowing for dynamic behavior. They can also handle errors by catching exceptions and modifying the response accordingly.

The Engine Behind Middleware

The chat engine is responsible for managing the middleware pipeline and executing the middlewares in the correct order. It maintains the context, which includes the chat messages, user information, and other relevant data. The engine provides methods to add middlewares and run the processing pipeline.

When the engine runs, it initializes the context and iterates through the middlewares, invoking each one with the context and the next function. The engine also supports caching, service resolution, and tracing, which can be utilized by the middlewares to enhance their functionality.

The engine's architecture allows for high flexibility and extensibility, making it easy to integrate new features and adapt to different use cases.

Example

Here is an example of how to use the chat engine with middlewares:

import { ChatEngineBuilder } from '@indigohive/cogfy-chat-engine'

const app = new ChatEngineBuilder()
  .instructions({ instructions: 'You are a helpful assistant' })
  .history({ maxCount: 20 })
  .openai({ model: 'gpt-4o-mini', openai: { apiKey: process.env.OPENAI_API_KEY } })
  .build()

;(async function main () {
  const result = await app.run({
    messages: [
      { role: 'user', content: 'Hello' }
    ]
  })

  console.log(result.messages)
  console.log(JSON.stringify(result.trace, null, 2))
})()

In this example, the chat engine is configured with three middlewares: instructions, history, and openai. Each middleware adds specific functionality to the chat processing pipeline, and the engine manages the execution flow.