Mead.Router (mead v0.1.1)

Copy Markdown View Source

Turn handler that sends each turn to a local or a cloud model.

The route is the one forced with Mead.submit_turn/3 (route: :local or route: :cloud) or, for route: :auto, the one Mead.Router.Classifier picks. The route names a slot, and each slot holds a {provider_module, provider_options} pair implementing Mead.Provider, read from the application environment:

config :mead, :router,
  slots: [
    local: {Mead.Provider.Ollama, model: "llama3.1"},
    cloud: {Mead.Provider.Anthropic, model: "claude-sonnet-5-5"}
  ]

A turn routed to an empty slot fails with :local_unavailable or :cloud_unavailable and nothing is sent anywhere. In particular a turn meant for the local model is never sent to the cloud on its own.

Tools

When the provider of the chosen slot exports converse/3, whichever the route, and Mead.MCP.Hub.tools_for/1 shows at least one tool for the turn, the turn becomes a loop: the model answers or asks for tools, each requested call goes to Mead.MCP.Hub.call_tool/3, and the results are sent back, until the model answers with text. The loop stops after eight provider calls with :tool_loop_limit. A tool that fails, is not allowed, or does not exist is reported to the model as an error result and is never retried; the turn does not fail for it. The turn option tools: :effect also shows tools that have side effects; see Mead.MCP.Policy. The whole loop runs inside the turn timeout, so a conversation that uses tools may need a larger :turn_timeout than the default of 30 seconds.

A tool result goes back to the model of the route that asked for it. A local turn therefore reaches every MCP server the cloud route reaches, including servers that talk to the network, and the result is seen only by the local model. A local model that does not support tools fails the turn with :tools_unsupported as soon as any tool is visible; the way out is route: :cloud, another model, or no tools configured.

The router keeps no state, so every conversation stays independent of the others. Each turn emits [:mead, :router, :route] with the measurements duration (in native time units), input_tokens, and output_tokens (summed over every provider call of the turn, and zero when the first call failed), and tool_calls (how many tool calls the model asked for), and the metadata route, route_source (:classified or :forced), provider (the module, or nil for an empty slot), status (:ok or :error), and session_id (the session the server passed in the options, nil outside a session). A provider that raises instead of returning an error emits no event.

Summary

Functions

Routes one turn through the given slots, with tools from hub.

Functions

route(history, input, options, slots, hub \\ Mead.MCP.Hub)

@spec route([Mead.Session.Turn.t()], term(), keyword(), keyword(), module()) ::
  {:ok, String.t()} | {:error, term()}

Routes one turn through the given slots, with tools from hub.

handle_turn/3 is this function with the slots read from the application environment and the default hub; it exists separately so tests can pass their own slots and hub without touching global state. A caller of Mead.submit_turn/3 cannot choose the hub.