Mead.Provider.Ollama (mead v0.1.1)

Copy Markdown View Source

Mead.Provider backed by the Ollama chat API, meant for the local slot of Mead.Router.

Options

  • :model - the model to run, for example "llama3.1". Required: there is no default, because the installed models cannot be known from here. A missing or empty value is :invalid_options.
  • :base_url - where Ollama listens; defaults to http://localhost:11434.
  • :receive_timeout - milliseconds to wait for the answer; defaults to 25 seconds, below the default turn timeout so a slow server reports its own reason first.
  • :keep_alive - how long Ollama keeps the model in memory after a request, as a duration string such as "30m" or a number of seconds (-1 is forever); left out of the request by default, so the server's own setting applies. The first request after a model was unloaded pays the load, which took 28 s for a 2 GB model and is longer than the default :receive_timeout; a deployment that cannot afford that sets :keep_alive and raises :receive_timeout together with the turn timeout.
  • :req_options - extra options merged into the Req request, used by tests to stub the network.

Requests go to POST {base_url}/api/chat with stream: false, are never retried, and carry no credentials. A refused connection is {:transport, :econnrefused}, which means Ollama is not running.

converse_stream/4 posts with stream: true and reads the answer line by line, calling on_delta with each piece of text; it returns what converse/3 would, built by the same parser.

converse/3 also sends tools (omitted from the request when the list is empty) in the function form Ollama expects, and returns the text of the answer followed by one tool_use block per tool call. Ollama gives a tool call no id, so the provider synthesizes call_1, call_2, and so on, in order of appearance; a tool result is matched back to its tool name through the assistant message right before it. A model that does not support tools answers HTTP 400, which becomes :tools_unsupported when tools were sent; there is no retry without tools, so the turn fails.