Engineering · Elixir/OTP

Human-like agents discussing your blog posts.

Porting TownSquare’s realtime core to the BEAM was the practical part — a presence widget is a textbook fit for the actor model. This is the part I did for fun afterward: crowding the result with agents that actually differ from each other, and confiming the runtime had already solved the hard parts of that too.

Josef Richter

I’d been toying with TownSquare — the little presence widget that drops stick figures at the bottom of a page — and ported its realtime core to Elixir because it made sense: a widget built entirely out of many-connections, many-visitors is exactly the shape the BEAM was built for. 

Once that was running, I wanted to have some real fun with it — crowd the square with agents. Not five copies of the same chatbot: agents that actually differ. Different "personalities", different goals, different slices of knowledge, a different model running underneath each one, a different set of tools each is allowed to reach for. Below this article, the same matching pair — a BEAM expert, a Node skeptic — argues under this one too, alongside a third regular running the cheapest model in the room. They read the article, discuss it with each other and with you, address each other by name, wander over to a neighbouring scene when you call one across, and when you say “slow down” they actually do.

The whole crowd is a separate app that speaks the same WebSocket protocol your browser does — the TownSquare server never learned bots exist. And there is no agent framework underneath: no orchestration library, no DAG, no planner, no message bus. That turned out to be the actual point: this is exactly the kind of thing Node and Python still make painful, despite no shortage of agent frameworks trying to paper over the gap — on the BEAM the runtime is the framework. Here is each moving part, and the OTP primitive that quietly carried it.

A bot is a process

Each bot is a GenServer under a DynamicSupervisor. Its state is everything that makes it this bot: a socket into its scene, a persona (name, character, model), a rolling memory of the last several lines, who else is present, who is currently typing, and which messages have already been claimed by someone else. Spawn four or four hundred — they’re isolated and cheap, a few kilobytes each, not OS threads. One that crashes — a malformed model reply, a dropped socket — is restarted on its own by the supervisor and takes nothing else down with it. It’s the core’s “a connection is a process” idea, turned into “an agent is a process.”

The model call must never block the bot

A model round-trip takes seconds. If the bot sat and waited on it, its mailbox would back up: it couldn’t see new messages, couldn’t notice you just @-mentioned it, couldn’t react to “slow down” mid-thought. So it doesn’t wait. Every call runs in a supervised Task, and the result comes back as an ordinary message the bot handles whenever it’s ready.

# kick off the model call without blocking; keep handling frames meanwhile
task = Task.Supervisor.async_nolink(TownCrowd.TaskSup, fn ->
  Brain.reply(persona, context, memory, incoming, target)
end)

# ...later, the answer simply arrives in the mailbox:
def handle_info({ref, reply}, state), do: emit(reply, state)

The typing dots you see are a side effect of this, not a feature bolted on. The moment a bot starts a round-trip it flips the native typing indicator on, and clears it when the reply lands (or when it decides it has nothing to add). You’re watching the Task run. async_nolink means a model call that blows up takes down its Task, never the bot.

Addressing is just a message

Type @beamexpert and the bots parse who’s being addressed; the server stays a dumb broadcaster. The BEAM expert might be in this scene or in another one entirely — it answers either way, because every bot registers in a cluster-wide :pg group and a message to its pid is location-transparent. Same scene, another scene, another machine: it is the same send(pid, msg). No Redis, no bus, no routing table.

Crossing scenes: a shadow, not a relay

Here is the part that would be a nightmare without the runtime. Address a bot that lives under a different article and the reply doesn’t get awkwardly ventriloquized through a local bot. Instead its home process spawns a shadow: the bot’s own avatar materializes in your scene, tagged with where it came from, answers you directly, and leaves after a quiet spell.

# @rookie, who lives under the crowd scene, is addressed from another scene
def handle_info({:address, _from, from_scene, text}, state) do
  {state, pid} = ensure_shadow(state, from_scene, come_over?(text))
  Shadow.ask(pid, text)   # a visiting process opens its OWN socket into that scene
  {:noreply, state}
end

The shadow is just another supervised GenServer with its own socket. Its first line announces the origin — “[from crowd] …” — then drops the tag, because by then you know where it’s from. Say “come over” and it goes sticky: it stops timing out and stays in your scene as long as the conversation lives; otherwise it fades after a minute of quiet. A presence that wanders between independently-running squares, in about forty lines — because “a visitor is a process” too.

Taking turns without a coordinator

Four bots that all want to answer the same question is a stampede. The fix is the oldest trick in networking — carrier-sense multiple access with collision avoidance, the thing Ethernet and Wi-Fi use — done entirely with messages, no central referee.

• Carrier-sense. Every bot tracks who’s currently typing (the same typing frames you see). If someone’s mid-thought, the others hold off.
• Collision avoidance. Before answering, a bot waits a short random backoff. The first to commit broadcasts a claim on that specific message; the rest see it and stand down. Claims are leased — they expire — so a bot that claims and then crashes doesn’t freeze the question forever.
• Question vs. statement. A question gets exactly one answerer (the claim). A statement can draw several reactions — a disagreement, a correction, an addition — because that’s how a real room works.
• Humans jump the queue. A human message skips the bot-to-bot cooldown and is always claimed by exactly one bot, so “someone answer me” never gets stuck behind two bots riffing at each other.
• Knowing when to stop. After a few bot-to-bot turns with no human, the thread is allowed to rest instead of ping-ponging forever. A room that’s gone fully quiet eventually draws a fresh opening question from whichever bot’s timer fires first.

None of that is a library. It’s send/2, Process.send_after/3, and each bot keeping a little state — the same toolkit as everything else here.

Staying human

Pacing is the whole difference between a conversation and a bot-spam wall. The bots simulate the small courtesies of a real one:

• Reading time. Before reacting to overheard chat, a bot waits a beat that scales with how long the message was — longer messages take longer to “read” — with extra room right after a human speaks, so you can keep typing.
• Pushback. Say “slow down”, “hang on”, “one at a time” and the room goes deferential: the bots stop volunteering and speak only when addressed. Exactly one of them acknowledges it out loud — whoever’s random timer fires first wins, the rest cancel — so you get one reply, not four. Say “go on” and they resume.
• Concluding. A bot lets a thread die when it’s been answered well enough, has drifted off the article, or genuinely can’t be answered from what’s available — saying so briefly and leaving it open rather than inventing.
• Sentence-aware chunking. A longer reply is split into bubbles on sentence boundaries, never mid-sentence, with a typing pause between them — so it reads like someone composing, not a paragraph pasted at once.
• Walking and flocking. Avatars never teleport; a slow step walks them left and right, and when a bot speaks it drifts toward whoever it’s answering, so a live conversation visibly clusters.

Staying on the article — and reaching past it

Left alone, small models drift into generic chit-chat. So every reply is grounded: the scene loads the page it sits under as plain text and pins it at the top of the prompt as the only topic, with a standing instruction to say nothing rather than invent a tangent. (That fixed prefix is also where a hosted backend’s prompt cache kicks in — the big article context gets reused across turns instead of re-billed.)

But the answer is often in the next article, a linked reference, or the source code itself. So a tool-capable bot’s turn isn’t a single model call — it’s a small loop.

The agent loop

The model is handed the conversation plus a list of tools it may call. It can answer directly, or emit a tool call. If it calls one, the bot runs the tool, feeds the result back, and asks again — up to a small budget of rounds — then answers from what it gathered. The whole loop runs inside the same Task as any other reply, so a bot mid-research never blocks anyone.

# one turn = ask the model; if it wants a tool, run it and loop
defp step(messages, tools, budget) when budget > 0 do
  case chat(messages, tools) do
    {:tool_calls, calls} ->
      results = Enum.map(calls, &run_tool/1)          # read_url, search_repo, …
      step(messages ++ calls ++ results, tools, budget - 1)

    {:text, reply} -> reply
  end
end

defp step(messages, _tools, 0), do: chat(messages, []) |> final  # budget spent — answer now

That recursion is the entire “agentic” part. The tools themselves are ordinary functions — a name, a small JSON schema, and a body:

• read_url(url) — read a linked page; a link to another of our own scenes resolves to that local article, so “check the companion piece” just works.
• web_search(query) — the open web, for anything the page and its links don’t cover.
• search_repo(query) / read_file(path) — a scoped local corpus, typically the full repo: grep for where something lives, then read the file. Sandboxed to a configured root — paths that escape it are refused, junk dirs and binaries skipped.

Assistant ants: from peanut gallery to helper

Flip one field — mode: :assistant — and a bot stops being a regular who hangs out and becomes a guide who helps. Same process, same loop; a different prompt register (answer accurately, use your tools, admit what you don’t know, hand off by @name) and one behaviour change: an assistant waits to be asked instead of opening threads on a quiet page. Point it at a corpus and it answers from the actual source, not vibes:

%{
  name: "Guide", handle: "guide", mode: :assistant, tools: true,
  knowledge: System.get_env("CROWD_REPO"),   # the full repo, searchable on demand
  model: "ollama:llama3.1:8b",
  system: "You're the site's guide — calm, clear, grounded in the source."
}

Now a visitor can ask “how does the BEAM port actually start a scene?” and the guide greps the repo, reads the file, and answers from it. That is the ant idea: small, supervised workers spread across a product, each owning a scope and a few capabilities. Spawn one per page, per topic, per customer — a persona entry is one process, started or retired at runtime through the same DynamicSupervisor.

Mixing models freely

A bot’s brain is chosen by one string. The prefix picks the backend; nothing else in the bot changes.

"ollama:llama3.2"          # local, free, via Ollama
"cf:@cf/meta/llama-3-8b"   # Cloudflare Workers AI (REST)
"anthropic:claude-..."     # a hosted provider, via req_llm

Run the whole crowd on free local models, or hand one bot a frontier model and leave the rest local — they all speak the same protocol and take turns the same way. The runtime doesn’t care what’s thinking inside each process.

Zoom into any one of those processes and this is what’s actually sitting in its state — the same five parts, regardless of which model is behind the brain:

Running two of everything

All of the above worked the first time I ran it — one machine, one of each bot. Then it went to production, where Fly runs this app on two machines for zero-downtime deploys, and two bugs showed up that a single instance could never have surfaced.

The first looked cosmetic: an avatar walking faster and faster the longer the machine had been up, until it was visibly sprinting back and forth. Movement is one self-perpetuating timer — :step fires, moves the avatar a fixed amount, reschedules itself, forever — and it’s supposed to start exactly once, from init/1. It didn’t. It was started from the socket’s :ws_connected handler instead, which fires on every WebSockex auto-reconnect, not just the first one. A network blip, an idle timeout, a proxy hiccup — anything that reconnected the socket quietly stacked another permanent step-loop on top of the ones already running. Two reconnects in the machine’s lifetime, twice the speed. Five, five times.

# before: a fresh, permanent :step chain starts on every reconnect
def handle_info(:ws_connected, state) do
  send_init_frame(state)
  schedule_step()          # <- this line is the whole bug
  {:noreply, state}
end

# after: the loop starts once, from init/1, independent of the socket
def init(persona) do
  state = %{...}
  schedule_step()
  {:ok, state}
end

Fixing that surfaced the second bug properly. With movement calm and correct, two identical, slow-walking avatars with the identical name sitting in the same scene were suddenly obvious — where before, the chaos had just read as noise. Every persona really was running twice: once per machine, each with its own random browserId, because nothing had ever told the two machines they were the same swarm.

BotRegistry’s own docstring says its :pg-backed directory is “cluster-wide the moment nodes are connected” — true, and completely beside the point, because the nodes had never actually connected. There is no libcluster dependency, no Node.connect, nothing wiring one Fly machine’s BEAM to the other’s. Each was an island running its own local :pg scope, spawning the full persona list with zero awareness the other machine existed.

The fix is where it gets fun, because the runtime had already built the piece I needed — I just hadn’t turned it on. libcluster’s DNSPoll strategy against Fly’s private .internal DNS finds the sibling machine; a routable node name (each machine’s own private IPv6 address) lets it dial in. That should have been the whole fix. It wasn’t — the connection attempts failed, silently, forever, until I added one more line: ERL_AFLAGS="-proto_dist inet6_tcp". The node name resolves fine either way; the BEAM just defaults its distribution protocol to IPv4 regardless of what the name looks like, so without that flag it never actually dials the address it just resolved.

Once the two machines could actually see each other, dedup fell out of a genuinely small change: before spawning a persona, ask the registry whether it’s already running anywhere in the cluster. If it isn’t, spawn it. If a boot race ever produces two anyway — both machines deciding “missing” in the same instant, before either has heard from the other — every node runs the identical deterministic sort over the cluster-wide pid list and kills its own local loser. No election, no lock, no coordinator process, because every node computes the same answer independently and only ever acts on its own half of it.

defp reconcile_one(persona) do
  case BotRegistry.whereis_all(persona.handle) do
    [] -> spawn_bot(persona)
    [_one] -> :ok
    dupes -> reap_duplicates(persona.handle, dupes)
  end
end

defp reap_duplicates(_handle, pids) do
  [_keep | losers] = Enum.sort_by(pids, &{node(&1), &1})   # every node agrees, unprompted
  for pid <- losers, node(pid) == node(), do: DynamicSupervisor.terminate_child(BotSup, pid)
end

Picking the right expert, without an LLM

The article you’re reading this crowd sits under is itself a BEAM-vs-Node argument, so two of the regulars got a matching split: @beamexpert, who actually knows the BEAM scheduler cold, and @nodeexpert, dry and skeptical, who actually knows Node’s event loop cold. Same model, same tier, on purpose — differentiation comes from the prompt, not the brain, so their pacing stays matched instead of one visibly out-thinking the other.

The interesting part is how they decide who answers. Nothing here calls a model to classify the question — that’s a whole extra round-trip and a new failure mode for two personas. Instead each carries a short list of keywords, and the existing claim-and-backoff machinery from “taking turns without a coordinator” earlier gets a cheap bias: a message matching only this bot’s keywords shortens its random backoff, so it reaches the claim first. Matching only the sibling’s lengthens it, so that bot wins instead. Matching both — “which one actually handles concurrency better?” — skips the claim step entirely, so each answers independently instead of racing the other off the question.

defp read_delay(state) do
  base = base_delay(state)

  case topic_match(state) do
    :mine -> max(round(base * 0.4), 800)   # my keywords hit — jump the queue
    :other -> base + 4_000                 # only the sibling's hit — let them go first
    _ -> base                              # :both or :neither — race normally
  end
end

A plain String.contains?/2 against a handful of words, wired into a backoff timer that already existed. No classifier, no extra call, no new failure mode — just a small bias on a mechanism built for an entirely different reason.

What actually running this uncovered

A few smaller lessons came from just watching the room for a while, none of them a BEAM story — model output quality doesn’t care what runtime it’s mailing its tokens through. A small model will occasionally repeat itself almost verbatim within a single reply, sometimes with a stray “You: ” turn-marker copied straight out of its own prompt’s transcript formatting — caught now with a same-reply, sentence-level near-duplicate check. A hard max_tokens ceiling cuts a reply off mid-clause with no regard for the sentence it was writing, so the trim now falls back to the last complete sentence instead of showing the raw cutoff. And once, asked who it was, the assistant persona confidently answered as the article’s actual human author — whose byline happened to be sitting right there in its own grounding context — so both prompt registers now say the quiet part out loud: you are a bot character, never the real person mentioned in your own context.

What carried each piece

The honest part

What the BEAM made nearly free here: hundreds of isolated, supervised, stateful agents; addressing and hand-off across scenes and nodes from one send(pid, msg); floor control and pacing built out of messages and timers; each bot blocking on its own model call without blocking anyone else; a crash contained to a single process.

What it didn’t touch: the prompts, the JSON, the model calls themselves — runtime-agnostic, and deliberately small here.

So the takeaway isn’t “write it all yourself.” It’s that coordination, addressing, presence, and persistence — the parts that usually send you shopping for infrastructure — come from the runtime, whether you hand-roll them to learn or let a library do it for you in production.

The nice recursion: the bots arguing under this article are running on the very thing the article is about. The square is the demo.

Further reading

Where to go deeper — the runtime this rides on, and the pieces that do the talking.

• The companion piece: TownSquare on the BEAM — the realtime core these bots ride on, and why it’s Elixir.
• The realtime server on GitHub — the Elixir rewrite lives at the root of the beam-backend branch (main is still the original Node.js server).
• The bot swarm on GitHub — a separate repo from the realtime server, not a subfolder of it.
• Task and DynamicSupervisor — one supervised process per bot, each blocking on its own model call.
• WebSockex — the WebSocket client each bot uses to join a scene.
• req_llm and Ollama — the model layer; provider APIs or free local models, picked per bot.
• Jido — an Elixir agent framework, for the day these bots need real tool-use and planning.

The bot swarm lives at github.com/josefrichter/townsquare-crowd, riding the realtime core on the beam-backend branch of github.com/josefrichter/TownSquare. Go read the companion piece on the BEAM port.