morph 0.1.0
Asynchronous bridge between GUI thread and business models
Loading...
Searching...
No Matches
remote.hpp
1// SPDX-License-Identifier: Apache-2.0
2
3#pragma once
4#include <algorithm>
5#include <array>
6#include <atomic>
7#include <cassert>
8#include <chrono>
9#include <core/async/StopToken.hpp>
10#include <cstddef>
11#include <cstdint>
12#include <functional>
13#include <iterator>
14#include <latch>
15#include <map>
16#include <memory>
17#include <mutex>
18#include <optional>
19#include <random>
20#include <set>
21#include <stdexcept>
22#include <string>
23#include <string_view>
24#include <unordered_map>
25#include <unordered_set>
26#include <utility>
27#include <vector>
28
29#include "../attributes.hpp"
30#include "../core/backend.hpp"
31#include "../journal/action_log.hpp"
32#include "../session/session.hpp"
33#include "changes.hpp"
34#include "completion.hpp"
35#include "connection_writer.hpp"
40#include "effects.hpp"
41#include "logger.hpp"
42#include "observability.hpp"
43#include "owner_strand.hpp"
44#include "profiler.hpp"
45#include "timeout_scheduler.hpp"
46#include "wire.hpp"
47
48namespace morph::backend {
49
50/// @brief Opt-in, connection-agnostic resource limits enforced by `RemoteServer`.
51///
52/// Every field defaults to `0`, meaning "unbounded": a default-constructed
53/// policy applies no limit. Given as `ServerConfig::limits`, at construction.
54/// See `docs/spec/core/backend.md`.
56 /// @brief Max wall-clock time a single `execute` may take before the server
57 /// sends an `err "timeout"` reply and discards the eventual strand
58 /// result. `0` = no timeout (today's behavior).
59 ///
60 /// A handler that succeeds after the timeout answered still tells the
61 /// connections watching what it changed, the caller included, by event.
62 ///
63 /// An ordinary handler is never interrupted — it keeps running to
64 /// completion on its strand, so this bounds the *caller's wait*, not the
65 /// model. A handler returning `core::async::Task` is also asked to stop: its
66 /// stop token is requested, and it unwinds at its next stop-aware
67 /// `co_await` (see `docs/spec/core/coroutines.md`).
68 std::chrono::milliseconds executeTimeout{0};
69
70 /// @brief Max models this `RemoteServer` will hold live at once, across all
71 /// callers. A `register` beyond this cap replies `err "too many
72 /// models"`. `0` = unbounded (today's behavior).
73 std::size_t maxLiveModels{0};
74
75 /// @brief Max concurrent in-flight `execute` calls this server will accept
76 /// before replying `err "server busy"` instead of dispatching.
77 /// `0` = unbounded (today's behavior).
78 std::size_t maxInFlightExecutes{0};
79};
80
81/// @brief Whether `RemoteServer` requires an `execute` body to carry every
82/// field the action's served schema marks `required`.
83///
84/// The one mechanical check `morph::wire`'s published action-evolution policy
85/// (docs/spec/core/wire.md, "Action-evolution policy") supports without
86/// contradicting itself. The policy's first bullet — *"New fields must be
87/// optional … so an older peer that omits them decodes cleanly"* — makes
88/// optionality, and only optionality, the wire's marker for *may be absent*.
89/// A field that is **not** optional therefore may not be absent, and the
90/// lenient `fromJson` cannot notice when it is: an absent field is
91/// default-constructed and an unknown one is dropped, so a client/server
92/// rename (`amount` → `amountCents`) decodes to a zero-valued action that
93/// `validate()` cannot tell from a legitimate zero.
94///
95/// @warning Not the default, and deliberately so. Turning it on rejects
96/// payloads a pre-existing client sends today, which is a non-additive change
97/// to the wire contract — precisely what the same policy says requires a
98/// `kProtocolVersion` bump. Enabling it by default would break the policy in
99/// the act of enforcing it. See docs/spec/core/wire.md, "Enforcing the policy".
100enum class PayloadCompleteness : std::uint8_t {
101 /// @brief Today's behaviour: the action codec's lenient decode is the only
102 /// gate, and a missing field is a default-constructed one.
103 Lenient,
104 /// @brief Reject an `execute` whose `body` omits a key the action's served
105 /// schema lists in `required`, with
106 /// `err "payload missing required field(s): …"`.
107 RequireDeclaredFields,
108};
109
110namespace detail {
111
112/// @brief Keyed 64-bit bijection that turns a monotonic counter into an
113/// unguessable, non-sequential id.
114///
115/// Implements a 4-round Feistel network over two 32-bit halves. A Feistel
116/// network is a bijection over its full domain for *any* round function —
117/// that is what guarantees `RemoteServer` never hands out the same id twice
118/// for two different counter values. What makes the permutation *opaque*
119/// rather than merely "scrambled" is that each round's mixing function folds
120/// in a secret round key drawn once, at construction, from
121/// `std::random_device`: an *unkeyed* public mixing function would be
122/// invertible by anyone reading the source, letting an attacker who observes
123/// one id recover the counter and predict the next; the secret per-round keys
124/// prevent that without needing the mixing function itself to be secret.
125///
126/// This is a self-contained reference construction (no external crypto
127/// dependency), in the same spirit as the hand-rolled HMAC-SHA256 in
128/// `session_auth.hpp`: adequate for the stated defence-in-depth goal (opaque
129/// ids are not the authorization boundary — `IAuthorizer::authorizeInstance`
130/// is), not a cryptographically-audited primitive.
131class OpaqueIdGenerator {
132public:
133 /// @brief Draws four independent 32-bit round keys from `std::random_device`.
134 OpaqueIdGenerator() {
135 std::random_device rd;
136 for (auto& key : _roundKeys) {
137 // std::random_device::result_type is unsigned int on this platform's
138 // standard library, so the cast below is a no-op here -- but the
139 // standard does not guarantee that, so it stays for portability to a
140 // standard library where result_type is wider than uint32_t.
141#if defined(__GNUC__) && !defined(__clang__)
142#pragma GCC diagnostic push
143#pragma GCC diagnostic ignored "-Wuseless-cast"
144#endif
145 key = static_cast<uint32_t>(rd());
146#if defined(__GNUC__) && !defined(__clang__)
147#pragma GCC diagnostic pop
148#endif
149 }
150 }
151
152 /// @brief Applies the keyed permutation to @p counter.
153 ///
154 /// Bijective over the full 64-bit domain: distinct @p counter values
155 /// always produce distinct results (the Feistel structure guarantees
156 /// this regardless of the round function), so a monotonically
157 /// increasing, non-repeating @p counter can never yield a collision.
158 /// @param counter Monotonic input, e.g. from an atomic counter.
159 /// @return A 64-bit value that is a bijective function of @p counter.
160 [[nodiscard]] uint64_t permute(uint64_t counter) const noexcept {
161 auto lo = static_cast<uint32_t>(counter & 0xffffffffULL);
162 auto hi = static_cast<uint32_t>(counter >> 32);
163 for (const uint32_t roundKey : _roundKeys) {
164 const uint32_t nextHi = lo;
165 lo = hi ^ mix(lo, roundKey);
166 hi = nextHi;
167 }
168 return (static_cast<uint64_t>(hi) << 32) | static_cast<uint64_t>(lo);
169 }
170
171private:
172 /// @brief Keyed avalanche mix (fmix32-style) used as the Feistel round function.
173 /// @param half Current 32-bit half being folded into the other half.
174 /// @param key This round's secret key.
175 /// @return A well-mixed 32-bit value depending non-linearly on both @p half and @p key.
176 [[nodiscard]] static uint32_t mix(uint32_t half, uint32_t key) noexcept {
177 uint32_t val = half ^ key;
178 val ^= val >> 16;
179 val *= 0x7feb352dU;
180 val ^= val >> 15;
181 val *= 0x846ca68bU;
182 val ^= val >> 16;
183 return val;
184 }
185
186 std::array<uint32_t, 4> _roundKeys{};
187};
188
189} // namespace detail
190
191/// @brief Snapshot of a `RemoteServer`'s current health.
193 /// @brief `true` if the server currently accepts and dispatches new work.
194 bool ready;
195 /// @brief Number of models currently registered on the server.
196 std::size_t liveModels;
197 /// @brief Number of executes currently dispatched but not yet replied.
198 std::size_t inFlight;
199};
200
201/// @brief Callable that supplies the action log to attach to a newly
202/// registered instance, given its model type and `contextKey`.
203///
204/// Return `nullptr` to register the instance with no log attached (e.g. for
205/// model types or context keys the host app doesn't want journaled).
206using LogProvider = std::function<std::shared_ptr<::morph::journal::IActionLog>(std::string_view modelType,
207 std::string_view contextKey)>;
208
209/// @brief What a `RemoteServer` knows about one connection that a service, not
210/// core, decides: handed to `IEnvelopeService::handle` with each envelope.
211///
212/// A service sets a fact while answering an envelope it serves, and core reads
213/// it later without parsing that service's bodies. The facts live as long as
214/// the connection; a message handled without a connection (`cid` 0) is handed
215/// fresh facts that are dropped after its reply.
217 /// @brief Whether the connection negotiated server-sent events: its
218 /// `ui-hello` offered them and the UI service accepted.
219 bool events = false;
220};
221
222/// @brief Envelope kinds a `RemoteServer` answers through an extension, such as
223/// the UI service's `ui-*` kinds.
224///
225/// Core does not know the kinds a higher layer adds, and must not include that
226/// layer. A server consults its services only for a kind it does not answer
227/// itself, so a service cannot take over `execute`, `hello` or any other
228/// built-in kind; a kind no service serves is still answered
229/// `err "unknown envelope kind: <kind>"`. See `docs/spec/core/backend.md`,
230/// "Envelope services".
231class IEnvelopeService {
232public:
233 IEnvelopeService() = default;
234 virtual ~IEnvelopeService() = default;
235 IEnvelopeService(const IEnvelopeService&) = delete;
236 IEnvelopeService& operator=(const IEnvelopeService&) = delete;
237 IEnvelopeService(IEnvelopeService&&) = delete;
238 IEnvelopeService& operator=(IEnvelopeService&&) = delete;
239
240 /// @brief Whether this service answers @p kind.
241 /// @param kind The envelope's `kind`.
242 /// @return `true` to have `handle` answer it.
243 [[nodiscard]] virtual bool serves(std::string_view kind) const = 0;
244
245 /// @brief Answers one envelope of a kind `serves` accepted.
246 ///
247 /// Runs on the server strand, so it must not block. The server has already
248 /// stamped the verified principal onto `request.session` (the client's
249 /// claim is never seen here); authorising the request is the service's,
250 /// through @p authorizer. A throw becomes an `err` reply carrying its
251 /// message.
252 /// @param request The decoded envelope.
253 /// @param authorizer The server's authoriser.
254 /// @return The encoded reply envelope (`wire::encode`), addressed to
255 /// `request.callId`.
256 [[nodiscard]] virtual std::string handle(const ::morph::wire::Envelope& request,
257 const ::morph::session::IAuthorizer& authorizer) = 0;
258
259 /// @brief Answers one envelope of a kind `serves` accepted, with the facts
260 /// of the connection it arrived on.
261 ///
262 /// What the server calls. A service that learns something about the
263 /// connection, as the UI service learns that `ui-hello` negotiated events,
264 /// overrides this one and records it in @p facts; the default forwards to
265 /// the two-parameter `handle` and leaves @p facts alone. A service
266 /// overriding only one of the two adds `using IEnvelopeService::handle;`,
267 /// so the other is not hidden. Same thread and error rules as the
268 /// two-parameter `handle`.
269 /// @param request The decoded envelope.
270 /// @param authorizer The server's authoriser.
271 /// @param facts The connection's facts, which the service may set;
272 /// fresh ones for a message handled without a connection.
273 /// @return The encoded reply envelope (`wire::encode`), addressed to
274 /// `request.callId`.
275 [[nodiscard]] virtual std::string handle(const ::morph::wire::Envelope& request,
276 const ::morph::session::IAuthorizer& authorizer, ConnectionFacts& facts) {
277 static_cast<void>(facts);
278 return handle(request, authorizer);
279 }
280};
281
282/// @brief Everything about a `RemoteServer` that is configured rather than
283/// learned: given once, at construction, and never changed.
284///
285/// The server reads these fields on its own strand and on its models' strands
286/// without a lock, which is sound only because nothing writes them after the
287/// constructor. A deployment that wants different limits constructs a
288/// different server. See `docs/spec/core/backend.md`, "`ServerConfig`".
290 /// @brief Opt-in resource limits. Default-constructed: unbounded.
292
293 /// @brief Consulted whenever an instance is constructed with a non-empty
294 /// `contextKey` — every `register` envelope, and every `attach` that
295 /// misses the shared directory and therefore creates the instance.
296 ///
297 /// `RemoteServer` owns the model instances of every remote client, so it is
298 /// the only place that can attach an action log to them. Null: no instance
299 /// gets a log. Runs on the server's strand.
300 LogProvider logProvider;
301
302 /// @brief Called with the server's health: once from the constructor, and
303 /// again, on the server's strand, whenever readiness changes
304 /// (`beginShutdown()`). Null: nothing is called.
305 ///
306 /// A deployment's transport can expose it over a probe endpoint; morph does
307 /// not embed an HTTP server.
308 std::function<void(const HealthStatus&)> healthHandler;
309
310 /// @brief Oldest protocol version this server accepts in reply to
311 /// `"hello"`. Must not exceed `maxProtocolVersion`.
312 std::uint32_t minProtocolVersion = ::morph::wire::kProtocolVersion;
313
314 /// @brief Newest protocol version this server accepts in reply to
315 /// `"hello"`. Widen the range when a `kProtocolVersion` bump must keep
316 /// serving older clients through their deprecation window (see
317 /// docs/spec/core/wire.md, "Action-evolution policy").
318 std::uint32_t maxProtocolVersion = ::morph::wire::kProtocolVersion;
319
320 /// @brief Whether an `execute` body must carry every field the action's
321 /// served schema marks `required`. See `PayloadCompleteness` for why
322 /// `Lenient` is the default.
323 PayloadCompleteness payloadCompleteness = PayloadCompleteness::Lenient;
324
325 /// @brief Extensions that answer envelope kinds the server does not know
326 /// itself, consulted in order. Empty: none, and every unknown kind is
327 /// answered `err "unknown envelope kind: <kind>"`. A null entry is
328 /// refused by the constructor.
329 std::vector<std::shared_ptr<IEnvelopeService>> services;
330
331 /// @brief How long a connection's changes are merged before they are sent
332 /// as one `changes` event.
333 ///
334 /// The window opens at the first change queued for the connection, and its
335 /// queue is sent when the window closes, or at once when 256 keys are
336 /// queued. Zero sends each fan-out's event at once, unmerged. Negative is
337 /// refused by the constructor.
338 std::chrono::milliseconds coalesceWindow{50};
339};
340
341/// @brief Server-side message handler that owns model instances and dispatches actions.
342///
343/// `RemoteServer` receives JSON envelopes (`morph::wire::Envelope`) from any
344/// transport (WebSocket, in-process simulation, …) and executes the corresponding
345/// model operations via an `ActionDispatcher`. Authorization is delegated to an
346/// `IAuthorizer` that defaults to allow-all.
347///
348/// @par One owner
349/// The server's state — the instance registry, the connection scopes, the
350/// in-flight count, readiness — belongs to one strand over the worker pool, the
351/// *server strand*, and is touched only in its tasks. Every public verb posts
352/// to it. An `execute` is admitted there and then posted to its model's own
353/// strand, so for one model the order `handle()` was called in is the order the
354/// model runs them: both hops are strands, and a strand runs in post order.
355/// Configuration is a `ServerConfig`, fixed at construction.
356///
357/// @par Heap allocation requirement
358/// `RemoteServer` **must** be heap-allocated via `std::make_shared`. Every task
359/// it posts captures `shared_from_this()`, so the server outlives its queued
360/// work however the last external reference is dropped.
361///
362/// @par Wire format
363/// All requests and replies are encoded as `morph::wire::Envelope` JSON. See
364/// `wire.hpp` for the field semantics. The `kind` field is the discriminator.
365class RemoteServer : public std::enable_shared_from_this<RemoteServer> {
366public:
367 /// @brief Alias of `morph::backend::LogProvider`, the type of
368 /// `ServerConfig::logProvider`.
369 using LogProvider = ::morph::backend::LogProvider;
370
371 /// @brief Constructs a server backed by @p workerPool with allow-all
372 /// authorization and a default `ServerConfig`.
373 ///
374 /// @param workerPool Pool the server strand and every model strand run on.
375 /// Borrowed, not owned: it must outlive this server and
376 /// keep running until teardown completes (see
377 /// `docs/spec/concurrency_and_lifetimes.md`, "Destruction
378 /// ordering").
379 /// @param dispatcher Action dispatcher; defaults to the process-level
380 /// singleton. Borrowed: it must outlive this server.
381 /// @param registry Model factory registry; defaults to the process-level
382 /// singleton. Borrowed: it must outlive this server.
383 /// @throws morph::model::EffectsRegistrationError if the process's record
384 /// and effects declarations are invalid: the server reads them
385 /// here, so such a server is never constructed.
386 explicit RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND,
387 ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND =
388 ::morph::model::detail::defaultDispatcher(),
389 ::morph::model::detail::ModelRegistryFactory& registry MORPH_LIFETIMEBOUND =
390 ::morph::model::detail::defaultRegistry())
391 : RemoteServer{workerPool, ::morph::session::allowAllAuthorizer(), ServerConfig{}, dispatcher, registry} {}
392
393 /// @brief Constructs a server with a custom authorizer and a default
394 /// `ServerConfig`.
395 ///
396 /// @param workerPool Pool the server's strands run on. Borrowed, on the
397 /// same terms as the constructor above.
398 /// @param authorizer Authorizer consulted for every envelope that makes an
399 /// authorization decision; null means allow-all.
400 /// @param dispatcher Action dispatcher; defaults to the process-level
401 /// singleton. Borrowed: it must outlive this server.
402 /// @param registry Model factory registry; defaults to the process-level
403 /// singleton. Borrowed: it must outlive this server.
404 /// @throws morph::model::EffectsRegistrationError if the process's record
405 /// and effects declarations are invalid: the server reads them
406 /// here, so such a server is never constructed.
407 RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND,
408 std::shared_ptr<::morph::session::IAuthorizer> authorizer,
409 ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND =
410 ::morph::model::detail::defaultDispatcher(),
411 ::morph::model::detail::ModelRegistryFactory& registry MORPH_LIFETIMEBOUND =
412 ::morph::model::detail::defaultRegistry())
413 : RemoteServer{workerPool, std::move(authorizer), ServerConfig{}, dispatcher, registry} {}
414
415 /// @brief Constructs a server with allow-all authorization and @p config.
416 ///
417 /// @param workerPool Pool the server's strands run on. Borrowed, on the
418 /// same terms as the first constructor.
419 /// @param config Limits, log provider, health handler, protocol range
420 /// and payload rule, fixed for the server's life.
421 /// @param dispatcher Action dispatcher; defaults to the process-level
422 /// singleton. Borrowed: it must outlive this server.
423 /// @param registry Model factory registry; defaults to the process-level
424 /// singleton. Borrowed: it must outlive this server.
425 /// @throws std::invalid_argument if `config.minProtocolVersion` exceeds
426 /// `config.maxProtocolVersion`, or `config.services` holds a null
427 /// entry.
428 /// @throws morph::model::EffectsRegistrationError if the process's record
429 /// and effects declarations are invalid: the server reads them
430 /// here, so such a server is never constructed.
431 RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND, ServerConfig config,
432 ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND =
433 ::morph::model::detail::defaultDispatcher(),
434 ::morph::model::detail::ModelRegistryFactory& registry MORPH_LIFETIMEBOUND =
435 ::morph::model::detail::defaultRegistry())
436 : RemoteServer{workerPool, ::morph::session::allowAllAuthorizer(), std::move(config), dispatcher, registry} {}
437
438 /// @brief Constructs a server with a custom authorizer and @p config.
439 ///
440 /// Calls `config.healthHandler`, if set, once, before returning, with the
441 /// initial status (ready, nothing registered, nothing in flight).
442 ///
443 /// @param workerPool Pool the server's strands run on. Borrowed, on the
444 /// same terms as the first constructor.
445 /// @param authorizer Authorizer consulted for every envelope that makes an
446 /// authorization decision; null means allow-all.
447 /// @param config Limits, log provider, health handler, protocol range
448 /// and payload rule, fixed for the server's life.
449 /// @param dispatcher Action dispatcher; defaults to the process-level
450 /// singleton. Borrowed: it must outlive this server.
451 /// @param registry Model factory registry; defaults to the process-level
452 /// singleton. Borrowed: it must outlive this server.
453 /// @throws std::invalid_argument if `config.minProtocolVersion` exceeds
454 /// `config.maxProtocolVersion`, or `config.services` holds a null
455 /// entry.
456 /// @throws morph::model::EffectsRegistrationError if the process's record
457 /// and effects declarations are invalid: the server reads them
458 /// here, so such a server is never constructed.
459 RemoteServer(::morph::exec::IExecutor& workerPool MORPH_LIFETIMEBOUND,
460 std::shared_ptr<::morph::session::IAuthorizer> authorizer, ServerConfig config,
461 ::morph::model::detail::ActionDispatcher& dispatcher MORPH_LIFETIMEBOUND =
462 ::morph::model::detail::defaultDispatcher(),
463 ::morph::model::detail::ModelRegistryFactory& registry MORPH_LIFETIMEBOUND =
464 ::morph::model::detail::defaultRegistry())
465 : _strand{workerPool},
466 _strands{std::make_shared<::morph::exec::detail::ModelStrands>(workerPool)},
467 _dispatcher{dispatcher},
468 _registry{registry},
469 _authorizer{authorizer ? std::move(authorizer) : ::morph::session::allowAllAuthorizer()},
470 _config{validated(std::move(config))},
471 _executeTimeouts{_config.limits.executeTimeout.count() > 0
472 ? std::make_unique<::morph::async::detail::TimeoutScheduler>()
473 : nullptr} {
474 // The first read of the record and effects registries: an invalid
475 // declaration fails the server's construction rather than its first
476 // successful execute, whose changes are derived from them.
477 static_cast<void>(::morph::effects::detail::resolved());
478 if (_config.healthHandler) {
479 _config.healthHandler(HealthStatus{.ready = true, .liveModels = 0, .inFlight = 0});
480 }
481 }
482
483 RemoteServer(const RemoteServer&) = delete;
484 RemoteServer& operator=(const RemoteServer&) = delete;
485 RemoteServer(RemoteServer&&) = delete;
486 RemoteServer& operator=(RemoteServer&&) = delete;
487
488 /// @brief Closes the server strand before any other member goes.
489 ///
490 /// Every task the server posts holds the server, so nothing of it is still
491 /// queued here; closing waits for a task still running on another thread
492 /// and returns at once from inside the server's own last task.
494 _strand.seal();
495 _strand.close();
496 }
497
498 /// @brief Asynchronously processes a JSON `Envelope` and calls @p reply with the response.
499 ///
500 /// Decodes on the calling thread and posts the envelope to the server
501 /// strand. @p reply is called exactly once, from a pool thread: the server
502 /// strand's, or, for an `execute`, the model strand's (or the I/O loop's,
503 /// when a `LimitPolicy::executeTimeout` answers first).
504 ///
505 /// Callable from any thread. For one connection, calls made one after the
506 /// other on one thread are handled in that order.
507 ///
508 /// @param msg JSON-encoded `morph::wire::Envelope` (via `wire::encode`).
509 /// @param reply Callback invoked with the JSON-encoded reply envelope.
510 void handle(std::string msg, std::function<void(std::string)> reply) {
511 handleImpl(std::move(msg), std::move(reply), 0);
512 }
513
514 /// @brief Like `handle(msg, reply)`, but additionally attributes any
515 /// `register` decoded from @p msg to a connection scope.
516 ///
517 /// A `register` processed under a non-zero @p cid records the new model in
518 /// that connection's scope, so a later `closeConnection(cid)` reclaims it.
519 /// Passing `cid == 0` is exactly the unscoped, two-argument `handle()` —
520 /// nothing is recorded and nothing is ever cleaned up automatically.
521 ///
522 /// Callable from any thread, on the same terms as `handle(msg, reply)`.
523 ///
524 /// @param msg JSON-encoded `morph::wire::Envelope` (via `wire::encode`).
525 /// @param reply Callback invoked with the JSON-encoded reply envelope.
526 /// @param cid Connection scope to attribute a `register` in @p msg to;
527 /// `0` means unscoped.
528 void handle(std::string msg, std::function<void(std::string)> reply, ConnectionId cid) {
529 handleImpl(std::move(msg), std::move(reply), cid);
530 }
531
532 /// @brief Processes a control `Envelope` on the server strand and returns
533 /// its reply, blocking the caller until it has one.
534 ///
535 /// For `register`, `deregister`, `attach`, `assign`, `instances`, `schemas`
536 /// and `hello` — the synchronous control path `SimulatedRemoteBackend`
537 /// uses. Runs inline when the caller is already on the server strand;
538 /// otherwise posts and waits. `execute` is rejected up front with an `err`
539 /// reply: its reply is produced later, on the model's strand.
540 ///
541 /// Callable from a pool thread, including one inside a running action (a
542 /// handler registered from an action handler): the server strand then runs
543 /// on another pool thread while this one waits. So the pool needs a thread
544 /// free for it — a one-thread pool calling this from inside its own task,
545 /// or every pool thread blocked here at once, never gets its reply. See
546 /// docs/spec/concurrency_and_lifetimes.md, "Synchronous re-entry".
547 ///
548 /// @param msg JSON-encoded `morph::wire::Envelope` (via `wire::encode`).
549 /// @return JSON-encoded reply envelope.
550 std::string handleInline(const std::string& msg) { return handleInline(msg, 0); }
551
552 /// @brief Like `handleInline(msg)`, but additionally attributes any
553 /// `register` decoded from @p msg to a connection scope.
554 ///
555 /// The synchronous counterpart to the scoped `handle(msg, reply, cid)`
556 /// overload, on the same terms as the one-argument `handleInline`.
557 /// Passing `cid == 0` is exactly the unscoped overload above.
558 ///
559 /// @param msg JSON-encoded `morph::wire::Envelope` (via `wire::encode`).
560 /// @param cid Connection scope to attribute a `register` in @p msg to;
561 /// `0` means unscoped.
562 /// @return JSON-encoded reply envelope.
563 std::string handleInline(const std::string& msg, ConnectionId cid) {
564 Decoded decoded = decodeEnvelope(msg);
565 if (decoded.env && decoded.env->kind == "execute") {
566 return ::morph::wire::encode(::morph::wire::makeErr(
567 "handleInline does not support execute (reply is asynchronous)", decoded.env->callId));
568 }
569 std::string out;
570 std::function<void(std::string)> capture = [&out](std::string reply) noexcept { out = std::move(reply); };
571 if (_strand.runningHere()) {
572 dispatchDecoded(std::move(decoded), capture, cid);
573 return out;
574 }
575 std::latch done{1};
576 _strand.postTask([self = shared_from_this(), &decoded, &capture, cid, &done] {
577 // Counted down however the dispatch ends, so the caller is never
578 // left waiting on a task that threw.
579 try {
580 self->dispatchDecoded(std::move(decoded), capture, cid);
581 } catch (...) {
582 done.count_down();
583 throw;
584 }
585 done.count_down();
586 });
587 done.wait();
588 return out;
589 }
590
591 /// @brief Opens a new connection scope and returns its id.
592 ///
593 /// Call once per accepted transport connection (e.g. from a WebSocket
594 /// server's "new connection" callback). The returned id is never `0`, so
595 /// it can always be distinguished from the reserved "unscoped" value. Pass
596 /// it to the scoped `handle(msg, reply, cid)` overload for every message
597 /// received on that connection, and to `closeConnection(cid)` once the
598 /// connection is gone.
599 ///
600 /// Callable from any thread. The id is drawn at once; the scope is opened
601 /// on the server strand, before any `handle()` the same thread makes next.
602 /// @return A fresh, non-zero `ConnectionId`.
603 [[nodiscard]] ConnectionId openConnection() {
604 MORPH_ZONE("RemoteServer::openConnection");
605 ConnectionId const cid{_nextConnectionId.fetch_add(1) + 1};
606 _strand.postTask([self = shared_from_this(), cid] {
607 self->noteOwner("RemoteServer::openConnection");
608 self->_connectionScopes.try_emplace(cid);
609 });
610 return cid;
611 }
612
613 /// @brief Reclaims every model still registered under @p cid, then drops the scope.
614 ///
615 /// Call once the transport observes the connection is gone (disconnect,
616 /// close, error). Releases exactly as many references as this connection
617 /// held, exactly as an explicit `deregister` would, then drops the scope
618 /// itself. A *private* instance (count 1, no directory entry) is erased
619 /// outright, so a later `execute` against its id replies
620 /// `err "model not found"`; a *shared* instance another connection is still
621 /// attached to survives, and an `execute` against it still succeeds — see
622 /// `InstanceDirectory::release`'s `retained` outcome.
623 ///
624 /// Idempotent: `cid == 0`, an unknown `cid`, or a `cid` already closed is a
625 /// no-op. Deliberately does **not** consult `IAuthorizer` — this is the
626 /// server's own housekeeping in reaction to a transport-level event, not
627 /// an action attributable to any caller (see docs/spec/core/backend.md).
628 ///
629 /// Safe while a model in the scope has an `execute` in flight: the model
630 /// strand's task holds its own `shared_ptr` to the model holder, so erasing
631 /// the registry entry here only prevents *new* lookups (see
632 /// docs/spec/concurrency_and_lifetimes.md).
633 ///
634 /// Callable from any thread; runs on the server strand, before any
635 /// `handle()` the same thread makes next.
636 /// @param cid Connection scope to close, as returned by `openConnection()`.
638 if (cid == 0) {
639 return;
640 }
641 _strand.postTask([self = shared_from_this(), cid] { self->closeConnectionHere(cid); });
642 }
643
644 /// @brief Hands the server the transport's writer for connection @p cid,
645 /// through which `post` reaches it.
646 ///
647 /// Replaces any writer @p cid already has, keeping a `resync` the
648 /// connection is owed. Kept only while the connection
649 /// is open: a writer for `0`, for an unknown `cid` or for one already
650 /// closed is dropped, and `closeConnection(cid)` drops it. A writer whose
651 /// `write` is empty removes the connection's writer.
652 ///
653 /// Callable from any thread; runs on the server strand, after an
654 /// `openConnection()` and before any `post` the same thread makes next.
655 /// @param cid Connection, as returned by `openConnection()`.
656 /// @param writer The connection's writer and its backlog read.
658 if (cid == 0) {
659 return;
660 }
661 _strand.postTask([self = shared_from_this(), cid, writer = std::move(writer)]() mutable {
662 self->noteOwner("RemoteServer::setConnectionWriter");
663 if (!self->_connectionScopes.contains(cid)) {
664 return;
665 }
666 if (!writer.write) {
667 self->discardChanges(cid);
668 self->_writers.erase(cid);
669 return;
670 }
671 // A resync already owed stays owed: the events it stands for were
672 // held back from the connection, not from the writer it had then.
673 self->_writers[cid].writer = std::move(writer);
674 });
675 }
676
677 /// @brief Writes @p envelope to connection @p cid, unless its reader has
678 /// fallen behind.
679 ///
680 /// On the server strand, while the connection's `backlog()` exceeds
681 /// `kEventBacklogHigh`, the envelope is dropped and the connection is owed
682 /// one `resync` event, which `connectionDrained(cid)` sends. Until then
683 /// every later envelope is dropped too, so the `resync` is the next thing
684 /// the connection is sent after the hold-back began. Replies do not pass
685 /// through here and are never held back.
686 ///
687 /// Dropped as well: an envelope larger than `wire::kMaxEnvelopeBytes`
688 /// (counted as `Metric::eventOversize`), and one for a connection with no
689 /// writer — `0`, unknown, or closed. A `write` or `backlog` that throws
690 /// is caught on the strand, and the envelope is not written.
691 ///
692 /// Callable from any thread. Envelopes one thread posts to one connection
693 /// are written in that order.
694 /// @param cid Connection to write to.
695 /// @param envelope The encoded envelope (via `wire::encode`).
696 void post(ConnectionId cid, std::string envelope) {
697 if (oversized(cid, envelope)) {
698 return;
699 }
700 if (cid == 0) {
701 return;
702 }
703 _strand.postTask([self = shared_from_this(), cid, envelope = std::move(envelope)]() mutable {
704 self->postHere(cid, std::move(envelope));
705 });
706 }
707
708 /// @brief Tells the server that connection @p cid's backlog has fallen
709 /// below `kEventBacklogLow`.
710 ///
711 /// What a transport calls when it has written enough of what it holds. If
712 /// `post` held an event back since the last drain, the connection is sent
713 /// one `resync` event (`wire::makeEvent(wire::kResyncTopic, "{}")`), meaning
714 /// everything it watches may have changed, and events flow again.
715 /// Otherwise nothing is sent. The drain, not a later write, sends it, so a
716 /// connection that only listens still receives it.
717 ///
718 /// Callable from any thread, the transport's included; it only posts to
719 /// the server strand.
720 /// @param cid Connection whose backlog drained.
722 if (cid == 0) {
723 return;
724 }
725 _strand.postTask([self = shared_from_this(), cid] { self->connectionDrainedHere(cid); });
726 }
727
728 /// @brief Whether connection @p cid negotiated server-sent events.
729 ///
730 /// Server strand only: call it from a task on `strand()`; a debug build
731 /// asserts that. The answer is what the UI service recorded when the
732 /// connection's `ui-hello` offered events, and stays until the connection
733 /// closes; a reconnect is a new connection and negotiates again.
734 /// @param cid Connection, as returned by `openConnection()`.
735 /// @return `true` once the connection negotiated events; `false` for `0`,
736 /// an unknown connection or a closed one.
737 [[nodiscard]] bool eventsNegotiated(ConnectionId cid) const {
738 assert(_strand.runningHere() && "RemoteServer::eventsNegotiated runs on the server strand");
739 auto const found = _facts.find(cid);
740 return found != _facts.end() && found->second.events;
741 }
742
743 /// @brief The topics each open connection watches.
744 ///
745 /// Server strand only: call it from a task on `strand()`; a debug build
746 /// asserts that. Filled by the `watch` and `unwatch` kinds, and emptied for
747 /// a connection when it closes or a write to it fails.
748 /// @return The watch table, valid for the server's life.
749 [[nodiscard]] detail::WatchTable const& watchTable() const {
750 assert(_strand.runningHere() && "RemoteServer::watchTable runs on the server strand");
751 return _watches;
752 }
753
754 /// @brief Tells every connection watching what @p changes names that it
755 /// changed, as one `changes` event per connection.
756 ///
757 /// Server strand only: call it from a task on `strand()`; a debug build
758 /// asserts that. A successful `execute` posts its changes here.
759 ///
760 /// A record topic, and the instance topic of a keyed shared instance, go
761 /// to every connection watching them, @p caller included: the caller
762 /// may never see its own reply (it timed out, the client discarded it, or
763 /// it went to an offline replay). An instance is matched by the directory
764 /// key it holds now, so one an `assign` keyed while its execute ran
765 /// reaches that key's watchers. A private instance's topic goes to every
766 /// connection whose scope holds that instance, which needs no watch, and
767 /// to no other. A connection that has not negotiated events is sent
768 /// nothing.
769 ///
770 /// Each recipient's watches are then re-authorised by the rule that
771 /// admitted them (`reauthorise`): a topic whose watch fails is dropped
772 /// from the table and from the event, and a connection that lost any
773 /// watch is sent one `resync` (`sendResync`), after whatever it had queued. A
774 /// record registered with `morph::record::privateKeys` reaches a principal
775 /// other than @p callerPrincipal without its keys.
776 ///
777 /// What is left is queued on the connection (`detail::ChangeCoalescer`),
778 /// not sent: the queue is merged for `ServerConfig::coalesceWindow` from
779 /// its first change, or until `detail::kCoalesceKeyLimit` keys are queued,
780 /// and then sent as one `changes` event through `post`'s backlog hold-back
781 /// and size cap. Nothing is queued for a connection with no writer, or one
782 /// held back behind its backlog, which its drain's `resync` covers.
783 /// @param changes What one execute changed, from `effects::deriveChanges`.
784 /// @param caller Connection the execute arrived on; `0` for unscoped.
785 /// @param callerPrincipal The execute's verified principal; empty when none was.
786 void fanOut(::morph::effects::Changes changes, ConnectionId caller, std::string_view callerPrincipal) {
787 assert(_strand.runningHere() && "RemoteServer::fanOut runs on the server strand");
788 noteOwner("RemoteServer::fanOut");
789 std::map<ConnectionId, ::morph::wire::ChangesBody> bodies;
790 for (auto const& change : changes.records) {
791 for (ConnectionId const cid : _watches.watchersOf(change.record)) {
792 if (eventsNegotiated(cid)) {
793 bodies[cid].records.push_back(change);
794 }
795 }
796 }
797 for (auto& change : changes.instances) {
798 addInstanceTopic(std::move(change), bodies);
799 }
800 Authentications authentications;
801 for (auto& [cid, body] : bodies) {
802 bool const dropped = reauthorise(cid, body, caller, callerPrincipal, authentications);
803 if (!body.records.empty() || !body.instances.empty()) {
804 queueChanges(cid, body);
805 }
806 if (dropped) {
807 // What was queued before the drop goes first, so the resync is
808 // the last word on everything the connection watched.
809 flushChanges(cid);
810 sendResync(cid);
811 }
812 }
813 }
814
815 /// @brief The server strand, as an executor.
816 ///
817 /// What `runningOn(server.strand())` asks about: true inside every task the
818 /// server runs on its own state, false elsewhere. A task posted to it runs
819 /// in order with the server's own, never beside one.
820 /// @return The strand every envelope, verb and `execute` admission runs on.
821 [[nodiscard]] ::morph::exec::IExecutor& strand() noexcept { return _strand; }
822
823 /// @brief Returns the configured payload-completeness rule.
824 /// @return `ServerConfig::payloadCompleteness`, as given at construction.
825 [[nodiscard]] PayloadCompleteness payloadCompleteness() const noexcept { return _config.payloadCompleteness; }
826
827 /// @brief The server's health, answered on the server strand.
828 ///
829 /// Posted, so the answer reflects every envelope and verb this thread
830 /// handed the server before calling. Callable from any thread; the caller
831 /// attaches to the answer on @p replyExec, the owner it names.
832 /// @param replyExec Executor the answer is delivered on, and the one the
833 /// caller attaches its callbacks from. Borrowed: it must outlive the
834 /// returned `Completion`'s delivery.
835 /// @return A `Completion` settled on the server strand with `ready`, the
836 /// live model count and the in-flight execute count; its callbacks
837 /// run on @p replyExec.
840 _strand.postTask([self = shared_from_this(), promise = std::move(settleable.second)]() mutable {
841 self->noteOwner("RemoteServer::health");
842 promise.resolve(self->snapshotHealth());
843 });
844 return std::move(settleable.first);
845 }
846
847 /// @brief Enters shutdown: from now on, `register`, `attach` and `execute`
848 /// envelopes are rejected with `err "server shutting down"`.
849 /// `deregister` is still served so clients can tear down cleanly.
850 /// `attach` is refused too, so a client cannot re-attach to a shared
851 /// instance during the drain window.
852 ///
853 /// Posted to the server strand: an envelope this thread hands `handle()`
854 /// after this call returns is refused. Idempotent. There is no way back: a
855 /// restarted service constructs a fresh `RemoteServer`.
856 ///
857 /// Also flips `health()`'s `ready` to `false` and calls
858 /// `ServerConfig::healthHandler`, if set, with the post-shutdown status, on
859 /// the server strand — what lets an orchestrator stop routing to this
860 /// server while `drainedWithin()`'s drain runs.
862 _strand.postTask([self = shared_from_this()] {
863 self->noteOwner("RemoteServer::beginShutdown");
864 bool const wasReady = self->_ready;
865 self->_shuttingDown = true;
866 self->_ready = false;
867 if (wasReady && self->_config.healthHandler) {
868 self->_config.healthHandler(self->snapshotHealth());
869 }
870 });
871 }
872
873 /// @brief Answers whether every in-flight `execute` delivers its reply
874 /// within @p deadline.
875 ///
876 /// "In-flight" is one count, kept on the server strand: incremented when an
877 /// `execute` is admitted for dispatch and decremented, on the strand, when
878 /// its reply is sent — on every resolving path (`ok`, `err`, or a
879 /// `LimitPolicy::executeTimeout` firing first). The same count
880 /// `LimitPolicy::maxInFlightExecutes` gates and `health()`'s `inFlight`
881 /// reads. Independent of `beginShutdown()`: it observes, it does not stop
882 /// new work from arriving. Callable from any thread; nothing blocks. The
883 /// caller attaches to the answer on @p replyExec, the owner it names.
884 /// @param deadline Longest time to wait; `0` answers from the count as it
885 /// stands.
886 /// @param replyExec Executor the answer is delivered on, and the one the
887 /// caller attaches its callbacks from. Borrowed: it must outlive the
888 /// returned `Completion`'s delivery.
889 /// @return A `Completion` settled on the server strand: `true` once the
890 /// count is zero, `false` if @p deadline elapses first. Its
891 /// callbacks run on @p replyExec.
892 [[nodiscard]] ::morph::async::Completion<bool> drainedWithin(std::chrono::milliseconds deadline,
893 ::morph::exec::IExecutor& replyExec) {
894 auto settleable = ::morph::async::Completion<bool>::makeSettleable(&replyExec);
895 _strand.postTask(
896 [self = shared_from_this(), deadline,
897 promise = std::make_shared<::morph::async::Completion<bool>::Promise>(std::move(settleable.second))] {
898 self->noteOwner("RemoteServer::drainedWithin");
899 self->addDrainWaiter(deadline, promise);
900 });
901 return std::move(settleable.first);
902 }
903
904private:
905 /// A settleable `bool` completion, held by a drain waiter.
906 using DrainPromise = std::shared_ptr<::morph::async::Completion<bool>::Promise>;
907
908 /// An envelope decoded on the transport's thread, or why it could not be.
909 struct Decoded {
910 std::optional<::morph::wire::Envelope> env;
911 /// The raw message, kept only when the decode failed, for the log line.
912 std::string raw;
913 /// The decode exception's message, when the decode failed.
914 std::string error;
915 };
916
917 /// One connection's writer, and whether `post` has held an event back
918 /// from it since the last drain.
919 struct ConnectionOutbox {
920 ConnectionWriter writer;
921 bool owesResync{false};
922 // The changes merged for the connection's next `changes` event.
923 detail::ChangeCoalescer queued;
924 // The armed coalescing window, `0` when none is: the timer's handle,
925 // and the number the flush it posts must match to send the queue.
926 ::morph::async::detail::TimeoutScheduler::Handle windowTimer{0};
927 std::uint64_t window{0};
928 };
929
930 /// @brief Rejects a configuration whose protocol range is empty.
931 /// @param config The configuration to check.
932 /// @return @p config, unchanged.
933 /// @throws std::invalid_argument if `minProtocolVersion > maxProtocolVersion`.
934 static ServerConfig validated(ServerConfig config) {
935 if (config.minProtocolVersion > config.maxProtocolVersion) {
936 throw std::invalid_argument("ServerConfig: minProtocolVersion must not exceed maxProtocolVersion");
937 }
938 if (config.coalesceWindow.count() < 0) {
939 throw std::invalid_argument("ServerConfig: coalesceWindow must not be negative");
940 }
941 if (std::ranges::find(config.services, nullptr) != config.services.end()) {
942 throw std::invalid_argument("ServerConfig: services must not hold a null entry");
943 }
944 return config;
945 }
946
947 /// @brief The service that answers @p kind, if any.
948 /// @param kind The envelope's `kind`.
949 /// @return The first configured service that serves it, or null.
950 [[nodiscard]] IEnvelopeService* serviceFor(std::string_view kind) const {
951 for (const auto& service : _config.services) {
952 if (service->serves(kind)) {
953 return service.get();
954 }
955 }
956 return nullptr;
957 }
958
959 /// @brief Decodes @p msg, keeping the message and the error when it fails.
960 /// @param msg JSON-encoded envelope.
961 /// @return The envelope, or the raw message and the decode error.
962 static Decoded decodeEnvelope(std::string msg) {
963 Decoded decoded;
964 try {
965 decoded.env = ::morph::wire::decode(msg);
966 } catch (const std::exception& exc) {
967 decoded.raw = std::move(msg);
968 decoded.error = exc.what();
969 }
970 return decoded;
971 }
972
973 /// @brief Shared body of both `handle()` overloads: one decode here, on the
974 /// transport's thread, and one post to the server strand.
975 /// @param msg JSON-encoded `morph::wire::Envelope` (via `wire::encode`).
976 /// @param reply Callback invoked with the JSON-encoded reply envelope.
977 /// @param cid Connection scope; `0` means unscoped (see `handle()`'s own doc).
978 void handleImpl(std::string msg, std::function<void(std::string)> reply, ConnectionId cid) {
979 _strand.postTask([self = shared_from_this(), decoded = decodeEnvelope(std::move(msg)),
980 reply = std::move(reply),
981 cid]() mutable { self->dispatchDecoded(std::move(decoded), reply, cid); });
982 }
983
984 /// @brief Records that the calling body touches server-strand state; a
985 /// debug build asserts it runs on the server strand.
986 /// @param site Static name of the body.
987 void noteOwner(char const* site) {
988 ::morph::exec::detail::noteOwner(site, _strand.coreExecutor(), _strand.runningHere());
989 }
990
991 /// @brief The health snapshot. On the server strand.
992 /// @return `ready`, the live instance count, the in-flight count.
993 [[nodiscard]] HealthStatus snapshotHealth() const {
994 return HealthStatus{.ready = _ready, .liveModels = _instances.size(), .inFlight = _inFlight};
995 }
996
997 /// @brief Body of `closeConnection`. On the server strand.
998 /// @param cid Connection scope to close.
999 void closeConnectionHere(ConnectionId cid) {
1000 MORPH_ZONE("RemoteServer::closeConnection");
1001 noteOwner("RemoteServer::closeConnection");
1002 discardChanges(cid);
1003 _writers.erase(cid);
1004 _facts.erase(cid);
1005 _watches.dropConnection(cid);
1006 _principalBaselines.erase(cid);
1007 auto scopeIter = _connectionScopes.find(cid);
1008 if (scopeIter == _connectionScopes.end()) {
1009 return;
1010 }
1011 for (const auto& [mid, refs] : scopeIter->second) {
1012 // Release exactly as many references as this connection held. A
1013 // shared instance another connection is still attached to survives;
1014 // a private one (count 1, no directory entry) is erased outright.
1015 for (std::size_t idx = 0; idx < refs; ++idx) {
1016 releaseInstance(mid);
1017 }
1018 }
1019 _connectionScopes.erase(scopeIter);
1020 }
1021
1022 /// @brief The facts a service records for connection @p cid. On the server
1023 /// strand.
1024 ///
1025 /// Fresh facts for `0`, an unknown `cid` or one already closed, so what a
1026 /// service sets there outlives no reply: a fact is kept only where the
1027 /// connection that negotiated it can be found again.
1028 /// @param cid Connection the envelope arrived on.
1029 /// @return The connection's facts, or throwaway ones.
1030 [[nodiscard]] ConnectionFacts& factsOf(ConnectionId cid) {
1031 if (cid == 0 || !_connectionScopes.contains(cid)) {
1032 _throwawayFacts = ConnectionFacts{};
1033 return _throwawayFacts;
1034 }
1035 return _facts[cid];
1036 }
1037
1038 /// @brief Whether @p envelope exceeds `wire::kMaxEnvelopeBytes`; such an
1039 /// envelope is counted as `Metric::eventOversize`, logged, and not sent.
1040 /// @param cid Connection it was meant for.
1041 /// @param envelope The encoded envelope.
1042 /// @return `true` when it must be dropped.
1043 static bool oversized(ConnectionId cid, std::string const& envelope) {
1044 if (envelope.size() <= ::morph::wire::kMaxEnvelopeBytes) {
1045 return false;
1046 }
1047 ::morph::observe::detail::emitMetric(::morph::observe::Metric::eventOversize, 1.0);
1048 ::morph::log::logWarn("[post] envelope for connection {} dropped: {} bytes exceeds {}", cid, envelope.size(),
1049 ::morph::wire::kMaxEnvelopeBytes);
1050 return true;
1051 }
1052
1053 /// @brief The connections whose scope holds @p mid. On the server strand.
1054 ///
1055 /// A private instance is held by the connection that registered it; every
1056 /// holder is named, so routing does not depend on which connection ran
1057 /// the action.
1058 /// @param mid Instance.
1059 /// @return The connections, ascending.
1060 [[nodiscard]] std::vector<ConnectionId> scopesHolding(::morph::exec::detail::ModelId mid) const {
1061 std::vector<ConnectionId> holders;
1062 for (auto const& [cid, scope] : _connectionScopes) {
1063 if (scope.contains(mid)) {
1064 holders.push_back(cid);
1065 }
1066 }
1067 return holders;
1068 }
1069
1070 /// One fan-out's `authenticate` answers, by (stored principal, token):
1071 /// whether the token still verifies as the principal it was admitted for.
1072 using Authentications = std::map<std::pair<std::string, std::string>, bool>;
1073
1074 /// @brief Whether @p watch's session still authenticates as the principal
1075 /// it was admitted for. On the server strand.
1076 ///
1077 /// An expired or revoked token verifies as no principal, which differs
1078 /// from a non-empty stored one. A watch admitted with no verified
1079 /// principal, as under an authorizer that never authenticates, was not
1080 /// admitted on authentication and still passes. Asked once per session
1081 /// per fan-out.
1082 /// @param watch What the watch was admitted on.
1083 /// @param asked This fan-out's answers so far; extended in place.
1084 /// @return True when the session still verifies as `watch.principal`.
1085 [[nodiscard]] bool stillAuthenticated(detail::Watch const& watch, Authentications& asked) const {
1086 auto [found, fresh] = asked.try_emplace({watch.principal, watch.context.token}, false);
1087 if (fresh) {
1088 found->second = verifiedPrincipal(watch.context) == watch.principal;
1089 }
1090 return found->second;
1091 }
1092
1093 /// @brief Adds one changed instance's topic to the body of every connection
1094 /// it reaches, for `fanOut`. On the server strand.
1095 ///
1096 /// An instance is matched by the directory key it holds now; a private one
1097 /// reaches the scopes that hold it.
1098 /// @param change The changed instance.
1099 /// @param bodies Each recipient's share of the fan-out; extended in place.
1100 void addInstanceTopic(::morph::effects::InstanceChange change,
1101 std::map<ConnectionId, ::morph::wire::ChangesBody>& bodies) {
1102 ::morph::exec::detail::ModelId const mid{change.instance};
1103 if (change.primary.empty()) {
1104 // Keyed after admission: the topic is the key it holds now.
1105 if (auto const* inst = _instances.find(mid); inst != nullptr && inst->sharedKey) {
1106 change.primary = inst->sharedKey->second;
1107 }
1108 }
1109 auto const recipients =
1110 change.primary.empty() ? scopesHolding(mid) : _watches.watchersOf(change.modelType, change.primary);
1111 ::morph::wire::InstanceTopic const topic{.modelType = std::move(change.modelType),
1112 .primary = std::move(change.primary),
1113 .instance = change.instance};
1114 for (ConnectionId const cid : recipients) {
1115 if (eventsNegotiated(cid)) {
1116 bodies[cid].instances.push_back(topic);
1117 }
1118 }
1119 }
1120
1121 /// @brief Narrows @p body to the topics connection @p cid may still be
1122 /// sent, re-authorising each watch by the rule that admitted it.
1123 /// On the server strand.
1124 ///
1125 /// For each watched topic, in order: the watch's stored session must
1126 /// still authenticate (`stillAuthenticated`), or every watch @p cid holds
1127 /// under that session is dropped; then the topic's admission rule is run
1128 /// again on that session, `mayWatchRecord` or `mayWatchInstance`, with
1129 /// the answers they cache per principal and token, or that one watch is
1130 /// dropped. A dropped watch's topic leaves @p body. Which action wrote
1131 /// the change is not checked: a principal that may list a record but not
1132 /// write it still receives its changes. The caller owes the connection a
1133 /// `resync` when this drops anything; the client answers it by sending
1134 /// its whole watch set again, which is admitted afresh.
1135 ///
1136 /// A private instance's topic has no watch: it is sent to the scope that
1137 /// holds the instance, and kept.
1138 ///
1139 /// A record registered with `morph::record::privateKeys` loses its keys
1140 /// unless @p cid is @p caller or its watch's principal is
1141 /// @p callerPrincipal, which must be a verified one: keys name rows the
1142 /// writer's principal may see, and a principal that filters rows
1143 /// differently must not learn them. Without keys, the recipient refetches
1144 /// every entry holding the record.
1145 /// @param cid Recipient.
1146 /// @param body The topics `fanOut` chose for it; narrowed in place.
1147 /// @param caller Connection the execute arrived on; `0` for unscoped.
1148 /// @param callerPrincipal The execute's verified principal; empty when none was.
1149 /// @param asked This fan-out's `authenticate` answers; extended in place.
1150 /// @return True when any of @p cid's watches was dropped.
1151 [[nodiscard]] bool reauthorise(ConnectionId cid, ::morph::wire::ChangesBody& body, ConnectionId caller,
1152 std::string_view callerPrincipal, Authentications& asked) {
1153 bool dropped = false;
1154 // Copied out before a drop: dropping frees the `Watch` it came from.
1155 auto const authenticated = [&](detail::Watch const& watch) {
1156 if (stillAuthenticated(watch, asked)) {
1157 return true;
1158 }
1159 std::string const principal = watch.principal;
1160 std::string const token = watch.context.token;
1161 _watches.dropSession(cid, principal, token);
1162 dropped = true;
1163 return false;
1164 };
1165 std::vector<::morph::effects::RecordChange> records;
1166 records.reserve(body.records.size());
1167 for (auto& change : body.records) {
1168 auto const* watch = _watches.watchOf(cid, change.record);
1169 if (watch == nullptr || !authenticated(*watch)) {
1170 continue;
1171 }
1172 if (!mayWatchRecord(watch->context, change.record)) {
1173 _watches.remove(cid, ::morph::wire::WatchBody{.records = {change.record}, .instances = {}});
1174 dropped = true;
1175 continue;
1176 }
1177 bool const writer = cid == caller || (!callerPrincipal.empty() && watch->principal == callerPrincipal);
1178 hidePrivateKeys(change, writer);
1179 records.push_back(std::move(change));
1180 }
1181 body.records = std::move(records);
1182 std::vector<::morph::wire::InstanceTopic> instances;
1183 instances.reserve(body.instances.size());
1184 for (auto& topic : body.instances) {
1185 if (!topic.primary.empty()) {
1186 auto const* watch = _watches.watchOf(cid, topic.modelType, topic.primary);
1187 if (watch == nullptr || !authenticated(*watch)) {
1188 continue;
1189 }
1190 if (!mayWatchInstance(watch->context, topic.modelType)) {
1191 _watches.remove(cid, ::morph::wire::WatchBody{
1192 .records = {},
1193 .instances = {{.modelType = topic.modelType, .primary = topic.primary}}});
1194 dropped = true;
1195 continue;
1196 }
1197 }
1198 instances.push_back(std::move(topic));
1199 }
1200 body.instances = std::move(instances);
1201 return dropped;
1202 }
1203
1204 /// @brief Drops @p change's keys when its record is registered with
1205 /// `morph::record::privateKeys` and the recipient is not the writer.
1206 /// @param change The change for one recipient; narrowed in place.
1207 /// @param writer True when the recipient is the execute's own connection, or
1208 /// watches as the execute's verified principal.
1209 static void hidePrivateKeys(::morph::effects::RecordChange& change, bool writer) {
1210 if (writer) {
1211 return;
1212 }
1213 auto const* info = ::morph::effects::recordNamed(change.record);
1214 if (info != nullptr && info->privateKeys) {
1215 change.keys.reset();
1216 }
1217 }
1218
1219 /// @brief Sends @p cid one `resync` event, owed because the server dropped
1220 /// some of its watches on its own. On the server strand.
1221 ///
1222 /// The client answers a `resync` by marking what it watches stale and
1223 /// sending its whole watch set again with `replace: true`; a topic still
1224 /// refused comes back in the `WatchReply`. Without it, a client whose
1225 /// watch was dropped would stop receiving that topic's changes and never
1226 /// know. Sent only to a connection that negotiated events, and through
1227 /// `postHere`, so a connection held back behind its backlog gets the one
1228 /// `resync` its drain already owes it. An `unwatch`, a `replace`, a close
1229 /// and a write failure drop watches without one: the first two are the
1230 /// client's own, and the last two leave no connection to tell.
1231 /// @param cid Connection whose watches were dropped.
1232 void sendResync(ConnectionId cid) {
1233 if (eventsNegotiated(cid)) {
1234 postHere(cid, ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kResyncTopic, "{}")));
1235 }
1236 }
1237
1238 /// @brief Queues @p body for connection @p cid's next `changes` event. On
1239 /// the server strand.
1240 ///
1241 /// Sent at once when the queue reaches `detail::kCoalesceKeyLimit` or the
1242 /// window is zero; otherwise the first change queued opens the window,
1243 /// whose close posts `closeWindow` back to the strand. Dropped for a
1244 /// connection with no writer, and for one owed a `resync`, which tells it
1245 /// more than the changes would.
1246 /// @param cid Recipient connection.
1247 /// @param body Its re-authorised share of one fan-out.
1248 void queueChanges(ConnectionId cid, ::morph::wire::ChangesBody const& body) {
1249 auto found = _writers.find(cid);
1250 if (found == _writers.end() || found->second.owesResync) {
1251 return;
1252 }
1253 ConnectionOutbox& outbox = found->second;
1254 outbox.queued.add(body);
1255 if (_config.coalesceWindow.count() == 0 || outbox.queued.keyCount() >= detail::kCoalesceKeyLimit) {
1256 flushChanges(cid);
1257 return;
1258 }
1259 if (outbox.window != 0) {
1260 return;
1261 }
1262 if (!_strandTimer) {
1263 _strandTimer = std::make_unique<::morph::async::detail::TimeoutScheduler>();
1264 }
1265 std::uint64_t const window = ++_nextWindow;
1266 outbox.window = window;
1267 // Weak, as a drain deadline is: the timer does not keep alive a server
1268 // nothing else refers to, whose connections are gone with it. The
1269 // strand task it posts holds the server, as every strand task does.
1270 outbox.windowTimer = _strandTimer->schedule(_config.coalesceWindow, [weak = weak_from_this(), cid, window] {
1271 if (auto self = weak.lock()) {
1272 self->_strand.postTask([self, cid, window] { self->closeWindow(cid, window); });
1273 }
1274 });
1275 }
1276
1277 /// @brief Sends connection @p cid's queue when its window @p window
1278 /// closes. On the server strand.
1279 ///
1280 /// Nothing is sent when the queue was sent since (by the key limit or a
1281 /// `resync`) and a later window is open, or when the connection closed.
1282 /// @param cid Connection whose window closed.
1283 /// @param window The number its `queueChanges` gave the window.
1284 // NOLINTNEXTLINE(bugprone-easily-swappable-parameters): a connection, then its window's number; swapped, nothing matches and nothing is sent.
1285 void closeWindow(ConnectionId cid, std::uint64_t window) {
1286 noteOwner("RemoteServer::closeWindow");
1287 auto found = _writers.find(cid);
1288 if (found == _writers.end() || found->second.window != window) {
1289 return;
1290 }
1291 found->second.window = 0;
1292 found->second.windowTimer = 0;
1293 flushChanges(cid);
1294 }
1295
1296 /// @brief Drops what connection @p cid has queued and closes its window,
1297 /// sending nothing. On the server strand.
1298 /// @param cid Connection whose queue is discarded.
1299 /// @return `true` when something was queued.
1300 bool discardChanges(ConnectionId cid) {
1301 auto found = _writers.find(cid);
1302 if (found == _writers.end()) {
1303 return false;
1304 }
1305 ConnectionOutbox& outbox = found->second;
1306 if (outbox.windowTimer != 0) {
1307 _strandTimer->cancel(outbox.windowTimer);
1308 }
1309 outbox.window = 0;
1310 outbox.windowTimer = 0;
1311 bool const queued = !outbox.queued.empty();
1312 (void)outbox.queued.take();
1313 return queued;
1314 }
1315
1316 /// @brief Sends what connection @p cid has queued as one `changes` event
1317 /// through `postHere`, and closes its window. On the server strand.
1318 /// @param cid Connection whose queue is sent.
1319 void flushChanges(ConnectionId cid) {
1320 auto found = _writers.find(cid);
1321 if (found == _writers.end()) {
1322 return;
1323 }
1324 ConnectionOutbox& outbox = found->second;
1325 if (outbox.windowTimer != 0) {
1326 _strandTimer->cancel(outbox.windowTimer);
1327 }
1328 outbox.window = 0;
1329 outbox.windowTimer = 0;
1330 if (outbox.queued.empty()) {
1331 return;
1332 }
1333 ::morph::wire::ChangesBody body = outbox.queued.take();
1334 std::string envelope = changesEnvelope(body);
1335 // The key count is bounded, the keys' size is not: past the envelope cap, the same records go as whole-type
1336 // changes, which name what changed in a few bytes each. A watcher must never miss a change, so what still
1337 // does not fit is a `resync`.
1338 if (envelope.size() > ::morph::wire::kMaxEnvelopeBytes) {
1339 for (::morph::effects::RecordChange& change : body.records) {
1340 change.keys.reset();
1341 }
1342 envelope = changesEnvelope(body);
1343 }
1344 if (oversized(cid, envelope)) {
1345 envelope = ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kResyncTopic, "{}"));
1346 }
1347 postHere(cid, std::move(envelope));
1348 }
1349
1350 /// @brief The encoded `changes` event carrying @p body.
1351 /// @param body The changes.
1352 /// @return The envelope.
1353 [[nodiscard]] static std::string changesEnvelope(::morph::wire::ChangesBody const& body) {
1354 std::string payload;
1355 (void)glz::write_json(body, payload);
1356 return ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kChangesTopic, std::move(payload)));
1357 }
1358
1359 /// @brief Body of `post`. On the server strand.
1360 /// @param cid Connection to write to.
1361 /// @param envelope The encoded envelope, within the size cap.
1362 void postHere(ConnectionId cid, std::string envelope) {
1363 noteOwner("RemoteServer::post");
1364 auto found = _writers.find(cid);
1365 if (found == _writers.end()) {
1366 return;
1367 }
1368 ConnectionOutbox& outbox = found->second;
1369 if (outbox.owesResync) {
1370 return;
1371 }
1372 std::size_t backlog = 0;
1373 try {
1374 backlog = outbox.writer.backlog ? outbox.writer.backlog() : 0;
1375 } catch (const std::exception& exc) {
1376 writeFailed(cid, exc.what());
1377 return;
1378 } catch (...) {
1379 writeFailed(cid, "unknown exception");
1380 return;
1381 }
1382 if (backlog > kEventBacklogHigh) {
1383 outbox.owesResync = true;
1384 return;
1385 }
1386 writeTo(cid, outbox, std::move(envelope));
1387 }
1388
1389 /// @brief Body of `connectionDrained`. On the server strand.
1390 /// @param cid Connection whose backlog drained.
1391 void connectionDrainedHere(ConnectionId cid) {
1392 noteOwner("RemoteServer::connectionDrained");
1393 auto found = _writers.find(cid);
1394 if (found == _writers.end() || !found->second.owesResync) {
1395 return;
1396 }
1397 found->second.owesResync = false;
1398 writeTo(cid, found->second,
1399 ::morph::wire::encode(::morph::wire::makeEvent(::morph::wire::kResyncTopic, "{}")));
1400 }
1401
1402 /// @brief Hands @p envelope to @p outbox's writer, catching what it
1403 /// throws. On the server strand.
1404 /// @param cid Connection the writer belongs to.
1405 /// @param outbox The connection's writer state.
1406 /// @param envelope The encoded envelope.
1407 void writeTo(ConnectionId cid, ConnectionOutbox& outbox, std::string envelope) {
1408 try {
1409 outbox.writer.write(std::move(envelope));
1410 } catch (const std::exception& exc) {
1411 writeFailed(cid, exc.what());
1412 } catch (...) {
1413 writeFailed(cid, "unknown exception");
1414 }
1415 }
1416
1417 /// @brief What a failed write to connection @p cid, or a failed read of
1418 /// its backlog, does. On the server strand.
1419 ///
1420 /// The connection keeps its writer: the transport reports the connection
1421 /// gone through `closeConnection`, which drops it. Its watches go now: a
1422 /// connection that cannot be written to is sent no events, and the client
1423 /// sends its whole watch set again when it reconnects.
1424 /// @param cid Connection whose `write` or `backlog` threw.
1425 /// @param reason The exception's message.
1426 void writeFailed(ConnectionId cid, std::string_view reason) {
1427 noteOwner("RemoteServer::writeFailed");
1428 ::morph::log::logWarn("[post] write to connection {} failed: {}", cid, reason);
1429 _watches.dropConnection(cid);
1430 }
1431
1432 /// @brief Answers a `watch` or `unwatch` envelope. On the server strand,
1433 /// after @p env's principal is stamped.
1434 ///
1435 /// Refused with an `err`: `cid == 0`, since an unscoped message has no
1436 /// connection whose watch set it could change; a connection already
1437 /// closed; and a body `readWatchBody` refuses. Otherwise `ok` with a
1438 /// `WatchReply` listing the topics not admitted, which `unwatch` never
1439 /// has. A topic listed twice in one body is judged, and reported, once.
1440 /// Topics past `wire::kMaxWatchTopics` on the connection are refused.
1441 /// @param env The envelope, its principal the verified one.
1442 /// @param reply Reply sink, invoked once.
1443 /// @param cid Connection the envelope arrived on.
1444 void handleWatch(const ::morph::wire::Envelope& env, const std::function<void(std::string)>& reply,
1445 ConnectionId cid) {
1446 noteOwner("RemoteServer::watch");
1447 if (cid == 0) {
1448 reply(::morph::wire::encode(::morph::wire::makeErr(env.kind + " needs a connection", env.callId)));
1449 return;
1450 }
1451 if (!_connectionScopes.contains(cid)) {
1452 reply(::morph::wire::encode(::morph::wire::makeErr("connection closed", env.callId)));
1453 return;
1454 }
1455 auto body = ::morph::wire::readWatchBody(env.body);
1456 if (!body) {
1457 reply(::morph::wire::encode(::morph::wire::makeErr(body.error(), env.callId)));
1458 return;
1459 }
1460 ::morph::wire::WatchReply refused;
1461 if (env.kind == ::morph::wire::kUnwatchKind) {
1462 _watches.remove(cid, *body);
1463 } else {
1464 ::morph::wire::WatchBody admitted{.records = {}, .instances = {}, .replace = body->replace};
1465 std::set<std::string_view> seenRecords;
1466 for (auto const& record : body->records) {
1467 if (seenRecords.insert(record).second) {
1468 (mayWatchRecord(env.session, record) ? admitted.records : refused.refusedRecords)
1469 .push_back(record);
1470 }
1471 }
1472 std::set<std::pair<std::string_view, std::string_view>> seenInstances;
1473 for (auto const& topic : body->instances) {
1474 if (seenInstances.emplace(topic.modelType, topic.primary).second) {
1475 (mayWatchInstance(env.session, topic.modelType) ? admitted.instances : refused.refusedInstances)
1476 .push_back(topic);
1477 }
1478 }
1479 auto overCap =
1480 _watches.add(cid, admitted, detail::Watch{.context = env.session, .principal = env.session.principal});
1481 std::ranges::move(overCap.refusedRecords, std::back_inserter(refused.refusedRecords));
1482 std::ranges::move(overCap.refusedInstances, std::back_inserter(refused.refusedInstances));
1483 }
1484 std::string json;
1485 (void)glz::write_json(refused, json);
1486 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, std::move(json))));
1487 }
1488
1489 /// @brief Answers an admission question through the watch table's cache.
1490 /// On the server strand.
1491 ///
1492 /// Cached per verified principal and token, since two tokens of one
1493 /// principal may carry different claims. A caller with no verified
1494 /// principal is judged afresh each time: an authorizer that does not
1495 /// authenticate may still tell such callers apart by their token.
1496 /// @tparam Judge Callable `() -> bool`, the uncached answer.
1497 /// @param session The `watch` envelope's session, its principal the verified one.
1498 /// @param kind What @p name names.
1499 /// @param name A registered record's wire name or model type name.
1500 /// @param judge Decides when no answer is cached.
1501 /// @return The answer.
1502 template <class Judge>
1503 [[nodiscard]] bool cachedAdmission(const ::morph::session::Context& session, detail::AdmissionTopic kind,
1504 std::string_view name, Judge judge) {
1505 bool const cacheable = !session.principal.empty();
1506 if (cacheable) {
1507 if (auto const cached = _watches.admission(session, kind, name)) {
1508 return *cached;
1509 }
1510 }
1511 bool const admitted = judge();
1512 if (cacheable) {
1513 _watches.noteAdmission(session, kind, name, admitted);
1514 }
1515 return admitted;
1516 }
1517
1518 /// @brief Whether @p session's principal may watch record type @p record:
1519 /// whether it may run at least one action whose result holds it.
1520 /// On the server strand.
1521 ///
1522 /// Each holding action is authorized as `authorize(session, modelType,
1523 /// actionId)` against every model type it is registered under in this
1524 /// server's dispatcher; which actions hold the record comes from the
1525 /// process effects registry. A name no record has is refused without
1526 /// caching, so a peer cannot grow the cache with names it makes up.
1527 /// @param session The `watch` envelope's session, its principal the verified one.
1528 /// @param record The record's wire name.
1529 /// @return True when the record may be watched.
1530 /// @throws morph::model::EffectsRegistrationError when the registered
1531 /// records and effects are invalid.
1532 [[nodiscard]] bool mayWatchRecord(const ::morph::session::Context& session, std::string_view record) {
1533 if (::morph::effects::recordNamed(record) == nullptr) {
1534 return false;
1535 }
1536 return cachedAdmission(session, detail::AdmissionTopic::record, record, [&] {
1537 for (auto const& actionId : ::morph::effects::actionsHolding(record)) {
1538 for (auto const& modelType : _dispatcher.modelsOf(actionId)) {
1539 if (_authorizer->authorize(session, modelType, actionId)) {
1540 return true;
1541 }
1542 }
1543 }
1544 return false;
1545 });
1546 }
1547
1548 /// @brief Whether @p session's principal may watch keyed instances of
1549 /// @p modelType. On the server strand.
1550 ///
1551 /// Three conditions: the model type is registered here (refused without
1552 /// caching otherwise); the principal may run at least one of its actions,
1553 /// `authorize(session, modelType, actionId)`, cached as for a record; and
1554 /// `authorizeInstance(session, modelType, {}, 0, "")` admits it. The last
1555 /// carries no instance id, because the instance may not exist yet, and no
1556 /// owner, because a shared instance has none.
1557 /// @param session The `watch` envelope's session, its principal the verified one.
1558 /// @param modelType The topic's model type.
1559 /// @return True when the instance topic may be watched.
1560 [[nodiscard]] bool mayWatchInstance(const ::morph::session::Context& session, std::string_view modelType) {
1561 if (!_registry.contains(modelType)) {
1562 return false;
1563 }
1564 bool const mayRun = cachedAdmission(session, detail::AdmissionTopic::modelType, modelType, [&] {
1565 return std::ranges::any_of(_dispatcher.actionsOf(modelType), [&](std::string const& actionId) {
1566 return _authorizer->authorize(session, modelType, actionId);
1567 });
1568 });
1569 return mayRun && _authorizer->authorizeInstance(session, modelType, {}, 0, {});
1570 }
1571
1572 /// @brief Registers a drain waiter, or answers it at once. On the server strand.
1573 /// @param deadline Longest time to wait for the in-flight count to reach zero.
1574 /// @param promise Settled `true` at zero, `false` at the deadline.
1575 void addDrainWaiter(std::chrono::milliseconds deadline, const DrainPromise& promise) {
1576 if (_inFlight == 0) {
1577 promise->resolve(true);
1578 return;
1579 }
1580 if (deadline.count() <= 0) {
1581 promise->resolve(false);
1582 return;
1583 }
1584 std::uint64_t const waiterId = ++_nextDrainWaiter;
1585 if (!_strandTimer) {
1586 _strandTimer = std::make_unique<::morph::async::detail::TimeoutScheduler>();
1587 }
1588 // Weak: a pending deadline does not keep a server alive that nothing
1589 // else refers to. The expiry is posted back, since the waiters are
1590 // strand state and the timer fires on its I/O loop.
1591 auto const handle = _strandTimer->schedule(deadline, [weak = weak_from_this(), waiterId] {
1592 if (auto self = weak.lock()) {
1593 self->_strand.postTask([self, waiterId] { self->expireDrainWaiter(waiterId); });
1594 }
1595 });
1596 _drainWaiters.push_back(DrainWaiter{.id = waiterId, .promise = promise, .timer = handle});
1597 }
1598
1599 /// @brief Answers `false` to the drain waiter @p waiterId, if it is still
1600 /// waiting. On the server strand.
1601 /// @param waiterId The waiter whose deadline elapsed.
1602 void expireDrainWaiter(std::uint64_t waiterId) {
1603 noteOwner("RemoteServer::drainedWithin");
1604 auto const found = std::ranges::find(_drainWaiters, waiterId, &DrainWaiter::id);
1605 if (found == _drainWaiters.end()) {
1606 return;
1607 }
1608 DrainPromise const promise = found->promise;
1609 _drainWaiters.erase(found);
1610 promise->resolve(false);
1611 }
1612
1613 /// @brief Counts one execute's reply as sent, and answers every drain
1614 /// waiter once none is left in flight. On the server strand.
1615 void executeFinished() {
1616 noteOwner("RemoteServer::executeFinished");
1617 _inFlight -= 1;
1618 ::morph::observe::detail::emitMetric(::morph::observe::Metric::executeInFlight,
1619 static_cast<double>(_inFlight));
1620 if (_inFlight != 0 || _drainWaiters.empty()) {
1621 return;
1622 }
1623 auto waiters = std::move(_drainWaiters);
1624 _drainWaiters.clear();
1625 for (auto& waiter : waiters) {
1626 _strandTimer->cancel(waiter.timer);
1627 waiter.promise->resolve(true);
1628 }
1629 }
1630
1631 /// One admitted execute, as `callFinished` and a cancel's lookup know it.
1632 struct CallTag {
1633 /// Connection the execute arrived on; `0` for unscoped.
1634 ConnectionId cid{0};
1635 /// The execute's own `callId`, as the client chose it.
1636 std::uint64_t callId{0};
1637 /// The run's stop source when the call is filed as cancellable, else
1638 /// null. Held, not just compared: it is what tells this call's entry
1639 /// from a later call's under a reused `callId`.
1640 std::shared_ptr<::core::async::StopSource> stop;
1641 };
1642
1643 /// Ends one admitted execute's bookkeeping once its reply has been sent:
1644 /// withdraws it from the cancellable calls, then counts it finished. On
1645 /// the server strand.
1646 /// @param tag The finished call.
1647 void callFinished(const CallTag& tag) {
1648 if (tag.stop) {
1649 auto const found = _cancellable.find(CancelKey{.cid = tag.cid, .callId = tag.callId});
1650 if (found != _cancellable.end() && found->second.stop == tag.stop) {
1651 _cancellable.erase(found);
1652 }
1653 }
1654 executeFinished();
1655 }
1656
1657 /// Files an admitted Task execute so the caller that made it can cancel
1658 /// it. On the server strand, after the execute's own authorization.
1659 ///
1660 /// Keeps what that authorization decided on — the verified principal, the
1661 /// model and action types, the instance and its recorded owner — so a
1662 /// cancel is judged by the same gates, against the same facts, as the
1663 /// execute it stops. A second execute under the same `callId` on the same
1664 /// connection replaces the first's entry; `callFinished` removes only its
1665 /// own.
1666 /// @param tag The call, with its stop source.
1667 /// @param env The admitted execute, its principal already stamped.
1668 /// @param owner The instance's recorded owner.
1669 void fileCancellable(const CallTag& tag, const ::morph::wire::Envelope& env, const std::string& owner) {
1670 _cancellable.insert_or_assign(CancelKey{.cid = tag.cid, .callId = tag.callId},
1671 CancelTarget{.stop = tag.stop,
1672 .principal = env.session.principal,
1673 .modelType = env.modelType,
1674 .actionType = env.actionType,
1675 .modelId = env.modelId,
1676 .owner = owner});
1677 }
1678
1679 /// Requests stop on the execute a `cancel` names, if the cancel may stop
1680 /// it; otherwise does nothing. On the server strand.
1681 ///
1682 /// The call is looked up under the connection the cancel arrived on, so a
1683 /// connection reaches only its own calls. The cancel must then carry the
1684 /// execute's verified principal, and pass the execute's own gates —
1685 /// `authorize` for its types and `authorizeInstance` for its instance —
1686 /// as the cancel's caller. Every refusal is silent: the reply is the same
1687 /// `ok` an unknown or finished call gets.
1688 /// @param env The cancel, its principal already stamped.
1689 /// @param cid Connection it arrived on; `0` reaches no call.
1690 void cancelCall(const ::morph::wire::Envelope& env, ConnectionId cid) {
1691 if (cid == 0) {
1692 return;
1693 }
1694 auto const found = _cancellable.find(CancelKey{.cid = cid, .callId = env.cancelCallId});
1695 if (found == _cancellable.end()) {
1696 return;
1697 }
1698 CancelTarget const& target = found->second;
1699 if (env.session.principal != target.principal ||
1700 !_authorizer->authorize(env.session, target.modelType, target.actionType) ||
1701 !_authorizer->authorizeInstance(env.session, target.modelType, target.actionType, target.modelId,
1702 target.owner)) {
1703 return;
1704 }
1705 auto const stop = target.stop;
1706 _cancellable.erase(found);
1707 // Runs the handler's stop callbacks here, on the server strand; each
1708 // resumes its coroutine through the handler's own resumer, on the
1709 // model's strand, as the execute timeout's stop does from its thread.
1710 static_cast<void>(stop->request_stop());
1711 }
1712
1713 /// @brief Authenticates @p env's session and makes the verified identity
1714 /// authoritative on it.
1715 ///
1716 /// A verifying `_authorizer` returns the principal it extracted from a
1717 /// valid token; that overwrites `env.session.principal` so model code
1718 /// reading `session::current()->principal` can trust it. If
1719 /// `authenticate()` returns `nullopt` the authorizer cannot vouch for the
1720 /// caller, so the client-asserted principal is cleared rather than passed
1721 /// through unverified — every envelope kind this is called for (register,
1722 /// attach, assign, instances, schemas, deregister, cancel, watch, unwatch,
1723 /// a service's kind) makes an authorization or ownership decision that
1724 /// must key on the verified identity, never the client's raw claim.
1725 /// `execute` does the same in `dispatchExecute`, from the principal its
1726 /// one `admit` returns. `hello`, which a client sends with no session, is not
1727 /// stamped. See docs/spec/security.md.
1728 ///
1729 /// It then compares the stamped principal with @p cid's baseline
1730 /// (`compareWithBaseline`).
1731 /// @param env Envelope whose `session.principal` is stamped or cleared in place.
1732 /// @param cid Connection @p env arrived on; `0` for unscoped.
1733 void stampVerifiedPrincipal(::morph::wire::Envelope& env, ConnectionId cid) {
1734 env.session.principal = verifiedPrincipal(env.session);
1735 compareWithBaseline(env.session.principal, cid);
1736 }
1737
1738 /// @brief The principal `_authorizer` verifies from @p session, or empty
1739 /// when it cannot vouch for one.
1740 /// @param session An envelope's session, its principal the client's claim.
1741 /// @return The verified principal, or an empty string.
1742 [[nodiscard]] std::string verifiedPrincipal(const ::morph::session::Context& session) const {
1743 auto verified = _authorizer->authenticate(session);
1744 return verified ? std::move(*verified) : std::string{};
1745 }
1746
1747 /// @brief Compares @p principal with @p cid's baseline, dropping the
1748 /// connection's watches when they differ. On the server strand.
1749 ///
1750 /// The baseline is the principal of the first envelope verified on the
1751 /// connection. A different one, including the empty principal of a token
1752 /// `authenticate` refuses, drops every watch @p cid holds and becomes the
1753 /// baseline: a watch was admitted for one principal, and must not keep
1754 /// delivering to another who now speaks on the same connection. A new
1755 /// token for the same principal keeps them. The changes queued for the
1756 /// connection are discarded, never sent: they were authorised for the
1757 /// previous principal. A connection that held any watch, or had changes
1758 /// queued, is sent one `resync` (`sendResync`), which covers them.
1759 /// `cid == 0` and a connection not open have no baseline.
1760 /// @param principal The envelope's verified principal, or empty.
1761 /// @param cid Connection the envelope arrived on; `0` for unscoped.
1762 void compareWithBaseline(const std::string& principal, ConnectionId cid) {
1763 if (cid == 0 || !_connectionScopes.contains(cid)) {
1764 return;
1765 }
1766 auto const [baseline, first] = _principalBaselines.try_emplace(cid, principal);
1767 if (!first && baseline->second != principal) {
1768 bool const watched = _watches.topicCount(cid) != 0;
1769 bool const queued = discardChanges(cid);
1770 _watches.dropConnection(cid);
1771 baseline->second = principal;
1772 if (watched || queued) {
1773 sendResync(cid);
1774 }
1775 }
1776 }
1777
1778 /// @brief Releases one reference to @p mid, destroying it at zero. On the server strand.
1779 ///
1780 /// A private instance carries no attachments and is erased outright. A
1781 /// shared instance is erased, and removed from the directory, only when its
1782 /// last attachment goes away, so one client's `deregister` or dropped
1783 /// connection never tears an instance out from under another client still
1784 /// using it.
1785 /// @param mid Instance to release.
1786 void releaseInstance(::morph::exec::detail::ModelId mid) { (void)_instances.release(mid); }
1787
1788 /// @brief Drops one of @p cid's references to @p mid and releases the
1789 /// instance by that one reference. On the server strand.
1790 ///
1791 /// A scoped connection releases only a reference it holds: if @p cid's
1792 /// scope records none for @p mid — it never attached it, already released
1793 /// every reference it had, or its scope is closed — nothing is released,
1794 /// because the reference that would go belongs to another connection.
1795 /// The unscoped path (`cid == 0`) records no references, so it always
1796 /// releases.
1797 /// @param mid Instance to release.
1798 /// @param cid Connection whose reference is being dropped; `0` for unscoped.
1799 void releaseScoped(::morph::exec::detail::ModelId mid, ConnectionId cid) {
1800 if (cid != 0) {
1801 auto scopeIter = _connectionScopes.find(cid);
1802 if (scopeIter == _connectionScopes.end()) {
1803 return;
1804 }
1805 auto refIter = scopeIter->second.find(mid);
1806 if (refIter == scopeIter->second.end()) {
1807 return;
1808 }
1809 refIter->second -= 1;
1810 if (refIter->second == 0) {
1811 scopeIter->second.erase(refIter);
1812 }
1813 }
1814 releaseInstance(mid);
1815 }
1816
1817 /// @brief Records a new attachment of @p mid to @p cid. On the server strand.
1818 /// @param mid Instance being attached.
1819 /// @param cid Connection attaching it; `0` for unscoped (records nothing).
1820 /// @return `false` if @p cid's scope was already closed, in which case nothing was recorded.
1821 bool noteScopeAttach(::morph::exec::detail::ModelId mid, ConnectionId cid) {
1822 if (cid == 0) {
1823 return true;
1824 }
1825 auto scopeIter = _connectionScopes.find(cid);
1826 if (scopeIter == _connectionScopes.end()) {
1827 return false;
1828 }
1829 scopeIter->second[mid] += 1;
1830 return true;
1831 }
1832
1833 /// @brief Attaches the configured `LogProvider`'s log to a freshly created
1834 /// holder. On the server strand.
1835 /// @param holder Newly created instance.
1836 /// @param env Envelope carrying `typeId` and `contextKey`.
1837 void attachLogIfConfigured(::morph::model::detail::IModelHolder& holder, const ::morph::wire::Envelope& env) {
1838 if (env.contextKey.empty() || !_config.logProvider) {
1839 return;
1840 }
1841 noteOwner("RemoteServer::attachLog");
1842 if (auto log = _config.logProvider(env.typeId, env.contextKey)) {
1843 holder.attachActionLog(std::move(log), env.contextKey);
1844 }
1845 }
1846
1847 /// @brief Attaches to an already-live directory entry, if there is one. On the server strand.
1848 /// @param dirKey Directory key being acquired.
1849 /// @param env Decoded request, for `callId`.
1850 /// @param reply Reply sink; invoked only when this returns `true`.
1851 /// @param cid Connection scope, or `0` for unscoped.
1852 /// @return `true` if an entry existed and @p reply was invoked; `false` to keep going.
1853 bool attachExisting(const detail::DirectoryKey& dirKey, const ::morph::wire::Envelope& env,
1854 const std::function<void(std::string)>& reply, ConnectionId cid) {
1855 // `nullopt` covers both a plain directory miss and an instance whose
1856 // first action failed: `InstanceDirectory::attach` evicts the latter and
1857 // reports it as a miss, so the caller falls through to creating a fresh
1858 // instance either way. The evicted instance stays live and is torn down
1859 // normally by whoever created it.
1860 auto const attached = _instances.attach(dirKey);
1861 if (!attached) {
1862 return false;
1863 }
1864 auto const mid = *attached;
1865 if (!noteScopeAttach(mid, cid)) {
1866 releaseInstance(mid);
1867 reply(::morph::wire::encode(::morph::wire::makeErr("connection closed", env.callId)));
1868 return true;
1869 }
1870 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, {}, mid.v)));
1871 return true;
1872 }
1873
1874 /// @brief Acquires (or creates) the shared instance for `(typeId, primary)`
1875 /// and replies. On the server strand.
1876 ///
1877 /// The register-or-attach core shared by the `register` branch (when
1878 /// `shared` is set) and by `attach`. A shared instance is recorded with an
1879 /// **empty owner principal**: `IAuthorizer::authorizeInstance`'s documented
1880 /// `ownerPrincipal == ctx.principal` policy would otherwise reject every
1881 /// client but the one that created it, defeating cross-client sharing
1882 /// outright. Gating access to a shared model is therefore `authorize`'s job
1883 /// (per type and action) or the model's own — see docs/spec/security.md.
1884 ///
1885 /// The directory check, the construction, the `maxLiveModels` admission and
1886 /// the insert all run in this one strand task. Only host code this task
1887 /// itself calls — the model's construction, the log provider — can file the
1888 /// key in between, through `handleInline`, so the directory is checked
1889 /// again after it.
1890 ///
1891 /// @param env Decoded request; uses `typeId`, `primary`, `contextKey`, `callId`.
1892 /// @param reply Reply sink; always invoked exactly once.
1893 /// @param cid Connection scope, or `0` for unscoped.
1894 /// @param releaseCurrent Instance to release once the target is confirmed
1895 /// acquired or about to be created (an `attach`
1896 /// re-point), or `ModelId{0}`. A throwing construction
1897 /// never touches it. It is released before the
1898 /// `maxLiveModels` admission check, so a sole holder's
1899 /// release frees the slot the re-point itself needs.
1900 void acquireSharedInstance(const ::morph::wire::Envelope& env, const std::function<void(std::string)>& reply,
1901 ConnectionId cid, ::morph::exec::detail::ModelId releaseCurrent) {
1902 detail::DirectoryKey dirKey{env.typeId, env.primary};
1903 if (attachExisting(dirKey, env, reply, cid)) {
1904 // A same-key re-attach lands on the exact mid attachExisting just
1905 // incremented, so releasing `releaseCurrent` here cancels only the
1906 // redundant reference that call took, never the caller's sole hold.
1907 // A different-key re-point releases the real old instance.
1908 if (releaseCurrent.v != 0U) {
1909 releaseScoped(releaseCurrent, cid);
1910 }
1911 return;
1912 }
1913 // env.primary is this instance's directory key -- always non-empty here
1914 // (acquireSharedInstance is only reached for a shared/keyed
1915 // register-or-attach), so the model learns its own key once, at
1916 // construction, via IModelHolder::attachIdentity.
1917 auto holder = _registry.create(env.typeId, env.primary);
1918 attachLogIfConfigured(*holder, env);
1919 // Checked again: the construction and the log provider are host code,
1920 // and one that registers or attaches through `handleInline` runs that
1921 // request inline, on this strand, possibly filing this very key.
1922 if (attachExisting(dirKey, env, reply, cid)) {
1923 if (releaseCurrent.v != 0U) {
1924 releaseScoped(releaseCurrent, cid);
1925 }
1926 return;
1927 }
1928 ::morph::exec::detail::ModelId const fresh{nextOpaqueId()};
1929 if (releaseCurrent.v != 0U) {
1930 releaseScoped(releaseCurrent, cid);
1931 }
1932 LimitPolicy const& limits = _config.limits;
1933 if (limits.maxLiveModels != 0 && _instances.size() >= limits.maxLiveModels) {
1934 reply(::morph::wire::encode(::morph::wire::makeErr("too many models", env.callId)));
1935 return;
1936 }
1937 if (!noteScopeAttach(fresh, cid)) {
1938 reply(::morph::wire::encode(::morph::wire::makeErr("connection closed", env.callId)));
1939 return;
1940 }
1941 // Filed with one attachment, an empty owner (shared instances are
1942 // ownerless, by design -- see this function's doc comment) and its
1943 // hydration pending, so the first action's outcome decides whether a
1944 // second client may ever be handed it.
1945 _instances.insertShared(fresh, std::move(holder), std::move(dirKey));
1946 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, {}, fresh.v)));
1947 }
1948
1949 /// @brief Files a live, still-anonymous instance under a primary key, in
1950 /// place. On the server strand.
1951 ///
1952 /// The existing holder of a key always wins: promoting onto a key another
1953 /// instance already holds is a silent no-op rather than a displacement.
1954 /// Symmetrically, an instance that already holds a *different* real key is
1955 /// left exactly where it is — also a silent no-op — since instances never
1956 /// change key (docs/spec/core/shared_instances.md); only a `mid` that has
1957 /// never held a directory key can ever be promoted. This server hands every
1958 /// keyed instance its own key once, at construction, through
1959 /// `_registry.create(typeId, primary)` -> `IModelHolder::attachIdentity`,
1960 /// and never updates it — so an instance evicted from its key as poisoned
1961 /// stays ineligible even though the directory no longer files it anywhere.
1962 /// @param env Decoded request; uses `typeId`, `primary`, `modelId`.
1963 void applyAssign(const ::morph::wire::Envelope& env) {
1964 if (env.primary.empty()) {
1965 return;
1966 }
1967 // A no-op unless the instance is live, has never held a directory key,
1968 // and the key is free -- `InstanceDirectory::promote` holds the guards.
1969 (void)_instances.promote(::morph::exec::detail::ModelId{env.modelId},
1970 detail::DirectoryKey{env.typeId, env.primary});
1971 }
1972
1973 /// @brief Names the fields `(modelType, actionType)`'s served schema marks
1974 /// `required` that @p body does not carry a key for.
1975 ///
1976 /// The whole of `PayloadCompleteness::RequireDeclaredFields`. Presence is
1977 /// judged on the **key**, not on the decoded value, because that is the
1978 /// only question the action codec cannot answer: `fromJson` turns an
1979 /// absent field and an explicitly-sent zero into the same action, which is
1980 /// exactly why a renamed field survives `validate()`.
1981 ///
1982 /// Conservative in both directions. A `body` that is not a JSON object at
1983 /// all reports nothing missing — malformed JSON is `fromJson`'s error to
1984 /// raise, with a better message, a moment later — and an action with no
1985 /// published requirement (`requiredFieldsFor` returning `nullptr`)
1986 /// contributes no rule, since the gate enforces what the schema published
1987 /// and cannot enforce what it could not publish.
1988 ///
1989 /// @param modelType Model type-id from the envelope.
1990 /// @param actionType Action type-id from the envelope.
1991 /// @param body The `execute` envelope's opaque payload JSON.
1992 /// @return A comma-separated list of missing field names, or `""` when
1993 /// nothing is missing or nothing is checkable.
1994 [[nodiscard]] std::string missingRequiredFields(const std::string& modelType, const std::string& actionType,
1995 const std::string& body) const {
1996 const auto* required = _dispatcher.requiredFieldsFor(modelType, actionType);
1997 if (required == nullptr || required->empty()) {
1998 return {};
1999 }
2000 glz::generic_u64 dom{};
2001 if (glz::read_json(dom, body) || !dom.is_object()) {
2002 return {};
2003 }
2004 std::string missing;
2005 for (const auto& field : *required) {
2006 if (!dom.contains(field)) {
2007 if (!missing.empty()) {
2008 missing += ", ";
2009 }
2010 missing += field;
2011 }
2012 }
2013 return missing;
2014 }
2015
2016 /// @brief Answers an `instances` request with the live shared keys of a
2017 /// type. On the server strand.
2018 /// @param env Decoded request; uses `typeId` and `callId`.
2019 /// @param reply Reply sink; always invoked exactly once.
2020 void handleInstances(const ::morph::wire::Envelope& env, const std::function<void(std::string)>& reply) {
2021 std::string body;
2022 (void)glz::write_json(_instances.keysOfType(env.typeId), body);
2023 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, std::move(body))));
2024 }
2025
2026 /// @brief Replies to an envelope that did not decode, and logs it. On the
2027 /// server strand.
2028 ///
2029 /// The one server-side record of a request that never dispatched at all. A
2030 /// client that swallows its own error (or is malformed precisely because it
2031 /// is confused) would otherwise leave no trace here. The payload prefix is
2032 /// the most useful field for diagnosing *why* the client sent something
2033 /// malformed -- and the most likely to carry application data, so it is
2034 /// capped at kLogPayloadPreviewBytes rather than logged in full. See
2035 /// docs/spec/core/backend.md, "Server-side observability".
2036 /// @param msg The raw message.
2037 /// @param error The decode exception's message.
2038 /// @param reply Reply sink.
2039 /// @param cid Connection the message arrived on.
2040 static void replyUndecodable(const std::string& msg, const std::string& error,
2041 const std::function<void(std::string)>& reply, ConnectionId cid) {
2042 constexpr std::size_t kLogPayloadPreviewBytes = 256;
2043 std::string_view const preview =
2044 std::string_view{msg}.substr(0, std::min(msg.size(), kLogPayloadPreviewBytes));
2045 ::morph::log::logError(
2046 "[dispatchMessage] undecodable envelope from connection {}: {} ({} bytes, "
2047 "payload prefix: {}{})",
2048 cid, error, msg.size(), preview, msg.size() > kLogPayloadPreviewBytes ? "..." : "");
2049 reply(::morph::wire::encode(::morph::wire::makeErr(error)));
2050 }
2051
2052 /// @brief Dispatches one decoded envelope, or replies to one that did not
2053 /// decode. On the server strand: every envelope's body starts here.
2054 /// @param decoded The envelope, or its decode failure.
2055 /// @param reply Reply sink; invoked exactly once, now or (for an admitted
2056 /// `execute`) from the model's strand.
2057 /// @param cid Connection scope; `0` means unscoped.
2058 void dispatchDecoded(Decoded decoded, std::function<void(std::string)>& reply, ConnectionId cid) {
2059 MORPH_ZONE("RemoteServer::dispatchMessage");
2060 noteOwner("RemoteServer::dispatch");
2061 if (!decoded.env) {
2062 replyUndecodable(decoded.raw, decoded.error, reply, cid);
2063 return;
2064 }
2065 MORPH_ZONE_TEXT(decoded.env->session.requestId);
2066 dispatchEnvelope(std::move(*decoded.env), reply, cid);
2067 }
2068
2069 // One flat switch over the wire's `kind` discriminator. Splitting it would
2070 // scatter the authorization sequence each branch depends on across helpers,
2071 // with no reader benefit. On the server strand.
2072 // NOLINTNEXTLINE(readability-function-cognitive-complexity)
2073 void dispatchEnvelope(::morph::wire::Envelope env, std::function<void(std::string)>& reply, ConnectionId cid) {
2074 // One line per successfully-decoded request -- the point every kind
2075 // funnels through, so a client stuck mid-handshake (or one that never
2076 // sent anything) is distinguishable from one whose requests are
2077 // arriving normally. Deliberately omits the session principal (opt-in
2078 // territory: personal data in many deployments) and the payload body
2079 // (already covered, truncated, on the decode-failure path; logging every
2080 // successful body by default would be far higher volume and duplicate
2081 // what dispatch already records via the action log for execute).
2082 ::morph::log::logDebug(
2083 "[dispatchMessage] connection {}: kind={} callId={} typeId={} modelId={} "
2084 "modelType={} actionType={} bodyBytes={}",
2085 cid, env.kind, env.callId, env.typeId, env.modelId, env.modelType, env.actionType, env.body.size());
2086 // Once shutdown has begun, new work is rejected fast — before any other
2087 // validation runs — while `deregister` (and any other kind) still flows
2088 // through unchanged, so a client can still tear its models down cleanly
2089 // during the drain window.
2090 if ((env.kind == "register" || env.kind == "execute" || env.kind == "attach") && _shuttingDown) {
2091 reply(::morph::wire::encode(::morph::wire::makeErr("server shutting down", env.callId)));
2092 return;
2093 }
2094 try {
2095 if (env.kind == "register") {
2096 MORPH_ZONE("RemoteServer::register");
2097 MORPH_ZONE_TEXT(env.typeId);
2098 ::morph::observe::detail::emitMetric(::morph::observe::Metric::registerCount, 1.0);
2099 if (env.typeId.empty()) {
2100 throw std::runtime_error("register requires a typeId");
2101 }
2102 LimitPolicy const& limits = _config.limits;
2103 // Cheap early rejection, so a server already at its cap does not
2104 // pay for authorize()/authenticate() and a model construction it
2105 // is about to discard.
2106 //
2107 // Skipped for a *shared* register, which may well create nothing:
2108 // if the key is already live it only takes another reference, and
2109 // `maxLiveModels` caps live models, not attachments to them.
2110 if (limits.maxLiveModels != 0 && (!env.shared || env.primary.empty()) &&
2111 _instances.size() >= limits.maxLiveModels) {
2112 reply(::morph::wire::encode(::morph::wire::makeErr("too many models", env.callId)));
2113 return;
2114 }
2115 // Authenticate the caller and make the verified identity
2116 // authoritative, exactly as dispatchExecute does for execute: a
2117 // verifying authorizer's returned principal overwrites
2118 // env.session.principal; a non-authenticating authorizer
2119 // (including allow-all) clears it, so the register decision
2120 // below — and the owner recorded from it — never key on the
2121 // client's unverified claim.
2122 stampVerifiedPrincipal(env, cid);
2123 // Bound *who may create* an instance. The default hook allows
2124 // all, so an unconfigured server registers any known type; a
2125 // deployer opts into gating registration by overriding
2126 // authorizeRegister. No instance is constructed on denial.
2127 if (!_authorizer->authorizeRegister(env.session, env.typeId)) {
2128 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2129 return;
2130 }
2131 // A `shared` register naming a primary is a register-or-attach
2132 // against the directory; everything below is the private path.
2133 if (env.shared && !env.primary.empty()) {
2134 acquireSharedInstance(env, reply, cid, ::morph::exec::detail::ModelId{0});
2135 return;
2136 }
2137 // env.primary here is either empty (a private/anonymous instance
2138 // -- attachIdentity is then a no-op) or a caller-supplied
2139 // identity for a non-shared instance, which is still this
2140 // instance's own key to learn.
2141 auto holder = _registry.create(env.typeId, env.primary);
2142 attachLogIfConfigured(*holder, env);
2143 // find(), never operator[]: the scope may already be gone. A
2144 // client that registers and immediately drops its socket
2145 // closes its scope before this register runs, and operator[]
2146 // would resurrect it -- a scope nothing closes a second time, so
2147 // this model and every later one on the dead cid would be
2148 // unreclaimable, and with `maxLiveModels` set the server would
2149 // wedge at `err "too many models"`.
2150 if (cid != 0 && !_connectionScopes.contains(cid)) {
2151 reply(::morph::wire::encode(::morph::wire::makeErr("connection closed", env.callId)));
2152 return;
2153 }
2154 // Re-tested after the construction, which is host code: a model
2155 // constructor that registers another model through
2156 // `handleInline` runs that register inline, on this strand.
2157 if (limits.maxLiveModels != 0 && _instances.size() >= limits.maxLiveModels) {
2158 reply(::morph::wire::encode(::morph::wire::makeErr("too many models", env.callId)));
2159 return;
2160 }
2161 ::morph::exec::detail::ModelId const mid{nextOpaqueId()};
2162 if (cid != 0) {
2163 _connectionScopes.at(cid)[mid] += 1;
2164 }
2165 // Private: no directory key, no sharing, no hydration tracking.
2166 // The owner is the principal stamped above, never the client's
2167 // raw claim; it is what lets `authorizeInstance` later deny a
2168 // different principal.
2169 _instances.insertPrivate(mid, std::move(holder), std::move(env.session.principal));
2170 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, {}, mid.v)));
2171 } else if (env.kind == "attach") {
2172 if (env.typeId.empty()) {
2173 throw std::runtime_error("attach requires a typeId");
2174 }
2175 stampVerifiedPrincipal(env, cid);
2176 if (!_authorizer->authorizeRegister(env.session, env.typeId)) {
2177 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2178 return;
2179 }
2180 acquireSharedInstance(env, reply, cid, ::morph::exec::detail::ModelId{env.modelId});
2181 } else if (env.kind == "assign") {
2182 if (env.typeId.empty()) {
2183 throw std::runtime_error("assign requires a typeId");
2184 }
2185 // Mirrors "attach"'s gate: filing an instance into the shared
2186 // directory -- whether by creating it (register) or by
2187 // promoting one already live (assign) -- is bounds-checked
2188 // identically. Unlike "attach"/"register", assign never
2189 // constructs a model, but it still changes what a future
2190 // attacher of `primary` reaches, so it must not be reachable
2191 // by an unauthenticated or unauthorized caller either.
2192 stampVerifiedPrincipal(env, cid);
2193 if (!_authorizer->authorizeRegister(env.session, env.typeId)) {
2194 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2195 return;
2196 }
2197 applyAssign(env);
2198 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, {}, env.modelId)));
2199 } else if (env.kind == "instances") {
2200 if (env.typeId.empty()) {
2201 throw std::runtime_error("instances requires a typeId");
2202 }
2203 stampVerifiedPrincipal(env, cid);
2204 // Enumeration is a read channel over the directory: gate it with
2205 // `authorize` for the model type (empty action id) so a deployer
2206 // can refuse listing without refusing use. It discloses the live
2207 // key set to anyone admitted — see docs/spec/security.md.
2208 if (!_authorizer->authorize(env.session, env.typeId, {})) {
2209 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2210 return;
2211 }
2212 handleInstances(env, reply);
2213 } else if (env.kind == "schemas") {
2214 if (env.typeId.empty()) {
2215 throw std::runtime_error("schemas requires a typeId");
2216 }
2217 stampVerifiedPrincipal(env, cid);
2218 // Gated exactly like `instances`, and for the same reason: a
2219 // description is a read channel over the model type, so
2220 // `authorize` with an empty action id lets a deployer refuse
2221 // *describing* a type without refusing *using* it. It is a
2222 // real disclosure — field names, bounds, rules and the
2223 // payload fingerprint of every action — so it must not be
2224 // reachable by a caller the server would not let execute.
2225 if (!_authorizer->authorize(env.session, env.typeId, {})) {
2226 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2227 return;
2228 }
2229 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, _dispatcher.schemasJson(env.typeId))));
2230 } else if (env.kind == "deregister") {
2231 MORPH_ZONE("RemoteServer::deregister");
2232 ::morph::observe::detail::emitMetric(::morph::observe::Metric::deregisterCount, 1.0);
2233 ::morph::exec::detail::ModelId const mid{env.modelId};
2234 // The ownership check below compares the recorded owner against
2235 // env.session.principal, so that principal must be the verified
2236 // one: without the stamp a caller holding any token, or none,
2237 // could claim the owner's name and release the owner's instance.
2238 stampVerifiedPrincipal(env, cid);
2239 // Type-level gate, as for `instances`/`schemas`. A deregister
2240 // names no type, so both ids are empty; it lets a verifying
2241 // authorizer refuse a caller with no valid token outright.
2242 if (!_authorizer->authorize(env.session, {}, {})) {
2243 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2244 return;
2245 }
2246 // Per-instance authorization: consult the hook with the recorded
2247 // owner before destroying the instance. The default hook allows
2248 // all; an ownership-enforcing authorizer rejects a caller tearing
2249 // down an instance it does not own.
2250 if (const auto* inst = _instances.find(mid);
2251 inst != nullptr && !_authorizer->authorizeInstance(env.session, {}, {}, mid.v, inst->owner)) {
2252 reply(::morph::wire::encode(::morph::wire::makeErr("unauthorized", env.callId)));
2253 return;
2254 }
2255 // Release exactly the reference *this* connection (`cid`, the
2256 // scope the deregister request itself carries) holds, not
2257 // whichever connection happened to attach the instance last --
2258 // a shared instance may have several owning connections at once,
2259 // and crediting the release to the wrong one either strands a
2260 // reference nobody will ever decrement, or lets one connection's
2261 // deregister silently consume another's hold. A connection
2262 // holding no reference releases nothing and is still answered
2263 // `ok`, the reply an unknown id gets, so it does not tell a live
2264 // id this connection does not hold apart from one that is gone.
2265 releaseScoped(mid, cid);
2266 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId)));
2267 } else if (env.kind == "execute") {
2268 dispatchExecute(std::move(env), reply, cid);
2269 } else if (env.kind == "cancel") {
2270 // Stamped before the ownership comparison in cancelCall, for
2271 // the reason deregister is: the comparison keys on the
2272 // verified principal, never on the client's claim.
2273 stampVerifiedPrincipal(env, cid);
2274 cancelCall(env, cid);
2275 // One answer whatever became of the call -- stopped, unknown,
2276 // finished, or another caller's -- so a cancel cannot be used
2277 // to learn which calls exist.
2278 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId)));
2279 } else if (env.kind == "hello") {
2280 MORPH_ZONE("RemoteServer::hello");
2281 const std::uint32_t minV = _config.minProtocolVersion;
2282 const std::uint32_t maxV = _config.maxProtocolVersion;
2283 if (env.protocolVersion < minV || env.protocolVersion > maxV) {
2284 reply(::morph::wire::encode(::morph::wire::makeErr("protocol version unsupported", env.callId)));
2285 } else {
2286 std::string body;
2287 (void)glz::write_json(
2288 ::morph::wire::ProtocolRange{
2289 .min = minV, .max = maxV, .capabilities = {std::string{::morph::wire::kCapabilityCancel}}},
2290 body);
2291 reply(::morph::wire::encode(::morph::wire::makeOk(env.callId, std::move(body))));
2292 }
2293 } else if (env.kind == ::morph::wire::kWatchKind || env.kind == ::morph::wire::kUnwatchKind) {
2294 // Stamped first: the watch keeps the verified principal, and
2295 // admission decides on it.
2296 stampVerifiedPrincipal(env, cid);
2297 handleWatch(env, reply, cid);
2298 } else if (IEnvelopeService* const service = serviceFor(env.kind); service != nullptr) {
2299 // Stamped first, as for every built-in kind that authorises:
2300 // the service decides on the verified principal, never on the
2301 // client's claim.
2302 stampVerifiedPrincipal(env, cid);
2303 reply(service->handle(env, *_authorizer, factsOf(cid)));
2304 } else {
2305 reply(::morph::wire::encode(::morph::wire::makeErr("unknown envelope kind: " + env.kind, env.callId)));
2306 }
2307 } catch (const std::exception& exc) {
2308 // Any throw from the branches above — including one out of
2309 // `dispatchExecute`'s `authorize`/`authenticate`/`authorizeInstance`/
2310 // `missingRequiredFields` steps, all reachable, non-`noexcept` code
2311 // — becomes this call's `err` reply. Nothing is held across it: a
2312 // later execute for the same model is simply the next task on this
2313 // strand.
2314 reply(::morph::wire::encode(::morph::wire::makeErr(exc.what(), env.callId)));
2315 }
2316 }
2317
2318 // Admission: one ordered gate sequence — limits, authorize, authenticate,
2319 // lookup, per-instance authorize, payload completeness, in-flight
2320 // reservation — whose *order* is the security contract itself (see
2321 // docs/spec/security.md), so it is deliberately not broken up. On the
2322 // server strand; the admitted run is posted to the model's strand.
2323 // NOLINTNEXTLINE(readability-function-cognitive-complexity)
2324 void dispatchExecute(::morph::wire::Envelope env, std::function<void(std::string)> reply, ConnectionId cid) {
2325 // The admission phase only, on the server strand: the run itself is
2326 // posted to the model's strand and zoned there, in startRemote.
2327 MORPH_ZONE("RemoteServer::dispatchExecute");
2328 MORPH_ZONE_TEXT(env.session.requestId);
2329 noteOwner("RemoteServer::admitExecute");
2330 auto reject = [&env, &reply](const char* message) {
2331 reply(::morph::wire::encode(::morph::wire::makeErr(message, env.callId)));
2332 };
2333 // Authenticated and authorized together, once, on entry: one look at
2334 // the token answers both, so a token cannot pass one check and expire
2335 // before the other. The principal is compared with the connection's
2336 // baseline before any check below can refuse the call: an execute under
2337 // another principal, or under none, drops the connection's watches even
2338 // when it is refused, and before its own fan-out when it is not.
2339 auto admission = _authorizer->admit(env.session, env.modelType, env.actionType);
2340 std::string verified = admission.principal ? std::move(*admission.principal) : std::string{};
2341 compareWithBaseline(verified, cid);
2342 LimitPolicy const& limits = _config.limits;
2343 // Exact, not advisory: the count is this strand's own state, so nothing
2344 // can be admitted between this check and the increment below.
2345 if (limits.maxInFlightExecutes != 0 && _inFlight >= limits.maxInFlightExecutes) {
2346 reject("server busy");
2347 return;
2348 }
2349 if (!admission.allowed) {
2350 reject("unauthorized");
2351 return;
2352 }
2353 // Make the identity authoritative: the principal verified on entry
2354 // replaces the client's claim, so model code reading
2355 // session::current()->principal can trust it. When the authorizer
2356 // could not vouch for the caller it is empty, never the client's claim:
2357 // an authorize-only or allow-all authorizer, which never authenticates,
2358 // never presents an unverified principal to the model. See
2359 // docs/spec/security.md.
2360 env.session.principal = std::move(verified);
2361 ::morph::exec::detail::ModelId const mid{env.modelId};
2362 // One lookup for the holder, its recorded owner and its hydration state
2363 // together: they are fields of one record.
2364 const auto* inst = _instances.find(mid);
2365 if (inst == nullptr) {
2366 // Answered in this task's turn: the server strand runs no handler,
2367 // so a lookup of a model that is gone never waits on a model that
2368 // is busy.
2369 reject("model not found");
2370 return;
2371 }
2372 std::shared_ptr<::morph::model::detail::IModelHolder> holder = inst->holder;
2373 std::shared_ptr<detail::HydrationState> hydration = inst->hydration;
2374 // Per-instance (row-level) authorization. `authorize` above only saw the
2375 // model *type*; this consults the optional ownership hook with the target
2376 // instance id and its recorded owner. The default hook allows all.
2377 // env.session carries the verified principal (stamped just above), so an
2378 // ownership authorizer compares the recorded owner against it.
2379 if (!_authorizer->authorizeInstance(env.session, env.modelType, env.actionType, mid.v, inst->owner)) {
2380 reject("unauthorized");
2381 return;
2382 }
2383 // Action-evolution policy gate (opt-in; see `PayloadCompleteness`).
2384 // After authorization — the diagnostic names the action's own field,
2385 // which is exactly what the `"schemas"` kind discloses and is therefore
2386 // owed the same gate — and before the in-flight slot is reserved, so a
2387 // payload that will not be dispatched never consumes one.
2388 if (_config.payloadCompleteness == PayloadCompleteness::RequireDeclaredFields) {
2389 const std::string missing = missingRequiredFields(env.modelType, env.actionType, env.body);
2390 if (!missing.empty()) {
2391 const std::string message = "payload missing required field(s): " + missing;
2392 reject(message.c_str());
2393 return;
2394 }
2395 }
2396 // Reserve the in-flight slot. From here the call is answered exactly
2397 // once, by `complete`, whichever of the model strand's finish and the
2398 // `executeTimeout` gets there first; `complete` posts the matching
2399 // `executeFinished` back to this strand.
2400 _inFlight += 1;
2401 ::morph::observe::detail::emitMetric(::morph::observe::Metric::executeInFlight,
2402 static_cast<double>(_inFlight));
2403 // The stop source a Task handler's token comes from. Requested by the
2404 // timeout alongside its reply, so a Task handler still suspended when
2405 // the caller is answered unwinds and leaves the action gate rather than
2406 // holding the model until its await completes; and by a `cancel` from
2407 // the caller that made the call. A call is cancellable only when its
2408 // handler is a Task, the one kind a stop reaches, and only on a
2409 // connection scope: an unscoped message has no connection to name the
2410 // call by, and callIds are the client's own, not unique across clients.
2411 bool const taskHandler = _dispatcher.dispatchesAsync(env.modelType, env.actionType);
2412 bool const cancellable = taskHandler && cid != 0;
2413 std::shared_ptr<::core::async::StopSource> stopSource;
2414 if (_executeTimeouts || cancellable) {
2415 stopSource = std::make_shared<::core::async::StopSource>();
2416 }
2417 CallTag const tag{.cid = cid, .callId = env.callId, .stop = cancellable ? stopSource : nullptr};
2418 auto self = shared_from_this();
2419 auto finished = std::make_shared<std::atomic_flag>();
2420 // Gives the slot back if this frame leaves by exception before the run
2421 // is handed to the model's strand: dispatchEnvelope's catch replies on
2422 // that path, so this only claims `finished` (making an armed timeout a
2423 // no-op) and decrements, here, on the strand.
2424 struct Reservation {
2425 RemoteServer& server;
2426 std::shared_ptr<std::atomic_flag> finished;
2427 CallTag tag;
2428 bool handedOff = false;
2429 Reservation(RemoteServer& owner, std::shared_ptr<std::atomic_flag> flag, CallTag call)
2430 : server{owner}, finished{std::move(flag)}, tag{std::move(call)} {}
2431 Reservation(const Reservation&) = delete;
2432 Reservation& operator=(const Reservation&) = delete;
2433 Reservation(Reservation&&) = delete;
2434 Reservation& operator=(Reservation&&) = delete;
2435 // NOLINTNEXTLINE(bugprone-exception-escape): callFinished runs the server's drain bookkeeping on its owner; a failure there has no recovery in a destructor.
2436 ~Reservation() {
2437 if (!handedOff && !finished->test_and_set()) {
2438 server.callFinished(tag);
2439 }
2440 }
2441 } reservation{*this, finished, tag};
2442 if (cancellable) {
2443 fileCancellable(tag, env, inst->owner);
2444 }
2445
2446 std::uint64_t const callId = env.callId;
2447 auto replySlot = std::make_shared<std::function<void(std::string)>>(std::move(reply));
2448 // The decrement is posted before the reply goes out, so anything posted
2449 // to the server strand by someone who has seen the reply runs after it.
2450 auto const complete = [self, finished, replySlot, tag](std::string msg) {
2451 if (!finished->test_and_set()) {
2452 self->_strand.postTask([self, tag] { self->callFinished(tag); });
2453 (*replySlot)(std::move(msg));
2454 }
2455 };
2456
2457 ::morph::async::detail::TimeoutScheduler::Handle timeoutHandle{};
2458 if (_executeTimeouts) {
2459 // Safe to fire after its own `cancel()`, which the model strand's
2460 // finish issues on both its success and its failure arm:
2461 // `TimeoutScheduler::cancel` stops a callback that has not started
2462 // but returns without waiting for one that already has. A timeout
2463 // callback already mid-flight when the dispatch finishes therefore
2464 // still calls `complete`, and `complete`'s reply-exactly-once flag
2465 // drops it rather than double-answering the call.
2466 timeoutHandle =
2467 _executeTimeouts->schedule(limits.executeTimeout, [complete, callId, stopSource]() mutable {
2468 complete(::morph::wire::encode(::morph::wire::makeErr("timeout", callId)));
2469 stopSource->request_stop();
2470 });
2471 }
2472
2473 RemoteRun run;
2474 run.self = self;
2475 run.env = std::move(env);
2476 run.holder = std::move(holder);
2477 run.hydration = std::move(hydration);
2478 run.complete = complete;
2479 run.timeoutHandle = timeoutHandle;
2480 run.stopSource = std::move(stopSource);
2481 run.mid = mid;
2482 run.cid = cid;
2483 run.sharedKey = inst->sharedKey;
2484 // Posted in this strand's order, which is the order `handle()` was
2485 // called in; the model's strand runs its tasks in the order they were
2486 // posted. That is the whole of per-model execute ordering.
2487 //
2488 // Through the instance's action gate: an action starts only once the one
2489 // before it has finished, which a Task handler does when its Task
2490 // completes rather than when the strand task that started it returns.
2491 if (taskHandler) {
2492 // A Task run is shared: its completion callback outlives the strand
2493 // task.
2494 auto shared = std::make_shared<RemoteRun>(std::move(run));
2495 _strands->post(mid,
2496 [shared] { shared->holder->actionGate().enter([shared] { startTaskRemote(shared); }); });
2497 } else {
2498 // An ordinary run travels by value in the strand task, so an
2499 // execute costs the post's one allocation, as LocalBackend's does.
2500 // See `ActionGate::tryEnter`.
2501 _strands->post(mid, [run = std::move(run)]() mutable {
2502 auto& gate = run.holder->actionGate();
2503 if (gate.tryEnter()) {
2504 startRemote(run);
2505 return;
2506 }
2507 gate.enter([waiting = std::make_shared<RemoteRun>(std::move(run))] { startRemote(*waiting); });
2508 });
2509 }
2510 // The model strand's task now owns `complete`, so the in-flight slot is
2511 // its responsibility rather than this frame's.
2512 reservation.handedOff = true;
2513 }
2514
2515 /// @brief Returns the next opaque model id. On the server strand.
2516 ///
2517 /// Runs an internal monotonic counter through `detail::OpaqueIdGenerator`,
2518 /// so distinct calls never collide (the permutation is a bijection) but
2519 /// the returned values are not sequential. Skips the one counter value
2520 /// (if any) whose permutation is exactly `0` — `ModelId`'s reserved
2521 /// "unbound" sentinel (see `strand.hpp`) — which is possible in principle
2522 /// (the permutation is a bijection over the *entire* 64-bit domain, so
2523 /// exactly one input maps to `0`) but has probability 1-in-2^64 for a
2524 /// random key; guarded defensively rather than ever handed out.
2525 /// @return A freshly-generated, non-zero, opaque `ModelId`.
2526 [[nodiscard]] ::morph::exec::detail::ModelId nextOpaqueId() {
2527 for (;;) {
2528 if (uint64_t const opaque = _idGen.permute(++_nextId); opaque != 0) {
2529 return ::morph::exec::detail::ModelId{opaque};
2530 }
2531 }
2532 }
2533
2534 /// Everything one dispatched execute carries from `dispatchExecute` to its
2535 /// reply. An ordinary handler's run is held by its model-strand task, or by
2536 /// the action gate's queue while it waits there; a Task handler's is
2537 /// shared, because its completion callback holds it too. Holding `self` is
2538 /// what keeps the server, and with it `_dispatcher`, alive until the reply
2539 /// is delivered.
2540 struct RemoteRun {
2541 std::shared_ptr<RemoteServer> self;
2542 ::morph::wire::Envelope env;
2543 std::shared_ptr<::morph::model::detail::IModelHolder> holder;
2544 std::shared_ptr<detail::HydrationState> hydration;
2545 std::function<void(std::string)> complete;
2546 ::morph::async::detail::TimeoutScheduler::Handle timeoutHandle{};
2547 /// Null without an `executeTimeout`.
2548 std::shared_ptr<::core::async::StopSource> stopSource;
2549 ::morph::exec::detail::ModelId mid{};
2550 /// Connection the execute arrived on; `0` for unscoped.
2551 ConnectionId cid{0};
2552 /// The instance's directory key at admission; empty for a private instance.
2553 std::optional<detail::DirectoryKey> sharedKey;
2554 std::chrono::steady_clock::time_point start;
2555 ::morph::observe::SpanId spanId{};
2556 };
2557
2558 /// Stamps an execute's start once it holds its instance's action gate.
2559 ///
2560 /// Metrics and endSpan are recorded before `complete(...)` runs (in
2561 /// finishRemote) so a caller observing completion — via handle()'s reply
2562 /// or the timeout path racing it — can never see the reply before this
2563 /// dispatch's own instrumentation is recorded. This mirrors the
2564 /// reply-exactly-once contract `complete` already provides: whichever
2565 /// path wins the race, the metrics for *this* dispatch are always
2566 /// emitted, exactly once, regardless of which path's reply the caller
2567 /// actually receives.
2568 ///
2569 /// A handler whose decoded action fails ActionValidator<Action>::ready(...)
2570 /// throws morph::model::ValidationError before Model::execute runs. No
2571 /// special-casing is needed: it becomes an ordinary `err` reply carrying its
2572 /// message and callId, exactly like any other dispatch failure. See
2573 /// docs/spec/core/registry.md.
2574 static void admitRemote(RemoteRun& run) {
2575 run.start = std::chrono::steady_clock::now();
2576 run.spanId =
2577 ::morph::observe::detail::beginSpan(run.env.session.requestId, run.env.modelType, run.env.actionType);
2578 }
2579
2580 /// Runs an ordinary handler's execute once it holds its instance's action
2581 /// gate, on the strand, and replies.
2582 static void startRemote(RemoteRun& run) {
2583 MORPH_ZONE("RemoteServer::startRemote");
2584 MORPH_ZONE_TEXT(run.env.session.requestId);
2585 admitRemote(run);
2586 std::string result;
2587 std::exception_ptr error;
2588 try {
2589 ::morph::session::detail::ScopedContext const scoped{run.env.session};
2590 result = run.self->_dispatcher.dispatch(run.env.modelType, run.env.actionType, *run.holder, run.env.body);
2591 } catch (...) {
2592 error = std::current_exception();
2593 }
2594 finishRemote(run, std::move(result), error);
2595 }
2596
2597 /// Starts a Task handler's execute once it holds its instance's action
2598 /// gate, on the strand. It replies when its Task completes.
2599 static void startTaskRemote(const std::shared_ptr<RemoteRun>& run) {
2600 MORPH_ZONE("RemoteServer::startTaskRemote");
2601 MORPH_ZONE_TEXT(run->env.session.requestId);
2602 admitRemote(*run);
2603 std::shared_ptr<::morph::exec::detail::TaskResumer> executor;
2604 try {
2605 ::morph::session::detail::ScopedContext const scoped{run->env.session};
2606 auto const& strands = run->self->_strands;
2607 executor = std::make_shared<::morph::exec::detail::TaskResumer>(strands, run->mid, run->env.session);
2608 strands->enroll(run->mid, executor);
2609 auto token = run->stopSource ? run->stopSource->get_token() : ::core::async::StopToken{};
2610 run->self->_dispatcher.dispatchAsync(
2611 run->env.modelType, run->env.actionType, *run->holder, run->env.body, executor, std::move(token),
2612 [run, resumer = executor.get()](std::string result, std::exception_ptr error) {
2613 // As on LocalBackend: a handler can end on another
2614 // executor, and leaving the gate belongs on the strand.
2615 // The strands are held here, not reached through the run:
2616 // the finish may release the last reference to this server.
2617 auto const held = run->self->_strands;
2618 held->withdraw(run->mid, resumer);
2619 held->runOnStrand(run->mid, [run, result = std::move(result), error = std::move(error)] {
2620 finishRemote(*run, result, error);
2621 });
2622 });
2623 } catch (...) {
2624 run->self->_strands->withdraw(run->mid, executor.get());
2625 finishRemote(*run, std::string{}, std::current_exception());
2626 }
2627 }
2628
2629 /// Derives what a successful execute changed and posts it to `fanOut` on
2630 /// the server strand. On the model's strand, before the reply, so it runs
2631 /// whether or not the caller is still waiting for that reply.
2632 ///
2633 /// A failure here is logged and goes no further: the reply and the action
2634 /// gate's release still follow.
2635 /// @param run The finished execute.
2636 /// @param result Its result JSON.
2637 static void postChanges(RemoteRun const& run, std::string_view result) {
2638 try {
2639 auto changes = ::morph::effects::deriveChanges(
2640 run.env.actionType, result,
2641 ::morph::effects::InstanceChange{.modelType = run.env.modelType,
2642 .instance = run.mid.v,
2643 .primary = run.sharedKey ? run.sharedKey->second : std::string{}});
2644 if (changes.empty()) {
2645 return;
2646 }
2647 // The execute's principal is the verified one by now (`dispatchExecute`).
2648 run.self->_strand.postTask([self = run.self, changes = std::move(changes), caller = run.cid,
2649 principal = run.env.session.principal]() mutable {
2650 self->fanOut(std::move(changes), caller, principal);
2651 });
2652 } catch (const std::exception& exc) {
2653 ::morph::log::logError("[fanOut] changes of {} not derived: {}", run.env.actionType, exc.what());
2654 } catch (...) {
2655 ::morph::log::logError("[fanOut] changes of {} not derived: unknown exception", run.env.actionType);
2656 }
2657 }
2658
2659 /// Records a finished execute, replies, and leaves the action gate so the
2660 /// next execute on the instance can start. On the strand.
2661 static void finishRemote(RemoteRun& run, std::string result, const std::exception_ptr& error) {
2662 MORPH_ZONE("RemoteServer::finishRemote");
2663 MORPH_ZONE_TEXT(run.env.session.requestId);
2664 auto& self = *run.self;
2665 if (self._executeTimeouts) {
2666 self._executeTimeouts->cancel(run.timeoutHandle);
2667 }
2668 bool const succeeded = error == nullptr;
2669 // Settle hydration before `endSpan`, before the metrics and before
2670 // `complete` -- each hands control to host code that is free to attach
2671 // to this instance's key, and an attacher reaching the directory while
2672 // the outcome is known but unrecorded is handed an instance whose first
2673 // action has already failed. See shared_instances.md's Failure modes.
2674 if (run.hydration) {
2675 run.hydration->settle(succeeded);
2676 }
2677 ::morph::observe::detail::endSpan(run.spanId, succeeded);
2678 auto const elapsedMs =
2679 std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - run.start).count();
2680 std::array<std::pair<std::string_view, std::string_view>, 2> const tags{
2681 {{"modelType", run.env.modelType}, {"actionType", run.env.actionType}}};
2682 ::morph::observe::detail::emitMetric(::morph::observe::Metric::executeLatencyMs, elapsedMs, tags);
2683 if (succeeded) {
2684 postChanges(run, result);
2685 run.complete(::morph::wire::encode(::morph::wire::makeOk(run.env.callId, std::move(result))));
2686 } else {
2687 ::morph::observe::detail::emitMetric(::morph::observe::Metric::executeErrors, 1.0, tags);
2688 std::string message = "unknown exception";
2689 try {
2690 std::rethrow_exception(error);
2691 } catch (const std::exception& exc) {
2692 message = exc.what();
2693 } catch (...) { // NOLINT(bugprone-empty-catch): the message stays "unknown exception"
2694 // Keeps "unknown exception": the reply still goes out, so the
2695 // caller's completion does not wait for its deadline.
2696 }
2697 run.complete(::morph::wire::encode(::morph::wire::makeErr(message, run.env.callId)));
2698 }
2699 run.holder->actionGate().leave();
2700 }
2701
2702 // The server strand. Declared before every member its tasks touch, and
2703 // closed first, in the destructor's body.
2705 // One strand per model instance, shared with the Task handlers' resumers.
2706 // Never closed before its last owner goes: a suspended handler's driver
2707 // frame holds its `RemoteRun`, and with it this server.
2708 std::shared_ptr<::morph::exec::detail::ModelStrands> _strands;
2709 ::morph::model::detail::ActionDispatcher& _dispatcher;
2710 ::morph::model::detail::ModelRegistryFactory& _registry;
2711 std::shared_ptr<::morph::session::IAuthorizer> _authorizer;
2712 // Fixed at construction; read on the server strand and on model strands
2713 // without a lock because nothing writes it afterwards.
2714 ServerConfig const _config;
2715 // Arms `LimitPolicy::executeTimeout`; present only when one is configured.
2716 // Fixed at construction, so a model strand's finish can cancel through it
2717 // without a lock.
2718 std::unique_ptr<::morph::async::detail::TimeoutScheduler> const _executeTimeouts;
2719
2720 // ── Server-strand state: touched only in the server strand's tasks ──────
2721 // Every live instance, private and shared alike, plus the shared-instance
2722 // directory over them — holder, owner principal, attach count, directory key
2723 // and hydration state as one record per instance, and the same type
2724 // LocalBackend owns. `HydrationState` is the one part with synchronisation
2725 // of its own: it is settled from the model strand's finish, reached through
2726 // a `shared_ptr` captured at admission rather than by looking the instance
2727 // up again.
2728 detail::InstanceDirectory _instances;
2729 // Connection-scope bookkeeping (opt-in; see openConnection/closeConnection
2730 // and the scoped handle(msg, reply, cid) overload). The value is a *count*
2731 // per instance, not a set: one connection may attach the same shared
2732 // instance from two handlers, and closing the connection must release both
2733 // references or the instance leaks. A private instance always has a count
2734 // of exactly 1.
2735 std::unordered_map<ConnectionId, std::unordered_map<::morph::exec::detail::ModelId, std::size_t,
2736 ::morph::exec::detail::ModelIdHash>>
2737 _connectionScopes;
2738 // The writer of each open connection that has one, set by
2739 // `setConnectionWriter` and dropped with the connection's scope.
2740 std::unordered_map<ConnectionId, ConnectionOutbox> _writers;
2741 // What services recorded about each open connection (`factsOf`), dropped
2742 // with the connection's scope.
2743 std::unordered_map<ConnectionId, ConnectionFacts> _facts;
2744 // Handed to a service for a message with no open connection behind it;
2745 // reset before each such message.
2746 ConnectionFacts _throwawayFacts;
2747 // What each open connection watches, filled by `watch`/`unwatch` and
2748 // dropped with the connection's scope or on a failed write.
2749 detail::WatchTable _watches;
2750 // The verified principal of the first stamped envelope on each open
2751 // connection, which a later one under another principal replaces after
2752 // dropping the connection's watches (`compareWithBaseline`). Dropped
2753 // with the connection's scope.
2754 std::unordered_map<ConnectionId, std::string> _principalBaselines;
2755 /// Names one cancellable call: the connection it arrived on and the
2756 /// `callId` its client chose.
2757 struct CancelKey {
2758 ConnectionId cid{0};
2759 std::uint64_t callId{0};
2760 bool operator==(const CancelKey&) const = default;
2761 };
2762 struct CancelKeyHash {
2763 std::size_t operator()(const CancelKey& key) const noexcept {
2764 // Golden-ratio multiply spreads the connection id before the mix,
2765 // so two connections' runs of small callIds do not collide.
2766 constexpr std::uint64_t kSpread = 0x9E3779B97F4A7C15ULL;
2767 return std::hash<std::uint64_t>{}((key.cid * kSpread) ^ key.callId);
2768 }
2769 };
2770 /// What a cancel is judged against: the facts the execute was admitted on.
2771 struct CancelTarget {
2772 std::shared_ptr<::core::async::StopSource> stop;
2773 std::string principal;
2774 std::string modelType;
2775 std::string actionType;
2776 std::uint64_t modelId{0};
2777 std::string owner;
2778 };
2779 // Admitted Task executes on a connection scope that have not replied yet,
2780 // filed by `fileCancellable` and withdrawn by `callFinished` or the cancel
2781 // that stops them.
2782 std::unordered_map<CancelKey, CancelTarget, CancelKeyHash> _cancellable;
2783 std::uint64_t _nextId{0};
2784 detail::OpaqueIdGenerator _idGen;
2785 // Admitted executes whose reply has not been sent: incremented at
2786 // admission, decremented by `executeFinished`, which each reply posts here.
2787 // What `maxInFlightExecutes` gates, `health()` reports, the
2788 // `executeInFlight` metric carries and `drainedWithin()` waits on.
2789 std::size_t _inFlight{0};
2790 // Set by beginShutdown() and never cleared.
2791 bool _shuttingDown{false};
2792 // `health()`'s `ready`; cleared by beginShutdown() and never set again.
2793 bool _ready{true};
2794 /// One `drainedWithin()` call still waiting for the in-flight count to reach zero.
2795 struct DrainWaiter {
2796 std::uint64_t id;
2797 DrainPromise promise;
2798 ::morph::async::detail::TimeoutScheduler::Handle timer;
2799 };
2800 std::vector<DrainWaiter> _drainWaiters;
2801 std::uint64_t _nextDrainWaiter{0};
2802 // Arms drain deadlines and coalescing windows; created by the first that
2803 // needs one. Its callbacks only post back to the strand.
2804 std::unique_ptr<::morph::async::detail::TimeoutScheduler> _strandTimer;
2805 // Numbers each coalescing window, so a window's late close cannot send a
2806 // queue a later window holds.
2807 std::uint64_t _nextWindow{0};
2808
2809 // Drawn off the strand by openConnection(), which answers at once.
2810 std::atomic<uint64_t> _nextConnectionId{0};
2811};
2812
2813/// @brief `IBackend` adapter that routes all calls through a `RemoteServer` as
2814/// wire `Envelope` messages.
2815///
2816/// Intended for testing and in-process simulation of remote execution.
2817/// Control envelopes (`bindModel`, `registerModel`, `deregisterModel`, ...) are
2818/// processed through `RemoteServer::handleInline`, which answers before it
2819/// returns, so a bind settles inside the call. `execute()` is asynchronous: it
2820/// sends the message through `RemoteServer::handle` and resolves the returned
2821/// `Completion` when the reply arrives.
2822///
2823/// Its pending list and session belong to the caller's owner (`setOwner`) and
2824/// are touched only there, without a lock; the server's reply callback touches
2825/// only the completion it settles.
2826///
2827/// A backend scoped to a connection also receives the server's events for it
2828/// (`setEventHandler`): it registers a `ConnectionWriter` whose `write` hands
2829/// each envelope to the handler, in the order the server posted them, with a
2830/// `backlog` that always reads `0` -- an in-process hand-off holds nothing
2831/// back, so the server never withholds an event or owes a `resync`.
2832class SimulatedRemoteBackend : public detail::IBackend {
2833public:
2834 /// @brief Constructs the backend targeting @p server, unscoped.
2835 ///
2836 /// Every `register`/`deregister`/`attach`/`assign`/`instances` call this
2837 /// backend makes uses `ConnectionId{0}` — the server's "unscoped" sentinel
2838 /// (`backend.md`, "Connection scopes"). Use the `ConnectionId` constructor
2839 /// below to give this backend its own scope instead, so tests can
2840 /// exercise connection-scoped state (rate limiting, connection-drop
2841 /// recovery, shared-instance attach/detach across connections)
2842 /// deterministically without a real socket.
2843 /// @param server The `RemoteServer` instance to forward calls to. Borrowed,
2844 /// not owned: it must outlive this backend — a backend holding
2845 /// a `RemoteServer&` that has been destroyed is a
2846 /// use-after-free on every call (see
2847 /// `docs/spec/concurrency_and_lifetimes.md`, "Destruction
2848 /// ordering").
2849 explicit SimulatedRemoteBackend(RemoteServer& server MORPH_LIFETIMEBOUND) : _server{server} {}
2850
2851 /// @brief Constructs the backend targeting @p server, scoped to @p cid.
2852 ///
2853 /// @p cid must have been obtained from `server.openConnection()` — the
2854 /// same connection-scope mechanism `QtWebSocketServer`/`morph::net::SocketServer`
2855 /// use for a real transport, made available for in-process simulation. Every
2856 /// `register`/`attach`/`assign`/`instances` call this backend makes is
2857 /// attributed to @p cid's scope (via the three-argument `RemoteServer::handle`/
2858 /// `handleInline` overloads), so a `deregisterModel`/destructor-driven
2859 /// `closeConnection` reclaims exactly what this backend registered — the
2860 /// same reclamation guarantee a dropped socket gives a real client.
2861 ///
2862 /// The backend does **not** call `closeConnection` itself on destruction:
2863 /// unlike a real transport, there is no single well-defined "this
2864 /// connection is gone" moment to hook here (the caller may want to keep
2865 /// the scope open past this object's lifetime, e.g. to construct a
2866 /// second `SimulatedRemoteBackend` against the same scope). Call
2867 /// `closeConnection(cid)` explicitly, or destroy `server` itself, when
2868 /// the simulated connection should be reclaimed.
2869 /// @param server The `RemoteServer` instance to forward calls to. Borrowed,
2870 /// on the same terms as the constructor above.
2871 ///
2872 /// The backend also becomes @p cid's event receiver: it gives the server a
2873 /// `ConnectionWriter` for @p cid, replacing any writer the connection had,
2874 /// so a second backend on the same scope takes the events over.
2875 /// @param cid Connection scope, as returned by `server.openConnection()`.
2876 SimulatedRemoteBackend(RemoteServer& server MORPH_LIFETIMEBOUND, ConnectionId cid)
2877 : _server{server}, _cid{cid}, _inbox{std::make_shared<EventInbox>()} {
2878 if (_cid == 0) {
2879 return;
2880 }
2881 _server.setConnectionWriter(
2882 _cid, ConnectionWriter{.write = [inbox = _inbox](std::string const& frame) { inbox->receive(frame); },
2883 .backlog = [] { return std::size_t{0}; }});
2884 }
2885
2886 /// @brief Stops delivering events: the server may still post to the
2887 /// connection, and those envelopes are dropped.
2889 if (_inbox) {
2890 _inbox->set(nullptr, nullptr);
2891 }
2892 }
2893
2895 SimulatedRemoteBackend& operator=(const SimulatedRemoteBackend&) = delete;
2897 SimulatedRemoteBackend& operator=(SimulatedRemoteBackend&&) = delete;
2898
2899 /// @brief Registers the model type on the server and returns its assigned id.
2900 ///
2901 /// Processed through `RemoteServer::handleInline`, which answers before it
2902 /// returns. The @p factory argument is ignored — model construction is
2903 /// delegated to the server's `ModelRegistryFactory`.
2904 ///
2905 /// @param typeId String type-id sent in the `register` message.
2906 /// @return `ModelId` assigned by the server.
2907 /// @throws std::runtime_error if the server replies with an error.
2908 ::morph::exec::detail::ModelId registerModel(
2909 const std::string& typeId,
2910 std::function<std::unique_ptr<::morph::model::detail::IModelHolder>()> /*factory*/) override {
2911 return registerModelWithContext(typeId, {}, {});
2912 }
2913
2914 /// @brief Registers the model type on the server, carrying @p contextKey across
2915 /// the wire so the server's `RemoteServer::LogProvider` (if configured)
2916 /// can attach an action log to the instance it creates.
2917 ///
2918 /// @p factory is still ignored — model construction is delegated to the
2919 /// server's `ModelRegistryFactory`, same as `registerModel()`.
2920 /// @param typeId String type-id sent in the `register` message.
2921 /// @param contextKey Stable identity of the new instance; empty if none.
2922 /// @return `ModelId` assigned by the server.
2923 /// @throws std::runtime_error if the server replies with an error.
2924 ::morph::exec::detail::ModelId registerModelWithContext(
2925 const std::string& typeId, std::function<std::unique_ptr<::morph::model::detail::IModelHolder>()> /*factory*/,
2926 std::string_view contextKey) override {
2927 note("SimulatedRemoteBackend::registerModelWithContext");
2928 auto env = ::morph::wire::makeRegister(typeId, std::string{contextKey});
2929 env.session = _session;
2930 auto reply = ::morph::wire::decode(_server.handleInline(::morph::wire::encode(env), _cid));
2931 if (reply.kind == "ok") {
2932 return ::morph::exec::detail::ModelId{reply.modelId};
2933 }
2934 throw std::runtime_error("register failed: " + reply.message);
2935 }
2936
2937 /// @brief Acquires an instance on the server for @p request and settles
2938 /// the returned `Completion` before returning.
2939 ///
2940 /// Each shape of `BindRequest` is one envelope, processed through
2941 /// `RemoteServer::handleInline` on the calling thread: a private
2942 /// `register`, a shared `register` (register-or-attach), or an `attach`
2943 /// that re-points from `current` in one request, so a re-pointing client
2944 /// cannot lose its slot to `LimitPolicy::maxLiveModels` between releasing
2945 /// the old instance and acquiring the new one. An empty `primary` with a
2946 /// live `current` releases it first and binds a private instance.
2947 /// @param request Owning bind request; moved from.
2948 /// @param cbExec Executor the continuation is delivered on. Borrowed: it
2949 /// must outlive the returned `Completion`.
2950 /// @return A settled `Completion`: the bound id, or the server's refusal as
2951 /// a `std::runtime_error`.
2953 ::morph::exec::IExecutor& cbExec) override {
2954 note("SimulatedRemoteBackend::bindModel");
2955 auto [completion, promise] =
2957 try {
2958 promise.resolve(bindNow(request));
2959 } catch (...) {
2960 promise.reject(std::current_exception());
2961 }
2962 return std::move(completion);
2963 }
2964
2965 /// @brief Records the owner every verb is called from.
2966 /// @param owner The caller's owner.
2967 void setOwner(const ::morph::exec::detail::OwnerAffinity& owner) override { _affinity.emplace(owner); }
2968
2969 /// @brief Files a live server-side instance under @p primary.
2970 /// @param mid Live instance to promote.
2971 /// @param typeId Model type id.
2972 /// @param primary Canonical string encoding of the key to file it under.
2973 void assignPrimary(::morph::exec::detail::ModelId mid, const std::string& typeId,
2974 std::string_view primary) override {
2975 note("SimulatedRemoteBackend::assignPrimary");
2976 if (primary.empty() || mid.v == 0U) {
2977 return;
2978 }
2979 auto env = ::morph::wire::makeAssign(typeId, std::string{primary}, mid.v);
2980 env.session = _session;
2981 (void)_server.handleInline(::morph::wire::encode(env), _cid);
2982 }
2983
2984 /// @brief Asks the server for the live shared primary keys of @p typeId.
2985 /// @param typeId String type-id to enumerate.
2986 /// @return Canonical key strings of the live shared instances.
2987 /// @throws std::runtime_error if the server replies with an error.
2988 std::vector<std::string> listInstances(const std::string& typeId) override {
2989 note("SimulatedRemoteBackend::listInstances");
2990 auto reply = ::morph::wire::decode(
2991 _server.handleInline(::morph::wire::encode(::morph::wire::makeInstances(typeId)), _cid));
2992 if (reply.kind != "ok") {
2993 throw std::runtime_error("instances failed: " + reply.message);
2994 }
2995 std::vector<std::string> keys;
2996 // Not covered by this file's own test suite: `_server` here is always
2997 // the same in-process `RemoteServer` this backend was constructed
2998 // against, and `handleInstances` (this file, above) only ever builds
2999 // `reply.body` via `glz::write_json` of a `std::vector<std::string>`
3000 // populated straight from the instance directory's own keys -- themselves
3001 // `env.primary` values that already round-tripped through a
3002 // successful envelope decode, so they are valid UTF-8 by
3003 // construction. There is no path through `SimulatedRemoteBackend` +
3004 // `RemoteServer` that hands this call a syntactically malformed
3005 // body, so this branch cannot be forced without directly fabricating
3006 // a fake "ok" reply -- not a real scenario for this backend. The
3007 // identical decode-failure shape is genuinely reachable (and
3008 // covered separately) on a backend that talks to an actual external
3009 // peer -- see `QtWebSocketBackend::listInstances`
3010 // (qt_websocket_backend.hpp) and `socket_backend.hpp`'s own
3011 // equivalent, both of which face a real wire and a peer this
3012 // process does not control.
3013 if (auto errCode = glz::read_json(keys, reply.body)) {
3014 throw std::runtime_error("instances decode failed: " + glz::format_error(errCode, reply.body));
3015 }
3016 return keys;
3017 }
3018
3019 /// @brief Deregisters the model on the server. Processed inline.
3020 ///
3021 /// Carries this backend's own `ConnectionId`, so a shared instance's
3022 /// attach count is decremented against *this* backend's scope entry —
3023 /// never a different connection's — exactly as a real transport's
3024 /// `deregister` does (`backend.md`, "Connection scopes").
3025 /// @param mid Id of the model to deregister.
3026 void deregisterModel(::morph::exec::detail::ModelId mid) override {
3027 note("SimulatedRemoteBackend::deregisterModel");
3028 auto env = ::morph::wire::makeDeregister(mid.v);
3029 env.session = _session;
3030 (void)_server.handleInline(::morph::wire::encode(env), _cid);
3031 }
3032
3033 /// @brief Sends a `"hello"` envelope to the server and classifies its reply.
3034 ///
3035 /// Processed inline on the calling thread via `RemoteServer::handleInline`,
3036 /// same as `registerModel`/`deregisterModel`. Intended to be called once,
3037 /// typically right after construction and before any
3038 /// `registerModel`/`execute` call — nothing enforces that ordering.
3039 ///
3040 /// @return `Negotiated` if the server accepted `kProtocolVersion`;
3041 /// `LegacyPeer` if the server does not understand `"hello"` (an
3042 /// un-upgraded `RemoteServer`).
3043 /// @throws std::runtime_error if the server explicitly rejects the version
3044 /// (e.g. `"protocol version unsupported"`).
3045 ::morph::wire::ProtocolNegotiationResult negotiateProtocolVersion() {
3046 auto reply = ::morph::wire::decode(_server.handleInline(::morph::wire::encode(::morph::wire::makeHello())));
3047 return ::morph::wire::interpretHelloReply(reply);
3048 }
3049
3050 /// @brief Fetches the server's `{actionType: schema}` document for @p typeId.
3051 ///
3052 /// Sends a `"schemas"` envelope over the same synchronous control path as
3053 /// `registerModel`/`negotiateProtocolVersion` (`handleInline`), carrying
3054 /// the current session so an authorizing server can refuse. Opt-in in the
3055 /// same sense `negotiateProtocolVersion()` is: nothing calls it for you.
3056 ///
3057 /// The document is what a client that is **not** linked against the
3058 /// model's C++ needs — each action's properties, `required` array, form
3059 /// metadata, and its `x-payloadFingerprint`/`x-payloadShape`. A client
3060 /// that *is* linked against it can compare that fingerprint against its
3061 /// own `morph::model::payloadFingerprint<A>()` to detect a build skew.
3062 ///
3063 /// @param typeId Model type id to describe.
3064 /// @return The raw `{actionType: schema}` JSON document.
3065 /// @throws std::runtime_error if the server replies `err` (e.g.
3066 /// `"unauthorized"`, or `"unknown envelope kind: schemas"` from a
3067 /// server predating this kind).
3068 [[nodiscard]] std::string fetchActionSchemas(std::string typeId) {
3069 auto env = ::morph::wire::makeSchemas(std::move(typeId));
3070 env.session = _session;
3071 auto reply = ::morph::wire::decode(_server.handleInline(::morph::wire::encode(env)));
3072 if (reply.kind != "ok") {
3073 throw std::runtime_error("schemas request failed: " +
3074 (reply.message.empty() ? std::string{"malformed reply"} : reply.message));
3075 }
3076 return reply.body;
3077 }
3078
3079 /// @brief Serialises the action, sends it to the server, and returns a `Completion`.
3080 ///
3081 /// The `Completion` resolves when the server's reply is received and
3082 /// deserialized. Callbacks are posted via @p cbExec. The session attached to
3083 /// the call (via `Bridge::setDefaultSession()` or the per-call API) is
3084 /// serialised into the envelope.
3085 ///
3086 /// @param mid Target model id on the server.
3087 /// @param call Bundled action; `serializeAction` and `deserializeResult` are used.
3088 /// @param cbExec Executor for delivering the completion callbacks.
3089 /// @return Completion that resolves with the deserialized result or an error.
3091 detail::ActionCall call,
3092 ::morph::exec::IExecutor* cbExec) override {
3093 note("SimulatedRemoteBackend::execute");
3094 auto state = std::make_shared<::morph::async::detail::CompletionState<std::shared_ptr<void>>>();
3096 trackPending(state);
3097
3099 env.kind = "execute";
3100 env.modelId = mid.v;
3101 env.modelType = call.modelTypeId;
3102 env.actionType = call.actionTypeId;
3103 env.body = call.serializeBody();
3104 env.session = std::move(call.session);
3105 auto deser = std::move(call.deserializeResult);
3106
3107 _server.handle(
3108 ::morph::wire::encode(env),
3109 [state, deser = std::move(deser)](const std::string& replyJson) mutable {
3110 try {
3111 auto reply = ::morph::wire::decode(replyJson);
3112 // Triage shared with net::SocketBackend and
3113 // QtWebSocketBackend (detail/reply_router.hpp), so the
3114 // three cannot drift apart. The error text stays
3115 // backend-specific: this one substitutes "malformed reply"
3116 // for an empty message, which the socket backends do not.
3117 switch (detail::classifyExecuteReply(reply)) {
3118 case detail::ExecuteReplyKind::Value:
3119 state->setValue(deser(reply.body));
3120 break;
3121 case detail::ExecuteReplyKind::Timeout:
3122 throw TimeoutError{};
3123 case detail::ExecuteReplyKind::Error:
3124 default:
3125 // `default:` only because the project builds with
3126 // -Wswitch-default; every enumerator of the closed
3127 // ExecuteReplyKind is handled explicitly above.
3128 throw std::runtime_error(reply.message.empty() ? "malformed reply" : reply.message);
3129 }
3130 } catch (...) {
3131 state->setException(std::current_exception());
3132 }
3133 },
3134 _cid);
3135 return comp;
3136 }
3137
3138 /// @brief No-op — models live in `RemoteServer`, not locally.
3139 void notifyBackendChanged() override {}
3140
3141 /// @brief Resolves every still-pending completion this backend produced with @p exc.
3142 /// @param exc Exception delivered to every pending completion's error sink.
3143 void cancelPending(const std::exception_ptr& exc) override {
3144 note("SimulatedRemoteBackend::cancelPending");
3145 auto snapshot = std::exchange(_pending, {});
3146 for (auto& weak : snapshot) {
3147 if (auto state = weak.lock()) {
3148 state->setException(exc);
3149 }
3150 }
3151 }
3152
3153 /// @brief Installs the handler each server-initiated event is posted to.
3154 /// See `IBackend::setEventHandler`.
3155 ///
3156 /// Only a backend scoped to a connection (the `ConnectionId` constructor)
3157 /// receives events; an unscoped one has no connection to post to. Events
3158 /// are posted to @p exec in the order the server posted them. An `event`
3159 /// envelope is recognised by its kind, so it never settles a call.
3160 /// @param handler Callable posted to @p exec once per event. Pass
3161 /// `nullptr` to clear.
3162 /// @param exec Executor the handler is posted to. Borrowed: it must
3163 /// outlive the installation. May be null only when
3164 /// @p handler is.
3165 void setEventHandler(std::function<void(::morph::wire::EventBody)> handler,
3166 ::morph::exec::IExecutor* exec) override {
3167 if (_inbox) {
3168 _inbox->set(std::move(handler), exec);
3169 }
3170 }
3171
3172 /// @brief Installs the session stamped onto every control envelope this
3173 /// backend subsequently builds (`register`, `registerShared`,
3174 /// `attach`, `assign`, `deregister`). See `IBackend::setSession`.
3175 /// @param session Session to stamp; typically pushed by `Bridge::setDefaultSession()`.
3176 void setSession(::morph::session::Context session) override {
3177 note("SimulatedRemoteBackend::setSession");
3178 _session = std::move(session);
3179 }
3180
3181private:
3182 /// @brief The event handler and the entry point the connection's writer
3183 /// calls, shared with that writer so the server can outlive the backend.
3184 ///
3185 /// The writer runs on the server strand and `setEventHandler` on the
3186 /// backend's owner, so both sit behind a mutex.
3187 struct EventInbox {
3188 /// @brief Replaces the handler and its executor.
3189 /// @param next New handler; empty clears it.
3190 /// @param target Executor it is posted to.
3191 void set(std::function<void(::morph::wire::EventBody)> next, ::morph::exec::IExecutor* target) {
3192 std::scoped_lock const lock{mutex};
3193 handler =
3194 next ? std::make_shared<std::function<void(::morph::wire::EventBody)>>(std::move(next)) : nullptr;
3195 exec = target;
3196 }
3197
3198 /// @brief Decodes one envelope the server wrote and posts an event to the handler.
3199 /// @param frame The encoded envelope. Anything that is not a well-formed event is dropped.
3200 void receive(const std::string& frame) {
3201 std::shared_ptr<std::function<void(::morph::wire::EventBody)>> current;
3202 ::morph::exec::IExecutor* target = nullptr;
3203 {
3204 std::scoped_lock const lock{mutex};
3205 current = handler;
3206 target = exec;
3207 }
3208 if (!current || target == nullptr) {
3209 return;
3210 }
3211 // By kind: an event's `callId` is not a reply's.
3212 static_cast<void>(detail::routeEvent(::morph::wire::decode(frame), [&](::morph::wire::EventBody event) {
3213 target->post([current, event = std::move(event)]() mutable { (*current)(std::move(event)); });
3214 }));
3215 }
3216
3217 std::mutex mutex;
3218 std::shared_ptr<std::function<void(::morph::wire::EventBody)>> handler;
3219 ::morph::exec::IExecutor* exec = nullptr;
3220 };
3221
3222 /// @brief Checks, in a debug build, that the caller is on the owner the
3223 /// backend was given. Nothing to check before `setOwner`.
3224 /// @param site Name of the calling body.
3225 void note(char const* site) const noexcept {
3226 if (_affinity) {
3227 _affinity->note(site);
3228 }
3229 }
3230
3231 /// @brief `bindModel`'s body: one control envelope per request shape.
3232 /// @param request The bind request.
3233 /// @return The bound id.
3234 /// @throws std::runtime_error if the server replies with an error.
3235 ::morph::exec::detail::ModelId bindNow(const detail::BindRequest& request) {
3236 if (request.primary.empty()) {
3237 if (request.current.v != 0U) {
3238 deregisterModel(request.current);
3239 }
3240 return registerModelWithContext(request.typeId, {}, request.contextKey);
3241 }
3242 auto env =
3243 request.current.v == 0U
3244 ? ::morph::wire::makeRegisterShared(request.typeId, request.primary, request.contextKey)
3245 : ::morph::wire::makeAttach(request.typeId, request.primary, request.current.v, request.contextKey);
3246 env.session = _session;
3247 auto reply = ::morph::wire::decode(_server.handleInline(::morph::wire::encode(env), _cid));
3248 if (reply.kind == "ok") {
3249 return ::morph::exec::detail::ModelId{reply.modelId};
3250 }
3251 throw std::runtime_error((request.current.v == 0U ? "register failed: " : "attach failed: ") + reply.message);
3252 }
3253
3254 void trackPending(const std::shared_ptr<::morph::async::detail::CompletionState<std::shared_ptr<void>>>& state) {
3255 std::erase_if(_pending, [](const auto& weak) { return weak.expired(); });
3256 _pending.emplace_back(state);
3257 }
3258
3259 RemoteServer& _server;
3260 // 0 = unscoped (the default constructor's behavior); non-zero when
3261 // constructed with a ConnectionId from server.openConnection(). Threaded
3262 // through every handle()/handleInline() call this backend makes.
3263 ConnectionId _cid{0};
3264 // Null for an unscoped backend, which has no connection to receive on.
3265 std::shared_ptr<EventInbox> _inbox;
3266 // The owner every verb is called from, once `setOwner` has named it. The
3267 // two fields below are touched only there.
3268 std::optional<::morph::exec::detail::OwnerAffinity> _affinity;
3269 std::vector<std::weak_ptr<::morph::async::detail::CompletionState<std::shared_ptr<void>>>> _pending;
3270 ::morph::session::Context _session;
3271};
3272
3273} // namespace morph::backend
AnyHandle env(std::string const &name)
A client environment value, {"env": name}: widthClass, locale, platform, online, backend.
Definition author.hpp:1209
Command run(MutationHandle< Action > const &mutation, Body< Action > const &body={}, RunOptions const &options={})
run: runs a mutation.
Definition author.hpp:2755
AnyHandle session(std::string const &name)
A session attribute, {"session": name}.
Definition author.hpp:1214
constexpr Self self
The dialog or drawer a command list runs in.
Definition author.hpp:1295
What RemoteServer has queued for one connection's next changes event.
Changes deriveChanges(std::string_view actionId, std::string_view resultJson, InstanceChange const &ranOn)
The changes a successful execute made, from the action's declared effects and its result.
Definition changes.hpp:265
Producer-side handle paired with a Completion<T> by makeSettleable().
Move-only handle representing the eventual result of an asynchronous operation.
static std::pair< Completion< T >, Promise > makeSettleable(::morph::exec::IExecutor *execPtr)
Constructs a Completion<T>/Promise<T> pair sharing one settleable state.
Envelope kinds a RemoteServer answers through an extension, such as the UI service's ui-* kinds.
Definition remote.hpp:231
virtual bool serves(std::string_view kind) const =0
Whether this service answers kind.
virtual std::string handle(const ::morph::wire::Envelope &request, const ::morph::session::IAuthorizer &authorizer)=0
Answers one envelope of a kind serves accepted.
virtual std::string handle(const ::morph::wire::Envelope &request, const ::morph::session::IAuthorizer &authorizer, ConnectionFacts &facts)
Answers one envelope of a kind serves accepted, with the facts of the connection it arrived on.
Definition remote.hpp:275
Server-side message handler that owns model instances and dispatches actions.
Definition remote.hpp:365
detail::WatchTable const & watchTable() const
The topics each open connection watches.
Definition remote.hpp:749
bool eventsNegotiated(ConnectionId cid) const
Whether connection cid negotiated server-sent events.
Definition remote.hpp:737
void setConnectionWriter(ConnectionId cid, ConnectionWriter writer)
Hands the server the transport's writer for connection cid, through which post reaches it.
Definition remote.hpp:657
RemoteServer(::morph::exec::IExecutor &workerPool, std::shared_ptr<::morph::session::IAuthorizer > authorizer, ::morph::model::detail::ActionDispatcher &dispatcher=::morph::model::detail::defaultDispatcher(), ::morph::model::detail::ModelRegistryFactory &registry=::morph::model::detail::defaultRegistry())
Constructs a server with a custom authorizer and a default ServerConfig.
Definition remote.hpp:407
void fanOut(::morph::effects::Changes changes, ConnectionId caller, std::string_view callerPrincipal)
Tells every connection watching what changes names that it changed, as one changes event per connecti...
Definition remote.hpp:786
::morph::async::Completion< HealthStatus > health(::morph::exec::IExecutor &replyExec)
The server's health, answered on the server strand.
Definition remote.hpp:838
void post(ConnectionId cid, std::string envelope)
Writes envelope to connection cid, unless its reader has fallen behind.
Definition remote.hpp:696
~RemoteServer()
Closes the server strand before any other member goes.
Definition remote.hpp:493
std::string handleInline(const std::string &msg, ConnectionId cid)
Like handleInline(msg), but additionally attributes any register decoded from msg to a connection sco...
Definition remote.hpp:563
void handle(std::string msg, std::function< void(std::string)> reply, ConnectionId cid)
Like handle(msg, reply), but additionally attributes any register decoded from msg to a connection sc...
Definition remote.hpp:528
void closeConnection(ConnectionId cid)
Reclaims every model still registered under cid, then drops the scope.
Definition remote.hpp:637
void beginShutdown()
Enters shutdown: from now on, register, attach and execute envelopes are rejected with err "server sh...
Definition remote.hpp:861
::morph::backend::LogProvider LogProvider
Alias of morph::backend::LogProvider, the type of ServerConfig::logProvider.
Definition remote.hpp:369
void connectionDrained(ConnectionId cid)
Tells the server that connection cid's backlog has fallen below kEventBacklogLow.
Definition remote.hpp:721
RemoteServer(::morph::exec::IExecutor &workerPool, std::shared_ptr<::morph::session::IAuthorizer > authorizer, ServerConfig config, ::morph::model::detail::ActionDispatcher &dispatcher=::morph::model::detail::defaultDispatcher(), ::morph::model::detail::ModelRegistryFactory &registry=::morph::model::detail::defaultRegistry())
Constructs a server with a custom authorizer and config.
Definition remote.hpp:459
std::string handleInline(const std::string &msg)
Processes a control Envelope on the server strand and returns its reply, blocking the caller until it...
Definition remote.hpp:550
PayloadCompleteness payloadCompleteness() const noexcept
Returns the configured payload-completeness rule.
Definition remote.hpp:825
::morph::exec::IExecutor & strand() noexcept
The server strand, as an executor.
Definition remote.hpp:821
::morph::async::Completion< bool > drainedWithin(std::chrono::milliseconds deadline, ::morph::exec::IExecutor &replyExec)
Answers whether every in-flight execute delivers its reply within deadline.
Definition remote.hpp:892
RemoteServer(::morph::exec::IExecutor &workerPool, ServerConfig config, ::morph::model::detail::ActionDispatcher &dispatcher=::morph::model::detail::defaultDispatcher(), ::morph::model::detail::ModelRegistryFactory &registry=::morph::model::detail::defaultRegistry())
Constructs a server with allow-all authorization and config.
Definition remote.hpp:431
ConnectionId openConnection()
Opens a new connection scope and returns its id.
Definition remote.hpp:603
void handle(std::string msg, std::function< void(std::string)> reply)
Asynchronously processes a JSON Envelope and calls reply with the response.
Definition remote.hpp:510
RemoteServer(::morph::exec::IExecutor &workerPool, ::morph::model::detail::ActionDispatcher &dispatcher=::morph::model::detail::defaultDispatcher(), ::morph::model::detail::ModelRegistryFactory &registry=::morph::model::detail::defaultRegistry())
Constructs a server backed by workerPool with allow-all authorization and a default ServerConfig.
Definition remote.hpp:386
IBackend adapter that routes all calls through a RemoteServer as wire Envelope messages.
Definition remote.hpp:2832
void setOwner(const ::morph::exec::detail::OwnerAffinity &owner) override
Records the owner every verb is called from.
Definition remote.hpp:2967
SimulatedRemoteBackend(RemoteServer &server, ConnectionId cid)
Constructs the backend targeting server, scoped to cid.
Definition remote.hpp:2876
void notifyBackendChanged() override
No-op — models live in RemoteServer, not locally.
Definition remote.hpp:3139
std::string fetchActionSchemas(std::string typeId)
Fetches the server's {actionType: schema} document for typeId.
Definition remote.hpp:3068
::morph::async::Completion<::morph::exec::detail::ModelId > bindModel(detail::BindRequest request, ::morph::exec::IExecutor &cbExec) override
Acquires an instance on the server for request and settles the returned Completion before returning.
Definition remote.hpp:2952
~SimulatedRemoteBackend() override
Stops delivering events: the server may still post to the connection, and those envelopes are dropped...
Definition remote.hpp:2888
void assignPrimary(::morph::exec::detail::ModelId mid, const std::string &typeId, std::string_view primary) override
Files a live server-side instance under primary.
Definition remote.hpp:2973
SimulatedRemoteBackend(RemoteServer &server)
Constructs the backend targeting server, unscoped.
Definition remote.hpp:2849
void cancelPending(const std::exception_ptr &exc) override
Resolves every still-pending completion this backend produced with exc.
Definition remote.hpp:3143
void setSession(::morph::session::Context session) override
Installs the session stamped onto every control envelope this backend subsequently builds (register,...
Definition remote.hpp:3176
std::vector< std::string > listInstances(const std::string &typeId) override
Asks the server for the live shared primary keys of typeId.
Definition remote.hpp:2988
::morph::wire::ProtocolNegotiationResult negotiateProtocolVersion()
Sends a "hello" envelope to the server and classifies its reply.
Definition remote.hpp:3045
void deregisterModel(::morph::exec::detail::ModelId mid) override
Deregisters the model on the server. Processed inline.
Definition remote.hpp:3026
::morph::exec::detail::ModelId registerModelWithContext(const std::string &typeId, std::function< std::unique_ptr<::morph::model::detail::IModelHolder >()>, std::string_view contextKey) override
Registers the model type on the server, carrying contextKey across the wire so the server's RemoteSer...
Definition remote.hpp:2924
::morph::exec::detail::ModelId registerModel(const std::string &typeId, std::function< std::unique_ptr<::morph::model::detail::IModelHolder >()>) override
Registers the model type on the server and returns its assigned id.
Definition remote.hpp:2908
::morph::async::Completion< std::shared_ptr< void > > execute(::morph::exec::detail::ModelId mid, detail::ActionCall call, ::morph::exec::IExecutor *cbExec) override
Serialises the action, sends it to the server, and returns a Completion.
Definition remote.hpp:3090
void setEventHandler(std::function< void(::morph::wire::EventBody)> handler, ::morph::exec::IExecutor *exec) override
Installs the handler each server-initiated event is posted to. See IBackend::setEventHandler.
Definition remote.hpp:3165
One strand over a morph executor, usable wherever a morph executor is: the owner a component's state ...
What a transport hands RemoteServer to write to one connection, and the backlog bounds both sides app...
constexpr std::size_t kEventBacklogHigh
Backlog above which RemoteServer::post holds a connection's events back and owes it one resync instea...
std::uint64_t ConnectionId
Opaque id a transport uses to scope model registrations to one connection so RemoteServer::closeConne...
Resolved const & resolved()
The process registries, matched and validated on the first call.
Definition effects.hpp:511
LowerCtx attached(DetachedLower const &held)
Puts a detached context's scope back.
The check every loop-owned body makes on entry: "am I on my owner?".
OwnerStrand: one core-cpp strand over a morph executor, which is itself a morph executor — the owner ...
Compile-time profiler instrumentation: Tracy zones, plots, messages and thread names,...
#define MORPH_ZONE_TEXT(text)
Attaches text to the enclosing scope's MORPH_ZONE; an empty text attaches nothing.
Definition profiler.hpp:126
#define MORPH_ZONE(name)
Opens a profiler zone named name that ends with the enclosing scope. At most one per scope.
Definition profiler.hpp:118
constexpr bool operator==(const Quantity< U, DecA > &lhs, const Quantity< U, DecB > &rhs) noexcept
Equality across same-unit quantities of any declared precision. Ignores the runtime precision tag; em...
Definition quantity.hpp:973
The callId-multiplexed execute reply router, shared by every backend that talks to a RemoteServer ove...
void request(Store< ViewState, Msg > &store, bridge::BridgeHandler< Model, Sharing > &handler, async::CallbackScope const &scope, Action action, ToMsg toMsg, ToFailMsg toFailMsg)
Executes action and sends its result, or its failure, to store as one message.
Definition store.hpp:202
What a RemoteServer knows about one connection that a service, not core, decides: handed to IEnvelope...
Definition remote.hpp:216
bool events
Whether the connection negotiated server-sent events: its ui-hello offered them and the UI service ac...
Definition remote.hpp:219
A transport's outbound side of one connection, for what the server sends unasked: events.
std::function< void(std::string)> write
Queues one encoded envelope for the connection. May throw when the connection can no longer be writte...
Snapshot of a RemoteServer's current health.
Definition remote.hpp:192
bool ready
true if the server currently accepts and dispatches new work.
Definition remote.hpp:194
std::size_t inFlight
Number of executes currently dispatched but not yet replied.
Definition remote.hpp:198
std::size_t liveModels
Number of models currently registered on the server.
Definition remote.hpp:196
Opt-in, connection-agnostic resource limits enforced by RemoteServer.
Definition remote.hpp:55
std::size_t maxLiveModels
Max models this RemoteServer will hold live at once, across all callers. A register beyond this cap r...
Definition remote.hpp:73
std::chrono::milliseconds executeTimeout
Max wall-clock time a single execute may take before the server sends an err "timeout" reply and disc...
Definition remote.hpp:68
std::size_t maxInFlightExecutes
Max concurrent in-flight execute calls this server will accept before replying err "server busy" inst...
Definition remote.hpp:78
Everything about a RemoteServer that is configured rather than learned: given once,...
Definition remote.hpp:289
PayloadCompleteness payloadCompleteness
Whether an execute body must carry every field the action's served schema marks required....
Definition remote.hpp:323
std::function< void(const HealthStatus &)> healthHandler
Called with the server's health: once from the constructor, and again, on the server's strand,...
Definition remote.hpp:308
std::uint32_t maxProtocolVersion
Newest protocol version this server accepts in reply to "hello". Widen the range when a kProtocolVers...
Definition remote.hpp:318
std::uint32_t minProtocolVersion
Oldest protocol version this server accepts in reply to "hello". Must not exceed maxProtocolVersion.
Definition remote.hpp:312
std::vector< std::shared_ptr< IEnvelopeService > > services
Extensions that answer envelope kinds the server does not know itself, consulted in order....
Definition remote.hpp:329
LimitPolicy limits
Opt-in resource limits. Default-constructed: unbounded.
Definition remote.hpp:291
LogProvider logProvider
Consulted whenever an instance is constructed with a non-empty contextKey — every register envelope,...
Definition remote.hpp:300
std::chrono::milliseconds coalesceWindow
How long a connection's changes are merged before they are sent as one changes event.
Definition remote.hpp:338
Thrown to a pending Completion when the server-side morph::backend::LimitPolicy::executeTimeout elaps...
Definition backend.hpp:636
What one successful execute changed.
std::vector< RecordChange > records
Record topics, one entry per record type.
std::vector< InstanceChange > instances
Instance topics.
A change to the state of one model instance.
std::uint64_t instance
The instance id.
std::string primary
The directory key's text for a keyed shared instance; empty for a private one.
std::string modelType
The model's registered type name.
std::string record
The record's wire name.
std::optional< std::vector< glz::raw_json > > keys
The keys of the rows that changed, each as raw JSON; absent when the whole type changed.
Abstract executor interface.
Definition executor.hpp:70
virtual void post(std::function< void()> task)=0
Schedules task for asynchronous execution.
Per-call session context carried through the bridge to the model.
Definition session.hpp:26
std::vector< effects::RecordChange > records
Record changes; a record with no keys changed as a whole.
Definition wire.hpp:701
std::vector< InstanceTopic > instances
Instance changes.
Definition wire.hpp:703
JSON wire envelope used between any client and RemoteServer.
Definition wire.hpp:91
The decoded body of an event envelope.
Definition wire.hpp:621
std::string primary
The directory key's text for a keyed shared instance; empty for a private one.
Definition wire.hpp:675
std::string modelType
The model's registered type name.
Definition wire.hpp:673
std::vector< InstanceTopic > instances
Instance topics to watch (or stop watching).
Definition wire.hpp:685
std::vector< std::string > records
Record wire names to watch (or stop watching).
Definition wire.hpp:683
std::vector< InstanceTopic > refusedInstances
Instance topics the server refused.
Definition wire.hpp:695
std::vector< std::string > refusedRecords
Record names the server does not publish or the caller may not read.
Definition wire.hpp:693
The topics each connection watches, as RemoteServer keeps them.