Bench/leios new#6622
Draft
fmaste wants to merge 11 commits into
Draft
Conversation
- MempoolCapacityBytesOverride (See https://github.com/input-output-hk/ouroboros-leios/blob/ebc1f7c76b34e3d4f1b28485b720ed324633a6b3/demo/proto-devnet/config/config.yaml#L9) - MempoolTimeoutCapacity
…etwork The Driver LogFormatting and MetaTrace instances for the Simple and Stateful TraceSendRecv were carried here as orphan copies of the cardano-node definitions. ouroboros-network now ships these instances itself, so the local copies are dropped and the upstream ones consumed directly.
Add an optional wait between builder pre-fill and worker connection with an optional top-level "startup_delay_seconds" field, defaulting to 0 (no wait). The delay lets multi-node benchmark clusters stabilise before traffic begins, so transmission ramps to the target TPS immediately rather than racing node startup. Plumbing follows the existing config tiers: parsed as a Maybe Natural in Config.Raw, resolved to a Natural (default 0) in Config.Validated, and consumed in Main after the builders are spawned and before the workers open their connections. A value of 0 skips both the sleep and its log lines; using Natural rejects negative values at parse time. Co-Authored-By: John Lotoski <john.lotoski@iohk.io>
Every stage is now a self-tracing component wired through trace-dispatcher, so
what the pipeline exposes, from queue depths to the submission protocol, is
chosen and tuned in the config file rather than in code. No recompile is needed
to change what is traced or at what detail.
The pipeline is split into single-responsibility parts, each owning its own STM
primitives and emitting its own structured trace:
* Pipe: a generic input + payload queue pair (an unbounded input TQueue and a
bounded payload TBQueue), pure structure with no async.
It emits InputEnqueued, InputDequeued, PayloadEnqueued and PayloadDequeued
with the resulting queue depth on every movement.
* Recycler: the sole owner of closed-loop input recycling, lifted out of the
pipe. It runs its own event queue and worker and takes strategy-agnostic
fire-and-forget hooks (onBuild, onDequeue, onConfirm). The worker is
order-independent, so a payload's build and release events may arrive in any
order without losing a recycle. It emits Recycle.
* Observer: an independent confirmation observer, decoupled from the pipe and
the recycler. It owns its own broadcast channel and emits Announce.
Runtime resolves the config into four top-level, name-keyed pools (builders,
pipes, recyclers, observers) and owns the interconnections between them: which
builder feeds which pipe, which recycler recycles into which pipe, and which
observer drives confirm-based recycling. The builder, recycler and observer each
run their own async, while the pipe stays pure structure, just the queues the
others act on.
Tracing is a set of structured trace-dispatcher namespaces under
TxCentrifuge.{Builder, Pipe, Recycler, Observer, TxSubmission}, each
independently tunable by severity, detail level, backend and maxFrequency.
Detail level gates the heavy fields, with DDetailed revealing the per-fund and
per-txid payloads. The README gains a Tracing section documenting every namespace
and how to configure it.
The Pipe and Recycler traces are event-driven rather than periodic: each fires
on a queue movement carrying the depth (or pending count) at that instant. A
maxFrequency cap turns any depth-carrying namespace into a periodic readout from
config alone, but samples only while traffic flows, so there is no heartbeat on
an idle pipeline and no cumulative counters. The README gains a Tracing section
documenting every namespace and how to configure it.
Co-Authored-By: John Lotoski <john.lotoski@iohk.io>
Each builder paid its outputs to an address derived from its index, so an
operator could neither choose the destination nor readily fund and inspect it.
Add an optional per-builder destination_signing_key naming a .skey (payment or
genesis UTxO key) whose address receives every output the builder produces and
under which it recycles.
* interpretBuilder takes the network id and builder index and resolves the
key: read the configured .skey, or fall back to a built-in per-index key
when the param is omitted. The resolved key and address are stored on
ValueBuilder (destinationSigningKey, destinationAddress) and used by
buildTx.
* Announce each builder's destination address on stderr at startup, so the
address to fund is discoverable.
* Carry the destination in BuilderNewTx, emitted as a "destination" field at
DMaximum.
* Export Fund.readSigningKey and extract deriveAddress for reuse.
This makes each destination explicit and independently fundable, rather than
deriving every workload's key from a single seed by munging a hex suffix. The
same keys are what on-chain discovery must be pointed at to re-pick-up recycled
funds on restart.
Co-Authored-By: John Lotoski <john.lotoski@iohk.io>
Restarting previously required a static funds file, which drifts from the live chain as the recycling loop moves UTxOs around. Add a second initial_inputs loader that discovers the starting funds on chain, making a restart stateless: point it at the builders' destination addresses and it re-picks-up whatever a previous run left there. A new "local_utxo_query" variant of InitialFunds carries a NodeToClient socket_path and a signing_keys array. Fund.discoverFunds reads each key, derives its address, deduplicates addresses so a UTxO is attributed to a single key, queries the local node for the UTxOs at those addresses, and tags each returned UTxO with the key that can spend it. The query lives in a new NodeToClient.UTxOQuery module over the LocalStateQuery mini-protocol. The era is detected at runtime via QueryCurrentEra and dispatched through caseByronOrShelleyBasedEra, so the query follows the chain across Shelley-based eras rather than being pinned to Conway. Results are joined back to the caller's Conway-typed addresses through an internal AddressAny reverse index, keeping the era-agnostic projection out of the interface, and the call is wrapped in try so an unreachable socket yields a readable error instead of a crash. The file-based genesis_utxo_keys loader is retained; both variants share the load-and-validate handling in loadConfig, which now runs after the protocol is built so the network id is available to derive query addresses and open the connection. Co-Authored-By: John Lotoski <john.lotoski@iohk.io>
Previously the builder called die whenever TxAssembly.buildTx returned Left,
most often because a batch of inputs was collectively worth at most the fee and
so could not fund a valid change output. One unbuildable batch would take down
the whole service.
buildTx now returns a typed BuildError, partitioning failures by recoverability:
* InsufficientValue: the batch's summed inputs do not cover the fee plus one
non-zero output each. This depends on the specific inputs, so the builder
drops the batch and continues with the next one.
* InvalidInput / LedgerFailure: a bad builder argument or an opaque
cardano-api construction error. These are constant across every batch, so
silently dropping would spin forever, here the builder dies loudly instead.
To let a builder abandon a batch, the builder loop now owns its own iteration
through a sealed BuilderApi (baTakeInputs, baAddPayload, baDropInputs);
BuilderHandle becomes a newtype wrapping that loop, and the engine owns only the
thread and the machinery behind the API. An InsufficientValue batch is released
with baDropInputs (neither recycled nor re-enqueued) and reported via a
structured Builder.InputsDropped trace (Warning) carrying the builder name,
dropped funds, and reason.
Dropping is deliberately whole-batch, not per-input: buildTx sums all inputs, so
a small UTxO is swept into a transaction alongside larger ones rather than
discarded. Only collectively-unbuildable batches are dropped, and only at build
time; no startup dust pre-filter is added.
Co-Authored-By: John Lotoski <john.lotoski@iohk.io>
Add a --dry-run startup mode that performs all config-time validation and exits 0 before any pipelines are created or traffic is generated, so the same production config can be checked, for example after a restart or fresh funding, without producing load. loadConfig now parses an optional leading --dry-run flag and returns it as an extra Bool at the head of its result tuple. When set, main prints "Dry run OK" and exits immediately after loadConfig returns, i.e. after config parsing, consensus protocol setup and on-chain fund discovery have all succeeded, but before Runtime.resolve creates any observers, pipelines or builders. Usage becomes: tx-centrifuge [--dry-run] <config.json> This validates config, protocol and fund discovery only; it does not exercise Runtime.resolve or observer connectivity, and prints no partition summary. The flag is accepted only in the leading position. Co-Authored-By: John Lotoski <john.lotoski@iohk.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add your description here, if it fixes a particular issue please provide a
link
to the issue.
Checklist
See Running tests for more details
CHANGELOG.mdfor affected package.cabalfiles are updatedhlint. See.github/workflows/check-hlint.ymlto get thehlintversionstylish-haskell. See.github/workflows/stylish-haskell.ymlto get thestylish-haskellversionghc-9.6andghc-9.12Note on CI
If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.