Group Selection, Health, and Warm-up Design
honk resolves groups to leaf outbounds, tracks their health, and retains bounded warm resources.
Scope
The scope is GroupManager, AliveDialerSet, the always-compiled Score scorer, cold URLTest preparation, and the warm-resource coordinators. Group fields and policy syntax belong in the group reference; process-wide health, warm-up, and dial keys belong in the global reference.
Group manager and selection pipeline
SharedGroupManager is a stable, hot-swappable handle:
SharedGroupManager = Arc<parking_lot::RwLock<Arc<GroupManager>>>
A reload builds a complete replacement GroupManager, migrates surviving per-network Selector member identities via migrate_selector_choices_from, installs interrupt, warm-up and persistence callbacks before publication, and swaps the inner Arc. A removed node is not retargeted to a same-named replacement. Native/Clash selection writes serialize with this publication, so they cannot acknowledge a write to an already-replaced manager. Readers see either complete manager.
Shared reload normalization in control/reload/fingerprint.rs preserves each configured group's UUID and created_at when its name is unchanged. Insertion, deletion and reordering do not affect this identity. A rename creates a new group rather than transferring the old identity; a programmatic rename carrying the old UUID receives a fresh one. This applies to SIGHUP, explicit activation and provider/runtime reloads, including no-op comparison, with or without native observation.
The src/group/ facade and its internals are split by responsibility:
| Module | Responsibility |
|---|---|
mod.rs | GroupManager types, shared handle, and selection-plan entry points |
resolver.rs | Nested-group expansion, member/leaf introspection, cycle cutting, and Selector-choice migration |
filter.rs | Network- and family-specific liveness filtering |
policy.rs | Selector, URLTest, LoadBalance, and Fallback picks and latency ranking |
score.rs, score/selection.rs | Shared Score state, public contracts and target-aware group selection |
score/ranking.rs, score/budget.rs | Ordinary ranking; retained business currency and in-flight accounting |
score/evidence.rs, score/feedback.rs | Scoped outcome/provenance rules and exact-once observation ownership |
score/comparison/, score/verification.rs | Bounded observation storage, paired response evidence, validation questions, the single optional-work dispatch walk and readonly pairwise relations; score/tests/ splits scenarios by ownership |
state.rs | URLTest/Fallback caches, Selector choices, and callbacks |
Selection follows one invariant (sing-box semantics): after resolution and liveness filtering, the dial path uses exactly the policy pick. Selector returns its effective manual choice, URLTest its current winner, LoadBalance its next member, Fallback its pin, and Score its ranked eligible leaf, including deterministic cold exploration. The only multi-candidate exception at selection time is an unmeasured top-level URLTest group; all warm URLTest and non-URLTest plans are authoritative single-leaf plans. Post-failure retry racing belongs to connection/, not here, and parallel racing is added nowhere else. If a group with no final has exactly one unique leaf and TCP liveness excludes it, that same leaf remains an authoritative last resort only when reachable through the current Selector choices: its health stays dead, but a real dial can prove recovery without leaking to another member or direct. UDP keeps normal liveness exclusion. Last-resort serving logs a rate-limited warning (60s per group); warm-up peeks stay silent.
UDP selection first excludes forwarding leaves whose canonical protocol/configuration does not support UDP, even before health evidence exists. A live TCP state cannot qualify VMess or a TCP-only node for UDP. This also removes those leaves from Score verification and from the capable-and-alive leaf witnesses used when publishing group connectivity. block remains a terminal action; a Selector's incapable choice may use its configured final, never an unchosen sibling. Target-specific refusals such as VLESS UDP/443 rejection still terminate at admission rather than being filtered into another route.
Policy semantics
| Policy | Runtime behavior |
|---|---|
| Selector | TCP and UDP resolve their separate concrete runtime member choices, then group.default, then the first declared member independently of health. Only missing/non-member choices fall through. An ineligible chosen member invokes only explicit final or the same-leaf TCP last resort; otherwise the plan is empty. Native writes choose TCP, UDP or atomic both; Clash writes both and displays TCP. PersistCallback stores each changed network's identity in the cache. |
| URLTest | Chooses the lowest halving moving average, keeps independent TCP and UDP selections, applies tolerance hysteresis, and re-evaluates lazily on dial and selection queries. A real selection change may invoke InterruptCallback. |
| LoadBalance | Round-robins eligible members in declaration order. Every group owns independent AtomicUsize cursors for TCP and UDP. Rotation never invokes InterruptCallback. |
| Fallback | Pins the first eligible member in declaration order independently for TCP and UDP. The pin stays until that member dies; recovery of an earlier member does not cause failback. |
| Score | With policy: score, chooses one health-eligible leaf using observed reliability, fresh target-aware quality and bounded validation. Historical sample volume is not a performance bonus. Selector remains the default. |
Selector validation/publication is one shared transition; callbacks run after synchronous guards are released. interrupt_connections captures the pre-transition transport owners by their actual selected group path and network, then closes them outside manager/entry locks. The response reports interruption only after confirmed completion. TCP closes its exact UUID owner; UDP retires its token/generation-bound view and waits for backend/driver/reply fences. Shared XUDP siblings and the carrier remain alive. This path is independent of optional native flow recording, not a deletion of tracking metadata.
Concurrent closes of the same captured owner share its real completion and inherit a failed retirement; they do not report Gone merely because it is already Closing or Failed. A missing or replaced owner remains Gone. Automatic interrupt callbacks only start retirement; callers that require confirmation wait on that owner's completion.
Native group observations are read-only, and icons are configured validated values. Restricted configuration PATCH belongs to the accepted source owner: parser spans, full offline admission, disk/dependency fences and reload. It does not create a second in-memory Group configuration. Its accepted revision is distinct from disk SHA-256 and is checked again before activation. An automatic-policy pin lives only in the running GroupManager, so an activation drops it; endpoint details are in the group API contract.
Score scoring and lifecycle
Score first runs the same liveness filter as every other policy. The filter's health family describes connectivity to the proxy server; the separately carried target family selects the scoring bucket. A server reached over IPv4 therefore remains eligible for an IPv6 business target, while no score can return a node already excluded as dead.
The exact key is (group, TCP/UDP, target IPv4/IPv6, normalized target, NodeId). Domains are ASCII-lowercased with one trailing dot removed and retain their port; IP targets retain the socket address. A second bounded (group, TCP/UDP, target family or no family, NodeId) aggregate supplies the prior for cold targets and receives targetless warm-up samples. Global aggregate, family aggregate, and exact-target evidence blend hierarchically until the more specific layer has enough evidence. Recursive selection carries the same target context and attributes the leaf outcome to every Score group traversed.
Target-path failures retain their numeric global/family/exact outcomes but open hard failure state only for the exact target. Typed proxy/auth/framing or shared-carrier failures, unknown pre-setup errors and targetless failures remain node-scoped. Target refusal before setup is still target-scoped. A node failure fences dependent target and probe evidence; an unrelated target failure does not invalidate configured probes. Probe failures invalidate their own current slot. Child cells inherit the node incarnation, failure time and shared-source episode identity: delayed fanout still counts each failed flow without restarting an already recovered episode. Every flow receiving one fanned-out carrier, source, packet-endpoint or shared-dial failure, including H2MUX streams of one connection, reports that failure's single episode; independently constructed failures stay separate.
Carrier provenance is attached where the proxy owns the connection, not inferred from a generic QUIC error: established end-to-end DoQ/DoH3 failures through a healthy packet proxy do not prove a proxy-carrier failure. Packet-backed DNS endpoints retain an actual adapter failure separately and project it only onto typed endpoint loss, not a buffered stream reset, EOF or malformed H3 response. Unknown pre-setup failures retain the conservative rule above. H2MUX stream resets/user errors remain flow-scoped; connection I/O and GOAWAY are node-scoped. An AnyTLS TCP SYNACK timeout on an otherwise active session is target-scoped; silent-session retirement and failed UoT service opens remain node-scoped. VMess EOF before any response-header byte is ambiguous and remains an I/O failure, while partial/malformed headers and actual carrier errors retain node provenance. XUDP END|ERROR closes a shared multi-target source, so its bound flows retain shared-source failure attribution rather than inventing one target.
Business attempt and outcome evidence decays with a 30-minute half-life. Setup failures carry extra weight in the Beta confidence bounds. Initial qualification requires four effective useful completions. Fresh targeted business RX can retain earned qualification for 60 seconds from its event time without adding completions; failure and reload revoke that lease. Below-threshold RX cannot recreate a cold or reload-only lease. Recovery from an actual failure instead uses the four-reporter rule below. Historical reliability still affects utility and promotion; overlapping global/family/exact counts are never added, and blending and hold maturity use actual decayed counts.
The first-choice performance heuristic keeps its event timestamps and bounded four-observation WeightedMean inertia. Each metric loses confidence after 60 seconds and expires at 120 seconds regardless of historical completion count. Configured probe RTT supplies a baseline; fresh comparable target response and directional goodput can override it. Probe RTT, setup and response latency are normalized within their own measurement scope, not pooled as interchangeable milliseconds. HTTP identity includes the normalized request URI and method; probe domain and health family remain distinct. This heuristic proposes candidates; it is not promotion proof.
Ordinary first choice uses serving-pool-relative utility and eligibility. An ineligible or unresolved-failure incumbent can escape immediately to another pool member. The first accepted targeted Traffic RX after setup/TX and strictly after the applicable failure/reload fence clears exploration delay, permitting only funded half-open work. Four distinct reporters in that cell's uninterrupted post-failure availability cohort clear its hard episode and grant a current 60-second qualification lease, without erasing historical failures or inventing terminal successes. Each targeted cell needs its own four credits even if another target recovered the node aggregate; reload alone cannot create failure provenance. Setup-only work, TX-only activity, probes, warm-up and cancellation alone cannot recover a node; cancellation preserves factual RX. Original cell and parent incarnations remain authoritative.
Only an ordinary escape to a different leaf suppresses optional validation. If the failed incumbent remains the ordinary winner, actionable alternatives can still receive funded trials; failure, backoff and currency checks remain authoritative.
For a trained, eligible, recovered incumbent, every eligible challenger in the evaluation set competes against that fixed incumbent. Promotion evidence is separate: paired exact-target observations, otherwise at most eight response-qualified common targets in canonical order and weighted equally, or the same current configured-probe cohort. Qualification precedes the target cap and never depends on the measured magnitude. Directional metrics must qualify across that same chosen response cohort; missing directions remain unknown. Skipped unqualified or capped matched common targets leave the comparison partial: the qualified subset may still inform promotion and readonly relations, but the member's response gap remains pending. Unmatched targets or targets without retained common time blocks remain outside this comparison cohort.
Business comparisons retain four shared 15-second blocks, expiring at block start plus 60 seconds. Configured probes carry the producer's actual interval I: their block width is max(15s, 2I), still with four retained blocks, so ordinary 30s/60s streams can accumulate four independent reporters without increasing probe frequency. Probe support expires at the earlier of the oldest supporting block's start plus four block widths and the weaker side's latest support plus max(60s, 2I). Unknown cadence retains the business timing; zero or unrepresentable cadence supplies no comparison proof. Cadence participates in probe identity alongside the configured request, and each metric still requires four distinct reporters per side. Unrelated aggregate averages, setup and warm-up are not proof; failure, reload, incarnation and probe-cohort fences remain authoritative.
Directional goodput contributes only with at least 10% gain in one known direction, no greater than 10% regression in any other known direction, response no more than 10% slower, and qualified terminal reliability at least as good. Unknown upload/download remains unknown, not zero or a win; an opposing directional gain/regression is reported as a tradeoff. Qualified observed-reliability and paired response gains still participate. The combined gain must clear the existing hold margin, capped at 0.005; support is the maximum of separately decayed global/family/exact completion counts, never their sum. Sparse exact evidence cannot reduce mature protection. Missing or expired performance alone is not a gain; holding does not certify availability or prevent budgeted trials, and there is no fixed residency timer.
ScoreFeedback is an immutable attribution/source factory; starting it creates an independent observation without changing any clone's meaning. Selected business plans instead carry ScoreAttempt: its fallible begin() returns the single admitted ScoreBusinessGuard, which owns cancellation until transferred to a ScoreReporter at the existing physical/logical I/O boundary. Setup and first response publish once at their observation time. Accepted TCP writes and successfully delivered UDP progress feed nonoverlapping, event-driven 1–10 second windows; a measured direction needs at least 64 KiB, with bidirectional flow progress and a response. Windows do not add Beta successes. Final settlement contributes at most one outcome and never adds already-published bytes again; dropping the last unfinished handle cancels. Rejection, cancellation and shutdown remove the attempt without a failure, while factual observations remain. Idle or application-limited traffic is not proof of congestion; no sampler task, payload replay or connection migration is added.
An ordinary pending attempt captured before a Score authority change may proceed unscored only when it owns no optional token. Its guard preserves the original business identity without publishing current evidence or currency. Invalid optional work still refuses; it never becomes a free challenger dial. Physical runtime admission, routing and typed refusal semantics remain unchanged.
TCP relay termination distinguishes client-side disconnects from upstream failures in both copy and splice paths. A client-side reset, abort, broken pipe, disconnected socket, unexpected EOF, socket timeout or host/network-unreachable error cancels Score neutrally without retracting accepted RX or clearing prior failure/backoff; the relay still returns its original I/O error and connection-error telemetry remains unchanged. The same errors on the upstream side, protocol errors and unclassified local failures retain their existing failure treatment. A low log severity alone never makes an error neutral.
Business-RX publication is separate from throughput windows: the first eligible positive RX publishes immediately, then at most once per second per reporter. Ordinary TX-only callbacks do not flush a suppressed receive, and no timer turns silence into progress. A shared source's first confirmed positive send can reconcile a reply already delivered after its serialized send began, using the actual RX timestamp rather than acknowledgement time; queued, failed or retired sends cannot authorize it. A post-failure receive inside the throttle interval can otherwise wait for the next publishable RX or terminal settlement, not necessarily one wall-clock second. Live publications update availability and the scoped recovery quorum without adding Beta outcomes.
The shared-source witness describes a transport-attempt interval, not UDP request/reply correlation: a matching peer can send delayed data during that interval. It is captured after the core source gate, before backend queue admission, and becomes usable only after successful send acceptance; it does not prove which datagram caused the reply.
Optional-work accounting begins when node-specific work starts, before DNS or physical-admission waits; this is distinct from the business reporter boundary. Admission-scoped TCP dialing starts its reporter at the first admitted physical attempt or immediately before a logical open on a reused session or QUIC connection, not while waiting for cold physical admission. The callback is one-shot; completed paths without either boundary retain the completion fallback.
An unstarted scope's timeout is a local capacity refusal only while a physical-admission acquisition is still pending. DNS resolution before admission remains an ordinary timeout, allowing the existing eligible retry path. Classification inspects the scope before cancelling its future; cancellation removes only that acquisition's pending registration.
Ordinary and post-race TCP ready/bare refills retain setup-quality evidence, but their successes and failures never alter real-flow failure streaks or exploration backoff. UDP drivers classify their terminal result before their health callback can synchronously retire the endpoint, so that callback cannot turn an error into neutral cancellation or success. Intentional retirement remains neutral without a reply and successful after a reply; process shutdown and packet-local congestion stay neutral, and reply-idle expiry is a timeout only when no reply was received.
On live endpoints, a proven QUIC path stall retires the shared carrier and is node-scoped for Score, even when the immediate error is packet-local congestion or idle-after-reply. Rejection, intentional retirement and local reply-delivery cancellation retain their prior precedence; Alive still classifies the original I/O error independently.
Traffic reporters cover transparent TCP/UDP, supported DNS exchanges and UI downloads. DNS attribution follows the carrier actually used, including TCP retry after UDP truncation. HealthProbe records configured measurement quality in global aggregate probe slots without settling business reliability or resetting real-flow streaks. Warmup records setup quality only. On-demand delay measurements retain their API/Alive latency history but do not create inert Score exchange reporters or convert failed arbitrary targets into real dial failures; their actual session preparation may still report warm-up quality.
Each retained (group, TCP/UDP, target family) budget freezes its initial member count n, cold allowance B (all members up to four, otherwise ceil(sqrt(n)) + 1) and earns currency from original business: each original start (their count is N) adds 1/q, with q = 16, or q = 8 while the group's recent original business on that network (five-minute half-life) implies at least two flows per second and the scope has at least three recent optional-trial outcomes (ten-minute half-life; decayed weight above 2.5) of which at least 80% succeed. Evidence needs a fixed number of trials, so busy groups whose trials pay off fund it sooner; quiet groups keep the smaller share. Target refusals are not trial outcomes here, and a reload forgets these outcomes. While the ordinary selection is observed usable and every scope behind a read has recent outcomes under 20% success, optional trials pause: with two or more recent failures they wait for those failures to decay, and with one they go out one at a time. A paused scope refuses trials like exhausted credit: each candidate it refuses counts in budgetBlocked and reads as a budget wait, while a member whose trial is already in flight still reads inFlight. Without a usable selection exploration continues, since it is how Score escapes; recovery continuations are unaffected. With earned currency E, spent optional trials V and outstanding reservations R obey V + R <= B + floor(E); unspent earned credit is capped at eight. N includes an original business assigned to a trial, but retries, clones, nested recounting, reads, elapsed time, evidence expiry and new targets cannot mint currency. Each scope traversed by the original plan counts that original once; continuations cannot earn again in another group, network or target family. The root business count deduplicates the whole nested path. Retained scopes preserve currency across membership changes and reload; deleting the group or restarting ends that lifetime.
Already-started trials still settle historical outcome and cost counters after reload. Their outcomes affect current trial yield only while their (group, node) membership remains configured; removed members cannot pause or accelerate replacement members' funding. Retained members may contribute across authority changes.
Reservations are spent only when node-specific work begins. Releasing or invalidating the last unbegun reservation refunds it; begun cancellation never refunds. Reload retains started in-flight tracking until settlement or its existing expiry. UI and subscription redirects, DNS retries and TCP replacements retain the original business identity and cannot mint trials or relabel an existing trial as free recovery. There is one optional-work path: an authorized targeted rank walks the evaluated challengers in dispatch order and reserves one credit for the first member the ledger admits; a budget refusal ends the walk, and an in-flight refusal tries the next member. Each member has at most one unanswered trial for its open question on a target; qualification counts across targets. Missing traffic stays inconclusive. Real failures retain 5-minute-to-6-hour backoff unless genuine scoped RX opens recovery; trials never acquire committed-incumbent protection.
A member's open question is recovery after a failure streak; availability without recent usability; a pair when ordinary selection could compare it but the selection's original pair with it lacks a fresh, non-partial business response of this scope (exact target, or common targets for an aggregate read) newer than the selection's latest degradation; otherwise qualification. Configured probes never settle a pair question. Dispatch order puts untrained members first, then the most progress toward the open question (for a pair, the challenger reporters already sharing live blocks with the reference), then the least recently selected member; the configured-probe hint breaks ties among never-selected members. Recency advances only when attributed work is admitted at begin, so unbegun, refused or unscored plans leave the order unchanged. Except for recovery, a member is dispatched only while its completions plus its unfinished work stay below the selection's completions: first choice follows evidence, so training challengers first would displace the probe-preferred selection, and cold exploration therefore starts once the selection has a completion. The readonly question, action and wait come from the first member of the same order; the wait adds only the ledger's current budget or in-flight refusal. Stored node provenance names failure time separately from invalidation; borrowed cell stamps share current-parent and effective-fence checks without adding a cache.
Carrier-pressure hints are separate from business results and performance scores. Score-bound runtimes observe sustained TCP RTT/retransmission or QUIC RTT/loss pressure on the honk-to-proxy carrier; the existing five-second heartbeat admits fresh episodes. A hint reopens comparison freshness only within the existing validation cadence, eligibility and backoff. It never changes utility, qualification, failure counts, Alive state or existing flows. Actual business observations must justify promotion.
Sampling uses active 1–10 second intervals with fresh ACK progress and at least 4 KiB payload-byte progress or 32 QUIC DATAGRAM frames. RTT trains on four active samples, then requires three consecutive intervals at least 1.5 times the baseline and 20 ms higher. Send-loss pressure requires meaningful outgoing payload, at least 32 transmissions, three retransmissions/lost packets, positive retransmitted/lost bytes and a 5% count ratio for three intervals. This is a pressure heuristic, not an application-loss estimate or a coherent packet cohort. Unknown/idle intervals prove neither pressure nor recovery. Two qualified normal intervals rearm an episode; each runtime/carrier family publishes at most once per 30 seconds. Hints expire after 60 seconds without being refreshed by reads.
The health-filter family is not necessarily the actual socket family chosen by address racing. Producers keep actual-family slots, while Score uses the latest fresh event as a node-owner comparison question, not a target/family performance penalty. Only carrier-backed networks receive it; Shadowsocks TCP pressure does not affect native Shadowsocks UDP. Warm configured probes can contribute carrier activity without becoming business evidence. Ordinary UDP loss, raw SOCKS/direct splice, reverse-path loss and proxy-to-target loss remain unknown. Reload fences prior episodes and replaced owners. No carrier or target identities are exported.
Score state is initialized with the manager, memory-only, process-local, and non-configurable; evidence entries are populated on demand. Exact cells use a 4,096-entry LRU and aggregate cells use a separate 4,096-entry LRU. Exact-target evidence is transport-quality evidence, not a semantic-unlock capability result; existing routing or geosite rules can select a dedicated service-specific Score group when that coarse cohort is desired. A successful in-process reload shares the same state Arc, publishes the new valid (group, member) set, and prunes removed cells; late feedback for deleted membership is ignored. Process restart clears everything. Score cells and scorer-only target data are never logged, persisted, or returned by Clash APIs; the existing /connections destination metadata remains unchanged.
The separate comparison store is capped at 512 cells and 1 MiB of logically accounted allocation, including store/vector capacity and owned key capacity. That bound is not process RSS: allocator overhead, other Score state and the rest of the process are outside it. Eviction or rejection leaves comparison evidence unknown rather than falling back to unrelated averages.
Neutral cells remain in the bounded LRUs rather than being eagerly deleted while another reporter may still own their incarnation. Reload clears configured-probe baselines and rejects old-generation probe observations; admitted business flows may continue reporting for surviving membership and matching cell incarnations. Eviction never permits an old reporter to recreate a cell.
Conditional verification
The default objective is response quality with observed business-availability protection; directional goodput participates in ordinary promotion only when comparable evidence exists. Forwarding remains immediate. provisional and observedUsable describe the selected path's business evidence. Readonly challengers separately report fresh pairwise response relations between the selected member and evaluated challengers; there is no group-wide comparison claim. A 10% metric tolerance represents practical equivalence, not an error probability or a proof of global optimality.
A relation does not use the ordinary switching margin or its completion maturity. It compares actual response values with an inclusive symmetric 10% band (high - low <= 0.1 × low), including zero and sub-millisecond responses, allowing only floating-point roundoff at the boundary: inside the band the pair is equivalent, otherwise the lower response is faster. Relations read the selection's original pairs, the same pairs ordinary promotion reads, and only their fresh qualified response metric; reliability, availability, directional goodput and holding stay with ordinary selection. Each relation names its basis and expires with that response metric.
Pairs need no joint alignment: each challenger's original pair with the selection is judged on its own shared blocks, and different challengers' histories create no validation demand against each other. No extra probes, copied requests or arbitrary-load completion guarantee are introduced.
The same evaluator identifies missing availability and response evidence, chooses the next budgeted real-flow validation, and produces readonly API summaries. An exact-target view never labels common-target or probe evidence as targetResponse. Successful HEAD/QUIC probes can support only their current configured measurement cohort, never target usability or bulk capacity. nextBusinessFlow names future real-work demand, not a reservation or dispatched I/O; question and waitReason distinguish missing evidence from budget, in-flight, comparable-traffic and backoff waits. Missing transfer evidence creates no question or work: directional goodput waits for real offered load, and no background bandwidth scan or copied user request is created. Aggregate reads inspect retained IPv4/IPv6 budget waits without creating or spending a scope.
Business comparison support retains its 60-second bound; configured-probe comparison follows the cadence-aware retention and latest-support bounds above. Failed business outcomes and committed reload revoke recent usability without erasing long-term reliability. API reads derive current validity without advancing clocks, trials or counters. Authorized Apply records provisional, usable and validation selections in existing bounded history. See API semantics.
Each Score group/network owns one committed serving pool, shared by every parent route and target. Ordinary first choice, failure escape, recovery, promotion and optional trials all select inside it; the winner gets no extra slot. Bootstrap puts the ordinary first-choice winner into a ranked slot, and refresh ordering prioritizes ordinary eligibility before utility and probe ties. Capacity k still follows offered business: original starts decay with a five-minute half-life and are projected to the 30-minute evidence half-life. Half the optional starts earned at q = 16 fund four effective completions per challenger, giving k = 1 + ⌊D/128⌋ for projected demand D, with a minimum of 3 and a ceiling of min(25, ⌈√n⌉ + 1) for views with more than four unique candidates (8 for 45). Smaller groups serve only available members. A view larger than k reserves one of those k slots for ten-minute rotation. A rotation member selected for ordinary service trades places with a ranked member, keeping the same membership and giving the displaced member a fresh rotation interval; expiration alone cannot evict that serving winner. The cap concentrates work without guaranteeing enough comparable traffic or globally optimal choices.
Only authorized Apply initializes or refreshes the pool. Periodic reranking is every five minutes; growth waits for that refresh and demand-driven shrinkage requires k to fall by at least two. Three node-wide failures or authoritative health exclusion may replace a member earlier without increasing capacity. Mere filtered/retry absence and exact-target failure do not themselves remove a shared member. Without a replacement, failed members retain access to funded recovery. Reload resets membership and capacity while preserving decayed demand; readonly/Peek uses committed membership, or a temporary bounded projection before initialization.
Nested slots retain their immediate subgroup's name, not its configurable UUID. When that subgroup emits a different eligible representative, Apply replaces its old representative in the same slot; absent subgroups and direct members are not mistaken for changed representatives. A Selector Applies only its chosen subgroup, even when the child's committed Peek has no visible member. Selector ancestors do not create another Score pool or wallet; independently configured Score ancestors still own their own budgets.
A full child resolution that yields no candidate can withdraw that child's retained slot, including health/capability filtering and an unavailable pinned Selector choice; retry-only omissions cannot. Before choosing an IPv6 final, the internal IPv4 preflight derives prospective membership through the same selection path without committing pools, reserving work or updating counters. This preflight requires current authority; it does not change public Peek semantics.
Every pending policy-selected attempt rechecks its captured ordinary-pool obligations at begin, under the evidence lock and before node-specific I/O. Departure or an uninitialized pool refuses the attempt with typed cancellation and refunds unbegun reservations; neither an ordinary stale plan nor a retry can downgrade that refusal into free work. Already-begun attempts and relays retain their lifetime and settlement rules. A new retry, although it retains its original business identity, must pass admission again. Active connections may therefore span retired pool members; k is not a limit on the historical union of live connection leaves.
Native pins that make an automatic group act as a Selector impose no ordinary Score-pool obligation for that owner. Attribution and any nested ordinary Score owners' pool obligations remain intact. Candidate observations retain member-specific health exclusions even when subgroup aliases share a leaf; Score utility and eligibility reasons update only surviving rows.
An explicit final bypasses only its own group's ordinary pool; any ordinary Score ancestor still constrains its selected leaf. The sole-TCP-leaf recovery path enters the same pool, without permitting an unchosen Selector sibling. Independent health probes and warm-up observations are not policy-selected business and keep their existing scope. Probe baselines remain available; comparison-cell admission follows serving membership, with the existing exceptions for four recent Apply winners and initial collection. Original currency, identity and TTL rules are unchanged. Membership checks are bounded by k; pool reranking sorts candidates and rotation scans them once.
A recent failure exclusion removes an ineligible member from pending validation until its failure is older than the performance evidence age. Exclusion does not measure the member as equivalent or nonregressing, and decaying layer weights can make it eligible again earlier. Response degradation, which reopens comparison freshness, compares a setup or response sample only with the same exact target's history; aggregate cells mix targets of unrelated latency, so aggregate comparisons reopen on carrier pressure rather than cross-target differences.
Recent usability requires four distinct Traffic reporters in the selected cell's uninterrupted availability cohort. Each must report targeted positive RX after setup and positive TX; open flows can qualify before termination. A reporter and its clones count once per cell/cohort, while later eligible RX refreshes the cohort's event time. This is neither four currently open flows nor statistical independence, and overlapping global/family/exact counts are never added. Probes, warm-up, TX-only work and RX preceding setup/TX cannot earn availability.
Applicable failure, committed reload or a gap of at least 60 seconds resets the cohort. One returning reporter cannot revive four old credits; surviving reporters may each contribute new RX under the failure/reload and incarnation fences. Terminal settlement shares live deduplication and may flush eligible RX at its actual event time, never cleanup time or across expired support. Neutral cancellation preserves factual RX without a success. Availability alone does not grant cold qualification; only a genuine failure cohort's four fresh reporters can earn the recovery lease above. Qualification and recovery gaps still receive bounded trials independently of comparison support.
An authorized applied multi-candidate rank increments one final reason: coldExplore, periodicExplore, incumbentIneligible, freshFailureBypass, insufficientEvidenceHeld, directionalTradeoffHeld, incumbentHeld, reliabilityWinner or performanceWinner. insufficientEvidenceHeld means no challenger earned promotion and the ordinary utility winner lacked a qualified shared performance comparison; it does not claim that reliability history is absent. directionalTradeoffHeld identifies opposing known directional changes when no challenger earned promotion. ordinarySwitch separately counts committed normal A→B identity changes within the same (group, network, family, target) history; first choices and trials do not count. switchFlap is its subset returning to the prior winner within eight ordinary selections. deadFiltered, failStreakExcluded and exploreBackedOff count affected candidates, not failed flows. History remains a 4,096-entry LRU; unknown/evicted history cannot prove a switch. Peek, proxy/stat reads, singleton bypass and last-resort selection remain neutral for these reason counters. Authenticated /stats.score exports fixed group/network reason, verification and budget fields plus evidence-cache totals, without scorer-private node/target/cell identities. Trial outcome and elapsed/setup cost counters describe observed trial work, not counterfactual extra failures or causal overhead.
Clash represents a Score group as type: "url_test", reports the current aggregate TCP winner in now, and rejects PUT /proxies/{name}.
URLTest ranking and hysteresis
Latency uses a halving moving average:
next = (previous + sample) / 2
The first sample initializes the average. This is dae min_moving_avg behavior: recent changes matter quickly without making one jitter sample authoritative.
SelectionNetwork::Tcp and SelectionNetwork::Udp retain separate winners. TCP uses the TCP probe average, or the (member tag, check_url) average when the group has a custom target. UDP first uses DataUdp, then DnsUdp; if no eligible candidate has real UDP ranking evidence in either domain's retained moving average for the selected address family, it mirrors the TCP selection. Synthetic dial-failure samples alone do not disable this mirror; evicting real samples from the history ring does not erase retained ranking evidence. This gives the effective fallback order DataUdp → DnsUdp → TCP.
The effective tolerance is max(configured tolerance, 1 ms) (group.tolerance.max(1)). The incumbent stays selected while:
best latency + tolerance >= incumbent current measured latency
The incumbent baseline is read again at selection time, not retained from the moment it won. A degraded incumbent can therefore be replaced; this matches sing-box Select() behavior. Hysteresis is skipped for an incumbent carrying failure strikes: a just-failed incumbent is replaced immediately.
Probe failures update only liveness and cooldown; they never create synthetic latency samples or ranking strikes. Two consecutive real dial failures append a display-excluded synthetic 10-second placeholder plus one failure strike. A lone transient failure rescued by the retry race leaves no selection state. Only a real dial success resets the streak, so a probe-alive but dial-dead node still accumulates failures. Real history and moving average are retained, but a candidate with pending dial-failure strikes ranks below every non-demoted candidate. Strikes clear only after max(strikes, 2) consecutive real successes. This flap guard stops a fast-but-flaky node from reclaiming first place with one lucky probe.
Real traffic also feeds ranking directly (TCP only). Each node keeps a self-referential EMA (α=1/8, after 3 warmup dials) of fresh dial latencies; pool-ready hits are excluded because they perform no network round trip. Three consecutive dials slower than max(min(2×EMA, EMA+500 ms), 250 ms) (max(min(2×ema, ema+500ms), 250ms) in report_dial_latency) append one failure strike and fire an emergency probe; the 250 ms floor keeps a fast incumbent's normal load jitter (e.g. 60→120 ms) from tripping the detector. The probe moving average is never touched, and a false positive (a shifted target mix rather than node decay) self-heals when the emergency probe succeeds and consecutive probe successes clear the strike. Gradual drift stays owned by the probe cycle; UDP degradation keeps the probe-cycle plus DataUdp traffic-threshold path.
An authoritative URLTest setup failure retains its existing one retry round over the latency-ordered top three. Its fresh retry deadline never creates another original Score business, even when the primary round finishes after its deadline. A Score-owned setup instead permits one sequential different leaf, excluding the failed NodeId before ranking even when it remains the ordinary winner. Both Score attempts share one absolute establishment deadline and physical admission budget. Canonical resolution preserves Selector choices and only final-group edges actually traversed by the primary; display names never authorize fallback. Local typed refusal, pre-admission capacity exhaustion, cancellation and generation shutdown are terminal. No retry occurs after application payload is written.
A group check_url creates independent TCP-only liveness and latency state keyed by (member tag, check_url). A failure removes that member only from groups using that target. Selector groups ignore check_url and emit a warning. URLTest probing sleeps after idle_timeout; an unset timeout uses the 30-minute health-layer default, and the next real selection wakes probes immediately.
Nested groups and member identity
Group.groups names sub-groups. Each sub-group contributes exactly one candidate: the leaf selected by that sub-group's own policy for the current network and address family. The parent ranks or pins that candidate as one member rather than merging every descendant into its policy.
Resolution is bounded by MAX_GROUP_DEPTH = 8 and a per-walk visited set. Construction also runs DFS over group edges and cuts every cycle-closing edge with a warning. These checks prevent a malformed graph from hanging selection or introspection.
Membership and explicit final edges share the bounded recursive resolver. A subgroup with no eligible policy pick follows only its configured final; the parent still sees that subgroup as the selected member. Final chains preserve selection chains and Score attribution. Ordinary member/display lists exclude final edges, while health registration, warm discovery, Score membership and datapath connectivity use the reachable-final closure. A leaf health transition recomputes every affected ancestor's alive slot, not only its first mapped group. IPv6 health-family retries prefer a usable ordinary IPv4 proxy path before a final, without changing the business target family. Missing or cyclic finals remain refusals; no transport error or terminal packet rejection is retried here.
Selector captures a concrete node or sub-group member before candidate expansion and health filtering. Name-based defaults and Clash writes bind the first matching declared member; native IDs can select a particular direct member even when display tags collide. Serving resolves only the chosen subgroup with Apply and then applies the parent health gate; it neither peeks at unchosen siblings nor requires a committed child Peek to succeed before Apply. For both TCP and UDP, an unavailable chosen path can continue only through its configured final. Candidates retain their originating subgroup reference rather than rediscovering it from a display tag. A chosen automatic sub-group may select another leaf within its own membership. The sole TCP leaf's last-resort walk also respects every nested Selector choice, while explicit delay tests may still inspect all members for recovery.
Display and API output retain member tags even when the physical dial reaches a deeper leaf; serving selection retains concrete node or subgroup identity separately:
| API | Output |
|---|---|
node_names_in_group | Direct node tags plus sub-group tags |
leaf_node_names_in_group | Deduplicated real leaf nodes reachable below the group |
delay_test_members | One (member tag, current leaf) pair ((tag, leaf)) per effective member |
selection_chain | Current chain from group through selected sub-groups to the leaf |
Custom-URL probes resolve delay_test_members again on every cycle. A sub-group is probed through its current pick, but the result is recorded under the sub-group tag. The parent therefore treats the sub-group as one stable member, matching sing-box RealTag semantics.
Cold URLTest UDP preparation
Only a top-level URLTest plan with no usable measurement may prepare several UDP transports. Candidate starts use absolute offsets 0 ms, 30 ms, 80 ms, then one every 80 ms; at most three preparations are in flight. Absolute scheduling prevents an earlier slow attempt from shifting all later starts.
The first successful candidate that is still eligible wins. On a winner or deadline, honk aborts and drains every started loser before the scheduler returns; it then rechecks winner eligibility and commits protocol state before endpoint publication or the first application send. Only an observed preparation Err affects traffic health. Never-started work, cancellation, an ineligible successful result, and successfully drained losers are neutral; a completed error discovered while draining is still an observed error and counts. AnyTLS uses caller-owned provisional pool slots so losers never publish sessions. QUIC protocols build detached clients and publish only the finalized winner; losing clients are closed with their speculative work.
Authoritative single-node and cold URLTest plans share one absolute transport-preparation deadline of max(10s, 4 × connect_timeout). It begins immediately before preparation and covers proxy-host resolution, physical-dial admission, protocol/control negotiation, stagger and full-capacity waits, plus the finalized winner's commit. Expiry starts no new candidate. Earlier sniffing/routing and later reply-socket creation, endpoint-driver readiness, and packet sends remain outside this deadline and retain their own lifecycle or I/O bounds.
Health state and probes
AliveDialerSet keys node health, registrations, histories, emergency triggers, and latency collections by the node's NodeId UUID (Uuid). Display names are metadata for logs and probe lookup, not identity. Every node has six independent states: three domains across IPv4 and IPv6.
| Failure source | Tcp | DnsUdp | DataUdp |
|---|---|---|---|
| Periodic probe | 3 | 3 | 3 |
| Real traffic | 10 | 3 | 50 |
Go dae's TCP=1 let transient probe loss eject URLTest incumbents before selection, bypassing tolerance hysteresis.
Probe and traffic failures have separate counters. Probe failures apply exponential cooldown from 5 seconds to 300 seconds. A separate min(5s, check_interval) recovery scheduler considers only dead domain/family states whose cooldown is due; deep-backoff states continue at the 300-second cadence rather than stopping permanently.
A dead state normally needs two consecutive probe successes to recover. notify_network_change clears stale cooldowns after a relevant link, address, or route change, primes dead states, and triggers probes so one fresh success can verify recovery. Newly registered nodes receive a 60-second grace period during which non-forced failures are recorded but do not count toward death. Probe history retains 100 entries per node, domain, and address family.
| Probe path | Behavior |
|---|---|
| TCP | Sends the configured HTTP method to tcp_check_url through the node, or performs a raw TCP connect when no HTTP probe applies. ProxyHttpProber delegates HTTP execution to the shared outbound measurement path below and reuses a runtime only when NodeRuntime::is_warm_or_stateless_for(WarmRequirement::Session); otherwise it closes a guarded cold runtime after the probe. Only a successful warm-path RTT enters the matching TCP family state; setup and target-exchange failures update liveness/cooldown without contributing latency or ranking strikes. |
| UDP health | Sends one minimal DNS query to the first udp_check_dns target through the node's packet path. It independently checks NodeRuntime::is_warm_or_stateless_for(WarmRequirement::Udp) and closes a guarded cold runtime when that requirement is not warm. Success records the measured RTT and marks both DnsUdp and DataUdp alive; failure adds one probe failure to each UDP domain unless the independent Score QUIC handshake succeeded in the same cycle, in which case only DnsUdp fails and DataUdp remains alive. It never changes TCP state. |
| Score QUIC evidence | Every periodic UDP cycle separately performs a real TLS-in-QUIC handshake with ALPN h3 through each Score leaf, targeting the first configured global.tcp_check_url only if it is valid HTTPS. Its measured duration supplies separate DataUdp probe quality, never business successes or invented throughput. A successful handshake may revive DataUdp when DNS probing failed. Missing/non-HTTPS URLs disable this probe; non-Score leaves create neither this handshake nor Score cells. |
| Per-group URL | Probes the dynamically resolved (member tag, current leaf) pairs with the same throwaway warm-path timing as the global TCP probe. State is TCP-only, dies after three consecutive failures, and uses the same cooldown and two-success recovery. sync_group_check_urls replaces the active group/URL registry on reload. |
has_udp_state(node) distinguishes a node with no UDP observations from one explicitly observed dead. For an ordinary endpoint, terminal send/receive errors and reply-idle expiry without any reply report DataUdp failure after the driver captures its terminal per-flow Score outcome. For source-shared protocols, the source owner reports transport health while every bound endpoint retains and settles its own Score reporter; matched replies belong to their endpoint, while foreign replies have no flow Score owner. A source terminal event retires its endpoints and fans the terminal outcome to those flows.
Typed policy, size, and PacketRejection::Capacity refusals are terminal for the candidate but health- and Score-neutral; CLI callers receive the capacity error rather than NotApplicable. Packet-local congestion, idle expiry after a reply, intentional retirement, node-death cancellation, and shutdown are also health-neutral. An alive-to-dead transition invokes the control-plane callback with (NodeId, name), purging pooled connections and UDP endpoints. Skip UDP-domain death while its sibling UDP domain is explicitly alive, so a blocked :53 probe does not purge working flows.
The last real TCP delay sample per node is written to the state db every 60 seconds and restored at startup only when it is at most 24 hours old. Liveness is never restored from the cache. Synthetic 10-second placeholders are flagged, excluded from display history and the moving average, and never persisted as the last real sample; selection demotion lives on the failure-strike counters, not on the placeholder.
src/alive/APIs includeregister_node,notify_check_*,report_*_traffic, andrecord_dial_failure. Group tables and(member tag, check_url)state remain name-keyed: groups have no NodeId; members may be sub-groups (sing-box RealTag).mod.rs: state, thresholds, registries, eBPF connectivity-push callback.probe.rs: HTTP/raw-connectprobe_nodeand DNS-through-dial_udp_transportprobe_node_udpmeasurements.runner.rs: health admission/cancellation/drain, concurrent cycles, and the periodic/recovery/emergency loop.collection.rs:DialerCollectionlatencies, moving averages, and dial-failure tracking.latencies.rs: O(1), cap-10 ring; measurementSystemTimegives real Clash history times.last_real_sample()excludes synthetic entries, preventing bogus dashboard 10000ms.src/urltest.rsowns shared HTTP request construction and measurement; URL interpretation delegates to the canonicalhonk-config::checkdecoders. Core and generation URLTest also share explicit cold-session preparation, retaining fallible node admission before resources or feedback; standalone tool calls retain handler-owned setup and their outer CLI deadline. Requests use credential-free authority with non-default ports, omit fragments, and preserve raw path/query including dot segments; query-only URLs use/?query. HTTPS verifies certificates and negotiatesh2,http/1.1, with server push disabled. A HEAD warm-up precedes the configured method (HEAD for delay tests), and both final responses must have valid decoded 200–499 statuses. HTTP/1 informational heads are consumed before the final response, except unsupported protocol switching; their cumulative size per round is capped at 16 KiB. HTTP/2 response header lists use the same size cap. Locally detected HTTP/2 protocol errors terminate the disposable probe connection on the first rejection; a later remote reset cannot turn rejected response headers into a warm-sample success. Cancelled-stream records use twice the per-round timeout, covering both request budgets so valid late warm-stream frames remain ignorable. At most two streams are retained, and completion or cancellation drops the connection without waiting for expiry. Every protocol reports the second request's warm-path RTT, excluding proxy dial, target TLS and session preparation. A second-round transport failure may return the validated first sample; HTTP/1 requires no partial response, and HTTP/2 accepts graceful GOAWAY or remoteREFUSED_STREAM. Invalid/partial heads still fail. Cold reusable probes close their guarded runtime, and HTTP/2 drivers stop on completion/cancellation. Preparation, dial, TLS, HTTP/2 startup and each request have separate phase budgets; core retains its connection timeout. Empty delay URLs usehttps://www.gstatic.com/generate_204.aliveowns health scheduling; only actual traffic feeds dial-failure strikes. Group delay concurrency remains capped at 10. The API's warm-up fallback is not published as Score evidence for the configured method: only a successfully measured second round updates that method's quality cohort.
UDP candidate eligibility
filter_alive_candidates decides UDP selection per node and address family:
DataUdpalive orDnsUdpalive: selectable.- Both UDP domains explicitly dead: excluded, even if TCP is alive.
- No UDP state has ever been recorded: inherit TCP liveness.
This keeps a TCP-healthy but UDP-broken node from attracting packet flows without penalizing deployments that have not enabled UDP probing yet.
eBPF connectivity publication
The eBPF alive slot belongs to a group, not to one node. For every domain and address family, publication uses the OR of all reachable leaf-member states, plus the existing TCP admission exception for exactly one unique leaf and no final. A callback caused by one node transition recomputes that group value; it never writes the transitioning node's value directly. Admission does not override userspace selection: a chosen empty Selector path still refuses traffic even when this slot remains alive.
Reload first sets every slot needed by the old or new group layout to alive, making the transition fail-open. After the new routing generation is published, honk writes the exact new group snapshot. Reordered groups therefore cannot inherit stale ordinal state; if exact publication fails partway, unfilled transition slots remain fail-open rather than falsely killing a group.
Warm-up and ownership
Warm-up has three independent mechanisms:
| Mechanism | Candidate and lifetime | Retained resource | Bounds |
|---|---|---|---|
| Startup preconnect | One startup-only pass; current group picks first, then config order. Only bare-TCP-poolable proxy nodes qualify. | One bare server TCP connection deposited in the pool | 'auto' selects at most 8 nodes; 0 disables it. It owns no policy-retention bit. |
| Selector pin | Tracks every Selector's TCP choice, including an unhealthy explicit choice; shared leaves are UUID-deduplicated. | The reusable session selected by the TCP path (AnyTLS or VLESS H2/shared Mux.Cool), one QUIC client/connection, or otherwise one bare server TCP | Effective-choice changes wake immediately; a 10-second pass repairs lost, consumed, or expired state. |
| UDP warm set | Opt-in; re-ranks each group's top min(N, 3) reusable UDP leaves for each address family on every pass, then UUID-deduplicates globally. | The reusable state selected by the UDP path, including a VLESS H2/shared/separate Mux.Cool pool, or a QUIC client | At most 4 warm attempts run concurrently; the retained process set is re-ranked and capped at 4 × N. |
Selector and UDP ownership are independent bits on reusable node runtimes. WarmRequirement::Session follows the TCP path and WarmRequirement::Udp follows the UDP path, so a VLESS UDP-only pool does not change direct-TCP warming or bare-TCP eligibility. Removing one owner leaves a shared pool retained for the other; only the final applicable owner release drains future reuse. Active flows are not cut. Startup preconnect remains only a pool seed.
On reload, unchanged node configurations transfer their existing NodeRuntime, including AnyTLS, VLESS pools/source key, and QUIC state. Changed configuration gets a fresh runtime. The existing outbound maintenance pass reaps unretained idle VLESS carriers together with its other idle resources; no new protocol timer is created.
Native probes capture member-to-leaf associations and generation owners without advancing policy state. Real raw-TCP, HTTP and TCP/UDP DNS measurements retain transport/purpose/family/warmth; obsolete or cancelled results do not publish fresh health. Probes, geodata and shared downloads use configured targets without an address or port allowlist. Whoever can write the configuration or control subscription content decides those targets; provider content is trusted configuration. Probe resolution remains pinned. See bounded probes.
Dial admission budget
max_concurrent_dials defaults to 64 and creates a generation-local semaphore for physical proxied connects and protocol handshakes. The configured value is clamped to the immutable process-wide descriptor gate computed at startup. Reload may change the replacement generation's local limit, but overlapping old and new generations still share that same process gate.
Ready-pool hits and logical streams opened on an already warm generation transport are exempt. block never dials; DirectHandler::dial goes through admit_physical_dial like other physical connects. Datapath-offloaded direct flows and direct UI downloads through the marked HTTP client do not use this handler's gate. A bare-TCP pool hit still runs its protocol handshake and therefore remains admitted by the dial budget.
Each VLESS physical carrier also takes a permit from the startup-sized process carrier gate shared by reload generations and DNS forks. The permit remains held through provisional, active, draining, and idle carrier I/O until task teardown. This global gate is authoritative; Mux.Cool does not impose a second per-node two-carrier limit.
Related docs
Source: honk upstream, licensed under GPL-3.0-only.