diff --git a/discv5/discv5-theory.md b/discv5/discv5-theory.md index 1b33fca4..76b16239 100644 --- a/discv5/discv5-theory.md +++ b/discv5/discv5-theory.md @@ -1,44 +1,56 @@ -# Node Discovery Protocol v5 - Theory +# Node Discovery Protocol v5 (Discv5) - Theory -**Protocol version v5.1** +**Protocol version: draft** This document explains the algorithms and data structures used by the protocol. +Discv5 is a service discovery protocol built on top of the node discovery DHT. +It allows nodes participating in a given service to advertise themselves to the network and allows other nodes to discover these advertisements. +The protocol uses pseudo-random advertisement placement and structured lookup over the DHT. +The protocol is also described in the [DiscNG research paper](ttps://ieeexplore.ieee.org/document/10629017). + + +## Services and Roles +Every node may perform all of the following roles at the same time: + +- **Advertiser:** a node participating in a service and trying to make itself discoverable. +- **Discoverer:** a node looking for peers participating in a service. +- **Registrar:** a node that accepts advertisements for arbitrary services and later returns them to discoverers. + +These roles are logical only. +The protocol does not require different classes of nodes. +Discv5 runs over the same global discovery DHT used by all nodes. +Each node can be a part of any number of services (i.e., applications) each identified by a service identifier. +The service identifier is a 32-byte hash of an application-defined service identifier. **TODO** do we need to specify the hash function? + + ## Nodes, Records and Distances -A participant in the Node Discovery Protocol is represented by a 'node record' as defined -in [EIP-778]. The node record keeps arbitrary information about the node. For the purposes -of this protocol, the node must at least provide an IP address (`"ip"` or `"ip6"` key) and -UDP port (`"udp"` key) in order to have it's record relayed in the DHT. +A participant in the Node Discovery Protocol is represented by a node record as defined in [EIP-778]. The node record keeps arbitrary information about the node. +The node must at least provide an IP address (`"ip"` or `"ip6"` key) and UDP port (`"udp"` key) in order to be reachable through discovery and to be returned to other nodes. -Node records are signed according to an 'identity scheme'. Any scheme can be used with -Node Discovery Protocol, and nodes using different schemes can communicate. +Node records are signed according to an identity scheme. +Any scheme can be used with Node Discovery Protocol, and nodes using different schemes can communicate. -The identity scheme of a node record defines how a 32-byte 'node ID' is derived from the -information contained in the record. The 'distance' between two node IDs is the bitwise -XOR of the IDs, taken as the big-endian number. +The identity scheme of a node record defines how a 32-byte 'node ID' is derived from the information contained in the record. +The distance between two node IDs is the bitwise XOR of the IDs, taken as the big-endian number. distance(n₁, n₂) = n₁ XOR n₂ -In many situations, the logarithmic distance (i.e. length of differing suffix in bits) is -used in place of the actual distance. +In many situations, the logarithmic distance is used in place of the actual distance. logdistance(n₁, n₂) = log2(distance(n₁, n₂)) ### Maintaining The Local Node Record -Participants should update their record, increase the sequence number and sign a new -version of the record whenever their information changes. This is especially important for -changes to the node's IP address and port. Implementations should determine the external -endpoint (the Internet-facing IP address and port on which the node can be reached) and +Participants should update their record, increase the sequence number and sign a new version of the record whenever their information changes. +This is especially important for changes to the node's IP address and port. +Implementations should determine the external endpoint (the Internet-facing IP address and port on which the node can be reached) and include it in their record. -If communication flows through a NAT device, the UPnP/NAT-PMP protocols or the mirrored -UDP envelope IP and port found in the [PONG] message can be used to determine the external -IP address and port. +If communication flows through a NAT device, the UPnP/NAT-PMP protocols or the mirrored UDP envelope IP and port found in the [PONG] message can be used to determine the external IP address and port. -If the endpoint cannot be determined (e.g. when the NAT doesn't support 'full-cone' -translation), implementations should omit IP address and UDP port from the record. +If the endpoint cannot be determined (e.g. when the NAT doesn't support 'full-cone' translation), implementations should omit IP address and UDP port from the record. ## Sessions @@ -240,8 +252,7 @@ strongly recommended to generate them by encoding the current outgoing message c the first 32 bits of the nonce and filling the remaining 64 bits with random data generated by a cryptographically secure random number generator. -## Node Table - +## Node and Service Tables Nodes keep information about other nodes in their neighborhood. Neighbor nodes are stored in a routing table consisting of 'k-buckets'. For each `0 ≤ i < 256`, every node keeps a k-bucket for nodes of `logdistance(self, n) == i`. The Node Discovery Protocol uses `k = @@ -254,284 +265,304 @@ the bucket already contains `k` entries, the liveness of the least recently seen the bucket, N₂, needs to be revalidated. If no reply is received from N₂ it is considered dead, removed and N₁ added to the front of the bucket. -Neighbors of very low distance are unlikely to occur in practice. Implementations may omit +Neighbors of very low distance re unlikely to occur in practice. Implementations may omit k-buckets for low distances. +For each service, a node acting as an Advertised or Discoverer for, additionally uses a service-specific table centered around the server identifier rather than around the local node ID. + +For service identifier `s`, the service table is written as + + B(s) = { b₀(s), b₁(s), ..., b₂₅₅(s) } + +where bucket `bᵢ(s)` contains nodes whose IDs share a common prefix of length `i` with `s`. + +Service tables are initialized from the local routing table and are refined opportunistically +during service advertising and lookup. +Responses from registrars include additional peers, which are used to populate service tables closer to the service. + +Implementations should treat service tables as soft state and rebuild them as needed. +Discv5 has two main operations: + +- **Advertisement placement:** advertisers place advertisements on registrars across the DHT. +- **Lookup:** discoverers query registrars to retrieve advertisements for a service. + +An advertisement is a data structure binding a service to an advertiser node. + + + + + ### Table Maintenance In Practice -Nodes are expected to keep track of their close neighbors and regularly refresh their -information. To do so, a lookup targeting the least recently refreshed bucket should be -performed at regular intervals. +Nodes are expected to keep track of their close neighbors and regularly refresh their information. +To do so, a lookup targeting the least recently refreshed bucket should be performed at regular intervals. + +Checking node liveness whenever a node is to be added to a bucket is impractical and creates a DoS vector. +Implementations should perform liveness checks asynchronously with bucket addition and occasionally verify that a random node in a random bucket is live by sending [PING]. +When the PONG response indicates that a new version of the node record is available, the liveness check should pull the new record and update it in the local table. + +If a node's liveness has been verified many times, implementations may consider occasional non-responsiveness permissible and assume the node is live. + +When responding to FINDNODE, implementations must avoid relaying any nodes whose liveness has not been verified. +This is easy to achieve by storing an additional flag per node in the table, tracking whether the node has ever successfully responded to a PING request. + +In order to keep all k-bucket positions occupied even when bucket members fail liveness checks, it is strongly recommended to maintain a 'replacement cache' alongside each bucket. +This cache holds recently-seen nodes which would fall into the corresponding bucket but cannot become a member of the bucket because it is already at capacity. Once a bucket member becomes unresponsive, a replacement can be chosen from the cache. + +## Advertisement Placement +For every service `s` a node advertises, a dedicated placement process is spawned. +The goal of this process is to maintain `Kregister` advertisements per bucket in `B(s)`. +The process starts from the furthest bucket `b0(s)` and progresses towards the closest bucket `b255(s)`. +While the number of ongoing or active registrations in bucket `i` is below `Kregister`, the advertiser picks a +random registrar from `bᵢ(s)` and starts an advertisement attempt. +The random selection must not repeatedly return the same registrar during the same placement cycle. +A single advertisement attempt is carried by [REGservice]. + +### REGservice + +[REGservice] registers the advertiser in a registrar's service queue using a ticket. The message contains: + +- `ReqID`: request identifier +- `serviceID`: the 32-byte serviceID +- `Ticket`: the previously issued ticket, or empty on the first attempt +- `ENR`: the advertiser ENR +- `Buckets`: distances from the service where the advertiser still has space in its + search table + +The registration process contains the following data exchange: + +1. The advertiser sends [REGservice] with `service = t`, `Ticket = ""`, and `ENR` set to its + current node record. +2. If the registrar can admit the advertisement immediately, it confirms registration. +3. Otherwise the registrar returns a ticket and waiting time. +4. The advertiser waits and sends [REGservice] again with the returned ticket. +5. The registrar can either admit the ticket or issue a new ticket with a new waiting time. +6. If the registrar is unreachable or rejects the request, the attempt fails and the advertiser moves to another registrar. + +An admitted advertisement remains stored for an expiry duration `E`. +Advertisers should continuously renew advertisements to keep the target number of active placements in each bucket. + +## Lookup +Lookup locates advertisers for a service by querying registrars along the service table. +The service specifies the target number of distinct advertisers the discoverer wants to collect `Flookup`. +The process starts from the furthest bucket `b0(s)` and progresses towards the closest bucket `b255(s)`. +The discoverer queries `Klookup` random registrars in each bucket of `B(s)`. +The discoverer repeats the process (going from to ) until it collest at least `Flookup` distinct advertisers, or until no unqueried +registrars remain. +A lookup request to a single registrar is carried by [serviceQUERY]. + + +### serviceQUERY +[serviceQUERY] asks a registrar for nodes matching the given service. The wire message contains: + +- `ReqID`: request identifier **TODO: how is this generated** +- `s`: the 32-byte service +- `Buckets`: distances from the service where the discoverer still has space in its search table `B(s)` and indicates where it still wants peer information for improving its service table. -Checking node liveness whenever a node is to be added to a bucket is impractical and -creates a DoS vector. Implementations should perform liveness checks asynchronously with -bucket addition and occasionally verify that a random node in a random bucket is live by -sending [PING]. When the PONG response indicates that a new version of the node record is -available, the liveness check should pull the new record and update it in the local table. +The queried registrar returns a maximum of `Freturn` advertisements and up to `XX` additional peers to enrich the service table. **TODO** do we have a limit on the number of returned peers? -If a node's liveness has been verified many times, implementations may consider occasional -non-responsiveness permissible and assume the node is live. -When responding to FINDNODE, implementations must avoid relaying any nodes whose liveness -has not been verified. This is easy to achieve by storing an additional flag per node in -the table, tracking whether the node has ever successfully responded to a PING request. +### serviceNODES +[serviceNODES] is a response to [serviceQUERY] and contains: -In order to keep all k-bucket positions occupied even when bucket members fail liveness -checks, it is strongly recommended to maintain a 'replacement cache' alongside each -bucket. This cache holds recently-seen nodes which would fall into the corresponding bucket -but cannot become a member of the bucket because it is already at capacity. Once a bucket -member becomes unresponsive, a replacement can be chosen from the cache. +- `ReqID`: request identifier +- `RespCount`: total number of responses to the request +- `Nodes`: ENRs matching the requested service +- **TODO: it should contain an advertisement list** -### Lookup +The discoverer accumulates distinct nodes from received [serviceNODES] messages to collect advertisements and populate its `B(s)` table. +Multiple responses may be sent for one request, as indicated by `RespCount`. -A 'lookup' locates the `k` closest nodes to a node ID. -The lookup initiator starts by picking `α` closest nodes to the target it knows of from -the local table. The initiator then sends [FINDNODE] requests to those nodes. `α` is an -implementation-defined concurrency parameter, typically `3`. As NODES responses are -received, the initiator resends FINDNODE to nodes it has learned about from previous -queries. Of the `k` nodes the initiator has heard of closest to the target, it picks `α` -that it has not yet queried and sends FINDNODE to them. The lookup terminates when the -initiator has queried and gotten responses from the `k` closest nodes it has seen. +## Admission Control +A registrar stores admitted advertisements in an ad cache. +Each entry has expiry time `E`, after which it is removed automatically. +The ad cache has fixed capacity `C`. +An advertiser may place at most one advertisement for a given service in the ad cache of a given registrar. +Registration requests for an advertisement already present in the cache are ignored. -To improve the resilience of lookups against adversarial nodes, the algorithm may be -adapted to perform network traversal on multiple disjoint paths. Not only does this -approach benefit security, it also improves effectiveness because more nodes are visited -during a single lookup. The initial `k` closest nodes are partitioned into multiple -independent 'path' buckets, and ​concurrent FINDNODE​ requests executed as described above, -with one difference: results discovered on one path are not reused on another, i.e. each -path attempts to reach the closest nodes to the lookup target independently without -reusing intermediate results found on another path. Note that it is still necessary to -track previously asked nodes across all paths to keep the paths disjoint. +When a registrar receives [REGservice], it computes a waiting time based on the current state of the ad cache and the incoming advertisement and replies with [REGservice]. +If the remaining waiting time is zero or negative, the advertisement is admitted immediately. +Otherwise the registrar returns a ticket and waiting time. +The registrar does not required to keep per-request state for pending registrations. +The waiting timee is held by the advertiser in the ticket signed by the advertiser. +The ticket waiting time is not binding and is reevaluated with every returning request. +If an advertiser resends [REGservice] after the indicated amount of time, it may be issued with an additional waiting time. +If an advertiser resends [REGservice] before the indicated amount of time, the request is ignored. -### Lookup Protocol +### REGCONFIRMATION +[REGCONFIRMATION] is a response to [REGservice]. The wire message contains: -This section shows how the wire protocol messages can be used to perform a lookup -interaction against a single node. +- `ReqID`: request identifier +- `RespCount`: total number of responses to the request +- `Ticket`: registrar-issued ticket; successful registration is indicated by zero-length + ticket +- `WaitTime`: how long to wait until sending the next [REGservice], in milliseconds -Node `A` is looking for target `x`. It selects node `B` from the local table or -intermediate lookup results. To query for nodes close to `x` on `B`, node `A` computes the -query distance `d = logdistance(B, x)` and sends its request. +This directly matches the admission protocol: - A -> B FINDNODE [d] +- if `len(Ticket) != 0`, the advertiser has not yet been admitted and must wait `WaitTime` + before retrying; +- if `len(Ticket) == 0`, the registration has succeeded. In the current implementation log + semantics, `WaitTime` then represents advertisement lifetime. +**TODO shouldn't we explicitely indicate 0 here? rather than relying on the length of the message?** -Node `B` responds with multiple nodes messages containing the nodes at the queried -distance. +The response may be split across multiple packets, as indicated by `RespCount`. - A <- B NODES [N₁, N₂, N₃] - A <- B NODES [N₄, N₅] +## Tickets -Depending on the value of `d` and the content of `B`s table, the response to the initial -query might contain very few nodes or no nodes at all. Should this be the case, `A` varies -the distance to retrieve more nodes from adjacent k-buckets on `B`: +A ticket proves that an advertiser has already waited for some amount of time with respect +to a given registrar and service advertisement. - A -> B FINDNODE [d+1] +Tickets as signed objects containing: -`B` responds with more nodes: +- a copy of the advertisement +- the ticket creation time `tinit` +- the ticket modification time `tmod` +- the remaining wait time `twait` - A <- B NODES [N₆, N₇] +When an advertiser receives a ticket, it waits for the indicated time and retries [REGservice] +with the returned ticket in the `Ticket` field. -Node `A` now sorts all received nodes by distance to the lookup target and proceeds by -repeating the lookup procedure on another, closer node. +A retry is valid only during the registration window: -## Topic Advertisement + tmod + twait ≤ now ≤ tmod + twait + δ -The topic advertisement subsystem indexes participants by their provided services. A -node's provided services are identified by arbitrary strings called 'topics'. A node -providing a certain service is said to 'place an ad' for itself when it makes itself -discoverable under that topic. Depending on the needs of the application, a node can -advertise multiple topics or no topics at all. Every node participating in the discovery -protocol acts as an advertisement medium, meaning that it accepts topic ads from other -nodes and later returns them to nodes searching for the same topic. +where `δ` is the registration window duration. If the advertiser retries too early, too +late, or without the latest ticket, it loses accumulated waiting time and must start over. -### Topic Table +The waiting time in a ticket is not binding. +Every retry causes the registrar to recompute the waiting time against the current cache contents. +The effective remaining waiting time is -Nodes store ads for any number of topics and a limited number of ads for each topic. The -data structure holding advertisements is called the 'topic table'. The list of ads for a -particular topic is called the 'topic queue' because it functions like a FIFO queue of -limited length. The image below depicts a topic table containing three queues. The queue -for topic `T₁` is at capacity. + tremaining = twait(current-cache, ad) - (now - tinit) -![topic table](./img/topic-queue-diagram.png) -The queue size limit is implementation-defined. Implementations should place a global -limit on the number of ads in the topic table regardless of the topic queue which contains -them. Reasonable limits are 100 ads per queue and 50000 ads across all queues. Since ENRs -are at most 300 bytes in size, these limits ensure that a full topic table consumes -approximately 15MB of memory. -Any node may appear at most once in any topic queue, that is, registration of a node which -is already registered for a given topic fails. Implementations may impose other -restrictions on the table, such as restrictions on the number of IP-addresses in a certain -range or number of occurrences of the same node across queues. +## Waiting Time Function -### Tickets +The waiting time determines admission order and shapes the contents of the ad cache. -Ads should remain in the queue for a constant amount of time, the `target-ad-lifetime`. To -maintain this guarantee, new registrations are throttled and registrants must wait for a -certain amount of time before they are admitted. When a node attempts to place an ad, it -receives a 'ticket' which tells them how long they must wait before they will be accepted. -It is up to the registrant node to keep the ticket and present it to the advertisement -medium when the waiting time has elapsed. -The waiting time constant is: - target-ad-lifetime = 15min +The waiting time is -The assigned waiting time for any registration attempt is determined according to the -following rules: + w(ad) = E * 1 / (1 - c/C)^Pocc * ( c(ad.service)/c + score(ad.IP) + G ) -- When the table is full, the waiting time is assigned based on the lifetime of the oldest - ad across the whole table, i.e. the registrant must wait for a table slot to become - available. -- When the topic queue is full, the waiting time depends on the lifetime of the oldest ad - in the queue. The assigned time is `target-ad-lifetime - oldest-ad-lifetime` in this - case. -- Otherwise the ad may be placed immediately. +where: -Tickets are opaque objects storing arbitrary information determined by the issuing node. -While details of encoding and ticket validation are up to the implementation, tickets must -contain enough information to verify that: +- `E` be the advertisement expiry duration +- `C` be the ad cache capacity +- `c` be the current number of ads in the cache +- `c(ad.service)` be the number of cached ads for the same service as `ad` +- `score(ad.IP)` be the IP similarity score of the advertiser +- `Pocc` be the occupancy exponent +- `G` be the safety constant -- The node attempting to use the ticket is the node which requested it. -- The ticket is valid for a single topic only. -- The ticket can only be used within the registration window. -- The ticket can't be used more than once. +### IP Similarity -Implementations may choose to include arbitrary other information in the ticket, such as -the cumulative wait time spent by the advertiser. A practical way to handle tickets is to -encrypt and authenticate them with a dedicated secret key: +Registrars maintain a binary tree over the IP addresses of admitted advertisements. +Each vertex stores a counter, and edges correspond to consecutive bits in the IP address. +For IPv4, the tree has 32 levels below the root. +To score an IP address, the registrar walks the tree along the address bits. +At each level, it compares the observed counter against the counter expected in a perfectly balanced tree. Prefixes that are overrepresented contribute to the score. +The resulting IP similarity score is normalized to the interval `[0,1]`. - ticket = aesgcm_encrypt(ticket-key, ticket-nonce, ticket-pt, '') - ticket-pt = [src-node-id, src-ip, topic, req-time, wait-time, cum-wait-time] - src-node-id = node ID that requested the ticket - src-ip = IP address that requested the ticket - topic = the topic that ticket is valid for - req-time = absolute time of REGTOPIC request - wait-time = waiting time assigned when ticket was created - cum-wait = cumulative waiting time of this node -### Registration Window +## Waiting Time Lower Bound +Discv5 enforces a waiting time lower bound. +A new waiting time must not improve on the old one by more than the elapsed time since the old ticket was issued. +Registrars keep lower-bound state only for bounded structures: -The image below depicts a single ticket's validity over time. When the ticket is issued, -the node keeping it must wait until the registration window opens. The length of the -registration window is 10 seconds. The ticket becomes invalid after the registration -window has passed. +- per-service lower bounds for services already present in the cache +- per-prefix lower bounds in the IP tree -![ticket validity over time](./img/ticket-validity.png) +When service s enters the cache for the first time, `bound(s)` is set to `0`, and a `timestamp(s)` is set to the current time. +When a ticket request arrives for the same service `s`, the registrar calculates the service waiting time `ws` and returns the value `ws = max(ws , bound(s) − timestamp(s))`. +The bound and the timestamp are updated when a new ticket is issued and `ws > (bound(s) − timestamp(s))`. -Since all ticket waiting times are assigned to expire when a slot in the queue opens, the -advertisement medium may receive multiple valid tickets during the registration window and -must choose one of them to be admitted in the topic queue. The winning node is notified -using a [REGCONFIRMATION] response. +For IP addresses in the IP tree structure, the state for an IP address is maintained at the node, which corresponds to the longest prefix match in the existing tree (without +introducing new nodes). +The advertiser also aggregates the lower bound states of multiple IPs mapping to the same node by applying a `max` function. -Picking the winner can be achieved by keeping track of a single 'next ticket' per queue -during the registration window. Whenever a new ticket is submitted, first determine its -validity and compare it against the current 'next ticket' to determine which of the two is -better according to an implementation-defined metric such as the cumulative wait time -stored in the ticket. -### Advertisement Protocol + -`C` replies with a ticket and waiting time. - A <- C TICKET [ticket, wait-time] -Node `A` now waits for the duration of the waiting time. When the wait is over, `A` sends -another registration request including the ticket. `C` does not need to remember its -issued tickets since the ticket is authenticated and contains enough information for `C` -to determine its validity. +## Parameters - A -> C REGTOPIC [T, ticket] +Discv5 uses the following main parameters: -Node `C` replies with another ticket. Node `A` must keep this ticket in place of the -earlier one, and must also be prepared to handle a confirmation call in case registration -was successful. +- `Kregister`: target number of active or pending registrations per bucket +- `Klookup`: maximum number of registrar queries per bucket during lookup +- `Freturn`: maximum number of ENRs returned by one registrar response +- `Flookup`: target number of distinct advertisers to collect before terminating lookup +- `E`: advertisement expiry duration +- `C`: ad cache capacity +- `δ`: registration window duration +- `Pocc`: occupancy exponent used in the waiting-time function +- `G`: safety constant used in the waiting-time function - A <- C TICKET [ticket, wait-time] +The concrete values belong in the wire or protocol-parameters document. -Node `C` waits for the registration window to end on the queue and selects `A` as the node -which is registered. Node `C` places `A` into the topic queue for `T` and sends a -[REGCONFIRMATION] response. +## Implementation Considerations - A <- C REGCONFIRMATION [T] +### ENR Freshness -### Ad Placement And Topic Radius +Advertisers should send their current ENR in [REGservice]. Registrars should store the ENR +that was accepted for the service and return ENRs in [serviceNODES]. -Since every node may act as an advertisement medium for any topic, advertisers and nodes -looking for ads must agree on a scheme by which ads for a topic are distributed. When the -number of nodes advertising a topic is at least a certain percentage of the whole -discovery network (rough estimate: at least 1%), ads may simply be placed on random nodes -because searching for the topic on randomly selected nodes will locate the ads quickly enough. +### Distinct Advertisers -However, topic search should be fast even when the number of advertisers for a topic is -much smaller than the number of all live nodes. Advertisers and searchers must agree on a -subset of nodes to serve as advertisement media for the topic. This subset is simply a -region of the node ID address space, consisting of nodes whose Kademlia address is within a -certain distance to the topic hash `sha256(T)`. This distance is called the 'topic -radius'. +Lookup should count distinct advertisers, not raw ENR count. Multiple registrars may return +the same advertiser. -Example: for a topic `f3b2529e...` with a radius of 2^240, the subset covers all nodes -whose IDs have prefix `f3b2...`. A radius of 2^256 means the entire network, in which case -advertisements are distributed uniformly among all nodes. The diagram below depicts a -region of the address space with topic hash `t` in the middle and several nodes close to -`t` surrounding it. Dots above the nodes represent entries in the node's queue for the -topic. +### Signature Validation -![diagram explaining the topic radius concept](./img/topic-radius-diagram.png) +The implementation uses ENRs as the advertised node object. Since ENRs are self-signed, +registrars and discoverers should validate returned ENRs according to the ENR rules before +using them. -To place their ads, participants simply perform a random walk within the currently -estimated radius and run the advertisement protocol by collecting tickets from all nodes -encountered during the walk and using them when their waiting time is over. +### Clocks -### Topic Radius Estimation +The protocol does not require clock synchronization between advertiser and registrar. +Tickets carry registrar-generated timing information, and the advertiser only needs to wait +for the reported duration. -Advertisers must estimate the topic radius continuously in order to place their ads on -nodes where they will be found. The radius mustn't fall below a certain size because -restricting registration to too few nodes leaves the topic vulnerable to censorship and -leads to long waiting times. If the radius were too large, searching nodes would take too -long to find the ads. +### Response Splitting -Estimating the radius uses the waiting time as an indicator of how many other nodes are -attempting to place ads in a certain region. This is achieved by keeping track of the -average time to successful registration within segments of the address space surrounding -the topic hash. Advertisers initially assume the radius is 2^256, i.e. the entire network. -As tickets are collected, the advertiser samples the time it takes to place an ad in each -segment and adjusts the radius such that registration at the chosen distance takes -approximately `target-ad-lifetime / 2` to complete. +[REGCONFIRMATION] and [serviceNODES] contain `RespCount`, indicating that a logical response +may span multiple packets. Implementations should collect all responses belonging to the +same `ReqID` until the announced count is reached or the request times out. -## Topic Search +## TODOs -Finding nodes that provide a certain topic is a continuous process which reads the content -of topic queues inside the approximated topic radius. This is a much simpler process than -topic advertisement because collecting tickets and waiting on them is not required. -To find nodes for a topic, the searcher generates random node IDs inside the estimated -topic radius and performs Kademlia lookups for these IDs. All (intermediate) nodes -encountered during lookup are asked for topic queue entries using the [TOPICQUERY] packet. +1. exact ticket encoding and signature format +2. exact rules for deriving the service from higher-level service identifiers +3. exact encoding of additional neighbor ENRs associated with `Buckets` +4. whether any application-specific payload besides ENR is part of an advertisement +5. parameter defaults for `Kregister`, `Klookup`, `Freturn`, `Flookup`, `E`, `C`, `δ`, + `Pocc`, and `G` -Radius estimation for topic search is similar to the estimation procedure for -advertisement, but samples the average number of results from TOPICQUERY instead of -average time to registration. The radius estimation value can be shared with the -registration algorithm if the same topic is being registered and searched for. +## References [EIP-778]: ../enr.md -[identity scheme]: ../enr.md#record-structure -[handshake message packet]: ./discv5-wire.md#handshake-message-packet-flag--2 -[WHOAREYOU packet]: ./discv5-wire.md#whoareyou-packet-flag--1 -[PING]: ./discv5-wire.md#ping-request-0x01 -[PONG]: ./discv5-wire.md#pong-response-0x02 -[FINDNODE]: ./discv5-wire.md#findnode-request-0x03 -[REGTOPIC]: ./discv5-wire.md#regtopic-request-0x07 -[REGCONFIRMATION]: ./discv5-wire.md#regconfirmation-response-0x09 -[TOPICQUERY]: ./discv5-wire.md#topicquery-request-0x0a +[REGservice]: ./discv5-wire.md#regservice-request-0x09 +[REGCONFIRMATION]: ./discv5-wire.md#regconfirmation-response-0x0a +[serviceQUERY]: ./discv5-wire.md#servicequery-request-0x0b +[serviceNODES]: ./discv5-wire.md#servicenodes-response-0x0c +[NODES]: ./discv5-wire.md#nodes-response-0x04 \ No newline at end of file