multi: Add addrv2 #2627

sefbkn · 2021-03-17T03:58:17Z

This PR modifies the wire protocol by adding a new message type addrv2, bumps to the wire protocol version 13 (TORv3Version), and adds support for sharing and connecting to V3 onion addresses on the peer to peer network.

High-level change summary:

Bump protocol version to 13
- Protocol version 12 (AddrV2Version) exposes the addrv2 message supporting only IPv4 and IPv6 addresses
- Protocol version 13 (TORv3Version) adds TORv3 address support to the addrv2 message.
  - This structure can be used as a template for adding new address types in the future.
Nodes can connect to V3 .onion addresses
- The --connect and --addpeer flags now accept V3 .onion addresses
- Nodes connecting to V3 .onion addresses require an onion proxy to be configured
Nodes can receive inbound connections from peers connecting via their V3 .onion address
Nodes can gossip V3 .onion addresses with peers having a negotiated protocol version >= 13 (TORv3Version)
The addr message fails for negotiated protocol version >= 12 (AddrV2Version), and is replaced by addrv2.

Several breaking changes were also introduced in this pull request.

Breaking change summary:

addrmgr.GetAddress now takes a filter parameter to restrict address types returned by the address manager prior to making outbound connections
Peer.na field type changed from wire.NetAddress to wire.NetAddressV2

Several package versions have also been upgraded.

Package upgrade summary:

addrmgr -> v4
peer -> v4

The structure of the addrv2 message is depicted below for reference.

addrv2 wire format

+---------------------------+
| Count (varint, [1..1000]) |
+---------------------------+
| network address[0]        |
| (see below)               |
+---------------------------+
| network address[1]        |
| (see below)               |
+---------------------------+
| ...                       |
+---------------------------+

The varint Count field of the serialized addrv2 payload describes how many network address entries exist in the message. This value must indicate at least 1 entry and no more than 1000 entries to be considered a valid message. This differs from the addr message in that it does not allow zero entries during serialization or deserialization.

network address wire format

+-----------+----------+--------+----------------+---------+
| Timestamp | Services |  Type  |     Address    |  Port   |
+-----------+----------+--------+----------------+---------+
|  8 bytes  | 8 bytes  | 1 byte | variable bytes | 2 bytes |
+-----------+----------+--------+----------------+---------+

The Type field of the network address payload informs the reader of both the network type that the address belongs to as well as allowing the reader to infer how many bytes to read from the Address field. This approach was chosen to simplify the wire format and to allow for the addition of new network address types based on the negotiated protocol version.

Closes #2035

dajohi · 2021-03-24T14:53:10Z

Does this handle torv3 addresses?

I also think the new addrmgr.NetAddress belongs in wire as NetAddressV2.

sefbkn · 2021-03-29T03:54:58Z

Does this handle torv3 addresses?

I also think the new addrmgr.NetAddress belongs in wire as NetAddressV2.

@dajohi No, torv3 addresses are not supported or broadcast in this PR, but the commit that introduces it and builds on these changes can be included if it seems appropriate for this PR.

Regarding putting this new type in the wire package, I need to follow back up on this one.

sefbkn · 2021-09-14T09:25:32Z

Converting this to a draft until #2596 is merged, since this pull request builds on that.

davecgh · 2021-09-14T19:52:30Z

#2596 has been merged.

davecgh · 2021-11-09T00:03:26Z

I was hoping to get this into v1.7, but it looks like I'm going to have to move this to 1.8 unless @sefbkn plans on finalizing it within the next few days.

server.go

davecgh

I'm reviewing this commit by commit and done with the first one. I've identified various minor things, but it looks pretty good overall thus far.

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

addrmgr/addrmanager.go

addrmgr/netaddress.go

addrmgr/netaddress_test.go

addrmgr/network.go

davecgh

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

addrmgr/addrmanager.go

server.go

davecgh

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

peer/go.mod

peer/peer.go

server.go

davecgh

I really don't understand the thought process of the changes being made here to remove the name resolution bits from the address manager. It's supposed to manage addresses, which, to me, includes name resolution using a resolver provided the caller.

As it stands, up to the point that I've reviewed, there is what seems like some kind of arbitrary line drawn where it sort of half parses hosts into IPs for certain things, but then not others and returns UnknownAddressType expecting the caller to do more with it, but every single caller (more than just dcrd) that uses the addrmgr will need to do that same logic.

Maybe there are some more upcoming changes that will provide more context, or it's that the naming of things is confusing me as to what the overall goal is, but at this point, the changes seem to be removing things that are useful to other callers and making them inaccessible.

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

server_test.go

davecgh

I've confirmed the Tor address encoding and parsing logic is correct per the Tor v3 address specification.

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

addrmgr/network.go

server.go

davecgh

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

addrmgr/network.go

server_test.go

davecgh

I would like to see the peer, wire, and server bits split up where the wire bits come first. You can take a look at previous PRs I've done to add new protocol messages for inspiration. For example (#1906). It helps the review process and also helps prevent accidental breaking changes to the protocol through the layers.

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

peer/peer.go

peer/peer_test.go

wire/common.go

wire/error.go

server.go

wire/msggetaddrv2.go

server.go

davecgh

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

connmgr/seed.go

server.go

davecgh

As before, I would prefer the wire, peer and server bits be split up into individual commits for the same reason.

Review progress:

addrmgr: Track network address types.
addrmgr: Add network address type filter.
peer: Partially decouple peer from wire.
addrmgr: Remove DNS lookups from address manager.
addrmgr: Add TORv3 network address type.
multi: Remove TORv2 network address type.
peer/wire: Add addrv2 and getaddrv2 message types.
connmgr: Allow seeding additional address types.
peer/wire: Relay TORv3 addresses.

server.go

wire/msgaddrv2.go

davecgh · 2022-03-25T15:21:07Z

Alright, I'm through with the initial review. This also needs to be rebased to resolve the conflicts. Once everything is addressed, I'll give it another pass and thoroughly test it manually.

davecgh · 2022-06-12T19:32:28Z

Reminder on this one. Will need a rebase to resolve conflicts as well.

sefbkn · 2025-11-16T18:35:17Z

Does this handle torv3 addresses?

This handles torv3 addresses now.

I also think the new addrmgr.NetAddress belongs in wire as NetAddressV2.

Done.

sefbkn · 2025-11-17T02:41:34Z

All feedback has been addressed and this is ready for review.

davecgh · 2025-11-17T03:18:40Z

Thanks for the updates. It will probably be about a week before I get to re-familiarize myself with this since it's been a while and review the updates properly.

I'm working on a consensus change that is time sensitive at the moment that I need to finish up first.

sefbkn · 2025-11-19T04:00:33Z

Cleaned up the PR description and split the commits probably more than necessary, but it looks cleaner to me and less scattered than before.

This adds the addrv2 message type to allow extensible peer address exchange. It also defines the protocol version AddrV2Version (12) for future negotiations, but does not advertise it on the peer to peer network or use it outside of the wire package. The structure of the addrv2 message is depicted below for reference. MsgAddrV2 wire format: +---------------------------+ | Count (varint, [1..1000]) | +---------------------------+ | NetAddressV2[N] | | (see below) | +---------------------------+ | ... | +---------------------------+ The varint Count prefix of the MsgAddrV2 payload describes how many NetAddressV2 entries exist in the message. This value must indicate at least 1 entry and no more than 1000 entries to be considered a valid message. This differs from the addr message in that it does not allow zero entries. NetAddressV2 wire format: +-----------+----------+--------+----------------+---------+ | Timestamp | Services | Type | Address | Port | +-----------+----------+--------+----------------+---------+ | 8 bytes | 8 bytes | 1 byte | variable bytes | 2 bytes | +-----------+----------+--------+----------------+---------+ The Type field of the NetAddressV2 payload informs the reader of both the network type that the address belongs to as well as allowing the reader to infer how many bytes to read from the address field. The addr message is also modified to now fail when encoding/decoding when the negotiated protocol version >= AddrV2Version. The following is an overview of the changes: - Add NetAddressV2 type - This includes a constructor NewNetAddressV2 that will be needed in a future commit for directly constructing v2 wire network addresses with varying types. - Add MsgAddrV2 type implementing Message interface - Define AddrV2Version constant (12) - Reject MsgAddr for negotiated protocol version >= 12 - Add ErrTooFewAddrs error - Add test coverage

Upcoming changes to the peer module will include breaking changes to the public API. Therefore, this follows the process for introducing major API changes, which consists of: - Bump the major version in the go.mod of the peer module - Update all imports in the repo to use the new major version as necessary - Tidy all modules in the repository to ensure they are updated to use the latest versions

This breaking change modifies the Peer.na field from wire.NetAddress to wire.NetAddressV2 to accommodate responses from addrv2. Since the addrmgr module uses its own NetAddress type, this adds conversion functions in server.go to translate between wire.NetAddressV2 and addrmgr.NetAddress. It also adds an NA function on serverPeer that returns an addrmgr.NetAddress. This is done because every instance of the peer's network address needs to be converted to an addrmgr.NetAddress anyways, so shadowing the embedded type removes boilerplate. Alternatives considered include updating the explicit conversion function in every usage of NA, coupling the peer module to the addrmgr module and make Peer.na an addrmgr.NetAddress, or creating a new netaddr module to avoid coupling wire and addrmgr, and adding NA to serverPeer appeared to be the cleanest approach for the immediate effort of integrating the addrv2 message into the codebase. The following is an overview of the changes: peer: - Change Peer.na from wire.NetAddress to wire.NetAddressV2 - Add supporting code to convert between NetAddress types server: - Move connectionsWithIP func definition closer to its usage - Update HostToNetAddress callback to return wire.NetAddressV2 - Add NA function on serverPeer to return addrmgr.NetAddress - Add conversion functions in server.go - Update OnAddr handler for new types

This adds support for the addrv2 message to the peer module, including the ability to send and receive addrv2 messages for IPv4 and IPv6 network address types. The following is an overview of the changes: - Bump peer.MaxProtocolVersion to AddrV2Version (12) - Add OnAddrV2 callback to MessageListeners - Add PushAddrV2Msg method - Add MsgAddrV2 handler in inHandler dispatch loop - Log addrv2 messages in messageSummary - Add test coverage for addrv2 functionality

This enables addrv2 message handling and makes the addrv2 message active and usable on the peer to peer network for IPv4 and IPv6 network address types. With this, the response to a getaddr message will either be an addr or addrv2 message, varying based on the negotiated protocol version. For negotiated protocol version < 12, respond to getaddr with addr >= 12, respond to getaddr with addrv2 The following is an overview of the changes: - Bump maxProtocolVersion to AddrV2Version (12) - Add conversion function wireToAddrmgrNetAddressesV2 - Add pushAddrV2Msg method - Update OnVersion to send addr or addrv2 based on negotiated version - Update OnGetAddr to respond with either addr or addrv2 - Add OnAddrV2 handler to process incoming addrv2 messages - Register OnAddrV2 message listener when constructing peer

This change modifies wire to - introduce a new TORv3Version protocol version - adds new TORv3Address network address type - ensure the TORv3Address network address type is only serialized and deserialized at protocol version >= 13

This updates the max protocol version supported by the peer module to TORv3Version (13).

Upcoming changes to the addrmgr module will include breaking changes to the public API. Therefore, this follows the process for introducing major API changes, which consists of: - Bump the major version in the go.mod of the addrmgr module - Update all imports in the repo to use the new major version as necessary - Tidy all modules in the repository to ensure they are updated to use the latest versions

This introduces a breaking change to the address manager by adding a filter parameter to GetAddress to allow callers to specify wanted address types. It also introduces per-bucket statistics to efficiently track address type counts, which allows tried and new buckets to be filtered without scanning their contents. This allows for identifying whether a bucket contains addresses matching the network address type filter in constant time rather than needing to do a linear scan of all addresses within the bucket. Note that deciding whether a bucket contains any addresses matching the filter is a distinct operation from retrieving an address from a bucket that is known to contain an address matching the filter. This prepares for new address types that will be added in the future. Filtering will allow callers to prevent address types that they cannot use or would otherwise be undesirable from being returned by the address manager. The following is an overview of the changes: - Add filter parameter to GetAddress (breaking change) - Add bucketStats to track address counts per bucket - Add and update tests

This adds TORv3 address support to the address manager, including serialization logic, validation, persistence and retrieval of TORv3 .onion addresses. Sharing TORv3 addresses over the peer to peer network is not yet supported and will be added in a later commit. The following is an overview of the changes: - Add TORv3Address constant (4) to NetAddressType - Add TORv3 parsing to EncodeHost - Add various TORv3 helper functions - Update ipString to serialze TORv3 addresses to .onion format - Update deriveNetAddressType to recognize TORv3 - Add Private constant to NetAddressReach for TORv3 - Update getRemoteReachabilityFromLocal for TORv3 - Add tests

This bumps the advertised protocol version to 13 and allows sharing TORv3 addresses on the peer to peer network for peers with a negotiated protocol version of TORv3Version (13). The following is an overview of the changes: - Modify converters to support translating between the wire and addrmgr TORv3Address network address type enums. - Add isSupportedNetAddressTypeV2 func to allow torv3 addresses to be read from the address manager for negotiated protocol version TORv3Version. - Ensure server only receives TORv3 addresses from address manager if an onion proxy is configured when calling GetAddress. - Avoid dns lookups for TORv3 .onion addresses - this allows the --connect and --addpeer flags to accept TORv3 .onion addresses.

sefbkn changed the title ~~[multi] Add getaddrv2 and addrv2.~~ multi: Add getaddrv2 and addrv2. Mar 17, 2021

davecgh added this to the 1.7.0 milestone Mar 17, 2021

davecgh added the wire protocol change Discussion and pull requests regarding items that require changes to the wire protocol. label Mar 17, 2021

sefbkn force-pushed the add_addrv2 branch from 7557e51 to d5bc0c1 Compare April 7, 2021 04:42

sefbkn marked this pull request as draft September 14, 2021 09:24

davecgh modified the milestones: 1.7.0, 1.8.0 Nov 10, 2021

sefbkn force-pushed the add_addrv2 branch from d5bc0c1 to 4bf338f Compare January 5, 2022 03:56

sefbkn force-pushed the add_addrv2 branch from 4bf338f to cf891d2 Compare February 7, 2022 13:56

sefbkn marked this pull request as ready for review February 9, 2022 05:11

sefbkn commented Feb 22, 2022

View reviewed changes

server.go Outdated Show resolved Hide resolved

sefbkn force-pushed the add_addrv2 branch from cf891d2 to 3632c29 Compare February 26, 2022 04:29

davecgh requested changes Mar 21, 2022

View reviewed changes

server_test.go Outdated Show resolved Hide resolved

server_test.go Outdated Show resolved Hide resolved

davecgh requested changes Mar 23, 2022

View reviewed changes

addrmgr/network.go Outdated Show resolved Hide resolved

addrmgr/network.go Outdated Show resolved Hide resolved

server.go Outdated Show resolved Hide resolved

davecgh requested changes Mar 23, 2022

View reviewed changes

addrmgr/network.go Show resolved Hide resolved

server_test.go Outdated Show resolved Hide resolved

server_test.go Outdated Show resolved Hide resolved

decred deleted a comment from sefbkn Mar 25, 2022

davecgh requested changes Mar 25, 2022

View reviewed changes

connmgr/seed.go Outdated Show resolved Hide resolved

server.go Outdated Show resolved Hide resolved

davecgh requested changes Mar 25, 2022

View reviewed changes

server.go Outdated Show resolved Hide resolved

wire/msgaddrv2.go Outdated Show resolved Hide resolved

davecgh added the waiting for changes Pull requests that are waiting for changes from the submitter. label Jun 24, 2022

davecgh modified the milestones: 1.8.0, 1.9.0 May 22, 2023

davecgh modified the milestones: 1.9.0, Future Version May 9, 2024

matthawkins90 mentioned this pull request Jul 10, 2024

addrmgr: Track network types and remove TorV2 support #3406

Merged

matthawkins90 mentioned this pull request Jul 21, 2024

addrmgr: Allow filtering by type, and remove DNS. #3409

Merged

sefbkn force-pushed the add_addrv2 branch from 3632c29 to 38f1ab2 Compare November 16, 2025 08:31

sefbkn changed the title ~~multi: Add getaddrv2 and addrv2.~~ multi: Add addrv2 Nov 16, 2025

sefbkn force-pushed the add_addrv2 branch from 38f1ab2 to 0ad6a33 Compare November 17, 2025 01:38

sefbkn requested a review from davecgh November 17, 2025 02:41

sefbkn force-pushed the add_addrv2 branch from 0ad6a33 to d0cfe23 Compare November 19, 2025 03:01

sefbkn added 11 commits November 28, 2025 00:14

wire: Add TORv3Version protocol constant

04bf860

This change modifies wire to - introduce a new TORv3Version protocol version - adds new TORv3Address network address type - ensure the TORv3Address network address type is only serialized and deserialized at protocol version >= 13

peer: Update to TORv3Version

8ad0a2b

This updates the max protocol version supported by the peer module to TORv3Version (13).

sefbkn force-pushed the add_addrv2 branch from d0cfe23 to 92d2a40 Compare November 29, 2025 03:47

multi: Add addrv2 #2627

Are you sure you want to change the base?

multi: Add addrv2 #2627

Uh oh!

Conversation

sefbkn commented Mar 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dajohi commented Mar 24, 2021

Uh oh!

sefbkn commented Mar 29, 2021

Uh oh!

sefbkn commented Sep 14, 2021

Uh oh!

davecgh commented Sep 14, 2021

Uh oh!

davecgh commented Nov 9, 2021

Uh oh!

Uh oh!

davecgh left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davecgh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davecgh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davecgh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

davecgh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davecgh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davecgh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sefbkn commented Mar 17, 2021 •

edited

Loading

davecgh left a comment •

edited

Loading