EIPs

EIP2481 - eth/66: request identifier

# Simple Summary

This document proposes a way to increase the efficiency of the eth networking protocol while at the same time reducing complexity in Ethereum node implementations. It does so by introducing a request id to all requests which their corresponding responses must include.

# Abstract

The eth protocol defines various request and response commands that are used to exchange data between Ethereum nodes. For example, to ask a peer node for a specific set of headers, a node sends it the GetBlockHeaders (opens new window) command.

Citing from the GetBlockHeaders spec definition (opens new window):

[block: {P, B_32}, maxHeaders: P, skip: P, reverse: P in {0, 1}]

Require peer to return a BlockHeaders message. Reply must contain a number of block headers, of rising number when reverse is 0, falling when 1, skip blocks apart, beginning at block block (denoted by either number or hash) in the canonical chain, and with at most maxHeaders items.

The node that receives the GetBlockHeaders command should answer it with the BlockHeaders (opens new window) response command accordingly.

Citing from the BlockHeaders spec definition (opens new window):

[blockHeader_0, blockHeader_1, ...]

Reply to GetBlockHeaders. The items in the list (following the message ID) are block headers in the format described in the main Ethereum specification, previously asked for in a GetBlockHeaders message. This may validly contain no block headers if none of the requested block headers were found. The number of headers that can be requested in a single message may be subject to implementation-defined limits.

Let's consider a client making many simultaneous requests for GetBlockHeaders to one of its peers. By nature it can not be guaranteed that the expected responses arrive in the same order as they were sent. For the client to associate the incoming responses to the correct requests it has to loop through all pending requests trying to match it with the incoming response based on its contents.

This can be particular tricky for responses that are ambiguous such as empty responses.

This EIP proposes to change the GetBlockHeaders and the BlockHeaders command to include a request_id.

The request_id is a 64-bit integer set by the client when it makes the request. On the responding side, the exact same request_id from the incoming request is put back into the response object.

This change allows the requesting client to match incoming responses directly back to their pending requests without going through all of the pending requests to check if they might match based on the response data.

The selected request/response pair serves as an example for many similar request/response pairs in the eth networking protocol.

# Motivation

The lack of request identifiers in the request / response paris of the eth protocol puts unnecessary burden of code complexity into every Ethereum client. It also makes the communication slightly less efficient. Another argument can be made that the addition of request identifiers makes the protocol more aligned with the les protocol which does already defines request identifiers for each request / response pair.

# Specification

Change the following message types in the eth protocol:

  • GetBlockHeaders (0x03)
    • Current (eth/65): [block: {P, B_32}, maxHeaders: P, skip: P, reverse: P in {0, 1}]
    • Then (eth/66): [request_id: P, [block: {P, B_32}, maxHeaders: P, skip: P, reverse: P in {0, 1}]]
  • BlockHeaders (0x04)
    • Current (eth/65): [blockHeader_0, blockHeader_1, ...]
    • Then (eth/66): [request_id: P, [blockHeader_0, blockHeader_1, ...]]
  • GetBlockBodies (0x05)
    • Current (eth/65): [hash_0: B_32, hash_1: B_32, ...]
    • Then (eth/66): [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]
  • GetPooledTransactions (0x09):
    • Current (eth/65)[hash_0: B_32, hash_1: B_32, ...]
    • Then (eth/66)[request_id: P, [hash_0: B_32, hash_1: B_32, ...]]
  • PooledTransactions (0x0a):
    • Current (eth/65)[[nonce: P, receivingAddress: B_20, value: P, ...], ...]
    • Then (eth/66)[request_id: P, [[nonce: P, receivingAddress: B_20, value: P, ...], ...]]
  • BlockBodies (0x06)
    • Current (eth/65): [hash_0: B_32, hash_1: B_32, ...]
    • Then (eth/66): [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]
  • GetNodeData (0x0d)
    • Current (eth/65): [hash_0: B_32, hash_1: B_32, ...]
    • Then (eth/66): [request_id: P, [hash_0: B_32, hash_1: B_32, ...]]
  • NodeData (0x0e)
    • Current (eth/65): [value_0: B, value_1: B, ...]
    • Then (eth/66): [request_id: P, [value_0: B, value_1: B, ...]]
  • GetReceipts (0x0f)
    • Current (eth/65): [blockHash_0: B_32, blockHash_1: B_32, ...]
    • Then (eth/66): [request_id: P, [blockHash_0: B_32, blockHash_1: B_32, ...]]
  • Receipts (0x10)
    • Current (eth/65): [[receipt_0, receipt_1], ...]
    • Then (eth/66): [request_id: P, [[receipt_0, receipt_1], ...]]

To elaborate, each command is altered in the following way:

  1. Create a list with the request_id being the first element.
  2. Make the second element the list that defines the whole command in the current scheme.

The request_id has the following characteristics:

  • 64 bit integer
  • Doesn't need to be sequential (can be random)
  • Does allow duplicates

# Rationale

Q: The efficiency gains might encourage clients to flood their peers with too many simultaneous requests

Peers can always throttle or disconnect if they don't feel treated well. This is the same as today.

Q: If les already defines the commands like this, why not just use the les protocol?

In practice, peers that serve the les protocol are much harder to find in the network. The reasons for this are varied but might boil down to client defaults, immature implementations or missing incentives.

Q: Networking works today, isn't this just adding bloat?

This is adding a single integer per command while at the same time reducing code complexity and improving networking efficiency. The addition seems justified.

Q: Why not demand request ids to be sequential?

Assuming request ids start always to count from 0 upon connection, things will become messy when connections are lost and clients reconnect and start over with the same request ids that they had used in the previous session.

Q: Why allow duplicate request ids?

The main benefit is flexibility and simplicity on the implementation side. Clients could decide to share the same ids across multiple different request types since they are naturally separated anyway. Clients could even decide to not rely on request ids at all, therefore using the same constant request id across all requests.

Q: Why choose a 64-bit integer for the request ids

64-bit integer were chosen to keep compatibility with the les protocol.

# Backwards Compatibility

This EIP extends the eth protocol in a backwards incompatible way and requires rolling out a new version, eth/66. However, devp2p supports running multiple versions of the same wire protocol side-by-side, so rolling out eth/66 does not require client coordination, since non-updated clients can keep using eth/65.

This EIP does not change the consensus engine, thus does not require a hard fork.

# Implementation

Trinity has a draft PR (opens new window) with an implementation. Geth PR (opens new window).

# Security Considerations

None

# Test cases

These testcases cover RLP-encoding of all the redefined messages types, where the rlp portion is the rlp-encoding of the message defined in the data portion.

{
  "type": "GetBlockHeadersPacket66",
  "rlp": "0xe8820457e4a000000000000000000000000000000000000000000000000000000000deadc0de050580",
  "data": {
    "RequestId": 1111,
    "Origin": {
      "Hash": "0x00000000000000000000000000000000000000000000000000000000deadc0de",
      "Number": 0
    },
    "Amount": 5,
    "Skip": 5,
    "Reverse": false
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
  "type": "GetBlockHeadersPacket66",
  "rlp": "0xca820457c682270f050580",
  "data": {
    "RequestId": 1111,
    "Origin": {
      "Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
      "Number": 9999
    },
    "Amount": 5,
    "Skip": 5,
    "Reverse": false
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
  "type": "BlockHeadersPacket66",
  "rlp": "0xf90202820457f901fcf901f9a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000940000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000b90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008208ae820d0582115c8215b3821a0a827788a00000000000000000000000000000000000000000000000000000000000000000880000000000000000",
  "data": {
    "RequestId": 1111,
    "BlockHeadersPacket": [
      {
        "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "sha3Uncles": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "miner": "0x0000000000000000000000000000000000000000",
        "stateRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "transactionsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "receiptsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
        "difficulty": "0x8ae",
        "number": "0xd05",
        "gasLimit": "0x115c",
        "gasUsed": "0x15b3",
        "timestamp": "0x1a0a",
        "extraData": "0x7788",
        "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "nonce": "0x0000000000000000",
        "hash": "0x8c2f2af15b7b563b6ab1e09bed0e9caade7ed730aec98b70a993597a797579a9"
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
{
  "type": "GetBlockBodiesPacket66",
  "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef",
  "data": {
    "RequestId": 1111,
    "GetBlockBodiesPacket": [
      "0x00000000000000000000000000000000000000000000000000000000deadc0de",
      "0x00000000000000000000000000000000000000000000000000000000feedbeef"
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
{
  "type": "BlockBodiesPacket66",
  "rlp": "0xf902dc820457f902d6f902d3f8d2f867088504a817c8088302e2489435353535353535353535353535353535353535358202008025a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10f867098504a817c809830334509435353535353535353535353535353535353535358202d98025a052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afba052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afbf901fcf901f9a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000940000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000000b90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008208ae820d0582115c8215b3821a0a827788a00000000000000000000000000000000000000000000000000000000000000000880000000000000000",
  "data": {
    "RequestId": 1111,
    "BlockBodiesPacket": [
      {
        "Transactions": [
          {
            "nonce": "0x8",
            "gasPrice": "0x4a817c808",
            "gas": "0x2e248",
            "to": "0x3535353535353535353535353535353535353535",
            "value": "0x200",
            "input": "0x",
            "v": "0x25",
            "r": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12",
            "s": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10",
            "hash": "0x588df025c4c2d757d3e314bd3dfbfe352687324e6b8557ad1731585e96928aed"
          },
          {
            "nonce": "0x9",
            "gasPrice": "0x4a817c809",
            "gas": "0x33450",
            "to": "0x3535353535353535353535353535353535353535",
            "value": "0x2d9",
            "input": "0x",
            "v": "0x25",
            "r": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb",
            "s": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb",
            "hash": "0xf39c7dac06a9f3abf09faf5e30439a349d3717611b3ed337cd52b0d192bc72da"
          }
        ],
        "Uncles": [
          {
            "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "sha3Uncles": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "miner": "0x0000000000000000000000000000000000000000",
            "stateRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "transactionsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "receiptsRoot": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
            "difficulty": "0x8ae",
            "number": "0xd05",
            "gasLimit": "0x115c",
            "gasUsed": "0x15b3",
            "timestamp": "0x1a0a",
            "extraData": "0x7788",
            "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "nonce": "0x0000000000000000",
            "hash": "0x8c2f2af15b7b563b6ab1e09bed0e9caade7ed730aec98b70a993597a797579a9"
          }
        ]
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
{
  "type": "GetNodeDataPacket66",
  "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef",
  "data": {
    "RequestId": 1111,
    "GetNodeDataPacket": [
      "0x00000000000000000000000000000000000000000000000000000000deadc0de",
      "0x00000000000000000000000000000000000000000000000000000000feedbeef"
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
{
  "type": "NodeDataPacket66",
  "rlp": "0xce820457ca84deadc0de84feedbeef",
  "data": {
    "RequestId": 1111,
    "NodeDataPacket": [
      "0xdeadc0de",
      "0xfeedbeef"
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
{
  "type": "GetReceiptsPacket66",
  "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef",
  "data": {
    "RequestId": 1111,
    "GetReceiptsPacket": [
      "0x00000000000000000000000000000000000000000000000000000000deadc0de",
      "0x00000000000000000000000000000000000000000000000000000000feedbeef"
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
{
  "type": "ReceiptsPacket66",
  "rlp": "0xf90172820457f9016cf90169f901668001b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000f85ff85d940000000000000000000000000000000000000011f842a0000000000000000000000000000000000000000000000000000000000000deada0000000000000000000000000000000000000000000000000000000000000beef830100ff",
  "data": {
    "RequestId": 1111,
    "ReceiptsPacket": [
      [
        {
          "root": "0x",
          "status": "0x0",
          "cumulativeGasUsed": "0x1",
          "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
          "logs": [
            {
              "address": "0x0000000000000000000000000000000000000011",
              "topics": [
                "0x000000000000000000000000000000000000000000000000000000000000dead",
                "0x000000000000000000000000000000000000000000000000000000000000beef"
              ],
              "data": "0x0100ff",
              "blockNumber": "0x0",
              "transactionHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
              "transactionIndex": "0x0",
              "blockHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
              "logIndex": "0x0",
              "removed": false
            }
          ],
          "transactionHash": "0x00000000000000000000000000000000000000000000000000000000deadc0de",
          "contractAddress": "0x0000000000000000000000000000000000011111",
          "gasUsed": "0x1b207",
          "blockHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
          "transactionIndex": "0x0"
        }
      ]
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
{
  "type": "GetPooledTransactionsPacket66",
  "rlp": "0xf847820457f842a000000000000000000000000000000000000000000000000000000000deadc0dea000000000000000000000000000000000000000000000000000000000feedbeef",
  "data": {
    "RequestId": 1111,
    "GetPooledTransactionsPacket": [
      "0x00000000000000000000000000000000000000000000000000000000deadc0de",
      "0x00000000000000000000000000000000000000000000000000000000feedbeef"
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
{
  "type": "PooledTransactionsPacket66",
  "rlp": "0xf8d7820457f8d2f867088504a817c8088302e2489435353535353535353535353535353535353535358202008025a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12a064b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10f867098504a817c809830334509435353535353535353535353535353535353535358202d98025a052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afba052f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb",
  "data": {
    "RequestId": 1111,
    "PooledTransactionsPacket": [
      {
        "nonce": "0x8",
        "gasPrice": "0x4a817c808",
        "gas": "0x2e248",
        "to": "0x3535353535353535353535353535353535353535",
        "value": "0x200",
        "input": "0x",
        "v": "0x25",
        "r": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c12",
        "s": "0x64b1702d9298fee62dfeccc57d322a463ad55ca201256d01f62b45b2e1c21c10",
        "hash": "0x588df025c4c2d757d3e314bd3dfbfe352687324e6b8557ad1731585e96928aed"
      },
      {
        "nonce": "0x9",
        "gasPrice": "0x4a817c809",
        "gas": "0x33450",
        "to": "0x3535353535353535353535353535353535353535",
        "value": "0x2d9",
        "input": "0x",
        "v": "0x25",
        "r": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb",
        "s": "0x52f8f61201b2b11a78d6e866abc9c3db2ae8631fa656bfe5cb53668255367afb",
        "hash": "0xf39c7dac06a9f3abf09faf5e30439a349d3717611b3ed337cd52b0d192bc72da"
      }
    ]
  }
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

Copyright and related rights waived via CC0 (opens new window).

▲ Powered by Vercel