William/Reticulum
1
0
Fork 0
mirror of https://github.com/markqvist/Reticulum.git synced 2024-11-05 05:40:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated documentation

Browse Source
This commit is contained in:
Mark Qvist 2021-05-20 18:37:12 +02:00
parent bf410e006f
commit e8ca88377a
9 changed files with 412 additions and 343 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
RNS/Transport.py
View File

@ -22,8 +22,6 @@ class Transport:
APP_NAME = "rnstransport" APP_NAME = "rnstransport"
# TODO: Document the addition of random windows
# and max local rebroadcasts.
PATHFINDER_M = 18 # Max hops PATHFINDER_M = 18 # Max hops
PATHFINDER_C = 2.0 # Decay constant PATHFINDER_C = 2.0 # Decay constant
PATHFINDER_R = 1 # Retransmit retries PATHFINDER_R = 1 # Retransmit retries

BIN
docs/Reticulum Manual.pdf
View File

Binary file not shown.

241
docs/manual/_sources/understanding.rst.txt
View File

@ -157,12 +157,16 @@ destinations. Reticulum uses three different basic destination types, and one sp
Destination Naming Destination Naming
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
Destinations are created and named in an easy to understand dotted notation of *aspects* , and Destinations are created and named in an easy to understand dotted notation of *aspects*, and
represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The
top level aspect should always be a unique identifier for the application using the destination. top level aspect should always be a unique identifier for the application using the destination.
The next levels of aspects can be defined in any way by the creator of the application. For example, The next levels of aspects can be defined in any way by the creator of the application.
a destination for a environmental monitoring application could be made up of the application name, a
device type and measurement type, like this: Aspects can be as long and as plentiful as required, and a resulting long destination name will not
impact efficiency, as names are always represented as truncated SHA-256 hashes on the network.
As an example, a destination for a environmental monitoring application could be made up of the
application name, a device type and measurement type, like this:
.. code-block:: text .. code-block:: text
@ -201,9 +205,8 @@ To recap, the different destination types should be used in the following situat
* **Single** * **Single**
When private communication between two endpoints is needed. Supports multiple hops. When private communication between two endpoints is needed. Supports multiple hops.
* **Group** * **Group**
When private communication between two or more endpoints is needed. More efficient in When private communication between two or more endpoints is needed. Supports multiple hops
data usage than *single* destinations. Supports multiple hops indirectly, but must first be indirectly, but must first be established through a *single* destination.
established through a *single* destination.
* **Plain** * **Plain**
When plain-text communication is desirable, for example when broadcasting information. When plain-text communication is desirable, for example when broadcasting information.
@ -214,9 +217,9 @@ an unknown public key from the network, as all participating nodes serve as a di
of public keys. of public keys.
Note that public key information can be shared and verified in many other ways than using the Note that public key information can be shared and verified in many other ways than using the
built-in methodology, and that it is therefore not required to use the announce/request functionality. built-in *announce* functionality, and that it is therefore not required to use the announce/request
It is by far the easiest though, and should definitely be used if there is not a good reason for functionality to obtain public keys. It is by far the easiest though, and should definitely be used
doing it differently. if there is not a good reason for doing it differently.
.. _understanding-keyannouncements: .. _understanding-keyannouncements:
@ -235,7 +238,7 @@ contain the following information:
* The announcers public key * The announcers public key
* Application specific data, in this case the users nickname and availability status * Application specific data, in this case the users nickname and availability status
* A random blob, making each new announce unique * A random blob, making each new announce unique
* A signature of the above information, verifying authenticity * An Ed25519 signature of the above information, verifying authenticity
With this information, any Reticulum node that receives it will be able to reconstruct an outgoing With this information, any Reticulum node that receives it will be able to reconstruct an outgoing
destination to securely communicate with that destination. You might have noticed that there is one destination to securely communicate with that destination. You might have noticed that there is one
@ -244,8 +247,9 @@ the aspect names of the destination. These are intentionally left out to save ba
will be implicit in almost all cases. If a destination name is not entirely implicit, information can be will be implicit in almost all cases. If a destination name is not entirely implicit, information can be
included in the application specific data part that will allow the receiver to infer the naming. included in the application specific data part that will allow the receiver to infer the naming.
It is important to note that announcements will be forwarded throughout the network according to a It is important to note that announces will be forwarded throughout the network according to a
certain pattern. This will be detailed later. certain pattern. This will be detailed in the section
:ref:`The Announce Mechanism in Detail<understanding-announce>`.
Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic. Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic.
@ -268,8 +272,8 @@ the identity first, and then link it to created destinations.
Building upon the simple messenger example, we could use an identity to represent the user of the Building upon the simple messenger example, we could use an identity to represent the user of the
application. Destinations created will then be linked to this identity to allow communication to application. Destinations created will then be linked to this identity to allow communication to
reach the user. In such a case it is of great importance to store the user’s identity securely and reach the user. In all cases it is of great importance to store the private keys associated with any
privately. Reticulum Identity securely and privately.
.. _understanding-gettingfurther: .. _understanding-gettingfurther:
@ -279,8 +283,9 @@ Getting Further
The above functions and principles form the core of Reticulum, and would suffice to create The above functions and principles form the core of Reticulum, and would suffice to create
functional networked applications in local clusters, for example over radio links where all interested functional networked applications in local clusters, for example over radio links where all interested
nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple
hops in the network. In the next sections, two concepts that allow this will be introduced, *paths* and hops in the network.
*links*.
In the following sections, two concepts that allow this will be introduced, *paths* and *links*.
.. _understanding-transport: .. _understanding-transport:
@ -298,85 +303,28 @@ useable over bandwidth-limited, high-latency links.
To overcome such challenges, Reticulum’s *Transport* system uses public-key cryptography to To overcome such challenges, Reticulum’s *Transport* system uses public-key cryptography to
implement the concept of *paths* that allow discovery of how to get information to a certain implement the concept of *paths* that allow discovery of how to get information to a certain
destination, and *resources* that help make reliable data transfer more efficient. destination. It is important to note that no single node in a Reticulum network knows the complete
path to a destination. Every Transport node participating in a Reticulum network will only
know what the most direct way to get a packet one hop closer to it's destination is.
.. _understanding-paths: .. _understanding-announce:
Reaching the Destination The Announce Mechanism in Detail
------------------------ --------------------------------
In networks with changing topology and trustless connectivity, nodes need a way to establish When an *announce* is transmitted by a node, it will be forwarded by any node receiving it, but
*verified connectivity* with each other. Since the network is assumed to be trustless, Reticulum according to some specific rules:
must provide a way to guarantee that the peer you are communicating with is actually who you
expect. Reticulum offers two ways to do this.
For exchanges of small amounts of information, Reticulum offers the *Packet* API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:
* | A packet is always created with an associated destination and some payload data. When the packet is sent to a *single* destination type, Reticulum will automatically create an ephemeral encryption key, perform an ECDH key exchange with the destinations public key, and encrypt the information.
* | It is important to note that this key exchange does not require any network traffic. The sender already knows the public key of the destination from an earlier received *announce*, and can thus perform the ECDH key exchange locally.
* | The public key part of the newly generated ephemeral key is included with the encrypted token, and sent along with the encrypted payload data in the packet.
* | When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the packet.
* | A new ephemeral key is used for every packet sent in this way, and forward secrecy is guaranteed on a per packet level.
* | In case the packet is addressed to a *group* destination type, the packet will be encrypted with the pre-shared AES-128 key associated with the destination. In case the packet is addressed to a *plain* destination type, the payload data will not be encrypted. Neither of these two destination types offer forward secrecy. In general, it is recommended to always use the *single* destination type, unless it is strictly necessary to use one of the others.
For exchanges of larger amounts of data, or when longer sessions of bidirectional communication is desired, Reticulum offers the *Link* API. To establish a *link*, the following process is employed: * | If this exact announce has already been received before, ignore it.
* | First, the node that wishes to establish a link will send out a special packet, that * | If not, record into a table which node the announce was received from, and how many times in
traverses the network and locates the desired destination. Along the way, the nodes that
forward the packet will take note of this *link request*.
* | Second, if the destination accepts the *link request* , it will send back a packet that proves the
authenticity of it’s identity (and the receipt of the link request) to the initiating node. All
nodes that initially forwarded the packet will also be able to verify this proof, and thus
accept the validity of the *link* throughout the network.
* | When the validity of the *link* has been accepted by forwarding nodes, these nodes will
remember the *link* , and it can subsequently be used by referring to a hash representing it.
* | As a part of the *link request* , a Diffie-Hellman key exchange takes place, that sets up an
efficient symmetrically encrypted tunnel between the two nodes, using elliptic curve
cryptography. As such, this mode of communication is preferred, even for situations when
nodes can directly communicate, when the amount of data to be exchanged numbers in the
tens of packets.
* | When a *link* has been set up, it automatically provides message receipt functionality, so the
sending node can obtain verified confirmation that the information reached the intended
recipient.
In a moment, we will discuss the details of how this methodology is implemented, but let’s first
recap what purposes this methodology serves. We first ensure that the node answering our request
is actually the one we want to communicate with, and not a malicious actor pretending to be so.
At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in
terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will
also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is
more suitable to the application. The procedure also inserts the *link id* , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this *link id*.
The total bandwidth cost of setting up a link is 409 bytes (more info in the :ref:`Binary Packet Format<understanding-packetformat>` section). The amount of bandwidth used on keeping a link open is practically negligible, at 0.62 bits per second. Even on a slow 1200 bits per second packet radio channel, 100 concurrent links will still leave 95% channel capacity for actual data.
Pathfinding in Detail
^^^^^^^^^^^^^^^^^^^^^
The pathfinding method builds on the *announce* functionality discussed earlier. When an announce
is sent out by a node, it will be forwarded by any node receiving it, but according to some specific
rules:
* | If this announce has already been received before, ignore it.
* | Record into a table which node the announce was received from, and how many times in
total it has been retransmitted to get here. total it has been retransmitted to get here.
* | If the announce has been retransmitted *m+1* times, it will not be forwarded. By default, *m* is * | If the announce has been retransmitted *m+1* times, it will not be forwarded. By default, *m* is
set to 18. set to 18.
* | The announce will be assigned a delay *d* = c\ :sup:`h` seconds, where *c* is a decay constant, by * | The announce will be assigned a delay *d* = c\ :sup:`h` seconds, where *c* is a decay constant, and *h* is the amount of times this packet has already been forwarded.
default 2, and *h* is the amount of times this packet has already been forwarded.
* | The packet will be given a priority *p = 1/d*. * | The packet will be given a priority *p = 1/d*.
@ -385,10 +333,11 @@ rules:
not utilized by other traffic, the announce will be forwarded. not utilized by other traffic, the announce will be forwarded.
* | If no other nodes are heard retransmitting the announce with a greater hop count than when * | If no other nodes are heard retransmitting the announce with a greater hop count than when
it left this node, transmitting it will be retried *r* times. By default, *r* is set to 2. Retries follow it left this node, transmitting it will be retried *r* times. By default, *r* is set to 1. Retries
same rules as above, with the exception that it must wait for at least *d* = c\ :sup:`h+1` + t seconds, ie., follow same rules as above, with the exception that it must wait for at least *d* = c\ :sup:`h+1` +
the amount of time it would take the next node to retransmit the packet. By default, *t* is set to t + rand(0, rw) seconds. This amount of time is equal to the amount of time it would take the next
10. node to retransmit the packet, plus a random window. By default, *t* is set to 10 seconds, and the
random window *rw* is set to 10 seconds.
* | If a newer announce from the same destination arrives, while an identical one is already in * | If a newer announce from the same destination arrives, while an identical one is already in
the queue, the newest announce is discarded. If the newest announce contains different the queue, the newest announce is discarded. If the newest announce contains different
@ -407,14 +356,95 @@ distance of *Lavg =* 15 kilometers, an announce will be able to propagate outwar
kilometers in 34 minutes, and a *maximum announce radius* of 270 kilometers in approximately 3 kilometers in 34 minutes, and a *maximum announce radius* of 270 kilometers in approximately 3
days. days.
.. _understanding-paths:
Reaching the Destination
------------------------
In networks with changing topology and trustless connectivity, nodes need a way to establish
*verified connectivity* with each other. Since the network is assumed to be trustless, Reticulum
must provide a way to guarantee that the peer you are communicating with is actually who you
expect. Reticulum offers two ways to do this.
For exchanges of small amounts of information, Reticulum offers the *Packet* API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:
* | A packet is always created with an associated destination and some payload data. When the packet is sent
to a *single* destination type, Reticulum will automatically create an ephemeral encryption key, perform
an ECDH key exchange with the destinations public key, and encrypt the information.
* | It is important to note that this key exchange does not require any network traffic. The sender already
knows the public key of the destination from an earlier received *announce*, and can thus perform the ECDH
key exchange locally, before sending the packet.
* | The public part of the newly generated ephemeral key-pair is included with the encrypted token, and sent
along with the encrypted payload data in the packet.
* | When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the
packet.
* | A new ephemeral key is used for every packet sent in this way, and forward secrecy is guaranteed on a
per packet level.
* | Once the packet has been received and decrypted by the addressed destination, that destination can opt
to *prove* its receipt of the packet. It does this by calculating the SHA-256 hash of the received packet,
and signing this hash with it's Ed25519 signing key. Transport nodes in the network can then direct this
*proof* back to the packets origin, where the signature can be verified against the destinations known
public signing key.
* | In case the packet is addressed to a *group* destination type, the packet will be encrypted with the
pre-shared AES-128 key associated with the destination. In case the packet is addressed to a *plain*
destination type, the payload data will not be encrypted. Neither of these two destination types offer
forward secrecy. In general, it is recommended to always use the *single* destination type, unless it is
strictly necessary to use one of the others.
For exchanges of larger amounts of data, or when longer sessions of bidirectional communication is desired, Reticulum offers the *Link* API. To establish a *link*, the following process is employed:
* | First, the node that wishes to establish a link will send out a special packet, that
traverses the network and locates the desired destination. Along the way, the nodes that
forward the packet will take note of this *link request*.
* | Second, if the destination accepts the *link request* , it will send back a packet that proves the
authenticity of it’s identity (and the receipt of the link request) to the initiating node. All
nodes that initially forwarded the packet will also be able to verify this proof, and thus
accept the validity of the *link* throughout the network.
* | When the validity of the *link* has been accepted by forwarding nodes, these nodes will
remember the *link* , and it can subsequently be used by referring to a hash representing it.
* | As a part of the *link request* , a Diffie-Hellman key exchange takes place, that sets up an
efficiently encrypted tunnel between the two nodes, using elliptic curve cryptography. As such,
this mode of communication is preferred, even for situations when nodes can directly communicate,
when the amount of data to be exchanged numbers in the tens of packets.
* | When a *link* has been set up, it automatically provides message receipt functionality, through
the same *proof* mechanism discussed before, so the sending node can obtain verified confirmation
that the information reached the intended recipient.
In a moment, we will discuss the details of how this methodology is implemented, but let’s first
recap what purposes this methodology serves. We first ensure that the node answering our request
is actually the one we want to communicate with, and not a malicious actor pretending to be so.
At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in
terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will
also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is
more suitable to the application. The procedure also inserts the *link id* , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this *link id*.
The combined bandwidth cost of setting up a link is 3 packets totalling 409 bytes (more info in the
:ref:`Binary Packet Format<understanding-packetformat>` section). The amount of bandwidth used on keeping
a link open is practically negligible, at 0.62 bits per second. Even on a slow 1200 bits per second packet
radio channel, 100 concurrent links will still leave 95% channel capacity for actual data.
Link Establishment in Detail Link Establishment in Detail
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
After seeing how the conditions for finding a path through the network are created, we will now After exploring the basics of the announce mechanism, finding a path through the network, and an overview
explore how two nodes can establish reliable communications over multiple hops. The *link* in of the link establishment procedure, this section will go into greater detail about the Reticulum link
Reticulum terminology should not be viewed as a direct node-to-node link on the physical layer, but establishment process.
as an abstract channel, that can be open for any amount of time, and can span an arbitrary number
of hops, where information will be exchanged between two nodes. The *link* in Reticulum terminology should not be viewed as a direct node-to-node link on the
physical layer, but as an abstract channel, that can be open for any amount of time, and can span
an arbitrary number of hops, where information will be exchanged between two nodes.
* | When a node in the network wants to establish verified connectivity with another node, it * | When a node in the network wants to establish verified connectivity with another node, it
@ -427,25 +457,25 @@ of hops, where information will be exchanged between two nodes.
considered as single public key for simplicity in this explanation.* considered as single public key for simplicity in this explanation.*
* | The *link request* is addressed to the destination hash of the desired destination, and * | The *link request* is addressed to the destination hash of the desired destination, and
contains the following data: The newly generated X25519 public key *LKi*. The contents contains the following data: The newly generated X25519 public key *LKi*.
are encrypted with the RSA public key of the destination and tramsitted over the network.
* | The broadcasted packet will be directed through the network according to the rules laid out * | The broadcasted packet will be directed through the network according to the rules laid out
previously. previously.
* | Any node that forwards the link request will store a *link id* in it’s *link table* , along with the * | Any node that forwards the link request will store a *link id* in it’s *link table* , along with the
amount of hops the packet had taken when received. The link id is a hash of the entire link amount of hops the packet had taken when received. The link id is a hash of the entire link
request packet. If the path is not *proven* within some set amount of time, the entry will be request packet. If the link request packet is not *proven* by the addressed destination within some
dropped from the *link table* again. set amount of time, the entry will be dropped from the *link table* again.
* | When the destination receives the link request packet, it will decrypt it and decide whether to * | When the destination receives the link request packet, it will decide whether to accept the request.
accept the request. If it is accepted, the destination will also generate a new X25519 private/public If it is accepted, the destination will also generate a new X25519 private/public key pair, and
key pair, and perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used to encrypt the
to encrypt the channel, once it has been established. channel, once it has been established.
* | A *link proof* packet is now constructed and transmitted over the network. This packet is * | A *link proof* packet is now constructed and transmitted over the network. This packet is
addressed to the *link id* of the *link*. It contains the following data: The newly generated X25519 addressed to the *link id* of the *link*. It contains the following data: The newly generated X25519
public key *LKr* and an RSA-1024 signature of the *link id* and *LKr*. public key *LKr* and an Ed25519 signature of the *link id* and *LKr* made by the signing key of
the addressed destination.
* | By verifying this *link proof* packet, all nodes that originally transported the *link request* * | By verifying this *link proof* packet, all nodes that originally transported the *link request*
packet to the destination from the originator can now verify that the intended destination received packet to the destination from the originator can now verify that the intended destination received
@ -465,11 +495,6 @@ reveal any identifying information about itself. The link initiator remains comp
When using *links*, Reticulum will automatically verify all data sent over the link, and can also When using *links*, Reticulum will automatically verify all data sent over the link, and can also
automate retransmissions if *Resources* are used. automate retransmissions if *Resources* are used.
Proven Delivery
^^^^^^^^^^^^^^^
TODO: Write
.. _understanding-resources: .. _understanding-resources:
Resources Resources

1
docs/manual/index.html
View File

@ -69,6 +69,7 @@ the development of Reticulum itself.</p>
</ul> </ul>
</li> </li>
<li class="toctree-l2"><a class="reference internal" href="understanding.html#reticulum-transport">Reticulum Transport</a><ul> <li class="toctree-l2"><a class="reference internal" href="understanding.html#reticulum-transport">Reticulum Transport</a><ul>
<li class="toctree-l3"><a class="reference internal" href="understanding.html#the-announce-mechanism-in-detail">The Announce Mechanism in Detail</a></li>
<li class="toctree-l3"><a class="reference internal" href="understanding.html#reaching-the-destination">Reaching the Destination</a></li> <li class="toctree-l3"><a class="reference internal" href="understanding.html#reaching-the-destination">Reaching the Destination</a></li>
<li class="toctree-l3"><a class="reference internal" href="understanding.html#resources">Resources</a></li> <li class="toctree-l3"><a class="reference internal" href="understanding.html#resources">Resources</a></li>
</ul> </ul>

BIN
docs/manual/objects.inv
View File

Binary file not shown.

20
docs/manual/reference.html
View File

@ -16,8 +16,8 @@
<link rel="index" title="Index" href="genindex.html" /> <link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" /> <link rel="search" title="Search" href="search.html" />
<link rel="next" title="Understanding Reticulum" href="understanding.html" /> <link rel="next" title="Examples" href="examples.html" />
<link rel="prev" title="Examples" href="examples.html" /> <link rel="prev" title="Understanding Reticulum" href="understanding.html" />
</head><body> </head><body>
<div class="related" role="navigation" aria-label="related navigation"> <div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3> <h3>Navigation</h3>
@ -26,10 +26,10 @@
<a href="genindex.html" title="General Index" <a href="genindex.html" title="General Index"
accesskey="I">index</a></li> accesskey="I">index</a></li>
<li class="right" > <li class="right" >
<a href="understanding.html" title="Understanding Reticulum" <a href="examples.html" title="Examples"
accesskey="N">next</a> |</li> accesskey="N">next</a> |</li>
<li class="right" > <li class="right" >
<a href="examples.html" title="Examples" <a href="understanding.html" title="Understanding Reticulum"
accesskey="P">previous</a> |</li> accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Reticulum Network Stack 0.2.1 beta documentation</a> &#187;</li> <li class="nav-item nav-item-0"><a href="index.html">Reticulum Network Stack 0.2.1 beta documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">API Reference</a></li> <li class="nav-item nav-item-this"><a href="">API Reference</a></li>
@ -1020,11 +1020,11 @@ will announce it.</p>
</ul> </ul>
<h4>Previous topic</h4> <h4>Previous topic</h4>
<p class="topless"><a href="examples.html"
title="previous chapter">Examples</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="understanding.html" <p class="topless"><a href="understanding.html"
title="next chapter">Understanding Reticulum</a></p> title="previous chapter">Understanding Reticulum</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="examples.html"
title="next chapter">Examples</a></p>
<div role="note" aria-label="source link"> <div role="note" aria-label="source link">
<h3>This Page</h3> <h3>This Page</h3>
<ul class="this-page-menu"> <ul class="this-page-menu">
@ -1053,10 +1053,10 @@ will announce it.</p>
<a href="genindex.html" title="General Index" <a href="genindex.html" title="General Index"
>index</a></li> >index</a></li>
<li class="right" > <li class="right" >
<a href="understanding.html" title="Understanding Reticulum" <a href="examples.html" title="Examples"
>next</a> |</li> >next</a> |</li>
<li class="right" > <li class="right" >
<a href="examples.html" title="Examples" <a href="understanding.html" title="Understanding Reticulum"
>previous</a> |</li> >previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Reticulum Network Stack 0.2.1 beta documentation</a> &#187;</li> <li class="nav-item nav-item-0"><a href="index.html">Reticulum Network Stack 0.2.1 beta documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">API Reference</a></li> <li class="nav-item nav-item-this"><a href="">API Reference</a></li>

2
docs/manual/searchindex.js
View File

File diff suppressed because one or more lines are too long

248
docs/manual/understanding.html
View File

@ -205,12 +205,14 @@ when a node is directly reachable.</p>
</ul> </ul>
<div class="section" id="destination-naming"> <div class="section" id="destination-naming">
<span id="understanding-destinationnaming"></span><h4>Destination Naming<a class="headerlink" href="#destination-naming" title="Permalink to this headline"></a></h4> <span id="understanding-destinationnaming"></span><h4>Destination Naming<a class="headerlink" href="#destination-naming" title="Permalink to this headline"></a></h4>
<p>Destinations are created and named in an easy to understand dotted notation of <em>aspects</em> , and <p>Destinations are created and named in an easy to understand dotted notation of <em>aspects</em>, and
represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The
top level aspect should always be a unique identifier for the application using the destination. top level aspect should always be a unique identifier for the application using the destination.
The next levels of aspects can be defined in any way by the creator of the application. For example, The next levels of aspects can be defined in any way by the creator of the application.</p>
a destination for a environmental monitoring application could be made up of the application name, a <p>Aspects can be as long and as plentiful as required, and a resulting long destination name will not
device type and measurement type, like this:</p> impact efficiency, as names are always represented as truncated SHA-256 hashes on the network.</p>
<p>As an example, a destination for a environmental monitoring application could be made up of the
application name, a device type and measurement type, like this:</p>
<div class="highlight-text notranslate"><div class="highlight"><pre><span></span>app name : environmentlogger <div class="highlight-text notranslate"><div class="highlight"><pre><span></span>app name : environmentlogger
aspects : remotesensor, temperature aspects : remotesensor, temperature
@ -246,9 +248,8 @@ receives.</p>
</dl> </dl>
</li> </li>
<li><dl class="simple"> <li><dl class="simple">
<dt><strong>Group</strong></dt><dd><p>When private communication between two or more endpoints is needed. More efficient in <dt><strong>Group</strong></dt><dd><p>When private communication between two or more endpoints is needed. Supports multiple hops
data usage than <em>single</em> destinations. Supports multiple hops indirectly, but must first be indirectly, but must first be established through a <em>single</em> destination.</p>
established through a <em>single</em> destination.</p>
</dd> </dd>
</dl> </dl>
</li> </li>
@ -264,9 +265,9 @@ nodes aware of your destinations public key, called the <em>announce</em>. It is
an unknown public key from the network, as all participating nodes serve as a distributed ledger an unknown public key from the network, as all participating nodes serve as a distributed ledger
of public keys.</p> of public keys.</p>
<p>Note that public key information can be shared and verified in many other ways than using the <p>Note that public key information can be shared and verified in many other ways than using the
built-in methodology, and that it is therefore not required to use the announce/request functionality. built-in <em>announce</em> functionality, and that it is therefore not required to use the announce/request
It is by far the easiest though, and should definitely be used if there is not a good reason for functionality to obtain public keys. It is by far the easiest though, and should definitely be used
doing it differently.</p> if there is not a good reason for doing it differently.</p>
</div> </div>
</div> </div>
<div class="section" id="public-key-announcements"> <div class="section" id="public-key-announcements">
@ -282,7 +283,7 @@ contain the following information:</p>
<li><p>The announcers public key</p></li> <li><p>The announcers public key</p></li>
<li><p>Application specific data, in this case the users nickname and availability status</p></li> <li><p>Application specific data, in this case the users nickname and availability status</p></li>
<li><p>A random blob, making each new announce unique</p></li> <li><p>A random blob, making each new announce unique</p></li>
<li><p>A signature of the above information, verifying authenticity</p></li> <li><p>An Ed25519 signature of the above information, verifying authenticity</p></li>
</ul> </ul>
<p>With this information, any Reticulum node that receives it will be able to reconstruct an outgoing <p>With this information, any Reticulum node that receives it will be able to reconstruct an outgoing
destination to securely communicate with that destination. You might have noticed that there is one destination to securely communicate with that destination. You might have noticed that there is one
@ -290,8 +291,9 @@ piece of information lacking to reconstruct full knowledge of the announced dest
the aspect names of the destination. These are intentionally left out to save bandwidth, since they the aspect names of the destination. These are intentionally left out to save bandwidth, since they
will be implicit in almost all cases. If a destination name is not entirely implicit, information can be will be implicit in almost all cases. If a destination name is not entirely implicit, information can be
included in the application specific data part that will allow the receiver to infer the naming.</p> included in the application specific data part that will allow the receiver to infer the naming.</p>
<p>It is important to note that announcements will be forwarded throughout the network according to a <p>It is important to note that announces will be forwarded throughout the network according to a
certain pattern. This will be detailed later.</p> certain pattern. This will be detailed in the section
<a class="reference internal" href="#understanding-announce"><span class="std std-ref">The Announce Mechanism in Detail</span></a>.</p>
<p>Seeing how <em>single</em> destinations are always tied to a private/public key pair leads us to the next topic.</p> <p>Seeing how <em>single</em> destinations are always tied to a private/public key pair leads us to the next topic.</p>
</div> </div>
<div class="section" id="understanding-identities"> <div class="section" id="understanding-identities">
@ -308,16 +310,16 @@ automatically. This may be desirable in some situations, but often you will prob
the identity first, and then link it to created destinations.</p> the identity first, and then link it to created destinations.</p>
<p>Building upon the simple messenger example, we could use an identity to represent the user of the <p>Building upon the simple messenger example, we could use an identity to represent the user of the
application. Destinations created will then be linked to this identity to allow communication to application. Destinations created will then be linked to this identity to allow communication to
reach the user. In such a case it is of great importance to store the user’s identity securely and reach the user. In all cases it is of great importance to store the private keys associated with any
privately.</p> Reticulum Identity securely and privately.</p>
</div> </div>
<div class="section" id="getting-further"> <div class="section" id="getting-further">
<span id="understanding-gettingfurther"></span><h3>Getting Further<a class="headerlink" href="#getting-further" title="Permalink to this headline"></a></h3> <span id="understanding-gettingfurther"></span><h3>Getting Further<a class="headerlink" href="#getting-further" title="Permalink to this headline"></a></h3>
<p>The above functions and principles form the core of Reticulum, and would suffice to create <p>The above functions and principles form the core of Reticulum, and would suffice to create
functional networked applications in local clusters, for example over radio links where all interested functional networked applications in local clusters, for example over radio links where all interested
nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple
hops in the network. In the next sections, two concepts that allow this will be introduced, <em>paths</em> and hops in the network.</p>
<em>links</em>.</p> <p>In the following sections, two concepts that allow this will be introduced, <em>paths</em> and <em>links</em>.</p>
</div> </div>
</div> </div>
<div class="section" id="reticulum-transport"> <div class="section" id="reticulum-transport">
@ -331,7 +333,70 @@ very limited. Existing routing protocols like BGP or OSPF carry too much overhea
useable over bandwidth-limited, high-latency links.</p> useable over bandwidth-limited, high-latency links.</p>
<p>To overcome such challenges, Reticulum’s <em>Transport</em> system uses public-key cryptography to <p>To overcome such challenges, Reticulum’s <em>Transport</em> system uses public-key cryptography to
implement the concept of <em>paths</em> that allow discovery of how to get information to a certain implement the concept of <em>paths</em> that allow discovery of how to get information to a certain
destination, and <em>resources</em> that help make reliable data transfer more efficient.</p> destination. It is important to note that no single node in a Reticulum network knows the complete
path to a destination. Every Transport node participating in a Reticulum network will only
know what the most direct way to get a packet one hop closer to it’s destination is.</p>
<div class="section" id="the-announce-mechanism-in-detail">
<span id="understanding-announce"></span><h3>The Announce Mechanism in Detail<a class="headerlink" href="#the-announce-mechanism-in-detail" title="Permalink to this headline"></a></h3>
<p>When an <em>announce</em> is transmitted by a node, it will be forwarded by any node receiving it, but
according to some specific rules:</p>
<ul>
<li><div class="line-block">
<div class="line">If this exact announce has already been received before, ignore it.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If not, record into a table which node the announce was received from, and how many times in
total it has been retransmitted to get here.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If the announce has been retransmitted <em>m+1</em> times, it will not be forwarded. By default, <em>m</em> is
set to 18.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">The announce will be assigned a delay <em>d</em> = c<sup>h</sup> seconds, where <em>c</em> is a decay constant, and <em>h</em> is the amount of times this packet has already been forwarded.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">The packet will be given a priority <em>p = 1/d</em>.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If at least <em>d</em> seconds has passed since the announce was received, and no other packets with a
priority higher than <em>p</em> are waiting in the queue (see Packet Prioritisation), and the channel is
not utilized by other traffic, the announce will be forwarded.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If no other nodes are heard retransmitting the announce with a greater hop count than when
it left this node, transmitting it will be retried <em>r</em> times. By default, <em>r</em> is set to 1. Retries
follow same rules as above, with the exception that it must wait for at least <em>d</em> = c<sup>h+1</sup> +
t + rand(0, rw) seconds. This amount of time is equal to the amount of time it would take the next
node to retransmit the packet, plus a random window. By default, <em>t</em> is set to 10 seconds, and the
random window <em>rw</em> is set to 10 seconds.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If a newer announce from the same destination arrives, while an identical one is already in
the queue, the newest announce is discarded. If the newest announce contains different
application specific data, it will replace the old announce, but will use <em>d</em> and <em>p</em> of the old
announce.</div>
</div>
</li>
</ul>
<p>Once an announce has reached a node in the network, any other node in direct contact with that
node will be able to reach the destination the announce originated from, simply by sending a packet
addressed to that destination. Any node with knowledge of the announce will be able to direct the
packet towards the destination by looking up the next node with the shortest amount of hops to the
destination.</p>
<p>According to these rules and default constants, an announce will propagate throughout the network
in a predictable way. In an example network utilising the default constants, and with an average link
distance of <em>Lavg =</em> 15 kilometers, an announce will be able to propagate outwards to a radius of 180
kilometers in 34 minutes, and a <em>maximum announce radius</em> of 270 kilometers in approximately 3
days.</p>
</div>
<div class="section" id="reaching-the-destination"> <div class="section" id="reaching-the-destination">
<span id="understanding-paths"></span><h3>Reaching the Destination<a class="headerlink" href="#reaching-the-destination" title="Permalink to this headline"></a></h3> <span id="understanding-paths"></span><h3>Reaching the Destination<a class="headerlink" href="#reaching-the-destination" title="Permalink to this headline"></a></h3>
<p>In networks with changing topology and trustless connectivity, nodes need a way to establish <p>In networks with changing topology and trustless connectivity, nodes need a way to establish
@ -341,27 +406,46 @@ expect. Reticulum offers two ways to do this.</p>
<p>For exchanges of small amounts of information, Reticulum offers the <em>Packet</em> API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:</p> <p>For exchanges of small amounts of information, Reticulum offers the <em>Packet</em> API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:</p>
<ul> <ul>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">A packet is always created with an associated destination and some payload data. When the packet is sent to a <em>single</em> destination type, Reticulum will automatically create an ephemeral encryption key, perform an ECDH key exchange with the destinations public key, and encrypt the information.</div> <div class="line">A packet is always created with an associated destination and some payload data. When the packet is sent
to a <em>single</em> destination type, Reticulum will automatically create an ephemeral encryption key, perform
an ECDH key exchange with the destinations public key, and encrypt the information.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">It is important to note that this key exchange does not require any network traffic. The sender already knows the public key of the destination from an earlier received <em>announce</em>, and can thus perform the ECDH key exchange locally.</div> <div class="line">It is important to note that this key exchange does not require any network traffic. The sender already
knows the public key of the destination from an earlier received <em>announce</em>, and can thus perform the ECDH
key exchange locally, before sending the packet.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">The public key part of the newly generated ephemeral key is included with the encrypted token, and sent along with the encrypted payload data in the packet.</div> <div class="line">The public part of the newly generated ephemeral key-pair is included with the encrypted token, and sent
along with the encrypted payload data in the packet.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the packet.</div> <div class="line">When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the
packet.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">A new ephemeral key is used for every packet sent in this way, and forward secrecy is guaranteed on a per packet level.</div> <div class="line">A new ephemeral key is used for every packet sent in this way, and forward secrecy is guaranteed on a
per packet level.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">In case the packet is addressed to a <em>group</em> destination type, the packet will be encrypted with the pre-shared AES-128 key associated with the destination. In case the packet is addressed to a <em>plain</em> destination type, the payload data will not be encrypted. Neither of these two destination types offer forward secrecy. In general, it is recommended to always use the <em>single</em> destination type, unless it is strictly necessary to use one of the others.</div> <div class="line">Once the packet has been received and decrypted by the addressed destination, that destination can opt
to <em>prove</em> its receipt of the packet. It does this by calculating the SHA-256 hash of the received packet,
and signing this hash with it’s Ed25519 signing key. Transport nodes in the network can then direct this
<em>proof</em> back to the packets origin, where the signature can be verified against the destinations known
public signing key.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">In case the packet is addressed to a <em>group</em> destination type, the packet will be encrypted with the
pre-shared AES-128 key associated with the destination. In case the packet is addressed to a <em>plain</em>
destination type, the payload data will not be encrypted. Neither of these two destination types offer
forward secrecy. In general, it is recommended to always use the <em>single</em> destination type, unless it is
strictly necessary to use one of the others.</div>
</div> </div>
</li> </li>
</ul> </ul>
@ -387,16 +471,15 @@ remember the <em>link</em> , and it can subsequently be used by referring to a h
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">As a part of the <em>link request</em> , a Diffie-Hellman key exchange takes place, that sets up an <div class="line">As a part of the <em>link request</em> , a Diffie-Hellman key exchange takes place, that sets up an
efficient symmetrically encrypted tunnel between the two nodes, using elliptic curve efficiently encrypted tunnel between the two nodes, using elliptic curve cryptography. As such,
cryptography. As such, this mode of communication is preferred, even for situations when this mode of communication is preferred, even for situations when nodes can directly communicate,
nodes can directly communicate, when the amount of data to be exchanged numbers in the when the amount of data to be exchanged numbers in the tens of packets.</div>
tens of packets.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">When a <em>link</em> has been set up, it automatically provides message receipt functionality, so the <div class="line">When a <em>link</em> has been set up, it automatically provides message receipt functionality, through
sending node can obtain verified confirmation that the information reached the intended the same <em>proof</em> mechanism discussed before, so the sending node can obtain verified confirmation
recipient.</div> that the information reached the intended recipient.</div>
</div> </div>
</li> </li>
</ul> </ul>
@ -407,76 +490,18 @@ At the same time we establish an efficient encrypted channel. The setup of this
terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will
also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is
more suitable to the application. The procedure also inserts the <em>link id</em> , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this <em>link id</em>.</p> more suitable to the application. The procedure also inserts the <em>link id</em> , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this <em>link id</em>.</p>
<p>The total bandwidth cost of setting up a link is 409 bytes (more info in the <a class="reference internal" href="#understanding-packetformat"><span class="std std-ref">Binary Packet Format</span></a> section). The amount of bandwidth used on keeping a link open is practically negligible, at 0.62 bits per second. Even on a slow 1200 bits per second packet radio channel, 100 concurrent links will still leave 95% channel capacity for actual data.</p> <p>The combined bandwidth cost of setting up a link is 3 packets totalling 409 bytes (more info in the
<div class="section" id="pathfinding-in-detail"> <a class="reference internal" href="#understanding-packetformat"><span class="std std-ref">Binary Packet Format</span></a> section). The amount of bandwidth used on keeping
<h4>Pathfinding in Detail<a class="headerlink" href="#pathfinding-in-detail" title="Permalink to this headline"></a></h4> a link open is practically negligible, at 0.62 bits per second. Even on a slow 1200 bits per second packet
<p>The pathfinding method builds on the <em>announce</em> functionality discussed earlier. When an announce radio channel, 100 concurrent links will still leave 95% channel capacity for actual data.</p>
is sent out by a node, it will be forwarded by any node receiving it, but according to some specific
rules:</p>
<ul>
<li><div class="line-block">
<div class="line">If this announce has already been received before, ignore it.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">Record into a table which node the announce was received from, and how many times in
total it has been retransmitted to get here.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If the announce has been retransmitted <em>m+1</em> times, it will not be forwarded. By default, <em>m</em> is
set to 18.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">The announce will be assigned a delay <em>d</em> = c<sup>h</sup> seconds, where <em>c</em> is a decay constant, by
default 2, and <em>h</em> is the amount of times this packet has already been forwarded.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">The packet will be given a priority <em>p = 1/d</em>.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If at least <em>d</em> seconds has passed since the announce was received, and no other packets with a
priority higher than <em>p</em> are waiting in the queue (see Packet Prioritisation), and the channel is
not utilized by other traffic, the announce will be forwarded.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If no other nodes are heard retransmitting the announce with a greater hop count than when
it left this node, transmitting it will be retried <em>r</em> times. By default, <em>r</em> is set to 2. Retries follow
same rules as above, with the exception that it must wait for at least <em>d</em> = c<sup>h+1</sup> + t seconds, ie.,
the amount of time it would take the next node to retransmit the packet. By default, <em>t</em> is set to
10.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">If a newer announce from the same destination arrives, while an identical one is already in
the queue, the newest announce is discarded. If the newest announce contains different
application specific data, it will replace the old announce, but will use <em>d</em> and <em>p</em> of the old
announce.</div>
</div>
</li>
</ul>
<p>Once an announce has reached a node in the network, any other node in direct contact with that
node will be able to reach the destination the announce originated from, simply by sending a packet
addressed to that destination. Any node with knowledge of the announce will be able to direct the
packet towards the destination by looking up the next node with the shortest amount of hops to the
destination.</p>
<p>According to these rules and default constants, an announce will propagate throughout the network
in a predictable way. In an example network utilising the default constants, and with an average link
distance of <em>Lavg =</em> 15 kilometers, an announce will be able to propagate outwards to a radius of 180
kilometers in 34 minutes, and a <em>maximum announce radius</em> of 270 kilometers in approximately 3
days.</p>
</div>
<div class="section" id="link-establishment-in-detail"> <div class="section" id="link-establishment-in-detail">
<h4>Link Establishment in Detail<a class="headerlink" href="#link-establishment-in-detail" title="Permalink to this headline"></a></h4> <h4>Link Establishment in Detail<a class="headerlink" href="#link-establishment-in-detail" title="Permalink to this headline"></a></h4>
<p>After seeing how the conditions for finding a path through the network are created, we will now <p>After exploring the basics of the announce mechanism, finding a path through the network, and an overview
explore how two nodes can establish reliable communications over multiple hops. The <em>link</em> in of the link establishment procedure, this section will go into greater detail about the Reticulum link
Reticulum terminology should not be viewed as a direct node-to-node link on the physical layer, but establishment process.</p>
as an abstract channel, that can be open for any amount of time, and can span an arbitrary number <p>The <em>link</em> in Reticulum terminology should not be viewed as a direct node-to-node link on the
of hops, where information will be exchanged between two nodes.</p> physical layer, but as an abstract channel, that can be open for any amount of time, and can span
an arbitrary number of hops, where information will be exchanged between two nodes.</p>
<ul> <ul>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">When a node in the network wants to establish verified connectivity with another node, it <div class="line">When a node in the network wants to establish verified connectivity with another node, it
@ -491,8 +516,7 @@ considered as single public key for simplicity in this explanation.</em></div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">The <em>link request</em> is addressed to the destination hash of the desired destination, and <div class="line">The <em>link request</em> is addressed to the destination hash of the desired destination, and
contains the following data: The newly generated X25519 public key <em>LKi</em>. The contents contains the following data: The newly generated X25519 public key <em>LKi</em>.</div>
are encrypted with the RSA public key of the destination and tramsitted over the network.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
@ -503,21 +527,22 @@ previously.</div>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">Any node that forwards the link request will store a <em>link id</em> in it’s <em>link table</em> , along with the <div class="line">Any node that forwards the link request will store a <em>link id</em> in it’s <em>link table</em> , along with the
amount of hops the packet had taken when received. The link id is a hash of the entire link amount of hops the packet had taken when received. The link id is a hash of the entire link
request packet. If the path is not <em>proven</em> within some set amount of time, the entry will be request packet. If the link request packet is not <em>proven</em> by the addressed destination within some
dropped from the <em>link table</em> again.</div> set amount of time, the entry will be dropped from the <em>link table</em> again.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">When the destination receives the link request packet, it will decrypt it and decide whether to <div class="line">When the destination receives the link request packet, it will decide whether to accept the request.
accept the request. If it is accepted, the destination will also generate a new X25519 private/public If it is accepted, the destination will also generate a new X25519 private/public key pair, and
key pair, and perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used to encrypt the
to encrypt the channel, once it has been established.</div> channel, once it has been established.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
<div class="line">A <em>link proof</em> packet is now constructed and transmitted over the network. This packet is <div class="line">A <em>link proof</em> packet is now constructed and transmitted over the network. This packet is
addressed to the <em>link id</em> of the <em>link</em>. It contains the following data: The newly generated X25519 addressed to the <em>link id</em> of the <em>link</em>. It contains the following data: The newly generated X25519
public key <em>LKr</em> and an RSA-1024 signature of the <em>link id</em> and <em>LKr</em>.</div> public key <em>LKr</em> and an Ed25519 signature of the <em>link id</em> and <em>LKr</em> made by the signing key of
the addressed destination.</div>
</div> </div>
</li> </li>
<li><div class="line-block"> <li><div class="line-block">
@ -541,10 +566,6 @@ reveal any identifying information about itself. The link initiator remains comp
<p>When using <em>links</em>, Reticulum will automatically verify all data sent over the link, and can also <p>When using <em>links</em>, Reticulum will automatically verify all data sent over the link, and can also
automate retransmissions if <em>Resources</em> are used.</p> automate retransmissions if <em>Resources</em> are used.</p>
</div> </div>
<div class="section" id="proven-delivery">
<h4>Proven Delivery<a class="headerlink" href="#proven-delivery" title="Permalink to this headline"></a></h4>
<p>TODO: Write</p>
</div>
</div> </div>
<div class="section" id="resources"> <div class="section" id="resources">
<span id="understanding-resources"></span><h3>Resources<a class="headerlink" href="#resources" title="Permalink to this headline"></a></h3> <span id="understanding-resources"></span><h3>Resources<a class="headerlink" href="#resources" title="Permalink to this headline"></a></h3>
@ -774,10 +795,9 @@ proof 11
</ul> </ul>
</li> </li>
<li><a class="reference internal" href="#reticulum-transport">Reticulum Transport</a><ul> <li><a class="reference internal" href="#reticulum-transport">Reticulum Transport</a><ul>
<li><a class="reference internal" href="#the-announce-mechanism-in-detail">The Announce Mechanism in Detail</a></li>
<li><a class="reference internal" href="#reaching-the-destination">Reaching the Destination</a><ul> <li><a class="reference internal" href="#reaching-the-destination">Reaching the Destination</a><ul>
<li><a class="reference internal" href="#pathfinding-in-detail">Pathfinding in Detail</a></li>
<li><a class="reference internal" href="#link-establishment-in-detail">Link Establishment in Detail</a></li> <li><a class="reference internal" href="#link-establishment-in-detail">Link Establishment in Detail</a></li>
<li><a class="reference internal" href="#proven-delivery">Proven Delivery</a></li>
</ul> </ul>
</li> </li>
<li><a class="reference internal" href="#resources">Resources</a></li> <li><a class="reference internal" href="#resources">Resources</a></li>

241
docs/source/understanding.rst
View File

@ -157,12 +157,16 @@ destinations. Reticulum uses three different basic destination types, and one sp
Destination Naming Destination Naming
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
Destinations are created and named in an easy to understand dotted notation of *aspects* , and Destinations are created and named in an easy to understand dotted notation of *aspects*, and
represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The represented on the network as a hash of this value. The hash is a SHA-256 truncated to 80 bits. The
top level aspect should always be a unique identifier for the application using the destination. top level aspect should always be a unique identifier for the application using the destination.
The next levels of aspects can be defined in any way by the creator of the application. For example, The next levels of aspects can be defined in any way by the creator of the application.
a destination for a environmental monitoring application could be made up of the application name, a
device type and measurement type, like this: Aspects can be as long and as plentiful as required, and a resulting long destination name will not
impact efficiency, as names are always represented as truncated SHA-256 hashes on the network.
As an example, a destination for a environmental monitoring application could be made up of the
application name, a device type and measurement type, like this:
.. code-block:: text .. code-block:: text
@ -201,9 +205,8 @@ To recap, the different destination types should be used in the following situat
* **Single** * **Single**
When private communication between two endpoints is needed. Supports multiple hops. When private communication between two endpoints is needed. Supports multiple hops.
* **Group** * **Group**
When private communication between two or more endpoints is needed. More efficient in When private communication between two or more endpoints is needed. Supports multiple hops
data usage than *single* destinations. Supports multiple hops indirectly, but must first be indirectly, but must first be established through a *single* destination.
established through a *single* destination.
* **Plain** * **Plain**
When plain-text communication is desirable, for example when broadcasting information. When plain-text communication is desirable, for example when broadcasting information.
@ -214,9 +217,9 @@ an unknown public key from the network, as all participating nodes serve as a di
of public keys. of public keys.
Note that public key information can be shared and verified in many other ways than using the Note that public key information can be shared and verified in many other ways than using the
built-in methodology, and that it is therefore not required to use the announce/request functionality. built-in *announce* functionality, and that it is therefore not required to use the announce/request
It is by far the easiest though, and should definitely be used if there is not a good reason for functionality to obtain public keys. It is by far the easiest though, and should definitely be used
doing it differently. if there is not a good reason for doing it differently.
.. _understanding-keyannouncements: .. _understanding-keyannouncements:
@ -235,7 +238,7 @@ contain the following information:
* The announcers public key * The announcers public key
* Application specific data, in this case the users nickname and availability status * Application specific data, in this case the users nickname and availability status
* A random blob, making each new announce unique * A random blob, making each new announce unique
* A signature of the above information, verifying authenticity * An Ed25519 signature of the above information, verifying authenticity
With this information, any Reticulum node that receives it will be able to reconstruct an outgoing With this information, any Reticulum node that receives it will be able to reconstruct an outgoing
destination to securely communicate with that destination. You might have noticed that there is one destination to securely communicate with that destination. You might have noticed that there is one
@ -244,8 +247,9 @@ the aspect names of the destination. These are intentionally left out to save ba
will be implicit in almost all cases. If a destination name is not entirely implicit, information can be will be implicit in almost all cases. If a destination name is not entirely implicit, information can be
included in the application specific data part that will allow the receiver to infer the naming. included in the application specific data part that will allow the receiver to infer the naming.
It is important to note that announcements will be forwarded throughout the network according to a It is important to note that announces will be forwarded throughout the network according to a
certain pattern. This will be detailed later. certain pattern. This will be detailed in the section
:ref:`The Announce Mechanism in Detail<understanding-announce>`.
Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic. Seeing how *single* destinations are always tied to a private/public key pair leads us to the next topic.
@ -268,8 +272,8 @@ the identity first, and then link it to created destinations.
Building upon the simple messenger example, we could use an identity to represent the user of the Building upon the simple messenger example, we could use an identity to represent the user of the
application. Destinations created will then be linked to this identity to allow communication to application. Destinations created will then be linked to this identity to allow communication to
reach the user. In such a case it is of great importance to store the user’s identity securely and reach the user. In all cases it is of great importance to store the private keys associated with any
privately. Reticulum Identity securely and privately.
.. _understanding-gettingfurther: .. _understanding-gettingfurther:
@ -279,8 +283,9 @@ Getting Further
The above functions and principles form the core of Reticulum, and would suffice to create The above functions and principles form the core of Reticulum, and would suffice to create
functional networked applications in local clusters, for example over radio links where all interested functional networked applications in local clusters, for example over radio links where all interested
nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple nodes can directly hear each other. But to be truly useful, we need a way to direct traffic over multiple
hops in the network. In the next sections, two concepts that allow this will be introduced, *paths* and hops in the network.
*links*.
In the following sections, two concepts that allow this will be introduced, *paths* and *links*.
.. _understanding-transport: .. _understanding-transport:
@ -298,85 +303,28 @@ useable over bandwidth-limited, high-latency links.
To overcome such challenges, Reticulum’s *Transport* system uses public-key cryptography to To overcome such challenges, Reticulum’s *Transport* system uses public-key cryptography to
implement the concept of *paths* that allow discovery of how to get information to a certain implement the concept of *paths* that allow discovery of how to get information to a certain
destination, and *resources* that help make reliable data transfer more efficient. destination. It is important to note that no single node in a Reticulum network knows the complete
path to a destination. Every Transport node participating in a Reticulum network will only
know what the most direct way to get a packet one hop closer to it's destination is.
.. _understanding-paths: .. _understanding-announce:
Reaching the Destination The Announce Mechanism in Detail
------------------------ --------------------------------
In networks with changing topology and trustless connectivity, nodes need a way to establish When an *announce* is transmitted by a node, it will be forwarded by any node receiving it, but
*verified connectivity* with each other. Since the network is assumed to be trustless, Reticulum according to some specific rules:
must provide a way to guarantee that the peer you are communicating with is actually who you
expect. Reticulum offers two ways to do this.
For exchanges of small amounts of information, Reticulum offers the *Packet* API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:
* | A packet is always created with an associated destination and some payload data. When the packet is sent to a *single* destination type, Reticulum will automatically create an ephemeral encryption key, perform an ECDH key exchange with the destinations public key, and encrypt the information.
* | It is important to note that this key exchange does not require any network traffic. The sender already knows the public key of the destination from an earlier received *announce*, and can thus perform the ECDH key exchange locally.
* | The public key part of the newly generated ephemeral key is included with the encrypted token, and sent along with the encrypted payload data in the packet.
* | When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the packet.
* | A new ephemeral key is used for every packet sent in this way, and forward secrecy is guaranteed on a per packet level.
* | In case the packet is addressed to a *group* destination type, the packet will be encrypted with the pre-shared AES-128 key associated with the destination. In case the packet is addressed to a *plain* destination type, the payload data will not be encrypted. Neither of these two destination types offer forward secrecy. In general, it is recommended to always use the *single* destination type, unless it is strictly necessary to use one of the others.
For exchanges of larger amounts of data, or when longer sessions of bidirectional communication is desired, Reticulum offers the *Link* API. To establish a *link*, the following process is employed: * | If this exact announce has already been received before, ignore it.
* | First, the node that wishes to establish a link will send out a special packet, that * | If not, record into a table which node the announce was received from, and how many times in
traverses the network and locates the desired destination. Along the way, the nodes that
forward the packet will take note of this *link request*.
* | Second, if the destination accepts the *link request* , it will send back a packet that proves the
authenticity of it’s identity (and the receipt of the link request) to the initiating node. All
nodes that initially forwarded the packet will also be able to verify this proof, and thus
accept the validity of the *link* throughout the network.
* | When the validity of the *link* has been accepted by forwarding nodes, these nodes will
remember the *link* , and it can subsequently be used by referring to a hash representing it.
* | As a part of the *link request* , a Diffie-Hellman key exchange takes place, that sets up an
efficient symmetrically encrypted tunnel between the two nodes, using elliptic curve
cryptography. As such, this mode of communication is preferred, even for situations when
nodes can directly communicate, when the amount of data to be exchanged numbers in the
tens of packets.
* | When a *link* has been set up, it automatically provides message receipt functionality, so the
sending node can obtain verified confirmation that the information reached the intended
recipient.
In a moment, we will discuss the details of how this methodology is implemented, but let’s first
recap what purposes this methodology serves. We first ensure that the node answering our request
is actually the one we want to communicate with, and not a malicious actor pretending to be so.
At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in
terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will
also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is
more suitable to the application. The procedure also inserts the *link id* , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this *link id*.
The total bandwidth cost of setting up a link is 409 bytes (more info in the :ref:`Binary Packet Format<understanding-packetformat>` section). The amount of bandwidth used on keeping a link open is practically negligible, at 0.62 bits per second. Even on a slow 1200 bits per second packet radio channel, 100 concurrent links will still leave 95% channel capacity for actual data.
Pathfinding in Detail
^^^^^^^^^^^^^^^^^^^^^
The pathfinding method builds on the *announce* functionality discussed earlier. When an announce
is sent out by a node, it will be forwarded by any node receiving it, but according to some specific
rules:
* | If this announce has already been received before, ignore it.
* | Record into a table which node the announce was received from, and how many times in
total it has been retransmitted to get here. total it has been retransmitted to get here.
* | If the announce has been retransmitted *m+1* times, it will not be forwarded. By default, *m* is * | If the announce has been retransmitted *m+1* times, it will not be forwarded. By default, *m* is
set to 18. set to 18.
* | The announce will be assigned a delay *d* = c\ :sup:`h` seconds, where *c* is a decay constant, by * | The announce will be assigned a delay *d* = c\ :sup:`h` seconds, where *c* is a decay constant, and *h* is the amount of times this packet has already been forwarded.
default 2, and *h* is the amount of times this packet has already been forwarded.
* | The packet will be given a priority *p = 1/d*. * | The packet will be given a priority *p = 1/d*.
@ -385,10 +333,11 @@ rules:
not utilized by other traffic, the announce will be forwarded. not utilized by other traffic, the announce will be forwarded.
* | If no other nodes are heard retransmitting the announce with a greater hop count than when * | If no other nodes are heard retransmitting the announce with a greater hop count than when
it left this node, transmitting it will be retried *r* times. By default, *r* is set to 2. Retries follow it left this node, transmitting it will be retried *r* times. By default, *r* is set to 1. Retries
same rules as above, with the exception that it must wait for at least *d* = c\ :sup:`h+1` + t seconds, ie., follow same rules as above, with the exception that it must wait for at least *d* = c\ :sup:`h+1` +
the amount of time it would take the next node to retransmit the packet. By default, *t* is set to t + rand(0, rw) seconds. This amount of time is equal to the amount of time it would take the next
10. node to retransmit the packet, plus a random window. By default, *t* is set to 10 seconds, and the
random window *rw* is set to 10 seconds.
* | If a newer announce from the same destination arrives, while an identical one is already in * | If a newer announce from the same destination arrives, while an identical one is already in
the queue, the newest announce is discarded. If the newest announce contains different the queue, the newest announce is discarded. If the newest announce contains different
@ -407,14 +356,95 @@ distance of *Lavg =* 15 kilometers, an announce will be able to propagate outwar
kilometers in 34 minutes, and a *maximum announce radius* of 270 kilometers in approximately 3 kilometers in 34 minutes, and a *maximum announce radius* of 270 kilometers in approximately 3
days. days.
.. _understanding-paths:
Reaching the Destination
------------------------
In networks with changing topology and trustless connectivity, nodes need a way to establish
*verified connectivity* with each other. Since the network is assumed to be trustless, Reticulum
must provide a way to guarantee that the peer you are communicating with is actually who you
expect. Reticulum offers two ways to do this.
For exchanges of small amounts of information, Reticulum offers the *Packet* API, which works exactly like you would expect - on a per packet level. The following process is employed when sending a packet:
* | A packet is always created with an associated destination and some payload data. When the packet is sent
to a *single* destination type, Reticulum will automatically create an ephemeral encryption key, perform
an ECDH key exchange with the destinations public key, and encrypt the information.
* | It is important to note that this key exchange does not require any network traffic. The sender already
knows the public key of the destination from an earlier received *announce*, and can thus perform the ECDH
key exchange locally, before sending the packet.
* | The public part of the newly generated ephemeral key-pair is included with the encrypted token, and sent
along with the encrypted payload data in the packet.
* | When the destination receives the packet, it can itself perform an ECDH key exchange and decrypt the
packet.
* | A new ephemeral key is used for every packet sent in this way, and forward secrecy is guaranteed on a
per packet level.
* | Once the packet has been received and decrypted by the addressed destination, that destination can opt
to *prove* its receipt of the packet. It does this by calculating the SHA-256 hash of the received packet,
and signing this hash with it's Ed25519 signing key. Transport nodes in the network can then direct this
*proof* back to the packets origin, where the signature can be verified against the destinations known
public signing key.
* | In case the packet is addressed to a *group* destination type, the packet will be encrypted with the
pre-shared AES-128 key associated with the destination. In case the packet is addressed to a *plain*
destination type, the payload data will not be encrypted. Neither of these two destination types offer
forward secrecy. In general, it is recommended to always use the *single* destination type, unless it is
strictly necessary to use one of the others.
For exchanges of larger amounts of data, or when longer sessions of bidirectional communication is desired, Reticulum offers the *Link* API. To establish a *link*, the following process is employed:
* | First, the node that wishes to establish a link will send out a special packet, that
traverses the network and locates the desired destination. Along the way, the nodes that
forward the packet will take note of this *link request*.
* | Second, if the destination accepts the *link request* , it will send back a packet that proves the
authenticity of it’s identity (and the receipt of the link request) to the initiating node. All
nodes that initially forwarded the packet will also be able to verify this proof, and thus
accept the validity of the *link* throughout the network.
* | When the validity of the *link* has been accepted by forwarding nodes, these nodes will
remember the *link* , and it can subsequently be used by referring to a hash representing it.
* | As a part of the *link request* , a Diffie-Hellman key exchange takes place, that sets up an
efficiently encrypted tunnel between the two nodes, using elliptic curve cryptography. As such,
this mode of communication is preferred, even for situations when nodes can directly communicate,
when the amount of data to be exchanged numbers in the tens of packets.
* | When a *link* has been set up, it automatically provides message receipt functionality, through
the same *proof* mechanism discussed before, so the sending node can obtain verified confirmation
that the information reached the intended recipient.
In a moment, we will discuss the details of how this methodology is implemented, but let’s first
recap what purposes this methodology serves. We first ensure that the node answering our request
is actually the one we want to communicate with, and not a malicious actor pretending to be so.
At the same time we establish an efficient encrypted channel. The setup of this is relatively cheap in
terms of bandwidth, so it can be used just for a short exchange, and then recreated as needed, which will
also rotate encryption keys. The link can also be kept alive for longer periods of time, if this is
more suitable to the application. The procedure also inserts the *link id* , a hash calculated from the link request packet, into the memory of forwarding nodes, which means that the communicating nodes can thereafter reach each other simply by referring to this *link id*.
The combined bandwidth cost of setting up a link is 3 packets totalling 409 bytes (more info in the
:ref:`Binary Packet Format<understanding-packetformat>` section). The amount of bandwidth used on keeping
a link open is practically negligible, at 0.62 bits per second. Even on a slow 1200 bits per second packet
radio channel, 100 concurrent links will still leave 95% channel capacity for actual data.
Link Establishment in Detail Link Establishment in Detail
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
After seeing how the conditions for finding a path through the network are created, we will now After exploring the basics of the announce mechanism, finding a path through the network, and an overview
explore how two nodes can establish reliable communications over multiple hops. The *link* in of the link establishment procedure, this section will go into greater detail about the Reticulum link
Reticulum terminology should not be viewed as a direct node-to-node link on the physical layer, but establishment process.
as an abstract channel, that can be open for any amount of time, and can span an arbitrary number
of hops, where information will be exchanged between two nodes. The *link* in Reticulum terminology should not be viewed as a direct node-to-node link on the
physical layer, but as an abstract channel, that can be open for any amount of time, and can span
an arbitrary number of hops, where information will be exchanged between two nodes.
* | When a node in the network wants to establish verified connectivity with another node, it * | When a node in the network wants to establish verified connectivity with another node, it
@ -427,25 +457,25 @@ of hops, where information will be exchanged between two nodes.
considered as single public key for simplicity in this explanation.* considered as single public key for simplicity in this explanation.*
* | The *link request* is addressed to the destination hash of the desired destination, and * | The *link request* is addressed to the destination hash of the desired destination, and
contains the following data: The newly generated X25519 public key *LKi*. The contents contains the following data: The newly generated X25519 public key *LKi*.
are encrypted with the RSA public key of the destination and tramsitted over the network.
* | The broadcasted packet will be directed through the network according to the rules laid out * | The broadcasted packet will be directed through the network according to the rules laid out
previously. previously.
* | Any node that forwards the link request will store a *link id* in it’s *link table* , along with the * | Any node that forwards the link request will store a *link id* in it’s *link table* , along with the
amount of hops the packet had taken when received. The link id is a hash of the entire link amount of hops the packet had taken when received. The link id is a hash of the entire link
request packet. If the path is not *proven* within some set amount of time, the entry will be request packet. If the link request packet is not *proven* by the addressed destination within some
dropped from the *link table* again. set amount of time, the entry will be dropped from the *link table* again.
* | When the destination receives the link request packet, it will decrypt it and decide whether to * | When the destination receives the link request packet, it will decide whether to accept the request.
accept the request. If it is accepted, the destination will also generate a new X25519 private/public If it is accepted, the destination will also generate a new X25519 private/public key pair, and
key pair, and perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used to encrypt the
to encrypt the channel, once it has been established. channel, once it has been established.
* | A *link proof* packet is now constructed and transmitted over the network. This packet is * | A *link proof* packet is now constructed and transmitted over the network. This packet is
addressed to the *link id* of the *link*. It contains the following data: The newly generated X25519 addressed to the *link id* of the *link*. It contains the following data: The newly generated X25519
public key *LKr* and an RSA-1024 signature of the *link id* and *LKr*. public key *LKr* and an Ed25519 signature of the *link id* and *LKr* made by the signing key of
the addressed destination.
* | By verifying this *link proof* packet, all nodes that originally transported the *link request* * | By verifying this *link proof* packet, all nodes that originally transported the *link request*
packet to the destination from the originator can now verify that the intended destination received packet to the destination from the originator can now verify that the intended destination received
@ -465,11 +495,6 @@ reveal any identifying information about itself. The link initiator remains comp
When using *links*, Reticulum will automatically verify all data sent over the link, and can also When using *links*, Reticulum will automatically verify all data sent over the link, and can also
automate retransmissions if *Resources* are used. automate retransmissions if *Resources* are used.
Proven Delivery
^^^^^^^^^^^^^^^
TODO: Write
.. _understanding-resources: .. _understanding-resources:
Resources Resources