William/Reticulum
1
0
Fork 0
mirror of https://github.com/markqvist/Reticulum.git synced 2024-11-05 05:40:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated docs

Browse Source
This commit is contained in:
Mark Qvist 2021-05-17 16:06:25 +02:00
parent aa93e475a4
commit 87a2c6b1fc
11 changed files with 90 additions and 138 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
docs/.nojekyll Normal file
View File

2
docs/Makefile
View File

@ -18,3 +18,5 @@ help:
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
rm -rf html
cp -r build/html ./

8
docs/html/_sources/index.rst.txt
View File

@ -1,7 +1,7 @@
*************************************
Reticulum Network Stack Documentation
*************************************
Welcome to the documentation for Reticulum. This document aims to provide you
******************************
Reticulum Network Stack Manual
******************************
Welcome to the manual for Reticulum. This document aims to provide you
with all the information you need to understand Reticulum, develop programs
using it, or to participate in the development of Reticulum itself.

89
docs/html/_sources/understanding.rst.txt
View File

@ -403,11 +403,12 @@ of hops, where information will be exchanged between two nodes.
* | When a node in the network wants to establish verified connectivity with another node, it
will create a *link request* packet, and broadcast it.
will randomly generate a new X25519 private/public key pair. It then creates a *link request*
packet, and broadcast it.
* | The *link request* packet contains the destination hash *Hd* , and an asymmetrically encrypted
part containing the following data: The source hash *Hs* , a symmetric key *Lk* , a truncated
hash of a random number *Hr* , and a signature *S* of the plaintext values of *Hd* , *Hs* , *Lk* and *Hr*.
* | The *link request* is addressed to the destination hash of the desired destination, and
contains the following data: The newly generated X25519 public key *LKi*. The contents
are encrypted with the RSA public key of the destination and tramsitted over the network.
* | The broadcasted packet will be directed through the network according to the rules laid out
previously.
@ -415,69 +416,51 @@ of hops, where information will be exchanged between two nodes.
* | Any node that forwards the link request will store a *link id* in it’s *link table* , along with the
amount of hops the packet had taken when received. The link id is a hash of the entire link
request packet. If the path is not *proven* within some set amount of time, the entry will be
dropped from the table again.
dropped from the *link table* again.
* | When the destination receives the link request packet, it will decide whether to accept the
request. If it is accepted, it will create a special packet called a *proof*. A *proof* is a simple
construct, consisting of a truncated hash of the message that needs to be proven, and a
signature (made by the destination’s private key) of this hash. This *proof* effectively verifies
that the intended recipient got the packet, and also serves to verify the discovered path
through the network. Since the *proof* hash matches the *path id* in the intermediary nodes’
*path tables* , the intermediary nodes can forward the proof all the way back to the source.
* | When the destination receives the link request packet, it will decrypt it and decide whether to
accept the request. If it is accepted, the destination will also generate a new X25519 private/public
key pair, and perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used
to encrypt the channel, once it has been established.
* | A *link proof* packet is now constructed and transmitted over the network. This packet is
addressed to the *link id* of the *link*. It contains the following data: The newly generated X25519
public key *LKr* and an RSA-1024 signature of the *link id* and *LKr*.
* | By verifying this *link proof* packet, all nodes that originally transported the *link request*
packet to the destination from the originator can now verify that the intended destination received
the request and accepted it, and that the path they chose for forwarding the request was valid.
In sucessfully carrying out this verification, the transporting nodes marks the link as active.
An abstract bi-directional communication channel has now been established along a path in the network.
* | When the source receives the *proof* , it will know unequivocally that a verified path has been
established to the destination, and that information can now be exchanged reliably and
securely.
established to the destination. It can now also use the X25519 public key contained in the
*link proof* to perform it's own Diffie Hellman Key Exchange and derive the symmetric key
that is used to encrypt the channel. Information can now be exchanged reliably and securely.
It’s important to note that this methodology ensures that the source of the request does not need to
reveal any identifying information. Only the intended destination will know “who called”, so to
speak. This is a huge improvement to protocols like IP, where by design, you have to reveal your
own address to communicate with anyone, unless you jump through a lot of hoops to hide it.
Reticulum offers initiator anonymity by design.
reveal any identifying information about itself. The link initiator remains completely anonymous.
When using *links* , Reticulum will automatically verify anything sent over the link, and also
automates retransmissions if parts of a message was lost along the way. Due to the caching features
of Reticulum, such a retransmission does not need to travel the entire length of an established path.
If a packet is lost on the 8th hop of a 12 hop path, it can be fetched from the last hop that received it
reliably.
When using *links*, Reticulum will automatically verify all data sent over the link, and can also
automate retransmissions if *Resources* are used.
.. _understanding-resources:
Resources
---------
TODO: Write
For exchanging small amounts of data over a Reticulum network, the :ref:`Packet<api-packet>` interface
is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate
the transfer is needed.
In traditional networks, large amounts of data is rapidly exchanged with very low latency. Links of
several thousand kilometers will often only have round-trip latency in the tens of milliseconds, and
as such, traditional protocols are often designed to not store any transmitted data at intermediary
hops. If a transmission error occurs, the sending node will simply notice the lack of a packet
acknowledgement, and retransmit the packet all the way, until it hears back from the receiver that it
got the intended data.
This is the purpose of the Reticulum :ref:`Resource<api-resource>`. A *Resource* can automatically
handle the reliable transfer of an arbitrary amount of data over an established :ref:`Link<api-link>`.
Resources can auto-compress data, will handle breaking the data into individual packets, sequencing
the transfer and reassembling the data on the other end.
In bandwidth-limited and high-latency conditions, such behaviour quickly causes congestion on the
network, and communications that span many hops become exceedingly expensive in terms of
bandwidth usage, due to the higher risk of some packets failing.
Reticulum alleviates this in part with it’s *path* discovery methodology, and in part by implementing
*resource* caching at all nodes that can support it. Network operation can be made much more
efficient by caching everything for a period of time, and given the availability of cheap memory and
storage, this is a very welcome tradeoff. A gigabyte of memory can store millions of Reticulum
packets, and since everything is encrypted by default, the storing poses very little privacy risk.
In a Reticulum network, any node that is able to do so, should cache as many packets as it’s
memory will allow for. When a packet is received, a timestamp and a hash of the packet is stored
along with the full packet itself, and it will be kept in storage until the allocated cache storage is
full, whereupon the packet that was last accessed in the cache will be deleted. If a packet is accessed
from the cache, it’s timestamp will be updated to the current time, to ensure that packets that are
used stay in the cache, and packets that are not used are dropped from memory.
Some packet types are stored in separate caching tables, that allow easier lookup for other nodes.
For example, an announce is stored in a way, that allows other nodes to request the public key for a
certain destination, and as such the network as a whole operates as a distributed key ledger.
For more details on how the caching works and is used, see the reference implementation source
code.
:ref:`Resources<api-resource>` are programmatically very simple to use, and only requires a few lines
of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory,
or stream data directly from files.
.. _understanding-referencesystem:

9
docs/html/genindex.html
View File

@ -47,7 +47,6 @@
| <a href="#I"><strong>I</strong></a>
| <a href="#K"><strong>K</strong></a>
| <a href="#L"><strong>L</strong></a>
| <a href="#M"><strong>M</strong></a>
| <a href="#N"><strong>N</strong></a>
| <a href="#P"><strong>P</strong></a>
| <a href="#R"><strong>R</strong></a>
@ -214,14 +213,6 @@
</ul></td>
</tr></table>
<h2 id="M">M</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>
<li><a href="reference.html#RNS.Resource.MAX_EFFICIENT_SIZE">MAX_EFFICIENT_SIZE (RNS.Resource attribute)</a>
</li>
</ul></td>
</tr></table>
<h2 id="N">N</h2>
<table style="width: 100%" class="indextable genindextable"><tr>
<td style="width: 33%; vertical-align: top;"><ul>

14
docs/html/index.html
View File

@ -5,7 +5,7 @@
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Reticulum Network Stack Documentation &#8212; Reticulum Network Stack 0.2.0 beta documentation</title>
<title>Reticulum Network Stack Manual &#8212; Reticulum Network Stack 0.2.0 beta documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/classic.css" />
@ -28,7 +28,7 @@
<a href="whatis.html" title="What is Reticulum?"
accesskey="N">next</a> |</li>
<li class="nav-item nav-item-0"><a href="#">Reticulum Network Stack 0.2.0 beta documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Reticulum Network Stack Documentation</a></li>
<li class="nav-item nav-item-this"><a href="">Reticulum Network Stack Manual</a></li>
</ul>
</div>
@ -37,9 +37,9 @@
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="reticulum-network-stack-documentation">
<h1>Reticulum Network Stack Documentation<a class="headerlink" href="#reticulum-network-stack-documentation" title="Permalink to this headline"></a></h1>
<p>Welcome to the documentation for Reticulum. This document aims to provide you
<div class="section" id="reticulum-network-stack-manual">
<h1>Reticulum Network Stack Manual<a class="headerlink" href="#reticulum-network-stack-manual" title="Permalink to this headline"></a></h1>
<p>Welcome to the manual for Reticulum. This document aims to provide you
with all the information you need to understand Reticulum, develop programs
using it, or to participate in the development of Reticulum itself.</p>
<div class="section" id="table-of-contents">
@ -134,7 +134,7 @@ using it, or to participate in the development of Reticulum itself.</p>
<div class="sphinxsidebarwrapper">
<h3><a href="#">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Reticulum Network Stack Documentation</a><ul>
<li><a class="reference internal" href="#">Reticulum Network Stack Manual</a><ul>
<li><a class="reference internal" href="#table-of-contents">Table of Contents</a></li>
<li><a class="reference internal" href="#current-status">Current Status</a></li>
<li><a class="reference internal" href="#caveat-emptor">Caveat Emptor</a></li>
@ -177,7 +177,7 @@ using it, or to participate in the development of Reticulum itself.</p>
<a href="whatis.html" title="What is Reticulum?"
>next</a> |</li>
<li class="nav-item nav-item-0"><a href="#">Reticulum Network Stack 0.2.0 beta documentation</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Reticulum Network Stack Documentation</a></li>
<li class="nav-item nav-item-this"><a href="">Reticulum Network Stack Manual</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">

BIN
docs/html/objects.inv
View File

Binary file not shown.

15
docs/html/reference.html
View File

@ -870,21 +870,6 @@ compression, coordination and checksumming.</p>
</ul>
</dd>
</dl>
<dl class="py attribute">
<dt class="sig sig-object py" id="RNS.Resource.MAX_EFFICIENT_SIZE">
<span class="sig-name descname"><span class="pre">MAX_EFFICIENT_SIZE</span></span><em class="property"> <span class="pre">=</span> <span class="pre">16777216</span></em><a class="headerlink" href="#RNS.Resource.MAX_EFFICIENT_SIZE" title="Permalink to this definition"></a></dt>
<dd><p>This is an indication of what the
maximum size a resource should be, if
it is to be handled within reasonable
time constraint, even on small systems.</p>
<p>A small system in this regard is
defined as a Raspberry Pi, which should
be able to compress, encrypt and hash-map
the resource in about 10 seconds.</p>
<p>This constant will be used when determining
how to sequence the sending of large resources.</p>
</dd></dl>
<dl class="py method">
<dt class="sig sig-object py" id="RNS.Resource.advertise">
<span class="sig-name descname"><span class="pre">advertise</span></span><span class="sig-paren">(</span><span class="sig-paren">)</span><a class="headerlink" href="#RNS.Resource.advertise" title="Permalink to this definition"></a></dt>

2
docs/html/searchindex.js
View File

File diff suppressed because one or more lines are too long

89
docs/html/understanding.html
View File

@ -450,13 +450,14 @@ of hops, where information will be exchanged between two nodes.</p>
<ul>
<li><div class="line-block">
<div class="line">When a node in the network wants to establish verified connectivity with another node, it
will create a <em>link request</em> packet, and broadcast it.</div>
will randomly generate a new X25519 private/public key pair. It then creates a <em>link request</em>
packet, and broadcast it.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">The <em>link request</em> packet contains the destination hash <em>Hd</em> , and an asymmetrically encrypted
part containing the following data: The source hash <em>Hs</em> , a symmetric key <em>Lk</em> , a truncated
hash of a random number <em>Hr</em> , and a signature <em>S</em> of the plaintext values of <em>Hd</em> , <em>Hs</em> , <em>Lk</em> and <em>Hr</em>.</div>
<div class="line">The <em>link request</em> is addressed to the destination hash of the desired destination, and
contains the following data: The newly generated X25519 public key <em>LKi</em>. The contents
are encrypted with the RSA public key of the destination and tramsitted over the network.</div>
</div>
</li>
<li><div class="line-block">
@ -468,66 +469,56 @@ previously.</div>
<div class="line">Any node that forwards the link request will store a <em>link id</em> in it’s <em>link table</em> , along with the
amount of hops the packet had taken when received. The link id is a hash of the entire link
request packet. If the path is not <em>proven</em> within some set amount of time, the entry will be
dropped from the table again.</div>
dropped from the <em>link table</em> again.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">When the destination receives the link request packet, it will decide whether to accept the
request. If it is accepted, it will create a special packet called a <em>proof</em>. A <em>proof</em> is a simple
construct, consisting of a truncated hash of the message that needs to be proven, and a
signature (made by the destination’s private key) of this hash. This <em>proof</em> effectively verifies
that the intended recipient got the packet, and also serves to verify the discovered path
through the network. Since the <em>proof</em> hash matches the <em>path id</em> in the intermediary nodes’
<em>path tables</em> , the intermediary nodes can forward the proof all the way back to the source.</div>
<div class="line">When the destination receives the link request packet, it will decrypt it and decide whether to
accept the request. If it is accepted, the destination will also generate a new X25519 private/public
key pair, and perform a Diffie Hellman Key Exchange, deriving a new symmetric key that will be used
to encrypt the channel, once it has been established.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">A <em>link proof</em> packet is now constructed and transmitted over the network. This packet is
addressed to the <em>link id</em> of the <em>link</em>. It contains the following data: The newly generated X25519
public key <em>LKr</em> and an RSA-1024 signature of the <em>link id</em> and <em>LKr</em>.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">By verifying this <em>link proof</em> packet, all nodes that originally transported the <em>link request</em>
packet to the destination from the originator can now verify that the intended destination received
the request and accepted it, and that the path they chose for forwarding the request was valid.
In sucessfully carrying out this verification, the transporting nodes marks the link as active.
An abstract bi-directional communication channel has now been established along a path in the network.</div>
</div>
</li>
<li><div class="line-block">
<div class="line">When the source receives the <em>proof</em> , it will know unequivocally that a verified path has been
established to the destination, and that information can now be exchanged reliably and
securely.</div>
established to the destination. It can now also use the X25519 public key contained in the
<em>link proof</em> to perform it’s own Diffie Hellman Key Exchange and derive the symmetric key
that is used to encrypt the channel. Information can now be exchanged reliably and securely.</div>
</div>
</li>
</ul>
<p>It’s important to note that this methodology ensures that the source of the request does not need to
reveal any identifying information. Only the intended destination will know “who called”, so to
speak. This is a huge improvement to protocols like IP, where by design, you have to reveal your
own address to communicate with anyone, unless you jump through a lot of hoops to hide it.
Reticulum offers initiator anonymity by design.</p>
<p>When using <em>links</em> , Reticulum will automatically verify anything sent over the link, and also
automates retransmissions if parts of a message was lost along the way. Due to the caching features
of Reticulum, such a retransmission does not need to travel the entire length of an established path.
If a packet is lost on the 8th hop of a 12 hop path, it can be fetched from the last hop that received it
reliably.</p>
reveal any identifying information about itself. The link initiator remains completely anonymous.</p>
<p>When using <em>links</em>, Reticulum will automatically verify all data sent over the link, and can also
automate retransmissions if <em>Resources</em> are used.</p>
</div>
</div>
<div class="section" id="resources">
<span id="understanding-resources"></span><h3>Resources<a class="headerlink" href="#resources" title="Permalink to this headline"></a></h3>
<p>TODO: Write</p>
<p>In traditional networks, large amounts of data is rapidly exchanged with very low latency. Links of
several thousand kilometers will often only have round-trip latency in the tens of milliseconds, and
as such, traditional protocols are often designed to not store any transmitted data at intermediary
hops. If a transmission error occurs, the sending node will simply notice the lack of a packet
acknowledgement, and retransmit the packet all the way, until it hears back from the receiver that it
got the intended data.</p>
<p>In bandwidth-limited and high-latency conditions, such behaviour quickly causes congestion on the
network, and communications that span many hops become exceedingly expensive in terms of
bandwidth usage, due to the higher risk of some packets failing.</p>
<p>Reticulum alleviates this in part with it’s <em>path</em> discovery methodology, and in part by implementing
<em>resource</em> caching at all nodes that can support it. Network operation can be made much more
efficient by caching everything for a period of time, and given the availability of cheap memory and
storage, this is a very welcome tradeoff. A gigabyte of memory can store millions of Reticulum
packets, and since everything is encrypted by default, the storing poses very little privacy risk.</p>
<p>In a Reticulum network, any node that is able to do so, should cache as many packets as it’s
memory will allow for. When a packet is received, a timestamp and a hash of the packet is stored
along with the full packet itself, and it will be kept in storage until the allocated cache storage is
full, whereupon the packet that was last accessed in the cache will be deleted. If a packet is accessed
from the cache, it’s timestamp will be updated to the current time, to ensure that packets that are
used stay in the cache, and packets that are not used are dropped from memory.</p>
<p>Some packet types are stored in separate caching tables, that allow easier lookup for other nodes.
For example, an announce is stored in a way, that allows other nodes to request the public key for a
certain destination, and as such the network as a whole operates as a distributed key ledger.</p>
<p>For more details on how the caching works and is used, see the reference implementation source
code.</p>
<p>For exchanging small amounts of data over a Reticulum network, the <a class="reference internal" href="reference.html#api-packet"><span class="std std-ref">Packet</span></a> interface
is sufficient, but for exchanging data that would require many packets, an efficient way to coordinate
the transfer is needed.</p>
<p>This is the purpose of the Reticulum <a class="reference internal" href="reference.html#api-resource"><span class="std std-ref">Resource</span></a>. A <em>Resource</em> can automatically
handle the reliable transfer of an arbitrary amount of data over an established <a class="reference internal" href="reference.html#api-link"><span class="std std-ref">Link</span></a>.
Resources can auto-compress data, will handle breaking the data into individual packets, sequencing
the transfer and reassembling the data on the other end.</p>
<p><a class="reference internal" href="reference.html#api-resource"><span class="std std-ref">Resources</span></a> are programmatically very simple to use, and only requires a few lines
of codes to reliably transfer any amount of data. They can be used to transfer data stored in memory,
or stream data directly from files.</p>
</div>
</div>
<div class="section" id="reference-system-setup">

0
docs/index.html Normal file
View File