Merge branch 'add-doc-contents' into 'development'

Add documentation contents

See merge request !3

docs/.gitlab-ci.yml

@@ -39,6 +39,4 @@ lint-documentation:
     - vale sync
 
   script:
-    - RESULT=$(vale $CI_PROJECT_DIR/docs/build/html/_sources)
-    - echo "$RESULT"
-    - if echo "$RESULT" | tail -n 1 | grep -q "✖" ; then echo -e "\n\n\n\033[0;31mPlease fix all Vale errors and warnings!\033[0m\n\n\n" && exit 1 ; else exit 0 ; fi
+    - vale $CI_PROJECT_DIR/docs/build/html/_sources

docs/architecture/index.md

 # Architecture
 
-We follow the principles of infrastructure as code, and we decompose configuration in data and templates.
+We follow the principles of infrastructure as code, and decompose configuration in data and templates. Git stores and 
+versions all configuration, and it's also responsible for managing mechanisms such as:
+
+* Automatic checks to validate data and data references.
+* Merge requests for change approval.
+
+The stack of tools is kept limited:
+
+* Workflow Orchestrator ({term}`WFO`) orchestrates all changes.
+* Ansible is the tool that deploys configuration.
+* If needed, custom Python scripts can support extra functionality.
+
+This approach works well for the deployment of 'base configuration'. For service fulfillment, three more components are
+introduced:
+
+* A resource database.
+* A service database.
+* A workflow engine.
+
+All three of these components are deployed as part of {term}`WFO`. More information about {term}`WFO` can be found on
+their website <a href="https://workfloworchestrator.org/" target="_blank">here</a>.
+
+(configuration-decomposition)=
+## Configuration (de)composition
+
+The configuration of a network element can be decomposed in different functional parts:
+
+* Base configuration.
+* Service configuration:
+  * {term}`IFS`
+  * {term}`CFS`
+
+The base configuration includes all configuration necessary to provision a new node, and to include it in the network.
+It covers aspects such as:
+
+* Standard routing configuration for node reachability.
+* User access configuration and AAA (Authentication, Authorisation, Accounting).
+* Standard configuration related to security and hardening of the device.
+* Monitoring and event management.
+
+The configuration is similar across all network elements, apart from some device-specific values such as hardware type 
+or loopback address.
+
+On top of this 'base layer' services can be deployed. Some examples of offered services are given in the table below.
+
+| Infrastructure facing services | Customer facing services          |
+|--------------------------------|-----------------------------------|
+| Backbone trunks                | Access point for R&E traffic      |
+| Transit provider               | Access point for internet traffic |
+| Internet exchange              |                                   |
+
+
+## Principles
+
+### For reference
+
+* All network operations are automated, requiring no operator steps beyond the instantiation of intent.
+* Changes applied to individual network element are fully declarative, vendor-neutral, and derived from the network
+infrastructure from the high-level, network-wide intent.
+* Any network changes are automatically halted and rolled back if the network displays unintended behaviour.
+* The infrastructure doesn't allow operations that violate network policies. 
+[Source.](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/45687.pdf)
+
+### Other principles
+
+* Automatic check evaluate all changes
+* After passing pipelines, all changes are merged into the `main` branch. These pipelines run once a change is 
+peer-reviewed and approved.
+
+- - -
+
+## Orchestration and service database
+
+All services offered can be summarised in the following four categories. Some examples are given in the table below.
+
+|         | Multipoint  | Point 2 point    |
+|---------|-------------|------------------|
+| Layer 2 | VPLS / EVPN | Layer 2 circuits |
+| Layer 3 | L3VPN       | Core links       |

docs/conf.py

@@ -13,9 +13,14 @@ source_suffix = {
 }
 
 # -- Options for Markdown support --------------------------------------------
-myst_enable_extensions = ['replacements', 'strikethrough']
+myst_enable_extensions = ['attrs_block', 'deflist', 'replacements', 'smartquotes', 'strikethrough']
 suppress_warnings = ['myst.strikethrough']
 
 # -- Options for HTML output -------------------------------------------------
 html_theme = 'sphinx_rtd_theme'
 html_static_path = ['static']
+html_theme_options = {
+    'style_nav_header_background': 'rgb(0 63 95)',
+}
+html_css_files = ['custom.css']
+html_logo = 'static/geant_logo_white.svg'

docs/glossary.md

+# Glossary of terms
+
+{.glossary}
+BGP
+: Border Gateway Protocol
+
+CFS
+: Customer Facing Service
+
+eBGP
+: External {term}`BGP`
+
+GA
+: Access Port
+
+GAN
+: Access Node
+
+GAP
+: GÉANT Automation Platform
+
+GP
+: Physical Port
+
+iBGP
+: Internal {term}`BGP`
+
+IFS
+: Interface Facing Service
+
+MPLS
+: Multi-Protocol Label Switching
+
+MTTR
+: Mean Time To Repair
+
+MTU
+: Maximum Transmission Unit
+
+OOB
+: Out-of-band
+
+PoP
+: Point of Presence
+
+SDP
+: Service Delivery Point
+
+WFO
+: <a href="https://workfloworchestrator.org/" target="_blank">Workflow Orchestrator</a>

docs/index.md

-# GÉANT Automation Platform (GAP)
+# GÉANT Automation Platform ({term}`GAP`)
 
-Welcome to the documentation of the GÉANT Automation Platform, also known as GAP. \
-The focus of this platform is configuration management and service orchestration for the GÉANT IP/MPLS network.
+Welcome to the documentation of the GÉANT Automation Platform, or {term}`GAP` for short. \
+The focus of this platform is configuration management and service orchestration for the GÉANT IP/{term}`MPLS` network.
+
+This documentation has the following sections:
 
-## Table of Contents
 ```{toctree}
+:caption: Background
+:maxdepth: 2
 overview/index.md
 architecture/index.md
 ```
+
+```{toctree}
+:caption: Modeling
+:maxdepth: 2
+modeling/index.md
+modeling/ports.md
+```
+
+```{toctree}
+:caption: Processes
+:maxdepth: 2
+processes/node_provisioning.md
+processes/deploy_router.md
+```
+
+```{toctree}
+:caption: —
+:maxdepth: 2
+glossary.md
+```

docs/modeling/index.md

+# Data models
+
+To be able to model network objects, a data model must exist for every configuration item in the network. This allows
+for describing the network, and the services as a composition of abstract objects.
+
+## Decomposition of objects
+
+Every object -- both services and access ports -- is composed of the following building blocks:
+* Administrative metadata
+  * Object ID
+  * Status
+  * Owner
+* Configuration data that depends on the specific service, some examples:
+  * Access port
+    * Access port type
+    * Physical interfaces
+  * IP trunk
+    * IPv4 network
+    * IPv6 network
+    * IS-IS metric
+* Placement metadata
+  * Access node
+  * Service delivery point

docs/modeling/ports.md

+# Services and ports
+
+While a port shouldn't be configured in case there is no service insisting on it, it could happen that more than one
+service is insisting on one port. For this reason, the following entities exist:
+
+{term}`SDP`
+: Service Delivery Point: the logical interface where a service is delivered
+
+{term}`GA`
+: Access Port: an access point into the GÉANT network
+
+{term}`GP`
+: Physical Port: the physical boundary for the {term}`GA`
+
+{term}`GAN`
+: Access Node: the node where a service is delivered
+
+These concepts apply to both {term}`CFS`es and {term}`IFS`es.
+
+```{figure} ../static/access_port_diagram.png
+:alt: Diagram that displays the different entities that make up a GÉANT service.
+
+A visualisation of how services insist on ports.
+```
+
+## Peer-to-peer and multipoint services
+
+There is a distinction between services -- as mentioned in [configuration (de)composition](#configuration-decomposition)
+-- between peer-to-peer and multipoint services. In both cases, {term}`SDP` is the delivery point of a service.
+However, for multipoint services customers are supposed to be dual-homed in at least two different {term}`GAN`s.
+
+To give some examples, the figure below shows a decomposition of a GeantIP service for an NREN.
+
+```{figure} ../static/geant_ip_ports_diagram.png
+:alt: Diagram that displays the different entities that make up a GeantIP service instance.
+
+A visualisation of an instance of a GeantIP service that consists of different configuration objects.
+```

docs/overview/index.md

 # Overview
 
 Configuration management is the process that maintains consistency and integrity of the network and of the services
-built on top of it ensuring that it meets requirements in terms of functionalities, performances, and security.
+built on top of it. It ensures that the network meets requirements in terms of functionalities, performances, and security.
 
-In the context of the IP/MPLS layer and particularly the backbone routers, different teams manage configuration in
-different ways: to deploy new nodes and to operate the network, they use various tools en methods. Compliance and
-quality are checked afterwards: this approach requires much manual work, and it is inherently error-prone.
+In the context of the IP/{term}`MPLS` layer and particularly the backbone routers, different teams manage configuration in
+different ways. To deploy new nodes and to operate the network, they use various tools en methods. These tools check 
+compliance and quality afterward: this approach requires much manual work, and it's inherently error-prone.
 
-This project aims to standardize the configuration and the configuration deployment process by using Open Source tools
-and DevOps strategies: one single framework to deploy and to operate. Considering the network configuration as code and
-managing it under a version control system (GitLab) will enable better visibility and control of changes and will help
-standardize the way of working.
+This project aims to standardize the configuration and the configuration deployment process. By using Open Source tools
+and DevOps strategies there is one single framework to deploy and to operate. Considering the network configuration as 
+code and managing it under a version control system (GitLab) will enable better visibility and control of changes and 
+will help standardize the way of working.
 
-The goal is to reduce configuration drifts and exceptions, Mean Time To Repair (MTTR) in case of fault, and possibly the
+The goal is to reduce configuration drifts and exceptions, {term}`MTTR` in case of fault, and possibly the
 number and the severity of incidents. To verify configuration compliance before it gets deployed, several policies are in
 place. Multiple non-production environments are used to automatically run regression and acceptance tests, in order
 to ensure the highest quality and to detect problems early.

docs/processes/deploy_router.md

+# Router deployment
+
+From a bird's-eye view, the process of deploying a new router in the network is as follows:
+
+1. Manually configure the router such that it's reachable from out-of-band ({term}`OOB`).
+2. Upgrade the router to the most recent OS.
+3. Deploy base configuration.
+4. Configure trunks to connect the router to the network.
+5. Update the protocol meshes (such as {term}`iBGP`).
+6. Promote the router to the production environment.
+
+```{figure} ../static/WFO_deploy_router.png
+:alt: Diagram that displays the steps that Workflow Orchestrator takes to provision a new router. 
+
+{term}`WFO` provisions a new router by following the steps shown here.
+```

docs/processes/node_provisioning.md

+# Node provisioning
+
+A node can either be a router, a switch, or a terminal server. In general -- as laid out more extensively 
+[here](https://wiki.geant.org/display/NETENG/001+-+Topology+and+physical+layout) (behind login) -- a {term}`PoP` 
+consists of:
+
+* One or two routers
+* One switch
+* One terminal server
+
+Globally, the workflow for a new site is as follows:
+
+1. Deploy terminal server:
+   1. Generate base configuration from GitLab
+   2. Ship the device to its location
+   3. Verify reachability and insert in LibreNMS
+2. Deploy {term}`PoP` router in a 'core' fashion
+   1. Rack it up and configure the hardware
+   2. Connect the router to the terminal server via both a console connection, and FXP
+   3. Deploy base configuration using {term}`GAP`
+3. Deploy {term}`PoP` switch
+   1. Rack it up and configure the hardware
+   2. Connect the switch to the terminal server via both a console connection, and FXP
+   3. Deploy base configuration using {term}`GAP`
+4. Deploy the {term}`PoP` interconnect between router and switch
+   1. Set up a physical connection between router and switch
+   2. Deploy configuration using {term}`GAP`
+5. Deploy IP trunks to connect the router to the rest of the network
+   1. Set up a physical connection
+   2. Deploy configuration using {term}`GAP`
+6. Update the {term}`iBGP` mesh to include the new router, promoting it to an edge router
+   1. Deploy configuration using {term}`GAP`
+   2. Using {term}`GAP`, insert the devices in LibreNMS
+
+In the context of the automation platform, the {term}`PoP` interconnects mentioned are modeled as separate objects.

docs/static/custom.css

+.wy-menu > p > span {
+    color: rgb(237 21 86);
+}

docs/static/geant_logo_white.svg

+<svg width="79" height="35" viewBox="0 0 79 35" fill="none" xmlns="http://www.w3.org/2000/svg">
+<g clip-path="url(#clip0_101_15)">
+<path d="M15.9 17.8C16.5 17.3 17 17.1 17.4 17.1C18.4 17.1 18.7 17.8 18.7 18.2C18.5 18.3 14.2 19.7 14 19.8C13.9 19.7 13.9 19.6 13.8 19.6C14 19.4 15.9 17.8 15.9 17.8Z" fill="white" fill-opacity="0.85"/>
+<path d="M0 27C0 31.8 2.1 34.3 6.3 34.3C9.1 34.3 10.8 33.1 10.8 33L10.9 32.9V26.2H5.2V28.1C5.2 28.1 8.1 28.1 8.6 28.1C8.6 28.6 8.6 31.6 8.6 31.9C8.3 32.1 7.4 32.4 6.2 32.4C3.7 32.4 2.5 30.6 2.5 27C2.5 24.9 3.1 22.4 5.9 22.4C7.8 22.4 8.5 23.5 8.5 24.5V24.8H11.1V24.5C11.1 22.1 9 20.5 5.9 20.5C2.2 20.5 0 22.9 0 27Z" fill="white" fill-opacity="0.85"/>
+<path d="M20.2 20.7H12.6V34.1H20.7V32.2C20.7 32.2 15.5 32.2 14.9 32.2C14.9 31.7 14.9 28.5 14.9 28C15.5 28 20.2 28 20.2 28V26.1C20.2 26.1 15.5 26.1 14.9 26.1C14.9 25.6 14.9 23 14.9 22.5C15.5 22.5 20.5 22.5 20.5 22.5V20.6H20.2V20.7Z" fill="white" fill-opacity="0.85"/>
+<path d="M54.5 20.7H42.9C42.9 20.7 42.9 28.7 42.9 30.6C42 29 37.2 20.7 37.2 20.7H34.5V34.1H36.8C36.8 34.1 36.8 26.1 36.8 24.2C37.7 25.8 42.5 34.1 42.5 34.1H45.2C45.2 34.1 45.2 23.2 45.2 22.6C45.7 22.6 48.4 22.6 48.9 22.6C48.9 23.2 48.9 34.1 48.9 34.1H51.3C51.3 34.1 51.3 23.2 51.3 22.6C51.8 22.6 54.9 22.6 54.9 22.6V20.7H54.5V20.7Z" fill="white" fill-opacity="0.85"/>
+<path d="M28.9 20.7H28.7H26.4L21.4 34.1H23.8C23.8 34.1 25.1 30.6 25.3 30.2C25.7 30.2 29.8 30.2 30.2 30.2C30.3 30.6 31.7 34.1 31.7 34.1H34L28.9 20.7ZM25.9 28.3C26.1 27.6 27.3 24.4 27.7 23.3C28.1 24.4 29.2 27.6 29.5 28.3C28.7 28.3 26.6 28.3 25.9 28.3Z" fill="white" fill-opacity="0.85"/>
+<path d="M77 8C68.2 -2.9 32.6 12.5 23 16.5C22.3 16.8 21.4 16.7 20.9 15.7C21.3 16.7 22.1 17.1 23.1 16.7C35.8 11.6 66.5 0.600002 74.2 10.7C77.7 15.3 76.7 20.9 72.9 28.8C72.7 29.1 72.6 29.4 72.6 29.4C72.6 29.4 72.6 29.4 72.6 29.5C72.6 29.5 72.6 29.5 72.6 29.6C72.3 30.1 71.9 30.3 71.6 30.4C72 30.4 72.5 30.2 72.9 29.6C73 29.5 73.1 29.3 73.3 29C78.7 19.5 80.7 12.5 77 8Z" fill="white" fill-opacity="0.85"/>
+<path d="M70.3 29.9C70.2 29.8 68.6 28.4 67 26.9C58.7 19 33.4 -5.3 22.4 1.1C19.3 2.9 18.8 8.2 20.7 15.2C20.7 15.3 20.8 15.4 20.8 15.5C21 16.2 21.5 16.7 22.2 16.7C21.7 16.6 21.3 16.2 21.1 15.7C21.1 15.6 21 15.5 21 15.5C21 15.4 20.9 15.3 20.9 15.1C20.9 15 20.9 14.9 20.8 14.9C19.8 9 21 5.1 23.4 3.5C32.3 -2.5 53.5 15.8 64.2 25C66.6 27.1 69.4 29.5 70.2 30.1C71.4 31 72.4 30 72.7 29.5C72.3 30.1 71.3 30.7 70.3 29.9Z" fill="white" fill-opacity="0.85"/>
+</g>
+<defs>
+<clipPath id="clip0_101_15">
+<rect width="78.9" height="34.3" fill="white"/>
+</clipPath>
+</defs>
+</svg>

docs/test_documentation.sh

@@ -3,5 +3,8 @@
 docker run -it --rm -v $(pwd):/docs sphinxdoc/sphinx:latest /bin/bash -c \
 "pip install sphinx_rtd_theme myst-parser;cd docs;make html"
 
-docker run -it --rm -v $(pwd):/gap jdkato/vale:latest --config="/gap/vale/.vale.ini" sync
+if [ ! -d ./vale/styles/proselint ] || [ ! -d ./vale/styles/Microsoft ]; then
+  docker run -it --rm -v $(pwd):/gap jdkato/vale:latest --config="/gap/vale/.vale.ini" sync
+fi
+
 docker run -it --rm -v $(pwd):/gap jdkato/vale:latest --config="/gap/vale/.vale.ini" /gap/build/html/_sources/

docs/vale/.vale.ini

@@ -4,13 +4,13 @@ MinAlertLevel = suggestion
 
 Vocab = geant-jargon, Sphinx
 
-Packages = RedHat, proselint
+Packages = proselint, Microsoft
 
 [formats]
 txt = md
 
 [*]
-BasedOnStyles = Vale, RedHat, proselint
+BasedOnStyles = Vale, proselint, Microsoft
 
 ; Ignore the table of contents directive
 BlockIgnores = (?s) *(\x60\x60\x60{toctree}.*?\x60\x60\x60)

docs/vale/styles/Vocab/geant-jargon/accept.txt

 IP/MPLS
 MPLS
+MTTR
 configuration as code
+[Rr]eachability
+[Ll]oopback
+Ansible
+[Bb]ackbone
+IFS
+CFS
+GAN
+SDP
+AAA
+GÉANT Automation Platform
+GAP
+[Mm]ultipoint
+WFO
+BGP
+FXP
+NREN
+MTU
+OOB