Skip to content
Snippets Groups Projects
Commit 6c09357b authored by Erik Reid's avatar Erik Reid
Browse files

Finished feature add-sphinx-skeleton.

parents e9e5ad2f 88313a54
No related branches found
No related tags found
No related merge requests found
Showing
with 312 additions and 69 deletions
# BRIAN Dashboard Manager # BRIAN Dashboard Manager
## Overview The BRIAN Dashboard Manager is used
provision Organizations and Dashboards in Grafana for BRIAN.
This module is used to provision Organizations and Dashboards in Grafana for BRIAN. Documentation can be generated by running sphinx:
It implements a Flask-based webservice used only to trigger the provisioning process.
The dashboards are generated from a list of interfaces obtained from Inventory Provider.
Jinja templates are populated with data from these interfaces to render Dashboard JSON definitions sent to the Grafana API.
Grafana API-related code lives in the `grafana` package.
The `brian_dashboard_manager/grafana/provision.py` file is responsible for the entire provisioning lifecycle.
Grafana API endpoints have wrapper functions in one file for relevant parts of the API, e.g. `brian_dashboard_manager/grafana/dashboard.py` for the dashboard API functions.
Another example is `brian_dashboard_manager/grafana/organization.py` for the organization API functions.
Some of the provisioned dashboards are not generated but are just static JSON files. These are put in the `brian_dashboard_manager/dashboards` directory. The same can be done for JSON datasource definitions in the `datasources` directory.
The `brian_dashboard_manager/templating` package contains the code and Jinja templates used to render dashboard JSON. Most dashboards reuse the same templates, with the exception of NREN-specific dashboards, which has its own template.
Of note is the `templating/helpers.py` file, which has all of the predicates and helper functions used to group interfaces together and generate the necessary data for the dashboard templates.
The `templating/render.py` has functions for rendering of the various Jinja templates from the given data.
## Configuration
This app allows specification of a few
example configuration parameters. These
parameters should stored in a file formatted
similarly to `config.json.example`, and the name
of this file should be stored in the environment
variable `CONFIG_FILENAME` when running the service.
## Running this module
This module has been tested in the following execution environments:
- As an embedded Flask application.
For example, the application could be launched as follows:
```bash ```bash
$ export FLASK_APP=/path/to/brian_dashboard_manager/app.py sphinx-build -M html docs/source docs/build
$ export CONFIG_FILENAME=/path/to/config.json
$ flask run
``` ```
- As a `gunicorn` wsgi service. The documents should be viewable in the
- Details of `gunicorn` configuration can be found in the brian_dashboard_manager Puppet repository. workspace of the most recent [Jenkins job](https://jenkins.geant.org/job/brian-dashboard-manager/ws/docs/build/html/index.html).
## Protocol Specification
The following resources can be requested from the webservice.
### resources
Any non-empty responses are JSON formatted messages.
## /update
This resource is used to trigger the provisioning to Grafana.
It responds to the request immediately after starting the provisioning process.
```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"message": {
"type": "string"
}
}
}
```
"""
Grafana module
===============
Grafana API-related code.
Provisioning
---------------
.. automodule:: brian_dashboard_manager.grafana.provision
Grafana API
-------------
.. automodule:: brian_dashboard_manager.grafana.dashboard
Organizations
----------------
.. automodule:: brian_dashboard_manager.grafana.organization
"""
"""
Grafana Dashhboard API endpoints wrapper functions.
"""
import logging import logging
import os import os
import json import json
......
"""
Grafana Organization management helpers.
"""
import random import random
import string import string
import logging import logging
......
"""
This module is responsible for the
entire provisioning lifecycle.
"""
import logging import logging
import time import time
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
......
...@@ -6,6 +6,16 @@ from brian_dashboard_manager import CONFIG_KEY ...@@ -6,6 +6,16 @@ from brian_dashboard_manager import CONFIG_KEY
routes = Blueprint("update", __name__) routes = Blueprint("update", __name__)
UPDATE_RESPONSE_SCHEMA = {
'$schema': 'http://json-schema.org/draft-07/schema#',
'type': 'object',
'properties': {
'message': {
'type': 'string'
}
}
}
@routes.after_request @routes.after_request
def after_request(resp): def after_request(resp):
...@@ -14,6 +24,19 @@ def after_request(resp): ...@@ -14,6 +24,19 @@ def after_request(resp):
@routes.route('/', methods=['GET']) @routes.route('/', methods=['GET'])
def update(): def update():
"""
This resource is used to trigger the provisioning to Grafana.
It responds to the request immediately after starting
the provisioning process.
The response will be formatted according to the following schema:
.. asjson::
brian_dashboard_manager.routes.update.UPDATE_RESPONSE_SCHEMA
:return: json
"""
executor = ThreadPoolExecutor(max_workers=1) executor = ThreadPoolExecutor(max_workers=1)
executor.submit(provision, current_app.config[CONFIG_KEY]) executor.submit(provision, current_app.config[CONFIG_KEY])
return {'data': {'message': 'Provisioning dashboards!'}} return {'data': {'message': 'Provisioning dashboards!'}}
"""
Code and
Jinja templates used to render dashboard JSON.
Most dashboards reuse the
same templates, with the exception of
NREN-specific dashboards, which has
its own template.
Templates
-----------
Some of the provisioned dashboards are not generated but are just static
JSON files. These are put in the
`brian_dashboard_manager/dashboards` directory.
The same can be done for JSON datasource definitions in
the `datasources` directory.
Helpers
---------
.. automodule:: brian_dashboard_manager.templating.helpers
Rendering
---------
.. automodule:: brian_dashboard_manager.templating.render
"""
"""
Predicates
and helper functions used to group interfaces together and generate the
necessary data for the dashboard templates.
"""
import re import re
import logging import logging
import json import json
......
"""
Methods for rendering of the
various Jinja templates from the given data.
"""
import os import os
import json import json
import jinja2 import jinja2
......
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
from importlib import import_module
from docutils.parsers.rst import Directive
from docutils import nodes
from sphinx import addnodes
import json
import os
import sys
sys.path.insert(0, os.path.abspath(
os.path.join(
os.path.dirname(__file__),
'..', '..', 'inventory_provider')))
class RenderAsJSON(Directive):
# cf. https://stackoverflow.com/a/59883833
required_arguments = 1
def run(self):
module_path, member_name = self.arguments[0].rsplit('.', 1)
member_data = getattr(import_module(module_path), member_name)
code = json.dumps(member_data, indent=2)
literal = nodes.literal_block(code, code)
literal['language'] = 'json'
return [
addnodes.desc_name(text=member_name),
addnodes.desc_content('', literal)
]
def setup(app):
app.add_directive('asjson', RenderAsJSON)
# -- Project information -----------------------------------------------------
project = 'BRIAN Dashboard Manager'
copyright = '2021, swd@geant.org'
author = 'swd@geant.org'
# The full version, including alpha/beta/rc tags
release = '0.0'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx_rtd_theme',
'sphinx.ext.autodoc',
'sphinx.ext.coverage'
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'
# html_theme = 'alabaster'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Both the class’ and the __init__ method’s docstring
# are concatenated and inserted.
autoclass_content = "both"
autodoc_typehints = "none"
Configuration and Running
=========================
Configuration
-------------
This app allows specification of a few
example configuration parameters. These
parameters should stored in a file formatted
similarly to `config.json.example`, and the name
of this file should be stored in the environment
variable `CONFIG_FILENAME` when running the service.
Running this module
---------------------
This module has been tested in the following execution environments:
* As an embedded Flask application.
For example, the application could be launched as follows:
.. code-block:: python
export FLASK_APP=/path/to/brian_dashboard_manager/app.py
export CONFIG_FILENAME=/path/to/config.json
flask run
* As a `gunicorn` wsgi service.
* Details of `gunicorn` configuration can be found in the
brian_dashboard_manager Puppet repository.
\ No newline at end of file
.. BRIAN Dashboard Manager documentation master file, created by
sphinx-quickstart on Tue Mar 16 14:42:57 2021.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Inventory Provider
==================
The BRIAN Dashboard Manager is used
provision Organizations and Dashboards in Grafana for BRIAN.
.. toctree::
:maxdepth: 3
:caption: Contents:
configuration
overview
protocol
Overview
=========================
This module is used to provision Organizations and Dashboards inGrafana for BRIAN.
The dashboards are generated from a list of interfaces obtained from Inventory Provider.
Jinja templates are populated with data from these interfaces to render
Dashboard JSON definitions sent to the Grafana API.
.. automodule:: brian_dashboard_manager.grafana
.. automodule:: brian_dashboard_manager.templating
Protocol
=========================
This module implements a Flask-based webservice used only to
trigger the provisioning process.
The following resources can be requested from the webservice.
resources
-----------
Any non-empty responses are JSON formatted messages.
/update
*********
.. autofunction:: brian_dashboard_manager.routes.update.update
...@@ -16,4 +16,5 @@ commands = ...@@ -16,4 +16,5 @@ commands =
coverage xml coverage xml
coverage html coverage html
coverage report --fail-under 75 coverage report --fail-under 75
flake8 flake8
\ No newline at end of file sphinx-build -M html docs/source docs/build
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment