1. [Inventory Provider]
   1. [Overview]
   2. [Configuration]
   3. [Running this module]
   4. [Protocol specification]
   5. [backend (Redis) storage schema]

Inventory Provider

Overview

This module implements a Flask-based webservice which provides some access to GÉANT network router inventory information.

The webservice is communicates with clients over HTTP. Responses to valid requests are returned as JSON messages. The server will therefore return an error unless application/json is in the Accept request header field.

HTTP communication and JSON grammar details are beyond the scope of this document. Please refer to RFC 2616 and www.json.org for more details.

Configuration

Several configuration files must be provided when launching the web service.

The runtime-accessible filename of the first configuration file must be stored in the environment variable SETTINGS_FILENAME.

The configuration file is python and defines a single variable. The following is an example:

INVENTORY_PROVIDER_CONFIG_FILENAME = "/somepath/config.json"

• INVENTORY_PROVIDER_CONFIG_FILENAME: run-time accessible filename of a json file containing the server configuration parameters. This file must be formatted according to the following json schema:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object",
      "properties": {
          "alarms-db": {
              "type": "object",
              "properties": {
                  "hostname": {"type": "string"},
                  "dbname": {"type": "string"},
                  "username": {"type": "string"},
                  "password": {"type": "string"}
              },
              "required": ["hostname", "dbname", "username", "password"],
              "additionalProperties": False
          },
          "oid_list.conf": {"type": "string"},
          "routers_community.conf": {"type": "string"},
          "ssh": {
              "type": "object",
              "properties": {
                  "private-key": {"type": "string"},
                  "known-hosts": {"type": "string"}
              },
              "required": ["private-key", "known-hosts"],
              "additionalProperties": False
          },
          "redis": {
              "type": "object",
              "properties": {
                  "hostname": {"type": "string"},
                  "port": {"type": "integer"}
              },
              "required": ["hostname", "port"],
              "additionalProperties": False
          }
      },
      "required": [
          "alarms-db",
          "oid_list.conf",
          "routers_community.conf",
          "ssh",
          "redis"],
      "additionalProperties": False
  }

Running this module

This module has been tested in the following execution environments:

• As an embedded Flask application. For example, the application could be launched as follows:

$ export FLASK_APP=app.py
$ export SETTINGS_FILENAME=settings.cfg
$ flask run

• As an Apache/mod_wsgi service.
  • Details of Apache and mod_wsgi configuration are beyond the scope of this document.

protocol specification

The following resources can be requested from the webservice.

resources

Any non-empty responses are JSON formatted messages.

• /data/version

  The response will be an object containing the module and protocol versions of the running server and will be formatted as follows:

  {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
            "api": {
                "type": "string",
                "pattern": r'\d+\.\d+'
            },
            "module": {
                "type": "string",
                "pattern": r'\d+\.\d+'
            }
        },
        "required": ["api", "module"],
        "additionalProperties": False
    }

• /data/routers

  The response will be a list of router hostnames for which information is available and will be formatted as follows:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {"type": "string"}
  }

• /data/interfaces/hostname

  The response will be a list of information about the interfaces present on the requested host and will be formatted as follows:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {
          "type": "object",
          "properties": {
              "name": {"type": "string"},
              "description": {"type": "string"},
              "ipv4": {
                  "type": "array",
                  "items": {"type": "string"}
              },
              "ipv6": {
                  "type": "array",
                  "items": {"type": "string"}
              }
          },
          "required": ["name", "description", "ipv4", "ipv6"],
          "additionalProperties": False
      }
  }

• /data/snmp/hostname

  The response will be a list of information about the interfaces discovered through snmp queries on the requested host and will be formatted as follows:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {
          "type": "object",
          "properties": {
              "index": {"type": "string"},
              "name": {"type": "string"}
          },
          "required": ["index", "name"],
          "additionalProperties": False
      }
  }

• /data/bgp/hostname

  The response will be a list of information about the bgp peerings configured for the requested host and will be formatted as follows:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {
          "type": "object",
          "properties": {
              "description": {"type": "string"},
              "as": {
                  "type": "object",
                  "properties": {
                      "peer": {
                          "type": "string",
                          "pattern": r'^\d+$'
                      },
                      "local": {
                          "type": "string",
                          "pattern": r'^\d+$'
                      },
                  },
                  "required": ["peer", "local"],
                  "additionalProperties": False
              },
          },
          "required": ["description", "as"],
          "additionalProperties": False
      }
  }

• /jobs/update

  This resource updates the inventory network data for juniper devices.

• /jobs/update-startup

  This resource updates data that should only be refreshed in case of system restart.

• /jobs/update-interfaces-to-services

  This resource updates the information that lists all the services for interfaces found in the external inventory system

• /jobs/update-service-hierarchy

  This resource updates all the information showing which services are related to one-another

• /jobs/update-equipment-locations

  This resource loads the location information for all the equipment from the external inventory system

• /jobs/update-from-inventory-system

  This resource updates the inventory data from the external inventory system, it is a short cut for executing the above three jobs

• /jobs/update-interface-statuses

  This resource updates the last known statuses of all interfaces extracted from the external inventory system, status information is retrieved from the alarms database

• /classifier/infinera-dna-addresses, /classifier/juniper-server-addresses

  Both of these resources return lists of source addresses of known senders of snmp traps. Responses will be formatted as follows:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {"type": "string"}
  }

• /classifier/type/source-equipment/source-interface

  The source-equipment is the equipment that causes the trap, not the NMS that sends it.

  The response will be an object containing the metadata

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object"
  }

• /poller/interfaces/hostname

  The response will be the list of active interfaces on the router hostname. Each element of the returned list contains information necessary for setting up snmp throughput counter polling.

  The response will be formatted according to the following schema:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
  
      "definitions": {
          "circuit": {
              "type": "object",
              "properties": {
                  "type": {"type": "string"},
                  "id": {"type": "integer"}
              },
              "required": ["type", "id"],
              "additionalProperties": False
          }
      },
  
      "type": "array",
      "items": {
          "type": "object",
          "properties": {
              "circuits": {
                  "type": "array",
                  "items": {"$ref": "#/definitions/circuit"}
              },
              "description": {"type": "string"},
              "name": {"type": "string"},
              "snmp-index": {"type": "integer"}
          },
          "required": ["circuits", "description", "name", "snmp-index"],
          "additionalProperties": False
      }
  }

backend (Redis) storage schema

netconf:<hostname>

• key example
  • netconf:mx1.ams.nl.geant.net
• value format
  • cf. validation in inventory_provider.juniper.load_config

snmp-interfaces:<hostname>

• key example:
  • snmp-interfaces:mx1.lon2.uk.geant.net
• value schema TODO: this needs to be checked ...

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
  
      "definitions": {
          "v4ifc": {
              "type": "object",
              "properties": {
                  "v4Address": {
                      "type": "string",
                      "pattern": "^(\d+\.){3}\d+$"
                  },
                  "v4Mask": {
                      "type": "string",
                      "pattern": "^(\d+\.){3}\d+$"
                  },
                  "v4InterfaceName": {:"type", "string"},
                  "index": {`
                      "type": "string",
                      "pattern": "^\d+$"
                  }
              },
              "required": [
                  "v4Address", "v4Mask", "v4InterfaceName", "index],
              "additionalProperties": False
          },
          "v6ifc": {
              "type": "object",
              "properties": {
                  "v6Address": {
                      "type": "string",
                      "pattern": "^[\d:]+$"
                  },
                  "v6Mask": {
                      "type": "string",
                      "pattern": "^\d+$"
                  },
                  "v6InterfaceName": {:"type", "string"},
                  "index": {`
                      "type": "string",
                      "pattern": "^\d+$"
                  }
              },
              "required": [
                  "v6Address", "v6Mask", "v6InterfaceName", "index],
              "additionalProperties": False
          }
      },
  
      "type": "array",
       "items": {
          "anyOf": {
              "$ref": "#/definitions/v4Ifc",
              "$ref": "#/definitions/v6Ifc"
          }
      }
  }

opsdb:interface_services:<equipment name>:<interface name>

• key examples
  • opsdb:interface_services:mx1.ams.nl.geant.net:ae15.1103
  • opsdb:interface_services:Lab node 2:1-1/5
• value schema

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
  
      "definitions": {
          "interface": {
              "type": "object",
              "properties": {
                  "id": {"type": "integer"},
                  "name": {"type": "string"},
                  "status": {
                      "type": "string",
                      "enum": ["operational", "installed", "planned", "ordered"]
                  },
                  "circuit_type": {
                      "type": "string",
                      "enum": ["path", "service", "l2circuit"]
                  },
                  "service_type": {"type": "string"},
                  "project": {"type": "string"},
                  "equipment": {"type": "string"},
                  "port": {
                      "anyOf": [
                          {
                              "type": "string",
                              "pattern": r'^[\d:]+$'
                          },
                          {
                              "type": "string",
                              "enum": ['console']
                          }
                      ]
                  },
                  "logical_unit": {
                      "anyOf": [
                          {"type": "integer"},
                          {"type": "string", "maxLength": 0}
                      ]
                  },
                  "manufacturer": {
                      "type": "string",
                      "enum": ["juniper", "coriant", "infinera",
                               "cisco", "hewlett packard",
                               "corsa", "graham smith uk ltd",
                               "unknown", ""]
                  },
                  "card_id": {"type": "string"},
                  "interface_name": {"type": "string"}
              },
              "required": ["id", "name", "status",
                           "circuit_type", "service_type",
                           "project", "equipment", "port",
                           "logical_unit", "manufacturer",
                           "card_id", "interface_name"],
              "additionalProperties": False
          }
      },
  
      "type": "object",
      "patternProperties": {
          "^.+$": {
              "type": "array",
              "items": {"$ref": "#/definitions/interface"}
          }
      }
  }

opsdb:location:<equipment name>

• key examples
  • opsdb:location:mx1.ams.nl.geant.net
  • opsdb:location:taas.srv1.par.fr.geant.net
  • opsdb:location:A test EX 3-Oct
• value schema

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "array",
      "items": {
          "type": "object",
          "properties": {
              "absid": {"type": "integer"},
              "equipment_name": {"type": "string"},
              "pop_name": {"type": "string"},
              "pop_abbreviation": {"type": "string"},
              "pop_site_id": {"type": "string"},
              "country": {"type": "string"},
              "longitude": {"type": "number"},
              "latitude": {"type": "number"},
          },
          "required": ["absid", "equipment_name",
                       "pop_name", "pop_abbreviation", "pop_site_id",
                       "country", "longitude", "latitude"],
          "additionalProperties": False
      }
  }

opsdb:services:children:<circuit db id> opsdb:services:parents:<circuit db id>

• key examples
  • opsdb:services:children:12363
  • opsdb:services:parents:14407
• value schema TODO: this needs to be checked ...

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
  
      "definitions": {
          "circuit_relationship": {
              "type": "object",
              "properties": {
                  "parent_circuit": {"type": "string"},
                  "parent_circuit_id": {"type": "integer"},
                  "parent_circuit_status": {
                      "type": "string",
                      "enum": ["operational", "installed", "planned", "ordered"]
                  },
                  "child_circuit": {"type": "string"},
                  "child_circuit_id": {"type": "integer"},
                  "child_circuit_status": {
                      "type": "string",
                      "enum": ["operational", "installed", "planned", "ordered"]
                  }
                  "circuit_group": {"type": "integer"},
              },
              "required": [
                  "parent_circuit",
                  "parent_circuit_id",
                  "parent_circuit_status",
                  "child_circuit",
                  "child_circuit_id",
                  "child_circuit_status",
                  "circuit_group"
              ],
              "additionalProperties": False
          }
      },
  
      "type": "object",
      "patternProperties": {
          "^\d+$": {
              "type": "array",
              "items": {"$ref": "#/definitions/circuit_relationship"}
          }
      }
  }

alarmsdb:interface_status:<equipment name>:<interface name>

• key examples
  • alarmsdb:interface_status:Lab node 1:1-1/3
  • alarmsdb:interface_status:mx1.ams.nl.geant.net:ae15.1500
• valid values: TODO: verify these values
  • unknown
  • up
  • down

ix_public_peer:<address>

• key examples
  • ix_public_peer:193.203.0.203
  • ix_public_peer:2001:07f8:00a0:0000:0000:5926:0000:0002
• valid values:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
  
      "definitions": {
          "ip-address": {
              "type": "string",
              "oneOf": [
                  {"pattern": r'^(\d+\.){3}\d+$'},
                  {"pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$'}
              ]
          }
      },
  
      "type": "object",
      "properties": {
          "name": {"$ref": "#/definitions/ip-address"},
          "description": {"type": "string"},
          "as": {
              "type": "object",
              "properties": {
                  "local": {"type": "integer"},
                  "peer": {"type": "integer"},
              },
              "required": ["local", "peer"],
              "additionalProperties": False
          }
      },
      "required": ["name", "description", "as"],
      "additionalProperties": False
  }

vpn_rr_peers/<address>

• key examples
  • ix_public_peer:193.203.0.203
• valid values:

  {
      "$schema": "http://json-schema.org/draft-07/schema#",
  
      "definitions": {
          "ip-address": {
              "type": "string",
              "oneOf": [
                  {"pattern": r'^(\d+\.){3}\d+$'},
                  {"pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$'}
              ]
          }
      },
  
      "type": "object",
      "properties": {
          "name": {"$ref": "#/definitions/ip-address"},
          "description": {"type": "string"},
          "peer-as": {"type": "integer"}
      },
      "required": ["name", "description"],
      "additionalProperties": False
  }