Inventory Provider
Overview
This module implements a Flask-based webservice which provides some access to GÉANT network router inventory information.
The webservice is communicates with clients over HTTP.
Responses to valid requests are returned as JSON messages.
The server will therefore return an error unless
application/json
is in the Accept
request header field.
HTTP communication and JSON grammar details are beyond the scope of this document. Please refer to RFC 2616 and www.json.org for more details.
Configuration
Several configuration files must be provided when launching the web service.
The runtime-accessible filename of the first configuration file
must be stored in the environment variable
SETTINGS_FILENAME
.
The configuration file is python and defines a single variable. The following is an example:
INVENTORY_PROVIDER_CONFIG_FILENAME = "/somepath/config.json"
ENABLE_TESTING_ROUTES = True
-
INVENTORY_PROVIDER_CONFIG_FILENAME
: [REQUIRED] Run-time accessible filename of a json file containing the server configuration parameters. This file must be formatted according to the following json schema:{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "database_credentials": { "type": "object", "properties": { "hostname": {"type": "string"}, "dbname": {"type": "string"}, "username": {"type": "string"}, "password": {"type": "string"} }, "required": ["hostname", "dbname", "username", "password"], "additionalProperties": False } }, "type": "object", "properties": { "alarms-db": {"$ref": "#/definitions/database_credentials"}, "ops-db": {"$ref": "#/definitions/database_credentials"}, "oid_list.conf": {"type": "string"}, "ssh": { "type": "object", "properties": { "username": {"type": "string"}, "private-key": {"type": "string"}, "known-hosts": {"type": "string"} }, "required": ["private-key", "known-hosts"], "additionalProperties": False }, "redis": { "type": "object", "properties": { "hostname": {"type": "string"}, "port": {"type": "integer"} }, "required": ["hostname", "port"], "additionalProperties": False }, "junosspace": { "api": {"type": "string"}, "username": {"type": "string"}, "password": {"type": "string"} }, "infinera-dna": { "type": "array", "items": { "type": "object", "properties": { "name": {"type": "string"}, "address": {"type": "string"} }, "required": ["name", "address"], "additionalProperties": False } }, "coriant-tnms": { "type": "array", "items": { "type": "object", "properties": { "name": {"type": "string"}, "address": {"type": "string"} }, "required": ["name", "address"], "additionalProperties": False } } }, "required": [ "alarms-db", "ops-db", "oid_list.conf", "ssh", "redis", "junosspace", "infinera-dna", "coriant-tnms"], "additionalProperties": False }
-
ENABLE_TESTING_ROUTES
: [OPTIONAL (default value: False)] Flat (can be any value that evaluates to True) to enable routes to special utilities used for testing. This must never be enabled in a production environment.
Running this module
This module has been tested in the following execution environments:
- As an embedded Flask application. For example, the application could be launched as follows:
$ export FLASK_APP=app.py
$ export SETTINGS_FILENAME=settings.cfg
$ flask run
-
As an Apache/
mod_wsgi
service.- Details of Apache and
mod_wsgi
configuration are beyond the scope of this document.
- Details of Apache and
-
As a
gunicorn
wsgi service.- Details of
gunicorn
configuration are beyond the scope of this document.
- Details of
Protocol Specification
The following resources can be requested from the webservice.
resources
Any non-empty responses are JSON formatted messages.
-
/data/version
The response will be an object containing the module and protocol versions of the running server and will be formatted as follows:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "api": { "type": "string", "pattern": r'\d+\.\d+' }, "module": { "type": "string", "pattern": r'\d+\.\d+' } }, "required": ["api", "module"], "additionalProperties": False }
-
/data/routers
The response will be a list of router hostnames for which information is available and will be formatted as follows:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "array", "items": {"type": "string"} }
-
/data/interfaces/
hostname
The response will be a list of information about the interfaces present on the requested host and will be formatted as follows:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "array", "items": { "type": "object", "properties": { "name": {"type": "string"}, "description": {"type": "string"}, "ipv4": { "type": "array", "items": {"type": "string"} }, "ipv6": { "type": "array", "items": {"type": "string"} } }, "required": ["name", "description", "ipv4", "ipv6"], "additionalProperties": False } }
-
/data/pop/
name
Returns location information for the equipment identified by
name
.The response will be formatted as follows:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "definitions": { "pop-info": { "type": "object", "properties": { "name": {"type": "string"}, "abbreviation": {"type": "string"}, "country": {"type": "string"}, "city": {"type": "string"}, "longitude": {"type": "number"}, "latitude": {"type": "number"} }, "required": [ "name", "abbreviation", "country", "city", "longitude", "latitude" ], "additionalProperties": False }, "equipment-info": { "type": "object", "properties": { 'equipment-name': {"type": "string"}, 'status': {"type": "string"}, 'pop': {"$ref": "#/definitions/pop-info"} }, "required": [ "equipment-name", "status", "pop" ], "additionalProperties": False } }, "type": "array", "items": {"$ref": "#/definitions/equipment-info"} }
-
/jobs/update
This resource updates the inventory network data for juniper devices. The function completes asynchronously and a list of outstanding task id's is returned so the caller can use
/jobs/check-task-status
to determine when all jobs are finished. The response will be formatted as follows:{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "array", "items": {"type": "string"} }
-
/jobs/reload-router-config/
equipment-name
This resource updates the inventory network data for the identified juniper device. This function completes asynchronously and returns the same value as
/jobs/update
, except the return contains exactly one task id. -
/jobs/check-task-status/
task-id
This resource returns the current status of an asynchronous task started by
/jobs/update
orjobs/reload-router-config
. The return value will be formatted as follows:{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "properties": { "id": {"type": "string"}, "status": {"type": "string"}, "exception": {"type": "boolean"}, "ready": {"type": "boolean"}, "success": {"type": "boolean"}, "result": {"type": "object"} }, "required": ["id", "status", "exception", "ready", "success"], "additionalProperties": False }
-
/classifier/juniper-link-info/
source-equipment
/source-interface
The source-equipment is the equipment that causes the trap, not the NMS that sends it.
The response will be an object containing the metadata, formatted according to the following schema:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "definitions": { "ip-address": { "type": "string", "oneOf": [ {"pattern": r'^(\d+\.){3}\d+$'}, {"pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$'} ] }, "ipv4-interface-address": { "type": "string", "pattern": r'^(\d+\.){3}\d+/\d+$' }, "ipv6-interface-address": { "type": "string", "pattern": r'^[a-f\d:]+/\d+$' }, "interface-info": { "type": "object", "properties": { "name": {"type": "string"}, "description": {"type": "string"}, "ipv4": { "type": "array", "items": {"$ref": "#/definitions/ipv4-interface-address"} }, "ipv6": { "type": "array", "items": {"$ref": "#/definitions/ipv6-interface-address"} } }, "required": ["name", "description", "ipv4", "ipv6"], "additionalProperties": False }, "service-info": { "type": "object", "properties": { "id": {"type": "integer"}, "name": {"type": "string"}, "status": { "type": "string", "enum": ["operational", "installed", "planned", "ordered"] }, "circuit_type": { "type": "string", "enum": ["path", "service", "l2circuit"] }, "service_type": {"type": "string"}, "project": {"type": "string"}, "equipment": {"type": "string"}, "other_end_equipment": {"type": "string"}, "port": {"type": "string"}, "other_end_port": {"type": "string"}, "logical_unit": { "oneOf": [ {"type": "integer"}, {"type": "string", "maxLength": 0} ] }, "other_end_logical_unit": { "oneOf": [ {"type": "integer"}, {"type": "string", "maxLength": 0} ] }, "manufacturer": { "type": "string", "enum": ["juniper", "coriant", "infinera", "cisco", "hewlett packard", "corsa", "graham smith uk ltd", "unknown", ""] }, "card_id": {"type": "string"}, "other_end_card_id": {"type": "string"}, "interface_name": {"type": "string"}, "other_end_interface_name": {"type": "string"} }, "additionalProperties": False } }, "type": "object", "properties": { "services": { "type": "array", "items": {"$ref": "#/definitions/service-info"} }, "interface": {"$ref": "#/definitions/interface-info"}, "related-services": { "type": "array", "items": {"$ref": "#/definitions/service-info"} } }, "additionalProperties": False }
-
/classifier/peer-info/
address
The
address
parameter should be the ip address of a remote peer. If this address is found in the system then information about the interface is returned, otherwise 404 is returned (or 422 if the address can't be parsed)The response will be formatted according to the following syntax:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "definitions": { "ip-address": { "type": "string", "oneOf": [ {"pattern": r'^(\d+\.){3}\d+$'}, {"pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$'} ] }, "interface-address": { "type": "string", "oneOf": [ {"pattern": r'^(\d+\.){3}\d+/\d+$'}, {"pattern": r'^[a-f\d:]+/\d+$'} ] }, "vpn-rr-peer": { "type": "object", "properties": { "name": {"$ref": "#/definitions/ip-address"}, "description": {"type": "string"}, "peer-as": {"type": "integer"}, "router": {"type": "string"} }, "required": ["name", "description"], "additionalProperties": False }, "ix-public-peer": { "type": "object", "properties": { "name": {"$ref": "#/definitions/ip-address"}, "description": {"type": "string"}, "router": {"type": "string"}, "as": { "type": "object", "properties": { "local": {"type": "integer"}, "peer": {"type": "integer"}, }, "required": ["local", "peer"], "additionalProperties": False } }, "required": ["name", "description", "as"], "additionalProperties": False }, "ix-public-peer-list": { "type": "array", "items": {"$ref": "#/definitions/ip-address"} }, "ix-public-peer-info": { "type": "object", "properties": { "peer": {"$ref": "#/definitions/ix-public-peer"}, "group": {"$ref": "#/definitions/ix-public-peer-list"}, "router": {"$ref": "#/definitions/ix-public-peer-list"} }, "required": ["peer", "group", "router"], "additionalProperties": False }, "interface-info": { "type": "object", "properties": { "name": {"$ref": "#/definitions/ip-address"}, "interface address": { "$ref": "#/definitions/interface-address"}, "interface name": {"type": "string"}, "router": {"type": "string"} }, "required": [ "name", "interface address", "interface name", "router"], "additionalProperties": False }, "service-info": { "type": "object" }, "interface-lookup-info": { "type": "object", "properties": { "interface": {"$ref": "#/definitions/interface-info"}, "services": { "type": "array", "items": {"$ref": "#/definitions/service-info"} } } } }, "type": "object", "properties": { "ix-public-peer-info": {"$ref": "#/definitions/ix-public-peer-info"}, "vpn-rr-peer-info": {"$ref": "#/definitions/vpn-rr-peer"}, "interfaces": { "type": "array", "items": {"$ref": "#/definitions/interface-lookup-info"} } }, "additionalProperties": False }
-
/classifier/infinera-lambda-info/
source-equipment
/source-interface
The source-equipment is the equipment that causes the trap, not the NMS that sends it.
The response will be an object containing the metadata, formatted according to the following schema:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "definitions": { "service-info": { "type": "object", "properties": { "id": {"type": "integer"}, "name": {"type": "string"}, "status": { "type": "string", "enum": ["operational", "installed", "planned", "ordered"] }, "circuit_type": { "type": "string", "enum": ["path", "service", "l2circuit"] }, "service_type": {"type": "string"}, "project": {"type": "string"}, "equipment": {"type": "string"}, "other_end_equipment": {"type": "string"}, "port": {"type": "string"}, "other_end_port": {"type": "string"}, "logical_unit": { "oneOf": [ {"type": "integer"}, {"type": "string", "maxLength": 0} ] }, "other_end_logical_unit": { "oneOf": [ {"type": "integer"}, {"type": "string", "maxLength": 0} ] }, "manufacturer": { "type": "string", "enum": ["juniper", "coriant", "infinera", "cisco", "hewlett packard", "corsa", "graham smith uk ltd", "unknown", ""] }, "card_id": {"type": "string"}, "other_end_card_id": {"type": "string"}, "interface_name": {"type": "string"}, "other_end_interface_name": {"type": "string"} }, "additionalProperties": False }, "geant-lambda": { "type": "object", "properties": { "id": {"type": "integer"}, "name": {"type": "string"}, "status": { "type": "string", "enum": ["operational", "installed", "planned", "ordered"] }, }, "additionalProperties": False } }, "type": "object", "properties": { "services": { "type": "array", "items": {"$ref": "#/definitions/service-info"} }, "geant-lambda": { "$ref": "#/definitions/geant-lambda" }, }, "additionalProperties": False }
-
/classifier/coriant-info/
equipment name
/entity name
Returns information about the effective path of a coriant card/port combination.
The response will be formatted according to the following schema:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "object", "definitions": { "pop-info": { "type": "object", "properties": { "name": {"type": "string"}, "abbreviation": {"type": "string"}, "country": {"type": "string"}, "city": {"type": "string"}, "longitude": {"type": "number"}, "latitude": {"type": "number"} }, "required": [ "name", "abbreviation", "country", "city", "longitude", "latitude" ], "additionalProperties": False }, "endpoint": { "type": "object", "properties": { "equipment name": {"type": "string"}, "card id": {"type": "string"}, "port number": {"type": "string"}, "pop": {"$ref": "#/definitions/pop-info"} }, "required": ["equipment name", "port number", "pop"], "additionalProperties": False }, "path": { "type": "object", "properties": { 'category': {"type": "string"}, 'circuit_type': {"type": "string"}, 'service_type': {"type": "string"}, 'peering_type': {"type": "string"}, 'status': {"type": "string"}, 'name': {"type": "string"}, 'a': {"$ref": "#/definitions/endpoint"}, 'b': {"$ref": "#/definitions/endpoint"} }, "required": [ "category", "circuit_type", "service_type", "peering_type", "status", "a", "b"], "additionalProperties": False } }, "type": "object", "properties": { "equipment name": {"type": "string"}, "card id": {"type": "string"}, "port number": {"type": "string"}, "path": {"$ref": "#/definitions/path"} }, "required": ["equipment name", "card id", "port number"], "additionalProperties": False }
-
/poller/interfaces/
hostname
The response will be the list of active interfac
es on the router
hostname`. Each element of the returned list contains information necessary for setting up snmp throughput counter polling.The response will be formatted according to the following schema:
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "circuit": { "type": "object", "properties": { "name": {"type": "string"}, "status": {"type": "string"}, "type": {"type": "string"}, "id": {"type": "integer"} }, "required": ["name", "status", "type", "id"], "additionalProperties": False } }, "type": "array", "items": { "type": "object", "properties": { "circuits": { "type": "array", "items": {"$ref": "#/definitions/circuit"} }, "description": {"type": "string"}, "name": {"type": "string"}, "snmp-index": {"type": "integer"} }, "required": ["circuits", "description", "name", "snmp-index"], "additionalProperties": False } }
Testing utilities
The following routes are only available if the server
was started with the ENABLE_TESTING_ROUTES
flag. These routes
are present only for testing, or for deprectated routes that
are intended to be deleted.
-
/testing/flushdb
This method erases all data in the backend redis database.
-
/testing/infinera-dna-addresses, /testing/coriant-tnmp-addresses, /testing/juniper-server-addresses`
All of these resources return lists of source addresses of known senders of snmp traps. Responses will be formatted as follows:
```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "array",
"items": {"type": "string"}
}
```
-
/data/snmp/
hostname
The response will be a list of information about the interfaces discovered through snmp queries on the requested host and will be formatted as follows:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "array", "items": { "type": "object", "properties": { "index": {"type": "string"}, "name": {"type": "string"} }, "required": ["index", "name"], "additionalProperties": False } }
-
/data/bgp/
hostname
The response will be a list of information about the bgp peerings configured for the requested host and will be formatted as follows:
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "array", "items": { "type": "object", "properties": { "description": {"type": "string"}, "as": { "type": "object", "properties": { "peer": { "type": "string", "pattern": r'^\d+$' }, "local": { "type": "string", "pattern": r'^\d+$' }, }, "required": ["peer", "local"], "additionalProperties": False }, }, "required": ["description", "as"], "additionalProperties": False } }
Backend (Redis) Storage Schema
-
netconf:<hostname>
- key example
netconf:mx1.ams.nl.geant.net
- value format
- cf. validation in
inventory_provider.juniper.load_config
- cf. validation in
- key example
-
snmp-interfaces:<hostname>
- key example:
snmp-interfaces:mx1.lon2.uk.geant.net
- value schema
TODO: this needs to be checked ...
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "v4ifc": { "type": "object", "properties": { "v4Address": { "type": "string", "pattern": r'^(\d+\.){3}\d+$' }, "v4Mask": { "type": "string", "pattern": r'^(\d+\.){3}\d+$' }, "v4InterfaceName": {"type": "string"}, "index": { "type": "string", "pattern": r'^\d+$' } }, "required": [ "v4Address", "v4Mask", "v4InterfaceName", "index"], "additionalProperties": False }, "v6ifc": { "type": "object", "properties": { "v6Address": { "type": "string", "pattern": r'^[a-f\d:]+$' }, "v6Mask": { "type": "string", "pattern": r'^\d+$' }, "v6InterfaceName": {"type": "string"}, "index": { "type": "string", "pattern": r'^\d+$' } }, "required": [ "v6Address", "v6Mask", "v6InterfaceName", "index"], "additionalProperties": False } }, "type": "array", "items": { "anyOf": [ {"$ref": "#/definitions/v4ifc"}, {"$ref": "#/definitions/v6ifc"} ] } }
- key example:
-
opsdb:interface_services:<equipment name>:<interface name>
- key examples
opsdb:interface_services:mx1.ams.nl.geant.net:ae15.1103
opsdb:interface_services:Lab node 2:1-1/5
- value schema
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "interface": { "type": "object", "properties": { "id": {"type": "integer"}, "name": {"type": "string"}, "status": { "type": "string", "enum": ["operational", "installed", "planned", "ordered"] }, "circuit_type": { "type": "string", "enum": ["path", "service", "l2circuit"] }, "service_type": {"type": "string"}, "project": {"type": "string"}, "equipment": {"type": "string"}, "port": { "anyOf": [ { "type": "string", "pattern": r'^[\d:]+$' }, { "type": "string", "enum": ['console'] } ] }, "logical_unit": { "anyOf": [ {"type": "integer"}, {"type": "string", "maxLength": 0} ] }, "manufacturer": { "type": "string", "enum": ["juniper", "coriant", "infinera", "cisco", "hewlett packard", "corsa", "graham smith uk ltd", "unknown", ""] }, "card_id": {"type": "string"}, "interface_name": {"type": "string"} }, "required": ["id", "name", "status", "circuit_type", "service_type", "project", "equipment", "port", "logical_unit", "manufacturer", "card_id", "interface_name"], "additionalProperties": False } }, "type": "object", "patternProperties": { "^.+$": { "type": "array", "items": {"$ref": "#/definitions/interface"} } } }
- key examples
-
opsdb:location:<equipment name>
- key examples
opsdb:location:mx1.ams.nl.geant.net
opsdb:location:taas.srv1.par.fr.geant.net
opsdb:location:A test EX 3-Oct
- value schema
{ "$schema": "http://json-schema.org/draft-07/schema#", "type": "array", "items": { "type": "object", "properties": { "absid": {"type": "integer"}, "equipment_name": {"type": "string"}, "pop_name": {"type": "string"}, "pop_abbreviation": {"type": "string"}, "pop_site_id": {"type": "string"}, "country": {"type": "string"}, "longitude": {"type": "number"}, "latitude": {"type": "number"}, }, "required": ["absid", "equipment_name", "pop_name", "pop_abbreviation", "pop_site_id", "country", "longitude", "latitude"], "additionalProperties": False } }
- key examples
-
opsdb:services:children:<circuit db id>
-
opsdb:services:parents:<circuit db id>
- key examples
opsdb:services:children:12363
opsdb:services:parents:14407
- value schema
TODO: this needs to be checked ...
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "circuit_relationship": { "type": "object", "properties": { "parent_circuit": {"type": "string"}, "parent_circuit_id": {"type": "integer"}, "parent_circuit_status": { "type": "string", "enum": ["operational", "installed", "planned", "ordered"] }, "child_circuit": {"type": "string"}, "child_circuit_id": {"type": "integer"}, "child_circuit_status": { "type": "string", "enum": ["operational", "installed", "planned", "ordered"] } "circuit_group": {"type": "integer"}, }, "required": [ "parent_circuit", "parent_circuit_id", "parent_circuit_status", "child_circuit", "child_circuit_id", "child_circuit_status", "circuit_group" ], "additionalProperties": False } }, "type": "object", "patternProperties": { "^\d+$": { "type": "array", "items": {"$ref": "#/definitions/circuit_relationship"} } } }
- key examples
-
alarmsdb:interface_status:<equipment name>:<interface name>
- key examples
alarmsdb:interface_status:Lab node 1:1-1/3
alarmsdb:interface_status:mx1.ams.nl.geant.net:ae15.1500
- valid values:
TODO: verify these values
unknown
up
down
- key examples
-
ix_public_peer:<address>
- key examples
ix_public_peer:193.203.0.203
ix_public_peer:2001:07f8:00a0:0000:0000:5926:0000:0002
- valid values:
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "ip-address": { "type": "string", "oneOf": [ {"pattern": r'^(\d+\.){3}\d+$'}, {"pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$'} ] } }, "type": "object", "properties": { "name": {"$ref": "#/definitions/ip-address"}, "description": {"type": "string"}, "as": { "type": "object", "properties": { "local": {"type": "integer"}, "peer": {"type": "integer"}, }, "required": ["local", "peer"], "additionalProperties": False } }, "required": ["name", "description", "as"], "additionalProperties": False }
- key examples
-
vpn_rr_peers/<address>
- key examples
vpn_rr_peers:193.203.0.203
- valid values:
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "ip-address": { "type": "string", "oneOf": [ {"pattern": r'^(\d+\.){3}\d+$'}, {"pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$'} ] } }, "type": "object", "properties": { "name": {"$ref": "#/definitions/ip-address"}, "description": {"type": "string"}, "peer-as": {"type": "integer"} }, "required": ["name", "description"], "additionalProperties": False }
- key examples
-
reverse_interface_addresses/<address>
- key examples
reverse_interface_addresses:193.203.0.203
reverse_interface_addresses:2001:07f8:00a0:0000:0000:5926:0000:0002
- valid values:
{ "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "v4a": { "type": "string", "pattern": r'^(\d+\.){3}\d+$' }, "v6a": { "type": "string", "pattern": r'^([a-f\d]{4}:){7}[a-f\d]{4}$' }, "v4i": { "type": "string", "pattern": r'^(\d+\.){3}\d+/\d+$' }, "v6i": { "type": "string", "pattern": r'^[a-f\d:]+/\d+$' } }, "type": "array", "items": { "type": "object", "properties": { "name": { "oneOf": [ {"$ref": "#/definitions/v4a"}, {"$ref": "#/definitions/v6a"} ] }, "interface address": { "oneOf": [ {"$ref": "#/definitions/v4i"}, {"$ref": "#/definitions/v6i"} ] }, "interface name": {"type": "string"}, }, "required": ["name", "interface address", "interface name"], "additionalProperties": False } }
- key examples