Skip to content
Snippets Groups Projects
Commit 9a2a1595 authored by David Schmitz's avatar David Schmitz
Browse files

Updated version number to v1.8.0

parent 3b2c577b
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 1472 additions and 98 deletions
ChangeLog
+ 120
− 94
View file @ 9a2a1595
=======
1.8.0 RELEASE (2023/05)
Improvements and enhancements
- Upgraded to newer Django
requires adaption of flowspy/settings.py regarding staticfiles handling
(check 'git log -p flowspy/settings.py.dist')
- Fixed unwanted possibility to mix IPv4 and IPv6 address prefixes in rules
- Fixed broken check_sync for IPv6 rules
- Support for implicit icmp6 in IPv6 rules
- Fixed improper UTF-8 encoding of Shibboleth attribute values
- Rule id shown on rule detail pages
- Installation support for DEBIAN/UBUNTU, including Systemd support
- Cleanup of log file handling
- Update of documentation
- Enhancement of the Dockerfiles for testing and reference-installation
- DEBIAN as base Docker image is now default: ./Dockerfile
(being linked to ./Dockerfile.fod.debian)
- CENTOS7 keeps to be supported: ./Dockerfile.fod.centos.new
- New experimental setting variable RULE_NAME_PREFIX:
possibility to prefix every FlowSpec rule
injected via JUNOS_specific NETCONF on the router with a given String,
to distinguish such FlowSpec rules from
FlowSpec rules injected via NETCONF by other tools or means
======= =======
2022/08 2022/08
...@@ -35,42 +61,42 @@ Updates and Enhancements ...@@ -35,42 +61,42 @@ Updates and Enhancements
======= =======
1.2 RELEASE 1.2 RELEASE
Updates and enhancements. Check documentation for updating from previous versions Updates and enhancements. Check documentation for updating from previous versions
- Code Cleanup - Code Cleanup
- Branding - Branding
- Rest Api - Rest Api
- Easier Configuration - Easier Configuration
======= =======
1.1.1 RELEASE 1.1.1 RELEASE
Minor changes release Minor changes release
- Change license to GPLv3 - Change license to GPLv3
- Minor documentation updates - Minor documentation updates
=========== ===========
1.1.0 RELEASE 1.1.0 RELEASE
Updates and enhancements. Check documentation for updating from previous versions Updates and enhancements. Check documentation for updating from previous versions
- Minor UI enhancements - Minor UI enhancements
- Patch user model to include user peer in __unicode__ - Patch user model to include user peer in __unicode__
- Include celery task exceptions in proxy - Include celery task exceptions in proxy
- Cleanup js files - Cleanup js files
- Resolve a major issue where the platform would start on 2nd refresh - Resolve a major issue where the platform would start on 2nd refresh
- Cleanup poller urls - Cleanup poller urls
- Enhancements to json routes export - Enhancements to json routes export
- Include timeout handling in tasks - Include timeout handling in tasks
- Fix issue with ports not updating - Fix issue with ports not updating
- Add longerusername plugin to replace user monkey patching - Add longerusername plugin to replace user monkey patching
- Handle a bug in South that causes the Peer table to become unusable after adding autoincrement - Handle a bug in South that causes the Peer table to become unusable after adding autoincrement
=========== ===========
1.0.3 RELEASE 1.0.3 RELEASE
Minor fixes. Check documentation for peers table handling Minor fixes. Check documentation for peers table handling
- Fix issue with altlogin redirection - Fix issue with altlogin redirection
- Switch peers primary key to AutoField - Switch peers primary key to AutoField
- Make peers tables management configurable - Make peers tables management configurable
=========== ===========
1.0.2 RELEASE 1.0.2 RELEASE
...@@ -81,214 +107,214 @@ Documentation enhancements ...@@ -81,214 +107,214 @@ Documentation enhancements
Minor fixes Minor fixes
Fixes: Fixes:
-Overview login theme -Overview login theme
-Add missing urls -Add missing urls
=========== ===========
1.0.0 RELEASE 1.0.0 RELEASE
Major UI redesign, Debian Wheezy version, Django 1.4 Major UI redesign, Debian Wheezy version, Django 1.4
Improvements: Improvements:
-New UI based on Bootstrap3 theme -New UI based on Bootstrap3 theme
-Minor fixes in long polling init -Minor fixes in long polling init
-Debian Wheezy - Django 1.4 ready -Debian Wheezy - Django 1.4 ready
=========== ===========
0.9.9 RELEASE 0.9.9 RELEASE
Major documentation improvements. Minor app enhancements. Minor bug fixes Major documentation improvements. Minor app enhancements. Minor bug fixes
Improvements: Improvements:
-Wrote documentation in rst format (http://flowspy.readthedocs.org/) -Wrote documentation in rst format (http://flowspy.readthedocs.org/)
-Update initial data with fragment types -Update initial data with fragment types
-Add current version in footer via context -Add current version in footer via context
-Add beanstalk client, as installation from package is fuzzy -Add beanstalk client, as installation from package is fuzzy
-Comment the helptext line in patched user model (django 1.3 issues) -Comment the helptext line in patched user model (django 1.3 issues)
=========== ===========
0.9.8 RELEASE 0.9.8 RELEASE
Minor functionality improvements (check Requirements) Minor functionality improvements (check Requirements)
Requirements: Requirements:
-south migration to include database changes if you are at -south migration to include database changes if you are at
<=0.9.5 <=0.9.5
Improvements: Improvements:
-Added more port validation checks -Added more port validation checks
-Added more IP address validation checks -Added more IP address validation checks
=========== ===========
0.9.7 RELEASE 0.9.7 RELEASE
Minor UI improvements (check Requirements) Minor UI improvements (check Requirements)
Requirements: Requirements:
-south migration to include database changes if you are at -south migration to include database changes if you are at
<=0.9.5 <=0.9.5
UI Improvements: UI Improvements:
-Added badges in rule status -Added badges in rule status
=========== ===========
0.9.6 RELEASE 0.9.6 RELEASE
New Feature and minor UI improvements (check Requirements) New Feature and minor UI improvements (check Requirements)
Requirements: Requirements:
-south migration to include database changes -south migration to include database changes
Features: Features:
-Added fragment type as an option in rule match statements -Added fragment type as an option in rule match statements
UI Improvements: UI Improvements:
-Changed wording;from 'Suspend' to 'Deactivate' -Changed wording;from 'Suspend' to 'Deactivate'
-Increased the size of Console and Add Rule buttons. Made Add Rule button -Increased the size of Console and Add Rule buttons. Made Add Rule button
stand out with different color. stand out with different color.
=========== ===========
0.9.5 RELEASE 0.9.5 RELEASE
Fixes Fixes
Fixes: Fixes:
-Fixed issue with page logo -Fixed issue with page logo
-Changed Shibboleth attributes from HTML to Shibboleth naming in error.html -Changed Shibboleth attributes from HTML to Shibboleth naming in error.html
-Minor change in the user activation procedure. Activation mail goes only to admins not TechCs -Minor change in the user activation procedure. Activation mail goes only to admins not TechCs
=========== ===========
0.9.4 RELEASE 0.9.4 RELEASE
Minor fixes Minor fixes
Fixes: Fixes:
-Change the name of the released file (Makefile) -Change the name of the released file (Makefile)
-Added copyright info plus updated the README file -Added copyright info plus updated the README file
-Added missing files in images -Added missing files in images
=========== ===========
0.9.3 RELEASE 0.9.3 RELEASE
Minor fix Minor fix
Fixes: Fixes:
-Fixed the population of "Any" in source field -Fixed the population of "Any" in source field
=========== ===========
0.9.2 RELEASE 0.9.2 RELEASE
Major enhancement and a minor fixes Major enhancement and a minor fixes
Enhancements: Enhancements:
-Added alternative view for helpdesk -Added alternative view for helpdesk
Fixes: Fixes:
-Fixed the static url for tinymce in settings -Fixed the static url for tinymce in settings
-Fixed an issue caused by multiple Shibboleth attributes -Fixed an issue caused by multiple Shibboleth attributes
=========== ===========
0.9.1 RELEASE 0.9.1 RELEASE
Major UI enhancements Major UI enhancements
Enhancements: Enhancements:
-Added bootstrap UI framework -Added bootstrap UI framework
-Added TinyMCE in flatpages -Added TinyMCE in flatpages
-Brought back flatpages with JS magic for translation switching -Brought back flatpages with JS magic for translation switching
-HomeOrganization is no longer required-user selects from dropdown -HomeOrganization is no longer required-user selects from dropdown
-Added Shibboleth mapping in settings -Added Shibboleth mapping in settings
-Added an Any button in source address -Added an Any button in source address
=========== ===========
0.9 RELEASE 0.9 RELEASE
Major enhancements Major enhancements
Enhancements: Enhancements:
-Added internationalization support -Added internationalization support
-Added Greek translation -Added Greek translation
=========== ===========
0.8.7 RELEASE 0.8.7 RELEASE
Minor enhancements Minor enhancements
Enhancements: Enhancements:
- Merged all mail txt files into one - Merged all mail txt files into one
- Added all routes in form cleaning (initially, EXPIRED, ADMININACTIVE and ERROR were excluded) - Added all routes in form cleaning (initially, EXPIRED, ADMININACTIVE and ERROR were excluded)
=========== ===========
0.8.6 RELEASE 0.8.6 RELEASE
Minor UI enhancements/Bug fix Minor UI enhancements/Bug fix
Fixes: Fixes:
- Fixed issue where rules in ERROR state would cause check_sync to check them - Fixed issue where rules in ERROR state would cause check_sync to check them
Enhancements: Enhancements:
- Added small dots to ongoing response field to indicate activity - Added small dots to ongoing response field to indicate activity
=========== ===========
0.8.5 RELEASE 0.8.5 RELEASE
Feature enhancement release/Minor UI fixes/Cleanup Feature enhancement release/Minor UI fixes/Cleanup
Fixes: Fixes:
- Changed javascript order to prevent unformated content in datatables - Changed javascript order to prevent unformated content in datatables
- Un-needed files cleanup - Un-needed files cleanup
- Error template is now based on base.html template - Error template is now based on base.html template
Enhancements: Enhancements:
- Administrator privileges apply on UI as well - Administrator privileges apply on UI as well
- Enhanced application security - Enhanced application security
=========== ===========
0.8.4 RELEASE 0.8.4 RELEASE
Vulnerability prevention/bug fixes release Vulnerability prevention/bug fixes release
Fixes: Fixes:
- Fixed a bug where the shib auth backend erased non-shibboleth users info - Fixed a bug where the shib auth backend erased non-shibboleth users info
- Added an authsource variable to prevent authentication backend overlapping - Added an authsource variable to prevent authentication backend overlapping
- Added exception handling for non-Shibboleth users that do not belong to a peer - Added exception handling for non-Shibboleth users that do not belong to a peer
=========== ===========
0.8.3 RELEASE 0.8.3 RELEASE
Feature enhancement release Feature enhancement release
Fixes: Fixes:
- User/username length monkey patching now works with admin forms as well - User/username length monkey patching now works with admin forms as well
=========== ===========
0.8.2 RELEASE 0.8.2 RELEASE
Bug Fix release Bug Fix release
Fixes: Fixes:
- Fixed bug with csrf cookie not being set while logged in for the first time - Fixed bug with csrf cookie not being set while logged in for the first time
=========== ===========
0.8.1 RELEASE 0.8.1 RELEASE
This is the latest alpha release operating on production network This is the latest alpha release operating on production network
Changes: Changes:
- Fixed bug with protected networks form cleaning - Fixed bug with protected networks form cleaning
=========== ===========
v0.8.0 RELEASE v0.8.0 RELEASE
New features New features
Changes: Changes:
- DB migration to protocol addition - DB migration to protocol addition
- Added protocol to match conditions plus check mechanism to form cleaning - Added protocol to match conditions plus check mechanism to form cleaning
=========== ===========
0.7.11 RELEASE 0.7.11 RELEASE
Bux fixes Bux fixes
Changes: Changes:
- Prevented a bug that would cause the rule application to throw exception - Prevented a bug that would cause the rule application to throw exception
=========== ===========
0.7.10 RELEASE 0.7.10 RELEASE
Got rid of another cronjob Got rid of another cronjob
Changes: Changes:
- Turned expiration notification cron job into celery job - Turned expiration notification cron job into celery job
- Added a preliminary draft for a Makefile facilitating various jobs - Added a preliminary draft for a Makefile facilitating various jobs
=========== ===========
0.7.9.7 RELEASE 0.7.9.7 RELEASE
Some minor changes mainly to reinforce security Some minor changes mainly to reinforce security
Changes: Changes:
- Added FQDN resolving in mail notification templates to denote the host that an action originated - Added FQDN resolving in mail notification templates to denote the host that an action originated
=========== ===========
0.7.9.5 RELEASE 0.7.9.5 RELEASE
Oops! Something was missing from form validation Oops! Something was missing from form validation
Changes: Changes:
- Added source address to required fields - Added source address to required fields
=========== ===========
0.7.9.2 RELEASE 0.7.9.2 RELEASE
Major changes (maybe version tag does not indicate that) Major changes (maybe version tag does not indicate that)
Changes: Changes:
- Added a custom command to fetch networks for each peer. Got rid of cronjob - Added a custom command to fetch networks for each peer. Got rid of cronjob
- Major change with db engine. Altered database storage engine to MYISAM to allow for software relations between tables and views - Major change with db engine. Altered database storage engine to MYISAM to allow for software relations between tables and views
=========== ===========
0.7.9.1 RELEASE 0.7.9.1 RELEASE
...@@ -300,33 +326,33 @@ Changes: ...@@ -300,33 +326,33 @@ Changes:
0.7.9 RELEASE 0.7.9 RELEASE
Bug fixes Bug fixes
Changes: Changes:
- Added a custom uknown_host_cb function to overcome ssh key errors - Added a custom uknown_host_cb function to overcome ssh key errors
=========== ===========
0.7.7 RELEASE 0.7.7 RELEASE
Modules cleanup Modules cleanup
Changes: Changes:
- Removed utils/beanstalkc as it is now a deb package - Removed utils/beanstalkc as it is now a deb package
=========== ===========
0.7.1 RELEASE 0.7.1 RELEASE
Code improvements Code improvements
Changes: Changes:
- Modified peer network range update mechanism - Modified peer network range update mechanism
=========== ===========
0.7 RELEASE 0.7 RELEASE
Major release/changes Major release/changes
Features: Features:
- Added registration to installed apps - Added registration to installed apps
- Removed user activation from shibboleth backend. Moved it to login view - Removed user activation from shibboleth backend. Moved it to login view
=========== ===========
Application features up to now: Application features up to now:
- Rule creation and application to device via netconf, nxpy - Rule creation and application to device via netconf, nxpy
- Match statements include source, destination addrs, src, dst ports - Match statements include source, destination addrs, src, dst ports
- Then statements include discard and rate limit for plain users - Then statements include discard and rate limit for plain users
- User authentication via Shibboleth - User authentication via Shibboleth
- Whois client determines user peer networks and user authority - Whois client determines user peer networks and user authority
_version.py
+ 1
− 1
View file @ 9a2a1595
#VERSION = '1.5' #VERSION = '1.5'
VERSION = '1.7' VERSION = '1.8.0'
if __name__ == "__main__": if __name__ == "__main__":
print(VERSION) print(VERSION)
doc/administration_and_usage/basic_administration_and_usage-v1.8.md 0 → 100644
+ 180
− 0
View file @ 9a2a1595
# Basic Administration and Usage
Early steps for administration and usage that can be done after FoD has been installed.
## 1: Setup NETCONF + FoD admin user
### 1 (local): Setup NETCONF + FoD admin user (FoD running locally)
admin user password and NETCONF connection has to be setup,
either by
A) via the setup page of FoD in container
(needs ENABLE_SETUP_VIEW=True in ./flowspy/settings.py;
only applicable once, i.e., as long as no admin user has been setup, for security reasons):
http://SERVERNAME:SERVERPORT/setup/
(/setup not only covers NETCONF connectivity and initial admin user/password,
but also the adding of a single peer range, i.e., an IP address prefix, (via a single peer)
to be assigned to the initial admin user,
in order to allow the creation/changing FlowSpec rules with a destination IP address prefix
falling within that single peer range)
or alternatively
B) manually
edit /srv/flowspy/flowspy/settings.py : settings NETCONF_DEVICE, NETCONF_PORT, NETCONF_USER, NETCONF_PASS
make sure IP connectivity to NETCONF_DEVICE is available
in FoD installation dir (default: /srv/flowspy): ./pythonenv ./manage.py createsuperuser ...
in FoD installation dir (default: /srv/flowspy): ./pythonenv ./manage.py changepassword ...
restart FoD: 'systemctl restart fod-gunicorn; systemctl restart fod-celeryd' (if installed and running with Systemd support)
or alternatively
C) already having been set by install-\*.sh parameters (check [Debian/Ubuntu Installation](../installation/v1.7/debian_ubuntu.md) or [CENTOS 7 Installation](../installation/v1.7/centos.md) )
### 1 (Docker): Setup NETCONF + FoD admin user (FoD running in a Docker container)
admin user password and NETCONF connection has to be setup,
either by
A) via the setup page of FoD in container
(only applicable once, i.e., as long as no admin user has been setup, for security reasons):
http://127.0.0.1:8001/setup/
(/setup not only covers NETCONF connectivity and initial admin user/password,
but also the adding of a single peer range, i.e., an IP address prefix, (via a single peer)
to be assigned to the initial admin user,
in order to allow the creation/change of FlowSpec rules with a destination IP address prefix
falling within that single peer range)
or alternatively
B) manually
in by entering the running container and editing
docker exec -ti "$DOCKERID" bash # find out DOCKERID of running container with "docker ps"
in docker: vi /srv/flowspy/flowspy/settings.py : settings NETCONF_DEVICE, NETCONF_PORT, NETCONF_USER, NETCONF_PASS
make sure docker container has IP connectivity to NETCONF_DEVICE
in docker: cd /srv/flowspy; ./pythonenv ./manage.py createsuperuser ...
in docker: cd /srv/flowspy; ./pythonenv ./manage.py changepassword ...
## 2: Accessing the FoD UI running in container after setup of admin user and password
http(s)://SERVERNAME:SERVERPORT/altlogin
(for use with Docker: http(s)://SERVERNAME:8000/altlogin)
(do not try to use the Shibboleth login, i.e., via /login, as it is not working without a set-up Shibboleth SP, see 2.1.2)
### 2.1 administration
via http(s)://SERVERNAME:SERVERPORT/admin (only accessible by admin users, e.g., the initial admin, see 1.)
#### 2.1.1 administration of peers, peer ranges, and users (via /admin)
Peer ranges and Peers:
(http(s)://SERVERNAME:SERVERPORT/admin/peers/peerrange/ and http(s)://SERVERNAME:SERVERPORT/admin/peers/peer/)
The 'peer' is a concept in FoD to support multi-tenancy.
Each peer has assigned a set of allowed destination IP address prefixes ('peer ranges')
for which the peer is allowed to deploy BGP FlowSpec rules.
It typically corresponds to a customer organization of the network operator organization running FoD
to provided to users of the different customer organizations.
For managing users (beyond the initial setup of the first admin user, under 1.) the /admin web UI interface can be used
as well, specifically http(s)://SERVERNAME:SERVERPORT/admin/auth/user/.
E.g., it allows adding/removing users, changing first/last name of a user, define whether it is an admin or not.
A 'user' in FoD (including every admin user) has assigned a set of peers (typically, often only 1).
Only for destination IP address prefixes of assigned peers the user has the right to
deploy or change BGP FlowSpec rules.
This restriction also applies for the initial admin user created initially (see 1.).
So before that admin user can deploy FlowSpec rules via FoD, a peer has to be created (maybe by this admin user).
and appropriate peer ranges have to assigned to the peer
and this peer has to assigned to the user.
Simple example: a single peer with peer ranges '0.0.0.0/0' and '0::0/0' assigned to an admin user,
allows to add/edit BGP FlowSpec rules with arbitrary destination IP prefixes.
For production deployments this is not generally recommended,
in order to avoid severe mitigation mistakes by admin users.
The basic setup via http(s)://SERVERNAME:SERVERPORT/setup (see 1.)
not only covers NETCONF connectivity and initial admin user, but also the adding of a single peer range for a single peer
to be assigned to the initial admin user.
#### 2.1.2 administration of user and their peer mapping (via Shibboleth + enrollment working for users)
TODO: User management via Shibboleth
TODO: enrollment workflow for users
TODO: auto update of enrolled users on login
### 2.2 usage
#### 2.2.1 usage via web UI
When logged into FoD UI via
http(s)://SERVERNAME:SERVERPORT/altlogin
(or via https://SERVERNAME:SERVERPORT/login for use with Shibboleth enrolled/registered users, see 2.1.2)
with a FoD user account which has assigned 1 or more peers with appropriate peer ranges (see 2.1.1)
the normal usage can start.
##### Rules list/table page
Provides a list/table of of all rule for all peers of the user.
BGP FlowSpec rule can have either status inactive (stored only in FoD database),
or active (stored in FoD database + installed on the router via NETCONF)
##### Rules dashboard
Provides a history of rule changes for all peers of the user.
##### Add New Rule
Allows to add a new rule, i.e., one not yet stored in the FoD database,
to FoD database and transfer it via NETCONF to the router(s).
##### Edit Existing Rule
Reachable from rules list page or dashboard page for all existing (active or inactive) displayed rules
for all peers of the user.
An edited route is changed in the FoD data base as well as updated on the router,
i.e., will be in active status after the edit operation.
##### Overview (for admin users)
TODO
##### Admin (only for admin users, see 2.1)
For admin users only, allows to perform (Django) admin actions (see 2.1)
##### User Profile
overview of the own user account, showing first/last name etc.
create REST API token (see [API v1.7](../api/api-v1.7.md))
#### 2.2.2 usage via REST API
see [API v1.7](../api/api-v1.7.md)
#### 3. Further/Regular Administration
### FoD run-time status
There is ./systemd/fod-status.sh, a generic script (not limited to Systemd) for determining the process status of FoD along with some further aspects, e.g., Database connection, NETCONF configuration and reachability
doc/api/api-v1.8.md 0 → 100644
+ 318
− 0
View file @ 9a2a1595
# REST API v1.8
(same as v1.7)
# Description
Current version of FoD officially has a REST API.
The API needs authentication. Out of the box the supported authentication
type is Token Authentication.
TO UPDATE
## Generating Tokens
A user can generate an API token using the FoD UI. Select "My Profile" from the
top right menu and on the "Api Token" section click "Generate One".
## Accessing the API
The API is available at `/api/`. One can see the available API endpoints for
each model by making a GET request there. An authentication token must be added
in the request:
* Using `cURL`, add the `-H "Authorization: Token <your-token>"`
parameter
* Using Postman, under the "Headers" add a header with name
"Authorization" and value "Token <your-token>".
# Endpoints
REST API provides the following endpoints that will be described in more detail in the next sections:
* `/api/matchprotocol/`
* `/api/fragmenttypes/`
* `/api/matchdscp/` not supported yet
* `/api/thenactions/`
* `/api/routes/`
* `/api/stats/routes/`
# Usage Examples
Some basic usage examples will be provided including available
actions. Examples will be provided in `cURL` form.
An example will be provided for `ThenAction`. This example applies to most other
models (`FragmentType`, `MatchProtocol`, `MatchDscp`) except
`Route` which is more complex and will be treated separately.
## ThenAction
### GET
#### All items
URL: `/api/thenactions/`
Example:
```
curl -X GET https://fod.example.com/api/thenactions/ -H "Authorization: Token <your-token>"
# or on the FoD host locally:
curl -X GET http://localhost:8000/api/thenactions/ -H "Authorization: Token <your-token>"
RESPONSE:
[
"discard",
"rate-limit:10000k",
"rate-limit:1000k",
"rate-limit:100k"
]
```
## Route
### GET
#### All items
URL: `/api/routes/`
Example:
```
curl -X GET https://fod.example.com/api/routes/ -H "Authorization: Token <your-token>"
RESPONSE:
[
{
"applier": "admin",
"comments": "test comment",
"destination": "1.0.0.4/32",
"destinationport": "124",
"dscp": [],
"expires": "2021-04-15",
"filed": "2021-03-17T13:39:04.244120Z",
"fragmenttype": [],
"icmpcode": null,
"id": 62,
"last_updated": "2021-03-17T13:39:04.244151Z",
"name": "test_66ML9G",
"packetlength": null,
"port": null,
"protocol": [
"udp"
],
"requesters_address": null,
"response": null,
"source": "0.0.0.0/0",
"sourceport": "123",
"status": "PENDING",
"tcpflag": null,
"then": [
"discard"
]
}
...
]
```
#### A specific item
One can also GET a specific `Route`, by using the `id` in the GET url
URL: `/api/routes/<route-id>/`
Example:
```
curl -X GET https://fod.example.com/api/routes/1/ -H "Authorization: Token <your-token>"
RESPONSE:
{
"applier": "admin",
"comments": "test comment",
"destination": "1.0.0.4/32",
"destinationport": "124",
"dscp": [],
"expires": "2021-04-15",
"filed": "2021-03-17T13:39:04.244120Z",
"fragmenttype": [],
"icmpcode": null,
"id": 62,
"last_updated": "2021-03-17T13:39:04.244151Z",
"name": "test_66ML9G",
"packetlength": null,
"port": null,
"protocol": [
"udp"
],
"requesters_address": null,
"response": null,
"source": "0.0.0.0/0",
"sourceport": "123",
"status": "PENDING",
"tcpflag": null,
"then": [
"discard"
]
}
```
### POST
Starting from FoD v1.7 the REST API accepts parameters for POST, PUT and PATCH only via JSON documents,
old method used in REST API of FoD v1.3 to specify parameters as single form values is not supported any more.
Required fields (to be specified via JSON document):
* `name`: a name for the route
* `source`: a source subnet in CIDR formation
* `destination`: a destination subnet in CIDR formation
* `comments`: a small comment on what this route is about
The response will contain all the additional fields
URL: `/api/routes/`
Example input data file `newroute.json`:
```
{
"name": "test1",
"comments": "test1 comment",
"source": "0.0.0.0/0",
"destination": "1.0.0.4/32",
"protocol": [
"udp", "tcp"
],
"sourceport": "123,126,250-270",
"destinationport": "27,124,300-400,500-600",
"then": ["discard"],
"expires": "2022-10-20"
}
```
```
# protocol: "icmp" | "udp" | "tcp" # actually choices returned by 'curl -X GET https://fod.example.com/api/matchprotocol/ ...)
# fragmenttype: "is-fragment" | "dont-fragment" | "first-fragment" | "last-fragment" | "not-a-fragment" # actually choices returned by 'curl -X GET https://fod.example.com/api/fragmenttype/ ...)
# dscp: not supported yet
# packetlength: not supported yet
# tcpflag: not supported yet
# icmptype: not supported yet
# icmpcode: not supported yet
# port, sourceport, destinationport
# then: "discard" | "rate-limit:10000k" | "rate-limit:1000k" | "rate-limit:100k" # actually choices returned by 'curl -X GET https://fod.example.com/api/thenactions/ ...)
```
```
curl -X POST https://fod.example.com/api/routes/ -H "Content-Type: application/json" -d@newroute.json -H "Authorization: Token <your-token>"
{
"name":"testroute_9Q5Y90",
"id":5,
"comments":"Route for testing",
"applier":"admin",
"source":"62.217.45.75/32",
"sourceport":[],
"destination":"62.217.45.94/32",
"destinationport":[],
"port":[],
"dscp":[],
"fragmenttype":[],
"icmpcode":null,
"packetlength":null,
"protocol":[],
"tcpflag":null,
"then":[
"https://fod.example.com/api/thenactions/4/"
],
"filed":"2017-03-29T14:21:03.261Z",
"last_updated":"2017-03-29T14:21:03.261Z",
"status":"PENDING",
"expires":"2017-04-05",
"response":null,
"requesters_address":null
}
```
Notice that the `Route` has a `PENDING` status. This happens because the `Route`
is applied asynchronously to the Flowspec device (the API does not wait for the
operation). After a while the `Route` application will be finished and the
`status` field will contain the updated status (`ACTIVE`, `ERROR` etc).
You can check this `Route`s status by issuing a `GET` request with the `id`
the API returned.
### PUT, PATCH
Starting from FoD v1.7 the REST API accepts parameters for POST, PUT and PATCH only via JSON documents,
old method used in REST API of FoD v1.3 to specify parameters as single form values is not supported any more.
`Route` objects can be modified using the `PUT` / `PATCH` HTTP methods.
```
# e.g., for rule id '10100'
curl -X PUT https://fod.example.com/api/routes/10100/ -d@data.json -H "Authorization: Token <your-token>"
curl -X PATCH https://fod.example.com/api/routes/10100/ -d@data.json -H "Authorization: Token <your-token>"
```
When using `PUT` all fields need to be specified (compare `POST` section).
However, when using `PATCH` one can specify single fields, too.
This is especially useful for changing the `status` of an `INACTIVE` `Route` to `ACTIVE`
or vice versa.
A PATCH or PUT of an active rule towards status INACTIVE will trigger removal of that active
rule from the configured NETCONF-linked router.
A PATCH or PUT of an inactive rule towards status ACTIVE will trigger commiting of that inactive
rule to the configured NETCONF-linked router.
A PATCH or PUT of an inactive rule with no status change will not trigger any change towards
the configured NETCONF-linked router.
The result of PUT and PATCH are different in the following case:
A PATCH without change of status value and change only of non-FlowSpec-specific
match and action fields will not trigger a recommiting of even an active FlowSpec rule
on the configured NETCONF-linked router.
So, e.g., the comments or the rule expiration period can be changed without interrupting
and active route.
In contrast, a PUT of an active rule will always trigger a recommiting of that route
on the configured NETCONF-linked router, independent of the attribute values changed.
So, this is useful
for ensuring that a rule is really still actively installed on the configured NETCONF-linked router
with all match and action parameters as is was last deployed by FoD,
and not changed by some other means without FoD's notice, e.g., other tools or directly by the CLI of the router.
### DELETE
```
# e.g., for rule id '10100'
curl -X DELETE https://fod.example.com/api/routes/10100/ -H "Authorization: Token <your-token>"
```
A DELETE method applied to a Flowspec rule has now always the semantics of fully removing the rule
also from data base (and so its history), not only deactivating it via NETCONF on the routers.
Furthermore, the application of this method can be restricted depending on the settings.
3 setting variables (ALLOW_DELETE_FULL_FOR_ADMIN, ALLOW_DELETE_FULL_FOR_USER_ALL, ALLOW_DELETE_FULL_FOR_USER_LIST)
control whether admin users, normal users in general, or a specificly selected
set of normal users is allowed to use DELETE method.
The default values of the settings (flowspy/settings.py.dist) by this means
allow DELETE only for admins, only for a specificly defined set of normal users.
### General notes on `Route` models (differences to REST API of FoD v1.7):
* Starting from FoD v1.7 the REST API accepts parameters for POST, PUT and PATCH only via JSON documents,
old method used in REST API of FoD v1.3 to specify parameters as single form values is not supported any more.
* In contrast to REST API of FoD v1.3, REST API of current version v1.7 will honor the status
value set in POST method calls.
So, it is possible to create a Flowspec rule with status INACTIVE,
i.e., a rule which will not be automatically commited via NETCONF as result of the POST method call.
* In contrast to REST API of FoD v1.3, REST API of current version v1.7,
the reuslt of PUT and PATCH method differs in some occasions (see above).
* In contrast to REST API of FoD v1.3, REST API of current version v1.7
will behave differently regarding the DELETE method (see above).
doc/configuration/configuration-v1.8.md 0 → 100644
+ 342
− 0
View file @ 9a2a1595
# Configuration v1.8
Time to configure flowspy.
First of all you have to copy the dist files to their proper position:
# cd /srv/flowspy/flowspy
# cp settings.py.dist settings.py
# cp urls.py.dist urls.py
Then, you have to edit the settings.py file to correspond to your needs. The settings one has to focus on are:
## Settings.py
Its time to configure `settings.py` in order to connect flowspy with a database, a network device and a broker.
So lets edit settings.py file.
It is strongly advised that you do not change the following to False
values unless, you want to integrate FoD with you CRM or members
database. This implies that you are able/have the rights to create
database views between the two databases:
PEER_MANAGED_TABLE = True
PEER_RANGE_MANAGED_TABLE = True
PEER_TECHC_MANAGED_TABLE = True
By doing that the corresponding tables as defined in peers/models will
not be created. As noted above, you have to create the views that the
tables will rely on.
### Administrators
ADMINS: set your admin name and email (assuming that your server can send notifications)
### Secret Key
Please put a random string in `SECRET_KEY` setting.
Make this *unique*, and don't share it with anybody. It's the unique identifier of this instance of the application.
### Allowed hosts
A list of strings representing the host/domain names that this Django site can serve. This is a security measure to prevent an attacker from poisoning caches and password reset emails with links to malicious hosts by submitting requests with a fake HTTP Host header, which is possible even under many seemingly-safe webserver configurations.
For example:
ALLOWED_HOSTS = ['*']
### Protected subnets
Subnets for which source or destination address will prevent rule creation and notify the `NOTIFY_ADMIN_MAILS`.
PROTECTED_SUBNETS = ['10.10.0.0/16']
### Minimum Prefix Length for submitted FlowSpec rules
IPv4 rules:
PREFIX_LENGTH = 29
IPv6 rules:
PREFIX_LENGTH_IPV6 = 64 - (32 - 29)
### Database
`DATABASES` should contain the database credentials:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'flowspy',
'USER': '<db user>',
'PASSWORD': '<db password>',
'HOST': '<db host>',
'PORT': '',
}
}
### Localization
By default Flowspy has translations for English and Greek. In case you want to add
another language, or remove one of the existing, you can change the `LANGUAGES`
variable and follow [django's localization documentation](https://docs.djangoproject.com/en/1.4/topics/i18n/translation/#localization-how-to-create-language-files)
You might want to change `TIME_ZONE` setting too. Here is a [list](http://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
### Cache
Flowspy uses cache in order to be fast. We recomend the usage of memcached, but
any cache backend supported by django should work fine.
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': '127.0.0.1:11211',
}
}
### Network device access for rule changes (NETCONF)
We have to inform django about the device we set up earlier.
NETCONF_DEVICE = "device.example.com"
NETCONF_USER = "<netconf user>"
NETCONF_PASS = "<netconf password>"
NETCONF_PORT = 830
### Network device access for mitigation counters (SNMP)
We have to inform django about the device we set up earlier.
SNMP_COMMUNITY = "abcd"
SNMP_IP = [
{"ip": "192.168.0.1", "port": 1000},
{"ip": "192.168.0.2", "port": 1001, "community": "abcdef"},
{"ip": "192.168.0.3", "port": 1002},
{"ip": "192.168.0.4", "port": 1002}
]
SNMP_CNTBYTES = "1.3.6.1.4.1.2636.3.5.2.1.5" # OID of bytes counter (currently unused)
SNMP_CNTPACKETS = "1.3.6.1.4.1.2636.3.5.2.1.4" # OID of packet counter
SNMP_RULESFILTER = ["__flowspec_default_inet__", "__flowspec_IAS_inet__", "__flowspec_default_inet6__", "__flowspec_IAS_inet6__"] # get only statistics of specified tables$
SNMP_POLL_INTERVAL = 8 #seconds # load new data into cache if it is older that a specified number of seconds
SNMP_TEMP_FILE = "/srv/flowspy/snmp_temp_data"
SNMP_POLL_LOCK = "/var/run/fod/snmppoll.lock"
SNMP_MAX_SAMPLECOUNT = 2016 # one month
SNMP_REMOVE_RULES_AFTER = 604800 # one month
### statistics calc based on the SNMP counters
GRAPHS_API_URL = 'http://127.0.0.1:8080/api/routes/'
STATISTICS_PER_RULE = True
STATISTICS_PER_RULE__ADD_INITIAL_ZERO = True
### redis
Redis connection configuration (as a broker for celery)
CELERY_BROKER_URL = "redis://localhost//"
POLLS_TUBE = 'polls'
BROKER_VHOST = "/"
CELERY_CONCURRENCY = 1
### Notifications
Outgoing mail address and prefix.
DISABLE_EMAIL_NOTIFICATION = False # only disable for testing
SERVER_EMAIL = "Example FoD Service <noreply@example.com>"
EMAIL_SUBJECT_PREFIX = "[FoD] "
NOTIFY_ADMIN_MAILS = ["admin@example.com"]
If you have not installed an outgoing mail server you can always use your own account (either corporate or gmail, hotmail ,etc) by adding the
following lines in settings.py:
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_USE_TLS = True #(or False)
EMAIL_HOST = 'smtp.example.com'
EMAIL_HOST_USER = 'username'
EMAIL_HOST_PASSWORD = 'yourpassword'
EMAIL_PORT = 587 #(outgoing)
### Whois servers
Add two whois servers in order to be able to get all the subnets for an AS.
PRIMARY_WHOIS = 'whois.example.com'
ALTERNATE_WHOIS = 'whois.example.net'
### Branding
Fill your company's information in order to show it in flowspy.
BRANDING = {
'name': 'Example.com',
'url': 'https://example.com',
'footer_iframe': 'https://example.com/iframes/footer/',
'facebook': '//facebook.com/example.com',
'twitter': '//twitter.com/examplecom',
'phone': '800-12-12345',
'email': 'helpdesk@example.com',
'logo': 'logo.png',
'favicon': 'favicon.ico',
}
### Shibboleth
Flowspy supports shibboleth authentication.
SHIB_ADMIN_DOMAIN = 'example.com'
SHIB_LOGOUT_URL = 'https://example.com/Shibboleth.sso/Logout'
SHIB_AUTH_ENTITLEMENT = 'urn:mace' # can also be set to '', if no filtering of users by entitlement is not needed
SHIB_SLUGIFY_USERNAME = False
attribute configuration:
SHIB_USERNAME = ['HTTP_EPPN'] # essential attribute, needed for identification of the user
SHIB_USERNAME_DISPLAY_NAME = "eduPersonPrincipalName" # only needed for displaying the error message of missing attribute
SHIB_USERNAME_DISPLAY_ADDINFO = "urn:mace:dir:attribute-def:eduPersonPrincipalName or urn:oid:1.3.6.1.4.1.5923.1.1.1.6" # only needed for displaying the error message of missing attribute
SHIB_MAIL = ['mail', 'HTTP_MAIL', 'HTTP_SHIB_INETORGPERSON_MAIL'] # essential attribute, because mail is needed for registration and notifications
SHIB_MAIL_DISPLAY_NAME = "MAIL" # only needed for displaying the error message of missing attribute
SHIB_MAIL_DISPLAY_ADDINFO = "urn:mace:dir:attribute-def:mail or urn:oid:0.9.2342.19200300.100.1.3 or SHIB_INETORGPERSON_MAIL" # only needed for displaying the error message of missing attribute
SHIB_FIRSTNAME = ['HTTP_SHIB_INETORGPERSON_GIVENNAME'] # not essential attribute
SHIB_FIRSTNAME_DISPLAY_NAME = "GIVENNAME" # only needed for displaying the error message of missing attribute
SHIB_FIRSTNAME_DISPLAY_ADDINFO = "urn:mace:dir:attribute-def:givenName or urn:oid:2.5.4.42" # only needed for displaying the error message of missing attribute
SHIB_LASTNAME = ['HTTP_SHIB_PERSON_SURNAME'] # not essential attribute
SHIB_LASTNAME_DISPLAY_NAME = "SURNAME" # only needed for displaying the error message of missing attribute
SHIB_LASTNAME_DISPLAY_ADDINFO = "urn:mace:dir:attribute-def:sn or urn:oid:2.5.4.4" # only needed for displaying the error message of missing attribute
SHIB_ENTITLEMENT = ['HTTP_SHIB_EP_ENTITLEMENT'] # not essential, if SHIB_AUTH_ENTITLEMENT='', otherwise essential
SHIB_ENTITLEMENT_DISPLAY_NAME = "eduPersonEntitlement" # only needed for displaying the error message of missing attribute
SHIB_ENTITLEMENT_DISPLAY_ADDINFO = "urn:oid:1.3.6.1.4.1.5923.1.1.1.7; the value of this attribute also has to include 'urn:mace:example.com:pki:user'" # only needed for displaying the error message of missing attribute
### Various Settings
TIME_ZONE = 'UTC'
LANGUAGE_CODE = 'en'
LANGUAGES = (
('el', _('Greek')),
('en', _('English')),
)
### Syncing the database
To create all the tables and fill with basic data needed by FoD we have to run the following commands:
cd /srv/flowspy
./manage.py migrate
./manage.py initial_data
## Create a superuser
A superuser can be added by using the following command from `/srv/flowspy/`:
./manage.py createsuperuser
## Propagate the flatpages
Inside the initial\_data/fixtures\_manual.xml file we have placed 4
flatpages (2 for Greek, 2 for English) with Information and Terms of
Service about the service. To import the flatpages, run from root
folder:
python manage.py loaddata initial_data/fixtures_manual.xml
### Celery
Celery is a distributed task queue, which helps FoD run some async tasks, like applying a flowspec rule to a router.
`Note` In order to check if celery runs or even debug it, you can run:
./manage.py celeryd --loglevel=debug
### Testing/Debugging
In order to see what went wrong you can check the following things.
#### Django
You can start the server manually by running:
./manage.py runserver 127.0.0.1:8081
By doing so, you can serve your application like gunicord does just to test that its starting properly. This command should not be used in production!
Of course you have to stop gunicorn and make sure that port 8081 is free.
#### Gunicorn
Just curl from the server http://localhost:8081
#### Celery
In order to check if celery is working properly one can start celery by typing:
./manage.py celeryd --loglevel=debug
Again this is for debug purposes.
#### Connectivity to flowspec device
Just try to connect with the credentials you entered in settings.py from the host that will be serving flowspy.
#### General Test
Log in to the admin interface via https://<hostname>/admin. Go to Peer ranges and add a new range (part of/or a complete subnet), eg. 10.20.0.0/19 Go to Peers and add a new peer, eg. id: 1, name: Test, AS: 16503, tag: TEST and move the network you have created from Available to Chosen. From the admin front, go to User, and edit your user. From the bottom of the page, select the TEST peer and save. Last but not least, modify as required the existing (example.com) Site instance (admin home->Sites). You are done. As you are logged-in via the admin, there is no need to go through Shibboleth at this time. Go to https://<hostname>/ and create a new rule. Your rule should be applied on the flowspec capable device after aprox. 10 seconds.
## Footer
Under the templates folder (templates), you can alter the footer.html
file to include your own footer messages, badges, etc.
## Welcome Page
Under the templates folder (templates), you can alter the welcome page -
welcome.html with your own images, carousel, videos, etc.
## Usage
### Web interface
FoD comes with a web interface, in which one can edit and apply new routes.
### Rest Api
FoD provides a rest api. It uses token as authentication method.
### Generating Tokens
A user can generate a token for his account on "my profile" page from FoD's
UI. Then by using this token in the header of the request he can list, retrieve,
modify and create rules.
### Example Usage
Here are some examples:
#### GET items
- List all the rules your user has created (admin users can see all the rules)
curl -X GET https://fod.example.com/api/routes/ -H 'Authorization: Token <Your users token>'
- Retrieve a specific rule:
curl -X GET https://fod.example.com/api/routes/<rule_id>/ -H 'Authorization: Token <Your users token>'
- In order to create or modify a rule you have to use POST/PUT methods.
#### POST/PUT rules
In order to update or create rules you can follow this example:
##### Foreign Keys
In order to create/modify a rule you have to connect the rule with some foreign keys:
###### Ports, Fragmentypes, protocols, thenactions
When creating a rule, one can specify:
- source port
- destination port
- port (if source = destination)
That can be done by getting the url of the desired port instance from `/api/ports/<port_id>/`
Same with Fragmentypes in `/api/fragmenttypes/<fragmenttype_id>/`, protocols in `/api/matchprotocol/<protocol_id>/` and then actions in `/api/thenactions/<action_id>/`.
Since we have the urls we want to connect with the rule we want to create, we can make a POST request like the following:
curl -X POST -H 'Authorization: Token <Your users token>' -F "name=Example" -F "comments=Description" -F "source=0.0.0.0/0" -F "sourceport=https://fod.example.com/api/ports/7/" -F "destination=203.0.113.12" https://fod.example.com/api/routes/
And here is a PUT request example:
curl -X PUT -F "name=Example" -F "comments=Description" -F "source=0.0.0.0/0" -F "sourceport=https://fod.example.com/api/ports/7/" -F "destination=83.212.9.93" https://fod.example.com/api/routes/12/ -H 'Authorization: Token <Your users token>'
doc/installation/v1.8/centos.md 0 → 100644
+ 52
− 0
View file @ 9a2a1595
# Installation on CENTOS
For the installation on CENTOS,
there is ./install-centos.sh, currently mainly used for installation inside a Docker container.
Currently, it understands several command line switches:
- --fod_dir DIRNAME : specify FoD run-time directory, where FoD files are to be copied and to be run from
- --venv_dir DIRNAME : specify where Python virtualenv used by FoD will be located
- --base-dir DIRNAME : specify base directory where FoD run-time directory and virtualenv directory will be sub directories
- --here : do not copy FoD files into a separate FoD run-time directory, but instead install and make to run from current directory (probably not so useful for production)
- --systemd : enable Systemd support explicitly, by default will be enabled if Systemd is detected
- --no_systemd : disable Systemd support explicitly, even if Systemd is detected
- --systemd_only_install : when enabling Systemd support, but do not try to contact a running Systemd to reload, only install Systemd service files (useful during Docker build)
- --systemd_check_start : when enabling Systemd support, do try to contact a running Systemd to reload and show status (default)
- --supversiord : enable Supversiord instead of Systemd, inside Docker Supversiord is now enabled by default in case no Systemd running s detected
- --no_supversiord : disable Supversiord explicitly
- --basesw : only install FoD dependencies (OS packages + Python Pip packages in virtualenv directory), do not install and setup FoD run-time directory (combines --basesw_os and --basesw_python)
- --basesw_os : only install FoD OS dependencies
- --basesw_python : only install FoD Python/Pip dependencies in virtualenv directory
- --fodproper : skip installing FoD dependencies, only install and setup FoD run-time directory
- --setup_admin_user : setup default admin
- --setup_admin_user5 : setup admin specified by user name, password, email address, peer name, first IP prefix
- --netconf : setup NETCONF specified by device address, TCP port, user name, password
- --exabgp : setup exabgp specified by BGP nodeid, IP address, AS number, peer IP address, peer AS number (only for the new FoD version, currently available in git branch "feature/exabgp_support2")
## Systemd support / Starting FoD
FoD installation currently supports Systemd, i.e., it installs Systemd startup files (./systemd/fod-\*.service) when a running Systemd is detected. Otherwise, in case of running inside docker Supversiord is enabled.
Also, Supversiord can be enabled instead of Systemd explicitly (check options above)
When not running under systemd,
alternatives for startup of FoD
are either
./runfod-fg.sh for directly starting FoD processes together with Redis task broker
or
./runfod-supervisord.sh for starting FoD processes and Redis task broker via Supervisord
or
the overall wrapper script ./runfod.sh which uses either of the startup method depending
on the choice selected by options for install-centos.sh.
The Systemd support also includes an experimental e-mail notification for startup-failures of the FoD processes
## FoD run-time status
There is ./systemd/fod-status.sh, a generic script (not limited to Systemd) for determining the process status of FoD along with some further aspects, e.g., Database connection, NETCONF configuration and reachability
doc/installation/v1.8/debian_ubuntu.md 0 → 100644
+ 60
− 0
View file @ 9a2a1595
# Installation on DEBIAN/UBUNTU
For the installation on recent DEBIAN or UBUNTU, e.g., UBUNTU focal (20.04),
there is ./install-debian.sh .
It understands several command line switches:
- --fod_dir DIRNAME : specify FoD run-time directory, where FoD files are to be copied and to be run from
- --venv_dir DIRNAME : specify where Python virtualenv used by FoD will be located
- --base-dir DIRNAME : specify base directory where FoD run-time directory and virtualenv directory will be sub directories
- --here : do not copy FoD files into a separate FoD run-time directory, but instead install and make to run from current directory (probably not so useful for production)
- --systemd : enable Systemd support explicitly, by default will be enabled if Systemd is detected
- --no_systemd : disable Systemd support explicitly, even if Systemd is detected
- --systemd_only_install : when enabling Systemd support, but do not try to contact a running Systemd to reload, only install Systemd service files (useful during Docker build)
- --systemd_check_start : when enabling Systemd support, do try to contact a running Systemd to reload and show status (default)
- --supversiord : enable Supversiord instead of Systemd, inside Docker Supversiord is now enabled by default in case no Systemd running s detected
- --no_supversiord : disable Supversiord explicitly
- --basesw : only install FoD dependencies (OS packages + Python Pip packages in virtualenv directory), do not install and setup FoD run-time directory (combines --basesw_os and --basesw_python)
- --basesw_os : only install FoD OS dependencies
- --basesw_python : only install FoD Python/Pip dependencies in virtualenv directory
- --fodproper : skip installing FoD dependencies, only install and setup FoD run-time directory
- --setup_admin_user : setup default admin
- --setup_admin_user5 : setup admin specified by user name, password, email address, peer name, first IP prefix
- --netconf : setup NETCONF specified by device address, TCP port, user name, password
- --exabgp : setup exabgp specified by BGP nodeid, IP address, AS number, peer IP address, peer AS number (only for the new FoD version, currently available in git branch "feature/exabgp_support2")
- --with-mta-postfix : if no MTA is installed, will try to install postfix (FoD operation is dependant on e-mail sending being available)
- --with-db-sqlite : use a SQLite DB for FoD DB (not recommended for production, but easy for testing)
- --with-db-mysql : will try to install MySQL and to setup FoD database in MySQL
- --with-db-mariadb : will try to install MariaDB and to setup FoD database in MariaDB
- --with-db-mysqllike : assumes that MySQL or MariaDB is installed and will try to setup FoD database based on this
## Systemd support / Starting FoD
FoD installation currently supports Systemd, i.e., it installs Systemd startup files (./systemd/fod-\*.service) when a running Systemd is detected. Otherwise, in case of running inside docker Supversiord is enabled.
Also, Supversiord can be enabled instead of Systemd explicitly (check options above)
When not running under systemd,
alternatives for startup of FoD
are either
./runfod-fg.sh for directly starting FoD processes together with Redis task broker
or
./runfod-supervisord.sh for starting FoD processes and Redis task broker via Supervisord
or
the overall wrapper script ./runfod.sh which uses either of the startup method depending
on the choice selected by options for install-debian.sh.
The Systemd support also includes an experimental e-mail notification for startup-failures of the FoD processes
## FoD run-time status
There is ./systemd/fod-status.sh, a generic script (not limited to Systemd) for determining the process status of FoD along with some further aspects, e.g., Database connection, NETCONF configuration and reachability
doc/installation/v1.8/docker.md 0 → 100644
+ 183
− 0
View file @ 9a2a1595
# Installing Flowspy v1.8 with Docker
Flowspy users may wish to test Flowspy in a container rather than directly onto a server or virtual host; this document shows how to build and deploy Flowspy in a Docker container.
Flowspy installs into `/srv/flowspy` by default; if you wish to use a different directory then you will need to update several configuration files. It is also assumed that the `root` user will perform every action.
# Requirements
Docker must be installed on your target OS; the installation of Docker is outside the scope of this document.
Docker images require disk space on the host OS; approximately 2 GB for Flowspy and approximately 200 MB for CentOS 7 which is the default OS used.
Although the default `Dockerfile` = `Dockerfile.fod.debian` is for Debian, `Dockerfile.fod.centos.new` for CENTOS 7, and `Dockerfile.fod.centos.old` left for compatibility,
several older Dockerfiles for both Debian and CentOS and can be found in the `Dockerfiles.d` directory. These are meant **for testing only** and are _not_ production-ready.
The containers run `gunicorn`, `celeryd` and Redis - and use a SQLite database (`/srv/flowspy/example-data`) to hold resources. They also make a web server available on port 8000.
# Building the Flowspy container
First build your Flowspy container. The default (`Dockerfile` = `Dockerfile.fod.debian`) is a container which uses Debian and `supervisord`, but there are other options; `Dockerfile.fod.centos.\*` uses CENTOS 7, `Dockerfile.fod.centos.new` with `supervisord`, `Dockerfile.fod.centos.old` is left for compatibility without `supervisord`.
`Dockerfile.fod.debian` and `Dockerfile.fod.centos.new` by default will split the installation of FoD during container build into 3 phases (OS dependencies, python/pip dependencies, FoD proper installation) to exploit Docker build cache for faster rebuilding.
Moreover, `Dockerfile.fod.debian` and `Dockerfile.fod.centos.new` can be edited for uncommenting/commenting to switch more options: e.g., use `systemd` instead of `supervisord` or not splitting the FoD installation into 3 phases
(first docker build will be a bit faster and the docker container building is more efficient, as the number of docker image layers is reduced, but subsequent build after some changes in code or settings will require nearly the same amount of time as the first build)
> Although the examples below use CentOS, just replace `Dockerfile.centos.new` with `Dockerfile.debian` if you wish to use Debian instead.
## CentOS with `supervisord`
```
docker build -f Dockerfile.centos.new -t fod-centos .
```
# Running Flowspy
You should now be able to see relevant images for Flowspy in your Docker environment;
```
[user@host-os ~]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
fod-centos latest 42da1eb2b4a5 3 minutes ago 1.73 GB
```
Although the web server listens on port 8000 by default you can map this to a different port on the Docker command line. Our examples map port 8000 on the docker container to port 80 on the host OS.
```
docker run -p 80:8000 fod-centos # run in foreground
or
docker run -d -p 80:8000 fod-centos # run in background
```
If you are using the Debian container then replace `fod-centos` with `fod-debian`.
You can confirm that the docker container is running with `docker ps` - this will also give you the autogenerated name of the container and show you port mappings if any;
```
[user@host-os Dockerfiles.d]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5b125f76470d fod-centos "/srv/flowspy/runfod.sh" 1 month ago Up 2 days 0.0.0.0:80->8000/tcp lucid_mcnulty
```
You can connect the container directly and start a shell using either the `CONTAINER ID` or the `NAME` and the `docker exec -it` command;
```
[user@host-os Dockerfiles.d]$ docker exec -ti lucid_mcnulty bash # replace with name of whose of your container
[root@5b125f76470d /]#
```
# Initial setup of Flowspy
You need to set a couple of options once the container is running; the password for the `admin` user and the details of the NETCONF-enabled router which Flowspy will use to inject FLOWSPEC routes.
- NETCONF_DEVICE
- NETCONF_PORT
- NETCONF_USER
- NETCONF_PASS
## via Web GUI
Access the Flowspy web server at `http://127.0.0.1:8000/setup/` (if you are connecting from _inside_ the container) or `http://your.host-os.ipaddress/` if you are connecting from _outside_ the container (and have mapped port 8000 to port 80).
Once you hit "Save", the config will be saved **in the container** as `/srv/flowspy/flowspy/settings_local.py`.
You will need to restart the container for Flowspy to pick up this config; from the Host OS
```
docker restart lucid_mcnulty # replace name with whose of your container
```
You can now access the Flowspy web server using your new credentials.
## via manual configuration
> This needs to be done from **inside** the container.
The relevant file is `/srv/flowspy/flowspy/settings.py`; the relevant variables are
- NETCONF_DEVICE
- NETCONF_PORT
- NETCONF_USER
- NETCONF_PASS
You will then need to use the `manage.py` script to change the admin password;
```
cd /srv/flowspy; ./pythonenv ./manage.py changepassword admin
```
You will need to restart the container for Flowspy to pick up this config; from the Host OS
```
docker restart lucid_mcnulty # replace name with whose of your container
```
You can now access the Flowspy web server using your new credentials.
## As fixed parameters in the Dockerfile
As a new alternative, admin user name/password and NETCONF parameters can be specified in the Dockerfile.fod.debian or Dockerfile.fod.centos.new:
Comment the line
RUN ./install-centos.sh --both --here --supervisord
and as replacement uncomment the line
\#RUN ./install-centos.sh --both --here --supervisord --setup_admin_user --setup_admin_user5 admin adminpwd admin@localhost.local testpeer 0.0.0.0/0 --netconf 172.17.0.3 830 netconf netconf
(In case of Dockerfile.fod.debian ./install-centos.sh is replaced by ./install-debian.sh)
Replace values as it suits you, the 2 parameters following "--setup_admin_user5" are admin username and password, the 4 parameters following "--netconf" are NETCONF_DEVICE, NETCONF_PORT, NETCONF_USER, NETCONF_PASS .
The docker container built with this changed Dockerfile will have the respective values setup and /setup URL will not be usable any more.
## Testing NETCONF connectivity
**Please** make sure that the docker container has IP connectivity to your NETCONF device!
If you wish to confirm this, then use `yum install telnet` to install telnet in your container and telnet to whichever port on which your NETCONF device is listening (`netconf-ssh`/830 by default);
```
[root@5b125f76470d flowspy]# telnet 192.0.2.1 netconf-ssh
Trying 192.0.2.1...
Connected to 192.0.2.1.
Escape character is '^]'.
SSH-2.0-OpenSSH_7.5
^]
telnet> close
Connection closed.
[root@5b125f76470d flowspy]#
```
# Misc
You may access the containerised Web GUI after the password of the admin user has been reset via `http://127.0.0.1:8000/altlogin`.
> Do not try to use the Shibboleth login via `/altlogin` as it will not work without a fully configured Shibboleth SP
# Persistent storage
The configuration will only last as long as the docker container itself; if the container is stopped or the host OS rebooted then the configuration will be lost.
You can use [Docker volumes](https://docs.docker.com/storage/volumes/) for persistent storage;
```
[user@host-os]$ docker volume create Flowspy-srv
Flowspy-srv
[user@host-os]$ docker volume inspect Flowspy-srv
[
{
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/Flowspy-srv/_data",
"Name": "Flowspy-srv",
"Options": {},
"Scope": "local"
}
]
```
Then use the Docker CLI argument `-v` (**v**olume) to mount the container at runtime;
```
docker run -d -p 80:8000 -v Flowspy-srv:/srv fod-centos
```
doc/installation/v1.8/docker_extra.md 0 → 100644
+ 53
− 0
View file @ 9a2a1595
# Extra Docker containers for testing FoD without router hardware
### NETCONF test server docker container
When no real NETCONF-enabled router supporting BGP FlowSpec is available, just for testing the NETCONF test server Docker container can be used:
In FoD-cloned installation dir, e.g., residing in /srv/flowspy, go to sub directory `router-container`
```
docker build -t juniper .
docker run -it --name juniper -p 830:830 --rm juniper:latest
```
To manually test NETCONF access to the test server, see `/router-container/run.txt`.
Now, find out IP address of the running test server container, e.g., by
docker inspect "$DOCKERID_NETCONF" | grep IPAddress # find out DOCKERID_NETCONF, e.g. by "docker ps"
In FoD test container (see above) configure this IP address of the NETCONF test server,
with NETCONF port 830, NETCONF_USER "netconf" and NETCONF_PASS "netconf"
Now, FoD can submit FlowSpec rules which are actually only stored inside the NETCONF test server
without an actual effect on any network, but FoD functionality of controlling rules can be tested.
### NETCONF test server docker container based on netconfd instead of netopeer
similar to router-container/Dockerfile but will use netconfd (DEBIAN package) instead of CESNET's netopeer NETCONF server
- ./Dockerfiles.d/Dockerfile.vnet_router0a :
### NETCONF test server docker container extended to virtual DDoS test network
Based on an instance of the NETCONF test server docker container
Extending with Mininet/OpenVSwitch,
a basic a script for syncing NETCONF rules stored in the test NETCONF server
to OpenFlow rules in Mininet simulating BGP FlowSpec rules (not all cases supported),
SNMPd and a Perl SNMPd statistic collector script
Yields a more complete simulation of a router for FoD.
- ./Dockerfiles.d/Dockerfile.vnet_router1 : based on netopeer2
- ./Dockerfiles.d/Dockerfile.vnet_router2 : similar to Dockerfile.vnet_router1, but will use netconfd (DEBIAN package) instead of CESNET's netopeer NETCONF server
- ./Dockerfiles.d/Dockerfile.vnet_router2.ubuntu: like ./Dockerfiles.d/Dockerfile.vnet_router2, but based on UBUNTU
- ./Dockerfiles.d/Dockerfile.vnet_router2.debian.exabgp1: extending ./Dockerfiles.d/Dockerfile.vnet_router2 to install some stuff for bgp testing inside the Mininet: exabgp, quagga, bird1
- ./Dockerfiles.d/Dockerfile.vnet_router2.debian.exabgp2: extending ./Dockerfiles.d/Dockerfile.vnet_router2 to install some stuff for bgp testing inside the Mininet: exabgp, bird2
- ./Dockerfiles.d/Dockerfile.vnet_router2.debian.exabgp2.topo2: extending ./Dockerfiles.d/Dockerfile.vnet_router2 to simple BGP testing vnet based on exabgp and bird2
(find instructions how to build and run inside the Dockerfiles)
doc/installation/v1.8/generic.md 0 → 100644
+ 151
− 0
View file @ 9a2a1595
# Installing FoD v1.7 Generic
This guide provides general information about the installation of FoD. In case you use Debian/UBUNTU, we provide detailed instructions for the installation.
Also it assumes that installation is carried out in `/srv/flowspy`
directory. If other directory is to be used, please change the
corresponding configuration files. It is also assumed that the `root` user
will perform every action.
TO UPDATE
## Requirements
### System Requirements
In order to install FoD properly, make sure the following software is installed on your computer.
- apache 2
- libapache2-mod-proxy-html
- gunicorn
- redis
- mysql
- python3
- python3-dev
- python-virtualenv
- pip
- gcc
### Download FoD
You can clone FoD from GEANT github repository.
mkdir -p /srv/
cd /srv/
git clone https://github.com/GEANT/FoD flowspy
### Python-virtualenv
cd /srv
virtualenv --python=python3 /srv/venv
. /srv/venv/bin/activate
### Pip
In order to install the required python packages for FoD you can use pip:
. /srv/venv/bin/activate
cd /srv/flowspy
pip install -r requirements.txt
### Create a database
If you are using mysql, you should create a database:
mysql -u root -p -e 'create database fod'
### Copy dist files
cd /srv/flowspy/flowspy
cp settings.py.dist settings.py
cp urls.py.dist urls.py
### Device Configuration
FoD generates and commits flowspec rules to a
device via netconf. You have to create an account
with rw access to flowspec and set these credentials
in settings.py. See Configuration for details.
### Adding some default data
Into `/srv/flowspy` you will notice that there is a directory called `initial_data`. In there, there is a file called `fixtures_manual.xml` which contains some default static pages for django's flatpages app. In this file we have placed 4 flatpages (2 for Greek, 2 for English) with Information and Terms of Service about the service. To import the flatpages, run from `/srv/flowspy`:
. /srv/venv/bin/activate
./manage.py loaddata initial_data/fixtures_manual.xml
### Redis
Just make sure redis is installed and started
# on DEBIAN/UBUNTU (>= DEBIAN 10/UBUNTU 18)
apt-get install redis-server
systemctl enable redis-server
systemctl start redis-server
### mkdocs-based internal documentation
Just make sure mkdocs is installed and prepare documentation with it
(the documentation is optional and non-essential for the operation of FoD)
# on DEBIAN/UBUNTU (>= DEBIAN 10/UBUNTU 18)
apt-get install mkdocs
cd /srv/flowspy
mkdocs build
### Apache2
Apache proxies gunicorn. Things are more flexible here as you may follow your own configuration and conventions.
#### Example config
Here is an example configuration.
<VirtualHost *:80>
ServerName fod.example.org
DocumentRoot /var/www
ErrorLog /var/log/httpd/fod_error.log
LogLevel debug
CustomLog /var/log/httpd/fod_access.log combined
RewriteEngine On
RewriteRule ^/(.*) https://fod.example.com/$1 [L,R]
</VirtualHost>
<VirtualHost *:443>
SSLEngine on
SSLProtocol TLSv1
SSLCertificateFile /home/local/GEANT/dante.spatharas/filename.crt
SSLCertificateKeyFile /home/local/GEANT/dante.spatharas/filename.key
SSLCACertificateFile /home/local/GEANT/dante.spatharas/filename.crt
Alias /static /srv/flowspy/static
AddDefaultCharset UTF-8
IndexOptions +Charset=UTF-8
#SSLProxyEngine off
ProxyErrorOverride off
ProxyTimeout 28800
ProxyPass /static !
ProxyPass / http://localhost:8080/ retry=0
ProxyPassReverse / http://localhost:8080/
</VirtualHost>
`Important!` You have to comment out/disable the default `Virtualhost` defined on line 74 until the end of this block at `/etc/httpd/conf.d/ssl.conf `.
### Gunicorn
FoD is served via gunicorn and is then proxied by Apache. If the above
directory conventions have been followed so far, then your configuration
should be:
CONFIG = {
'mode': 'django',
'working_dir': '/srv/flowspy',
'args': (
'--bind=127.0.0.1:8081',
'--workers=1',
'--worker-class=egg:gunicorn#gevent',
'--timeout=30',
'--debug',
'--log-level=debug',
'--log-file=/var/log/gunicorn/fod.log',
),
}
mkdocs.yml
+ 10
− 1
View file @ 9a2a1595
...@@ -2,7 +2,7 @@ site_name: Firewall On Demand ...@@ -2,7 +2,7 @@ site_name: Firewall On Demand
#repo_url: https://github.com/grnet/flowspy/ #repo_url: https://github.com/grnet/flowspy/
repo_url: https://github.com/GEANT/FOD/ repo_url: https://github.com/GEANT/FOD/
#site_author: Stavros Kroustouris #site_author: Stavros Kroustouris
site_author: GÉANT4-3 WP8 T3 site_author: GÉANT5-1 WP8-T3
site_dir: static/site site_dir: static/site
docs_dir: doc docs_dir: doc
#theme: readthedocs #theme: readthedocs
...@@ -21,14 +21,23 @@ nav: ...@@ -21,14 +21,23 @@ nav:
- 'CentOS': 'installation/v1.7/centos.md' - 'CentOS': 'installation/v1.7/centos.md'
- 'Docker': 'installation/v1.7/docker.md' - 'Docker': 'installation/v1.7/docker.md'
- 'Docker Extra for Testing': 'installation/v1.7/docker_extra.md' - 'Docker Extra for Testing': 'installation/v1.7/docker_extra.md'
- 'v1.8':
- 'Generic': 'installation/v1.8/generic.md'
- 'DEBIAN/UBUNTU': 'installation/v1.8/debian_ubuntu.md'
- 'CentOS': 'installation/v1.8/centos.md'
- 'Docker': 'installation/v1.8/docker.md'
- 'Docker Extra for Testing': 'installation/v1.8/docker_extra.md'
- 'Configuration': - 'Configuration':
- 'v1.3': 'configuration/configuration-v1.3.md' - 'v1.3': 'configuration/configuration-v1.3.md'
- 'v1.7': 'configuration/configuration-v1.7.md' - 'v1.7': 'configuration/configuration-v1.7.md'
- 'v1.8': 'configuration/configuration-v1.8.md'
- 'Administration and Usage': - 'Administration and Usage':
- 'Basic Administration and Usage (v1.7)': 'administration_and_usage/basic_administration_and_usage-v1.7.md' - 'Basic Administration and Usage (v1.7)': 'administration_and_usage/basic_administration_and_usage-v1.7.md'
- 'Basic Administration and Usage (v1.8)': 'administration_and_usage/basic_administration_and_usage-v1.8.md'
- 'API': - 'API':
- 'v1.3': 'api/api-v1.3.md' - 'v1.3': 'api/api-v1.3.md'
- 'v1.7': 'api/api-v1.7.md' - 'v1.7': 'api/api-v1.7.md'
- 'v1.8': 'api/api-v1.8.md'
- 'Development': - 'Development':
- 'Design Overview': 'development/design-overview.md' - 'Design Overview': 'development/design-overview.md'
templates/500.html
+ 1
− 1
View file @ 9a2a1595
...@@ -3362,7 +3362,7 @@ n6PVRQV1z4Xb5R2/ig2gdF2X+48iDxWt0GPAb2xj+2RYXbWgbiBKtWpQ/T913YsPOb733o93/rGN ...@@ -3362,7 +3362,7 @@ n6PVRQV1z4Xb5R2/ig2gdF2X+48iDxWt0GPAb2xj+2RYXbWgbiBKtWpQ/T913YsPOb733o93/rGN
<div id="footer"> <div id="footer">
If you have any questions or need help, contact GÉANT OC at <a href='mailto:support@oc.geant.net'>support@oc.geant.net</a> or +44 1223 733033. If you have any questions or need help, contact GÉANT OC at <a href='mailto:support@oc.geant.net'>support@oc.geant.net</a> or +44 1223 733033.
<a href="https://github.com/GEANT/FOD">FoD</strong></a> - <p> <a href="https://github.com/GEANT/FOD">FoD</strong></a> - <p>
Designed and developed in GÉANT4-3 WP8 T3; Originally designed and developed by GRNET NOC Designed and developed in GÉANT5-1 WP8-T3; Originally designed and developed by GRNET NOC
</div> </div>
</div> </div>
......
templates/footer.html
+ 1
− 1
View file @ 9a2a1595
...@@ -7,7 +7,7 @@ ...@@ -7,7 +7,7 @@
{% endif %} {% endif %}
<!--<div style="padding-top: 10px;"><a href="https://code.grnet.gr/projects/flowspy">Version: <strong>{{ settings.SW_VERSION }}</strong></a> --> <!--<div style="padding-top: 10px;"><a href="https://code.grnet.gr/projects/flowspy">Version: <strong>{{ settings.SW_VERSION }}</strong></a> -->
<div style="padding-top: 10px;"><a href="https://github.com/GEANT/FOD">Version: <strong>{{ settings.SW_VERSION }}</strong></a> - <div style="padding-top: 10px;"><a href="https://github.com/GEANT/FOD">Version: <strong>{{ settings.SW_VERSION }}</strong></a> -
{% trans "Designed and developed in GÉANT4-3 WP8 T3; Originally designed and developed by GRNET NOC" %} {% trans "Designed and developed in GÉANT5-1 WP8-T3; Originally designed and developed by GRNET NOC" %}
</a> </a>
</div> </div>
<div style="padding-top: 10px;"> <div style="padding-top: 10px;">
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment