Skip to content
Snippets Groups Projects
Commit 42d22788 authored by Ian Galpin's avatar Ian Galpin
Browse files

Added documentation for getting survey-db populated

parent 256e1ec0
Branches
Tags
No related merge requests found
.. development
Development Setup Guide
=======================
This is a short description of what is required to setup a development
environment for Compendium-V2. This guide is a work-in-progress as the
project is still in the early development phase.
.. contents:: :local:
Database Setup
--------------
Compendium makes use of two schemas:
- **survey**: This stores past and current survey questions and answers
- **presentation**: This stores the processed reporting information that will be used for presentation side
.. note::
For the time being, the survey data is being treated as an external entity with
no project defined SQLAlchemy entities. This will change once the project reaches
the import and data migration phase.
Create the database
+++++++++++++++++++
Create the new PostgreSQL user and database:
.. code-block:: shell
sudo -u postgres createuser 'compendium_v2'
sudo -u postgres createdb 'compendium_v2' -O 'compendium_v2'
sudo -u postgres psql
Then assign a random password for the user:
.. code-block:: psql
alter user 'compendium_v2' with encrypted password '<password>';
At this point you need to update your ``config-local.json`` with the new database credentials.
Setup survey schema
+++++++++++++++++++
For setting up an development environment, duplicating the data from the test environment is the best idea.
.. code-block:: shell
pg_dump --host=test-postgres01.geant.org --dbname=compendium_v2 --username=compendium_v2 --schema survey > "test-db_compendium_v2_survey-$(date +%Y-%m-%d_%H%M%S).sql"
psql --host=localhost --dbname=compendium_v2 --username=compendium_v2 -v ON_ERROR_STOP=1 --file=test-db_compendium_v2_survey-2022-11-21_224041.sql
.. warning::
There is no need to perform the next steps in this section for a dev environment. The steps are provided as a record for creating the survey schema on *test* and *uat* systems.
.. warning::
Please note that should you run these commands on a GNU/Linux system, you need to remove the empty quotes after the ``-i`` switch in the ``sed`` commands.
The survey data can be extracted with the following steps:
#. Dump out the following tables
* compendia
* sections
* questions
* countries
* nrens
* nrens_mappings
* answers
* budgets
* total_budgets_nren
* ``pg_dump --no-owner --host=uat-postgres01.geant.org --dbname=compendium --username=compendium -t answers -t budgets -t compendia -t countries -t nrens -t nrens_mappings -t questions -t sections -t total_budgets_nren > "uat-db_compendium_export-$(date +%Y-%m-%d_%H%M%S).sql"``
#. Replace mention of the `public` schema with `survey` schema. This breaks some of the data, which needs to be reversed. The first command shows what lines will be affected by the first sed command and should be used to update the fix invalid data
* ``grep 'public\.' uat-db_compendium_export-2022-11-21_235823.sql`` # Check what non-SQL data will change
* ``sed -i '' 's/public\./survey\./g' uat-db_compendium_export-2022-11-21_235823.sql`` # Rename schema
* ``sed -i '' 's/survey\.html/public\.html/g' uat-db_compendium_export-2022-11-21_235823.sql`` # Revert 'public.html' change
* ``sed -i '' 's/survey\. /public\. /g' uat-db_compendium_export-2022-11-21_235823.sql`` # Revert 'public.' change
#. Create the `survey` schema, repeat for `uat` server
* ``echo 'CREATE SCHEMA IF NOT EXISTS survey;' | psql --host=test-postgres01.geant.org --dbname=compendium_v2 --username=compendium_v2``
* ``echo "CREATE DOMAIN survey.udt_country_code AS character varying(2) CONSTRAINT udt_country_code_check CHECK ((((VALUE)::text ~ '([A-Za-z]+){2}'::text) AND (length((VALUE)::text) <= 2)));" | psql --host=test-postgres01.geant.org --dbname=compendium_v2 --username=compendium_v2``
* ``echo "CREATE DOMAIN survey.udt_m49_code AS character varying(7) CONSTRAINT udt_m49_code_check CHECK ((length((VALUE)::text) <= 7));" | psql --host=test-postgres01.geant.org --dbname=compendium_v2 --username=compendium_v2``
#. Import modified sql file into test and uat, repeat for `uat` server
* ``psql --host=test-postgres01.geant.org --dbname=compendium_v2 --username=compendium_v2 -v ON_ERROR_STOP=1 --file=uat-db_compendium_export-2022-11-21_235823.sql``
Setup presentation schema
+++++++++++++++++++++++++
.. code-block:: shell
flask db prepare
\ No newline at end of file
......@@ -10,3 +10,4 @@ a React web application that consumes and renders the json data.
:caption: Contents:
api
development
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment