Administrator guide#
This document describes the access to the docker registry and repository needed for running the SIRA application, followed by the actual instructions on how to set-up and start the app.
The last version of the application is always available, if not in the middle of the update step, at https://nicelabel.docker-e9.ijs.si. You can visit https://nicelabel.docker-e9.ijs.si/issues/index for an application entrypoint.
Prerequisites#
It is recommended that a new Linux host machine is prepared for the application deployment. Ubuntu and Ubuntu-based distributions were tested to work.
The following applications have to be pre-installed before installing the application:
It is recommended to set-up the docker command to be used without elevating the permission to sudo before invoking it.
Docker registry access#
Credentials
You should have already received the Docker image registry credentials. Contact your contact point for JSI - DIS in order to obtain new credentials.
URL#
The URL for the Docker registry, where the application images are hosted:
Usage#
First, log in to the Docker registry by entering into the terminal
docker login registry-dis.ijs.si
Then enter the required credentials. You should now have access to the necessary Docker images.
Git Repository#
Credentials
You should have already received the Git repository credentials. Contact your contact point for JSI - DIS to obtain new credentials.
URL#
The URL for the Git repository, where the application images are hosted:
Usage#
Clone the repository with the terminal command
git clone --depth 1 https://<USERNAME>:<PASSWORD>@repo.ijs.si/nicelabel/email-app.git
Usage of the --depth 1 option will speed up the code download by downloading only the latest version of the code.
Architecture#
The application consist of running three docker containers:
- Database container
- Events daemon container
- Application container
To keep the images small and secure, each image used as the basis for container creating includes only the required artifacts for running the code.
Database container#
The database container is based on the official Postgresql image. Additionally, it includes the wal2Json extension and has the following parameters changed from defaults: wal_level = logical and max_wal_senders = 4.
By default, the database container only listens on the application docker network and is, therefore, not accessible from other applications. If access from other applications is required, the relevant docker-compose file sections should be updated. This might be necessary in order to enable database synchronization.
Events daemon container#
The events daemon container is responsible for keeping the database up to date with transformed email text that is suitable for search. It is based on the official Python 3.7 image and includes the SIRA logic.
By default, the process for observing the database changes through the logical replication protocol is launched, and any data change on the database that requires the transformation of text for search triggers the transformation computation.
By default, the events daemon container only listens on the application docker network and is, therefore, not accessible from other applications. If access from other applications is required, the relevant docker-compose file sections should be updated.
Application container#
The application container packages the Flask application with all the logic required to provide the SIRA web application. It is based on the official Python 3.7 docker image. The containers for events daemon and application are created from the same image, only the commands for process start are different.
By default, the application container only listens on the application network. In the docker-compose-deploy-nicelabel.yaml, the container port 3000 is mapped to the host port 3000, making the application accessible at http://localhost:3000.
The application container also includes the generated front-end single page application (SPA), used for similar issue search served at the index endpoint, e.g., http://localhost:3000/?issueID=76746. React framework is used for SPA development.
Clean app start#
Open the terminal and descend into the application directory:
cd email-app
Start the application and relevant services:
docker-compose -f docker-compose.yaml -f docker-compose-deploy-nicelabel.yaml up -d
You can observe the logs with:
docker-compose logs -f
When the events container daemon finishes processing, the application should be available at http://localhost:3000. You can visit http://localhost:3000/issues/index to view all available issues in one interactive table.
Database synchronization#
The SIRA backend supports the automatic transformation of email text to prepare it for search when new emails are made available in the database. This is achieved through the logical replication protocol supported by the Postgresql, the database software that powers SIRA.
On the initial run, the database will be populated with issues and emails received up to April 24th, 2019. This means that the data in the system may be old and not so relevant anymore.
For the SIRA to include new emails and issues in the search results, specific data from the existing issue tracking system database has to be synchronized to the SIRA database.
SIRA requires the following tables and their columns.
| issues |
|---|
| id |
| emails |
|---|
| id |
| issue_id - ForeignKey(issues.id) |
| subject |
| full_text |
| sender |
| sent_time |
The mapping of column names should be done case by case since it depends on the existing issue tracking system database.
After the synchronization is set up, and the SIRA database is updated, the required transformation for search should be computed automatically by the events daemon container.
Update#
Descend into the application directory:
cd email-app
Obtain the latest version from the git repository master branch:
git pull
Download latest docker images:
docker-compose -f docker-compose.yaml -f docker-compose-deploy-nicelabel.yaml pull
Start the application and relevant services:
docker-compose -f docker-compose.yaml -f docker-compose-deploy-nicelabel.yaml up -d
The containers should be automatically recreated and based on new images. You can observe the logs with:
docker-compose logs -f
Troubleshooting#
Send an email to one of the JSI - DIS contact point for help in troubleshooting.
FAQ#
Here will be a list of frequently asked questions and issues and provided answers.