diff --git a/.gitignore b/.gitignore
index bbd7a3a..ca7e6d6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -434,3 +434,13 @@ FodyWeavers.xsd
*.msix
*.msm
*.msp
+
+times.md
+docs/traffic_light_examples
+misc/mysql-workbench-community_**
+misc/berths_and_terminals.csv
+src/notebooks_metz
+src/server/BreCal.egg-info
+src/lib_brecal_utils/**.egg-info/**
+
+
diff --git a/Ampelfunktion.md b/Ampelfunktion.md
new file mode 100644
index 0000000..b655d12
--- /dev/null
+++ b/Ampelfunktion.md
@@ -0,0 +1,33 @@
+# Ampelfunktion Bremen Calling
+
+| Ampelfunktionen | Beschreibung | Definition | Bemerkungen |
+|-----------------|-----------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|
+| | | | |
+| 0001 | Nicht alle Zeiten sind zugeordnet | Bedingungen:
- Header der Zeile ist zugeordnet (Agentur, Festmacher usw.)
- Zugeordnete Zeit ist leer | |
+| 0001 - A | Agentur / einkommend | times_agency:
- participant_id = ausgefüllt
- ETA Berth = leer
___zum Zeitpunkt 20 Std vor___
shipcall:
- eta | gelb |
+| 0001 - B | Agentur / ausgehend + Verholung | times_agency:
- participant_id = ausgefüllt
- ETD Berth = leer
___zum Zeitpunkt 20 Std vor___
shipcall:
- etd | gelb |
+| 0001 - C | Festmacher / einkommend | times_mooring:
- participant_id = ausgefüllt
- ETA Berth = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETA Berth | gelb |
+| 0001 - D | Festmacher / ausgehend + Verholung | times_mooring:
- participant_id = ausgefüllt
- ETD Berth = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETD Berth | gelb |
+| 0001 - F | Hafenamt / einkommend | times_portauthority:
- participant_id = ausgefüllt
- ETA Berth = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETA Berth | gelb |
+| 0001 - G | Hafenamt / ausgehend + Verholung | times_portauthority:
- participant_id = ausgefüllt
- ETD Berth = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETD Berth | gelb |
+| 0001 - H | Lotsen / einkommend | times_pilot:
- participant_id = ausgefüllt
- ETA Berth = leer
______
times_agency:
- ETA Berth < 16 Stunden entfernt | gelb |
+| 0001 - I | Lotsen / ausgehend + Verholung | times_pilot:
- participant_id = ausgefüllt
- ETD Berth = leer
______
times_agency:
- ETD Berth < 16 Stunden entfernt | gelb |
+| 0001 - J | Schlepper / einkommend | times_tug:
- participant_id = ausgefüllt
- ETA Berth = leer
______
times_agency:
- ETA Berth < 16 Stunden entfernt | gelb |
+| 0001 - K | Schlepper / ausgehend + Verholung | times_tug:
- participant_id = ausgefüllt
- ETD Berth = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETD Berth | gelb |
+| 0001 - L | Terminal / einkommend | times_terminal:
- participant_id = ausgefüllt
- Operation Start = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETA Berth | gelb |
+| 0001 - M | Terminal / ausgehend + Verholung | times_terminal:
- participant_id = ausgefüllt
- Operation Ende = leer
___zum Zeitpunkt 16 Std vor___
times_agency:
- ETD Berth | gelb |
+| 0002 | Zeiten für einen Eintrag weichen voneinander ab | Bedingungen:
- Header der Zeile ist zugeordnet (Agentur, Festmacher usw. - außer BSMD-Spalte)
- Zeiten ungleich (leere Einträge nicht berücksichtigen => 0001) | |
+| 0002 - A | Agentur + Festmacher + Hafenamt + Lotsen + Schlepper / einkommend | times_agency:
- ETA Berth
____ungleich____
times_mooring:
- ETA Berth
____ungleich____
times_portauthority:
- ETA Berth
____ungleich____
times_pilot:
- ETA Berth
____ungleich____
times_tug:
- ETA Berth | rot |
+| 0002 - B | Agentur + Festmacher + Hafenamt + Lotsen + Schlepper / ausgehend | times_agency:
- ETD Berth
____ungleich____
times_mooring:
- ETD Berth
____ungleich____
times_portauthority:
- ETD Berth
____ungleich____
times_pilot:
- ETD Berth
____ungleich____
times_tug:
- ETD Berth | rot |
+| 0002 - C | Agentur + Festmacher + Hafenamt + Lotsen + Schlepper / Verholung | times_agency:
- ETA Berth
- ETD Berth
____ungleich____
times_mooring:
- ETA Berth
- ETD Berth
____ungleich____
times_portauthority:
- ETA Berth
- ETD Berth
____ungleich____
times_pilot:
- ETA Berth
- ETD Berth
____ungleich____
times_tug:
- ETA Berth
- ETD Berth | rot |
+| 0003 | Arbeitszeit überschneidet sich mit Fahrtzeit | Bedingungen:
- Header der Zeile ist zugeordnet (Terminal)
- Zeiten passt nicht zu Ankunft / Abfahrt (leere Einträge nicht berücksichtigen => 0001) | |
+| 0003 - A | Terminal / einkommend | times_terminal:
- Operation Start
___vor (kleiner als)____
times_agency:
- ETA Berth | rot |
+| 0003 - B | Terminal / ausgehend + Verholung | times_terminal:
- Operation Ende
___nach (größer als)____
times_agency:
- ETD Berth | rot |
+| 0004 | Tidezeiten passen nicht zu Fahrzeiten | Bedingungen:
- Header der Zeile ist zugeordnet (Agentur)
- Tidezeit ausgefüllt | |
+| 0004 - A | Agentur / einkommend | times_agency:
- ETA Berth
___vor (kleiner als)____
times_agency:
- Tidefenster von
___und/oder___nach (größer als)____
times_agency:
- Tidefenster bis | rot |
+| 0004 - B | Agentur / ausgehend + Verholung | times_agency:
- ETD Berth
___vor (kleiner als)____
times_agency:
- Tidefenster von
___und/oder___nach (größer als)____
times_agency:
- Tidefenster bis | rot |
+| 0005 | Zu viele Schiffe mit gleicher Fahrtzeit | Bedingungen:
- Header der Zeile ist zugeordnet (Agentur)
- Übergreifend über die Einträge | |
+| 0005 - A | Agentur / einkommend + ausgehend + Verholung | ____Zählen wenn gleich:____
times_agency:
- ETA Berth
___und____
times_agency:
- ETD Berth
___mehr als 3____
| gelb |
+| 0006 | Agentur und Terminal planen mit unterschiedlichen Liegeplätzen | Bedingungen:
- Header der Zeile ist zugeordnet (Agentur / Terminal)
- LP jeweils ausgefüllt | |
+| 0006 - A | Agentur + Terminal (Liegeplatz) / einkommend + ausgehend + Verholung | times_agency:
- Liegeplatz
____ungleich____
times_terminal:
- Liegeplatz | gelb |
+| 0006 - B | Agentur + Terminal (Anlegeseite) / einkommend + ausgehend + Verholung | times_agency:
- Anlegeseite
____ungleich____
times_terminal:
- Anlegeseite | gelb |
diff --git a/change_log_metz.md b/change_log_metz.md
new file mode 100644
index 0000000..99a6a06
--- /dev/null
+++ b/change_log_metz.md
@@ -0,0 +1,140 @@
+
+
+# 20231012:
+Building out the remaining validation rules, which are all finalized now (0001 - 0006).
+Made sure that the ValidationRuleState.NONE is 0. Within the notification check, a notification requirement is based on new_state>old_state. To prevent
+future none->green changes to trigger a notification, a 'max' call is used. When max(GREEN, NONE), then the old state is considered at least 'green'. This solves the issue.
+
+Renaming 'ValidationRuleState' to StatusFlags, which better suits the client version and improves understanding.
+The traffic status (StatusFlags) will be stored in 'shipcall.evaluation'. The potential error message (str) is stored in 'shipcall.evaluation_message'.
+Create a oneline function that applies all validation rules and stores the results in 'evaluation' and 'evaluation_message'.
+
+# TODO: determine, how the results will be made accessible to the API. Options are: overwriting the MySQL database or calling the api's /shipcalls route
+
+
+# 20231006
+Focusing entirely on the traffic light. The ValidationRules object is inherited from ValidationRuleFunctions, which is connected to the SQLHandler. This makes
+it possible to compute all functions within ValidationRules. The object's methods have a default naming convention and input/output convention. The outputs
+always return a tuple of (enumeration object, method name). The method name is computed dynamically and can be used to indicate, what violation took place.
+The names are chosen in a way, where they are understandable immediately. However, it might still make sense to map these to genuine error messages, that
+users can understand better (and use German language). The functions were mapped out and the necessary variables were selected. Finally,
+a shipcall object is enough to select the respected 'times'. It is still somewhat unclear, how the participant's role will be selected (contacted D. Schick).
+
+Creating tests for the validation rules, so the logic is verified. This will need extensive testing and stub objects.
+
+
+# 20231004
+Working on the pandas dataframes, obtained from the mysql database. The list of participants is loaded and automatically assigned to each shipcall.
+A blank column 'validation_state' (str) and 'validation_state_changed' (datetime) are added upon initialization.
+Created methods and tests for simple retrieval of data models based on the pandas dataframe. This allows for quick and easy connection to any
+rule-based verification system (e.g., ValidationRules or InputValidation).
+(Note & #TODO: according to Christin, there should be LocalValidationRules and GlobalValidationRules. Local: focus only on a single shipcall & time, Global: compare multiple shipcalls & times)
+
+Konfliktbehebung nach dem Merge mit 'develop'.
+
+# 20230929
+
+Removing 'tug' from the library due to feedback of Daniel Schick.
+Creating an InputValidation object, which uses a dictionary of supported dataclass models. It maps directly towards
+the respective input validation objects of the dataclass, so the correct rule-set is applicable.
+Creating RequestStatusCode objects (parent and child classes) for the default status codes within the Flask HTTP framework (200, 201, 400, 403, 404, 500). This
+ensures that responses always follow the default format. It also makes debugging easier and more reproducible.
+Creating an SQLHandler object, which can read data from the SQL database and store it in a pandas DataFrame. These dataframes are much easier to handle and provide
+access to numpy, which allows for simple and quick boolean comparisons or computations.
+
+# 20230915
+
+Installed the 'schedule' library, which proves access to continuous routines in defined intervals. The library will be particularly useful to
+monitor alarm event notifications, as the threshold (e.g., 30 minutes) can be taken into account. When creating such a schedule object, an interval
+of 60 seconds may be sufficient, so that all notifications are checked every minute, whether they are eligible to be released (e.g., via E-Mail).
+
+Created a queue item (collections.deque) to store notifications in the queue. This queue has been connected to the 'schedule', so one can monitor it
+in fixed intervals. The queue item is likely going to be shifted towards NotificationList instead.
+
+Created a helper function and tests for time differences, so the different between an event's .modified date to the current time can quickly be
+obtained. A respective notification stub object has been provided, so testing becomes straight forward.
+
+created a suggestion to update the list objects for dataclasses (e.g., BerthList)
+
+apt installed
+ - MySQL Workbench (see brecal/misc/mysql-workbench-community_8.0.34-1ubuntu22.04_amd64.deb)
+ - heidisql-wine
+ - mysql-server
+ - mysql-client
+https://stackoverflow.com/questions/39281594/error-1698-28000-access-denied-for-user-rootlocalhost
+scope, Max123 ---- HalloWach_2323XXL!!
+ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'HalloWach_2323XXL!!';
+
+# 20230911
+Added git SSH key, updated the documentation. Successfully connected to the brecal git repository. The configurations are successful.
+
+
+# 20230908
+Created a library 'brecal_utils', which will contain some assisting functions and reusable objects. This also enables creating a tests/ folder.
+The setup.py file currently does not suggest the use of a license ('to be determined').
+(#TODO_license: clarify)
+
+__init__ & _version are created and refer to another. The _version suggests '0.0.1' initially.
+
+
+Creating a conda environment 'brecal' and an environment.yml file.
+Preparing the environment.yml file to contain only relevant libraries (for now) and create a {brecal} conda environment.
+
+Auto-installing the local lib_brecal_utils package via pip. However, the path to that file and the conda environment are currently hardcoded (#TODO_hardcoded_conda_paths)
+
+Setup the git connection, local configurations and pre-conditioned access to the git remote repository.
+(#TODO_SSH_KEY: Will talk to Daniel about the connection key)
+(20230911: solved)
+
+Adding file_handling.py and test_handling.py with updated docstrings to the library. These are copied from other projects and give baseline functionality, that is commonly used in my workflow.
+Also encorporating the notebook {run_unit_tests.ipynb}, which will, together with the {notebooks_metz} folder be ignored for .git versioning.
+
+Removing the 'jwt' dependency in server/requirements.txt, as it raised an error on my operation system. This is likely caused by jwt and PyJWT overlapping. When removing it from the requirements.txt,
+the flask-jwt-extended dependency automatically pulls the correct PyJWT version (instead of jwt).
+
+Definition of rules for validation as pre-suggestions in a jupyter notebook.
+Converting the 'Liegeplätze_sample_format.xlsx' to a .csv file, which may be used in the validation rules (mapping between berth and terminal).
+
+Setting up stub object for every currently defined dataclass. These will serve as the foundation for the traffic-light tests.
+Created an object {ValidationRules}, which takes shipcalls and times as variables and determines validation_state and notification_state. This will likely
+be the foundation for the traffic light system.
+
+Created a wrapper for validation function, whereas every function could return a tuple of (validation_state:bool, validation_name:str). This could be used as an
+indicator to determine the root cause of a traffic light issue.
+
+Suggestion: there is a library, which seems interesting. It builds on marshmallow and extends it by 'validators', which can be applied directly on the marshmallow.field objects.
+(https://apiflask.com/schema/). This requires the apiflask dependency.
+
+Adding schema_validation functions (Ship and Participant), time_logic (will be used in the validation_rules later), validation_rules (covers time-overlap handling).
+Further, creating simple unit tests for stubs (will be extended soon). (note: copied this text from the commit message)
+
+Creating a large, very extensive fleet of unit tests for the Ship object and its keys.
+
+
+################ Continuous Building ################
+
+Ensure that the active environment can run mamba installations (much preferably the 'base' environment)
+Note: this part of the documentation is incomplete. The commands currently only cover updating an existing environment.
+
+1.) Locate to the project root folder
+ cd ~/brecal
+2.) Update the environment. Note: it is explicitly mandatory to use the --prefix argument, which pin-points towards the correct conda environment (in this case {brecal}),
+as otherwise mamba does not understand the location of the environment and does not update the packages. PIP_NO_DEPS=1 ignores the dependencies of every pip package. Make sure
+to incorporate these packages in the conda dependencies then, which is much cleaner.
+ PIP_NO_DEPS=1 mamba env update --file environment.yml --prefix /home/scope/anaconda3/envs/brecal
+
+# Note: (UNTESTED): to create an environment instead, use this:
+PIP_NO_DEPS=1 mamba env create --prefix [conda_path]/envs/[new_env_name] -f [environment_file]
+
+
+### Building Steps ###
+Create the environment, activate it, and install mamba before continuing, as it is much faster. The locate the environment yml and install
+
+ conda create --name brecal python=3.11
+ conda activate brecal
+ conda install -c conda-forge mamba
+ cd ~/brecal
+ PIP_NO_DEPS=1 mamba env update --file environment.yml --prefix /home/scope/anaconda3/envs/brecal
+
+
+