merging 0.7.0 fixed
This commit is contained in:
max_metz
parent
5ed0a08ca2
commit
dddc4a1c9b
10
.gitignore
vendored
10
.gitignore
vendored
@ -434,3 +434,13 @@ FodyWeavers.xsd
|
||||
*.msix
|
||||
*.msm
|
||||
*.msp
|
||||
|
||||
times.md
|
||||
docs/traffic_light_examples
|
||||
misc/mysql-workbench-community_**
|
||||
misc/berths_and_terminals.csv
|
||||
src/notebooks_metz
|
||||
src/server/BreCal.egg-info
|
||||
src/lib_brecal_utils/**.egg-info/**
|
||||
|
||||
|
||||
|
||||
33
Ampelfunktion.md
Normal file
33
Ampelfunktion.md
Normal file
@ -0,0 +1,33 @@
|
||||
# Ampelfunktion Bremen Calling
|
||||
|
||||
| Ampelfunktionen | Beschreibung | Definition | Bemerkungen |
|
||||
|-----------------|-----------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|
|
||||
| | | | |
|
||||
| 0001 | Nicht alle Zeiten sind zugeordnet | Bedingungen:<br/> - Header der Zeile ist zugeordnet (Agentur, Festmacher usw.)<br/> - Zugeordnete Zeit ist leer | |
|
||||
| 0001 - A | Agentur / einkommend | times_agency:<br/> - participant_id = ausgefüllt<br/> - ETA Berth = leer<br/>___zum Zeitpunkt 20 Std vor___<br/> shipcall:<br/> - eta | gelb |
|
||||
| 0001 - B | Agentur / ausgehend + Verholung | times_agency:<br/> - participant_id = ausgefüllt<br/> - ETD Berth = leer<br/>___zum Zeitpunkt 20 Std vor___<br/> shipcall:<br/> - etd | gelb |
|
||||
| 0001 - C | Festmacher / einkommend | times_mooring:<br/> - participant_id = ausgefüllt<br/> - ETA Berth = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETA Berth | gelb |
|
||||
| 0001 - D | Festmacher / ausgehend + Verholung | times_mooring:<br/> - participant_id = ausgefüllt<br/> - ETD Berth = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETD Berth | gelb |
|
||||
| 0001 - F | Hafenamt / einkommend | times_portauthority:<br/> - participant_id = ausgefüllt<br/> - ETA Berth = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETA Berth | gelb |
|
||||
| 0001 - G | Hafenamt / ausgehend + Verholung | times_portauthority:<br/> - participant_id = ausgefüllt<br/> - ETD Berth = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETD Berth | gelb |
|
||||
| 0001 - H | Lotsen / einkommend | times_pilot:<br/> - participant_id = ausgefüllt<br/> - ETA Berth = leer<br/>______<br/> times_agency:<br/> - ETA Berth < 16 Stunden entfernt | gelb |
|
||||
| 0001 - I | Lotsen / ausgehend + Verholung | times_pilot:<br/> - participant_id = ausgefüllt<br/> - ETD Berth = leer<br/>______<br/> times_agency:<br/> - ETD Berth < 16 Stunden entfernt | gelb |
|
||||
| 0001 - J | Schlepper / einkommend | times_tug:<br/> - participant_id = ausgefüllt<br/> - ETA Berth = leer<br/>______<br/> times_agency:<br/> - ETA Berth < 16 Stunden entfernt | gelb |
|
||||
| 0001 - K | Schlepper / ausgehend + Verholung | times_tug:<br/> - participant_id = ausgefüllt<br/> - ETD Berth = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETD Berth | gelb |
|
||||
| 0001 - L | Terminal / einkommend | times_terminal:<br/> - participant_id = ausgefüllt<br/> - Operation Start = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETA Berth | gelb |
|
||||
| 0001 - M | Terminal / ausgehend + Verholung | times_terminal:<br/> - participant_id = ausgefüllt<br/> - Operation Ende = leer<br/>___zum Zeitpunkt 16 Std vor___<br/> times_agency:<br/> - ETD Berth | gelb |
|
||||
| 0002 | Zeiten für einen Eintrag weichen voneinander ab | Bedingungen:<br/> - Header der Zeile ist zugeordnet (Agentur, Festmacher usw. - außer BSMD-Spalte)<br/> - Zeiten ungleich (leere Einträge nicht berücksichtigen => 0001) | |
|
||||
| 0002 - A | Agentur + Festmacher + Hafenamt + Lotsen + Schlepper / einkommend | times_agency:<br/> - ETA Berth <br/>____ungleich____<br/>times_mooring:<br/> - ETA Berth <br/>____ungleich____<br/>times_portauthority:<br/>- ETA Berth <br/>____ungleich____<br/>times_pilot:<br/> - ETA Berth <br/>____ungleich____<br/>times_tug:<br/> - ETA Berth | rot |
|
||||
| 0002 - B | Agentur + Festmacher + Hafenamt + Lotsen + Schlepper / ausgehend | times_agency:<br/> - ETD Berth <br/>____ungleich____<br/>times_mooring:<br/> - ETD Berth <br/>____ungleich____<br/>times_portauthority:<br/>- ETD Berth <br/>____ungleich____<br/>times_pilot:<br/> - ETD Berth <br/>____ungleich____<br/>times_tug:<br/> - ETD Berth | rot |
|
||||
| 0002 - C | Agentur + Festmacher + Hafenamt + Lotsen + Schlepper / Verholung | times_agency:<br/> - ETA Berth <br/> - ETD Berth <br/>____ungleich____<br/>times_mooring:<br/> - ETA Berth <br/> - ETD Berth <br/>____ungleich____<br/>times_portauthority:<br/> - ETA Berth <br/>- ETD Berth <br/>____ungleich____<br/>times_pilot:<br/> - ETA Berth <br/> - ETD Berth <br/>____ungleich____<br/>times_tug:<br/> - ETA Berth <br/> - ETD Berth | rot |
|
||||
| 0003 | Arbeitszeit überschneidet sich mit Fahrtzeit | Bedingungen:<br/> - Header der Zeile ist zugeordnet (Terminal)<br/> - Zeiten passt nicht zu Ankunft / Abfahrt (leere Einträge nicht berücksichtigen => 0001) | |
|
||||
| 0003 - A | Terminal / einkommend | times_terminal:<br/> - Operation Start<br/>___vor (kleiner als)____<br/> times_agency:<br/> - ETA Berth | rot |
|
||||
| 0003 - B | Terminal / ausgehend + Verholung | times_terminal:<br/> - Operation Ende<br/>___nach (größer als)____<br/> times_agency:<br/> - ETD Berth | rot |
|
||||
| 0004 | Tidezeiten passen nicht zu Fahrzeiten | Bedingungen:<br/> - Header der Zeile ist zugeordnet (Agentur)<br/> - Tidezeit ausgefüllt | |
|
||||
| 0004 - A | Agentur / einkommend | times_agency:<br/> - ETA Berth <br/>___vor (kleiner als)____<br/> times_agency:<br/> - Tidefenster von <br/>___und/oder___nach (größer als)____<br/> times_agency:<br/> - Tidefenster bis | rot |
|
||||
| 0004 - B | Agentur / ausgehend + Verholung | times_agency:<br/> - ETD Berth <br/>___vor (kleiner als)____<br/> times_agency:<br/> - Tidefenster von <br/>___und/oder___nach (größer als)____<br/> times_agency:<br/> - Tidefenster bis | rot |
|
||||
| 0005 | Zu viele Schiffe mit gleicher Fahrtzeit | Bedingungen:<br/> - Header der Zeile ist zugeordnet (Agentur)<br/>- Übergreifend über die Einträge | |
|
||||
| 0005 - A | Agentur / einkommend + ausgehend + Verholung | ____Zählen wenn gleich:____<br/> times_agency:<br/> - ETA Berth <br/>___und____<br/> times_agency:<br/> - ETD Berth<br/>___mehr als 3____<br/> | gelb |
|
||||
| 0006 | Agentur und Terminal planen mit unterschiedlichen Liegeplätzen | Bedingungen:<br/> - Header der Zeile ist zugeordnet (Agentur / Terminal)<br/>- LP jeweils ausgefüllt | |
|
||||
| 0006 - A | Agentur + Terminal (Liegeplatz) / einkommend + ausgehend + Verholung | times_agency:<br/> - Liegeplatz<br/>____ungleich____<br/>times_terminal:<br/> - Liegeplatz | gelb |
|
||||
| 0006 - B | Agentur + Terminal (Anlegeseite) / einkommend + ausgehend + Verholung | times_agency:<br/> - Anlegeseite<br/>____ungleich____<br/>times_terminal:<br/> - Anlegeseite | gelb |
|
||||
140
change_log_metz.md
Normal file
140
change_log_metz.md
Normal file
@ -0,0 +1,140 @@
|
||||
|
||||
|
||||
# 20231012:
|
||||
Building out the remaining validation rules, which are all finalized now (0001 - 0006).
|
||||
Made sure that the ValidationRuleState.NONE is 0. Within the notification check, a notification requirement is based on new_state>old_state. To prevent
|
||||
future none->green changes to trigger a notification, a 'max' call is used. When max(GREEN, NONE), then the old state is considered at least 'green'. This solves the issue.
|
||||
|
||||
Renaming 'ValidationRuleState' to StatusFlags, which better suits the client version and improves understanding.
|
||||
The traffic status (StatusFlags) will be stored in 'shipcall.evaluation'. The potential error message (str) is stored in 'shipcall.evaluation_message'.
|
||||
Create a oneline function that applies all validation rules and stores the results in 'evaluation' and 'evaluation_message'.
|
||||
|
||||
# TODO: determine, how the results will be made accessible to the API. Options are: overwriting the MySQL database or calling the api's /shipcalls route
|
||||
|
||||
|
||||
# 20231006
|
||||
Focusing entirely on the traffic light. The ValidationRules object is inherited from ValidationRuleFunctions, which is connected to the SQLHandler. This makes
|
||||
it possible to compute all functions within ValidationRules. The object's methods have a default naming convention and input/output convention. The outputs
|
||||
always return a tuple of (enumeration object, method name). The method name is computed dynamically and can be used to indicate, what violation took place.
|
||||
The names are chosen in a way, where they are understandable immediately. However, it might still make sense to map these to genuine error messages, that
|
||||
users can understand better (and use German language). The functions were mapped out and the necessary variables were selected. Finally,
|
||||
a shipcall object is enough to select the respected 'times'. It is still somewhat unclear, how the participant's role will be selected (contacted D. Schick).
|
||||
|
||||
Creating tests for the validation rules, so the logic is verified. This will need extensive testing and stub objects.
|
||||
|
||||
|
||||
# 20231004
|
||||
Working on the pandas dataframes, obtained from the mysql database. The list of participants is loaded and automatically assigned to each shipcall.
|
||||
A blank column 'validation_state' (str) and 'validation_state_changed' (datetime) are added upon initialization.
|
||||
Created methods and tests for simple retrieval of data models based on the pandas dataframe. This allows for quick and easy connection to any
|
||||
rule-based verification system (e.g., ValidationRules or InputValidation).
|
||||
(Note & #TODO: according to Christin, there should be LocalValidationRules and GlobalValidationRules. Local: focus only on a single shipcall & time, Global: compare multiple shipcalls & times)
|
||||
|
||||
Konfliktbehebung nach dem Merge mit 'develop'.
|
||||
|
||||
# 20230929
|
||||
|
||||
Removing 'tug' from the library due to feedback of Daniel Schick.
|
||||
Creating an InputValidation object, which uses a dictionary of supported dataclass models. It maps directly towards
|
||||
the respective input validation objects of the dataclass, so the correct rule-set is applicable.
|
||||
Creating RequestStatusCode objects (parent and child classes) for the default status codes within the Flask HTTP framework (200, 201, 400, 403, 404, 500). This
|
||||
ensures that responses always follow the default format. It also makes debugging easier and more reproducible.
|
||||
Creating an SQLHandler object, which can read data from the SQL database and store it in a pandas DataFrame. These dataframes are much easier to handle and provide
|
||||
access to numpy, which allows for simple and quick boolean comparisons or computations.
|
||||
|
||||
# 20230915
|
||||
|
||||
Installed the 'schedule' library, which proves access to continuous routines in defined intervals. The library will be particularly useful to
|
||||
monitor alarm event notifications, as the threshold (e.g., 30 minutes) can be taken into account. When creating such a schedule object, an interval
|
||||
of 60 seconds may be sufficient, so that all notifications are checked every minute, whether they are eligible to be released (e.g., via E-Mail).
|
||||
|
||||
Created a queue item (collections.deque) to store notifications in the queue. This queue has been connected to the 'schedule', so one can monitor it
|
||||
in fixed intervals. The queue item is likely going to be shifted towards NotificationList instead.
|
||||
|
||||
Created a helper function and tests for time differences, so the different between an event's .modified date to the current time can quickly be
|
||||
obtained. A respective notification stub object has been provided, so testing becomes straight forward.
|
||||
|
||||
created a suggestion to update the list objects for dataclasses (e.g., BerthList)
|
||||
|
||||
apt installed
|
||||
- MySQL Workbench (see brecal/misc/mysql-workbench-community_8.0.34-1ubuntu22.04_amd64.deb)
|
||||
- heidisql-wine
|
||||
- mysql-server
|
||||
- mysql-client
|
||||
https://stackoverflow.com/questions/39281594/error-1698-28000-access-denied-for-user-rootlocalhost
|
||||
scope, Max123 ---- HalloWach_2323XXL!!
|
||||
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'HalloWach_2323XXL!!';
|
||||
|
||||
# 20230911
|
||||
Added git SSH key, updated the documentation. Successfully connected to the brecal git repository. The configurations are successful.
|
||||
|
||||
|
||||
# 20230908
|
||||
Created a library 'brecal_utils', which will contain some assisting functions and reusable objects. This also enables creating a tests/ folder.
|
||||
The setup.py file currently does not suggest the use of a license ('to be determined').
|
||||
(#TODO_license: clarify)
|
||||
|
||||
__init__ & _version are created and refer to another. The _version suggests '0.0.1' initially.
|
||||
|
||||
|
||||
Creating a conda environment 'brecal' and an environment.yml file.
|
||||
Preparing the environment.yml file to contain only relevant libraries (for now) and create a {brecal} conda environment.
|
||||
|
||||
Auto-installing the local lib_brecal_utils package via pip. However, the path to that file and the conda environment are currently hardcoded (#TODO_hardcoded_conda_paths)
|
||||
|
||||
Setup the git connection, local configurations and pre-conditioned access to the git remote repository.
|
||||
(#TODO_SSH_KEY: Will talk to Daniel about the connection key)
|
||||
(20230911: solved)
|
||||
|
||||
Adding file_handling.py and test_handling.py with updated docstrings to the library. These are copied from other projects and give baseline functionality, that is commonly used in my workflow.
|
||||
Also encorporating the notebook {run_unit_tests.ipynb}, which will, together with the {notebooks_metz} folder be ignored for .git versioning.
|
||||
|
||||
Removing the 'jwt' dependency in server/requirements.txt, as it raised an error on my operation system. This is likely caused by jwt and PyJWT overlapping. When removing it from the requirements.txt,
|
||||
the flask-jwt-extended dependency automatically pulls the correct PyJWT version (instead of jwt).
|
||||
|
||||
Definition of rules for validation as pre-suggestions in a jupyter notebook.
|
||||
Converting the 'Liegeplätze_sample_format.xlsx' to a .csv file, which may be used in the validation rules (mapping between berth and terminal).
|
||||
|
||||
Setting up stub object for every currently defined dataclass. These will serve as the foundation for the traffic-light tests.
|
||||
Created an object {ValidationRules}, which takes shipcalls and times as variables and determines validation_state and notification_state. This will likely
|
||||
be the foundation for the traffic light system.
|
||||
|
||||
Created a wrapper for validation function, whereas every function could return a tuple of (validation_state:bool, validation_name:str). This could be used as an
|
||||
indicator to determine the root cause of a traffic light issue.
|
||||
|
||||
Suggestion: there is a library, which seems interesting. It builds on marshmallow and extends it by 'validators', which can be applied directly on the marshmallow.field objects.
|
||||
(https://apiflask.com/schema/). This requires the apiflask dependency.
|
||||
|
||||
Adding schema_validation functions (Ship and Participant), time_logic (will be used in the validation_rules later), validation_rules (covers time-overlap handling).
|
||||
Further, creating simple unit tests for stubs (will be extended soon). (note: copied this text from the commit message)
|
||||
|
||||
Creating a large, very extensive fleet of unit tests for the Ship object and its keys.
|
||||
|
||||
|
||||
################ Continuous Building ################
|
||||
|
||||
Ensure that the active environment can run mamba installations (much preferably the 'base' environment)
|
||||
Note: this part of the documentation is incomplete. The commands currently only cover updating an existing environment.
|
||||
|
||||
1.) Locate to the project root folder
|
||||
cd ~/brecal
|
||||
2.) Update the environment. Note: it is explicitly mandatory to use the --prefix argument, which pin-points towards the correct conda environment (in this case {brecal}),
|
||||
as otherwise mamba does not understand the location of the environment and does not update the packages. PIP_NO_DEPS=1 ignores the dependencies of every pip package. Make sure
|
||||
to incorporate these packages in the conda dependencies then, which is much cleaner.
|
||||
PIP_NO_DEPS=1 mamba env update --file environment.yml --prefix /home/scope/anaconda3/envs/brecal
|
||||
|
||||
# Note: (UNTESTED): to create an environment instead, use this:
|
||||
PIP_NO_DEPS=1 mamba env create --prefix [conda_path]/envs/[new_env_name] -f [environment_file]
|
||||
|
||||
|
||||
### Building Steps ###
|
||||
Create the environment, activate it, and install mamba before continuing, as it is much faster. The locate the environment yml and install
|
||||
|
||||
conda create --name brecal python=3.11
|
||||
conda activate brecal
|
||||
conda install -c conda-forge mamba
|
||||
cd ~/brecal
|
||||
PIP_NO_DEPS=1 mamba env update --file environment.yml --prefix /home/scope/anaconda3/envs/brecal
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user