Software Stack

• Python FastAPI
• HTML
• Jinja2
• JavaScript
  • Since rather old Android tablets are used (oldest has Android 4.1), any JavaScript needs to be supported in Chrome 71 or earlier
• Bootstrap

Running the program

Background

In order to develop new features for the machine support consoles application, 2 types of applications need to be running locally:

• a running Machine support consoles application (see this repository in Set-up section)
• a locally running CRM (see) CRM Repository
  • create an API-key on localhost page of CRM localhost:portnum/admin
  • add API-key to .env file in machine support consoles repository by using .env.example file
  • create a test user so the local-crm-database has at least 1
  • run the command: uv run python manage.py runserver 8001

Set-up

To get the project running:

1. install uv (if you haven't already) by executing the command in uv Instructions;
   • This project uses uv instead of pip in an attempt to avoid dependency versioning problems.
2. install this projects' dependencies by running uv sync;
3. if you're not using an IDE that loads the virtual environment automatically: run source .venv/bin/activate on macOS/Linux or .venv\Scripts\activate on Windows to activate it manually;
4. run the server with fastapi dev.
   • If you want to access the webpage from another device (like one of the tablets):
     1. find out your IP address:
        • on macOS, use ipconfig getifaddr en0;
        • on Windows, use ipconfig /all;
        • on Linux, use ip a.
     2. run the server with the --host 0.0.0.0 flag: fastapi dev --host 0.0.0.0;
     3. open the browser of your other device to the IP address from step 1 with the port displayed when running step 2 (defaults to 8000). Example for if IP is 1.2.3.4 link in browser should be 1.2.3.4:8000
   • If you want to access the webpage from the computer you are developing with, then write localhost:8000 in the browser
5. If not installed already, install chrome (the CI pipeline as can be seen in gitlab-ci.yml also does so for linux)
6. Add the API-key created in the locally running CRM to the local .env file and add the rest of the .env like stated in the .env.example file

Docker

You can also get the project running in Docker:

1. copy your .env to .env.docker;
2. change the BASE_URL in .env.docker from http://localhost:8001 to http://host.docker.internal:8001;
3. install Docker or Podman for Windows/macOS;
4. run docker compose --profile development up --watch.
   • this will hot-reload files in ./app, but nothing outside of that. So if you change the .env or whatever, you need to run docker compose --profile development up --watch --build once.

Production

The production Docker Compose service can be started with docker compose --profile production up (and --build can be added to rebuild). This configuration does not have hot-reloading. It will only expose the service through the Apache reverse proxy, so make sure to have followed the below certificate steps, else it will not work.

Kiosk mode on the tablets

This application contains a Progressive Web App. To install this:

1. go to the webpage on the tablets;
   • note: this only works on the root page (where you can select which machine to use)
2. click the three dots in the top-right;
3. click Add to Home screen;
   • you should get a pop-up that prefills the application name with either MSC or Machine Support Consoles. If it does not, the browser is not recognising the PWA manifest, and something is wrong.
     • (manually filling in a different name will not break the PWA)
4. click the ADD button.

The home screen will now have a PWA. Click to open it, and the application can be used from there. To reset this PWA back to the root page (where you can select which machine to use), click the recent apps/app switcher button and close the PWA. After re-opening, it will be on the root page again.

Note: it appears Chrome 119 (most recent version supported by Android 7) might not have any button to install the PWA. This is despite PWAs being supported in much older versions of Chrome, like 71, which is the most recent version supported by Android 4.1-4.3. This appears to be a problem for all sites, not just ours. Examples of these other sites are music.youtube.com and reddit.com.
Sadly, we have not yet been able to find a solution to this yet. Perhaps downgrading to an older version of Chrome might work?

Certificates on the tablets

General note for if you want to generate your own keys: the tablets seem to like .pfx files (always PKCS#12 format), and .crt files in the DER format.

HTTPS

This is the section to follow if you get the NET::ERR_CERT_AUTHORITY_INVALID error when trying to visit a webpage that is not the one for this solution.
You can check whether this is a problem on your tablet by opening up an HTTPS webpage like https://wiki.makerspaceleiden.nl. If this loads with no security warnings and with a green padlock in the top left of the Chrome window, great! You can skip this section.

The reason these pages fail to load is that the root CA certificate for Let's Encrypt is not pre-installed on the tablets. As Let's Encrypt signs quite a lot of websites nowadays, this absence causes many pages to not load (without errors).

You can follow these steps to install the root CA certificate manually, and afterwards webpages will work like they should:

1. on your computer: go to https://letsencrypt.org/certificates;
2. under Root CAs -> ISRG Root X1 -> Certificate details (self-signed), download the .pem file;
3. open a terminal into your downloads folder;
   • if you are on Windows, use Git Bash. This is installed by default with Git.
4. run openssl x509 -in isrgrootx1.pem -out isrgrootx1.crt -outform der;
5. follow the Installing certificates instructions with the isrgrootx1.crt file you just generated.

It should work now! You can test this by refreshing the page from the start of this section (probably https://wiki.makerspaceleiden.nl).

Authentication

Our authentication is also handled through certificates. If you want to test our authentication or connect through the Apache reverse proxy, you will need to install them. If you want to connect to our production environment, you will need to run the relevant scripts on the server.

There's scripts to achieve this in ./apache-configuration/self-signed-keys. These are sh scripts, so they will work natively on Linux and macOS. On Windows, you might be able to use Git Bash.

• if you get a new IP address or want to set up a new server, run 1-generate-server-keys.sh;
• 2-generate-tablet-ca-keys.sh only needs to be run once during initial setup;
• if you want to add a new tablet, run 3-generate-tablet-keys.sh;
• for the initial setup, run all of the above (in order).

The scripts will tell you what steps you need to take next.

If a script exits unexpectedly, you might need to run chmod +x ./1-generate-server-keys.sh ./2-generate-tablet-ca-keys.sh ./3-generate-tablet-keys.sh to make them executable, then try again.

Installing certificates

You can install a certificate on the tablet by following these steps:

1. grab the following:
   • the certificate(s) you want to import;
   • the tablet;
   • your computer;
   • a microSD card;
   • if necessary: an adapter so the microSD card fits into your computer;
2. insert the microSD card into your laptop;
3. copy the certificate(s) you want to install from your local computer onto the microSD card;
4. in your file explorer, click the eject button next to the microSD card;
5. physically eject the microSD card from your laptop;
6. insert the microSD card into the tablet;
7. on the tablet: open the My Files app and copy the certificate(s) from the microSD card to /storage/emulated/0;
8. open Settings -> Security -> click Install from device storage (under the Credential storage section);
9. for each certificate that you want to install:
   1. select your certificate;
   2. if prompted: enter the password for this certificate;
   3. give it a name and click OK;
   4. if you see a dropdown, choose VPN and apps or VPN & app user certificate or similar, not Wi-Fi (certificate);
   5. if prompted: enter the PIN code for the tablet (0000), or if there is no screen lock yet: set up a screen lock for the tablet with PIN code 0000.
10. prepare the microSD card for ejection from the tablet through Settings -> Storage -> Unmount SD card (under the SD card section) and click OK;
11. physically remove the microSD card from the tablet.

Error: No certificate file found on the SD card.

You might have forgotten to copy your certificate file from the SD card to the internal storage at /storage/emulated/0.
Note: the error message is misleading: it's not actually looking for certificates on the SD card. It means to say that no certificates could be found on the internal storage.

Error: Type the correct password.

This error shows up more often than if you just type in your password incorrectly, so your real problem may be different. It also shows up when you use openssl pkcs12 without the -legacy flag.

Code style

We have a Python linter called ruff. It can recognise common bugs and bad patterns in your code, and will help you clean them up. To run the linter, run ruff check. To have it fix the issues that can automatically be safely fixed, you can ruff check --fix.

ruff also has a formatter. This doesn't fix bugs or bad code logic patterns, but instead helps to maintain consistent code style across different team members writing code. It does this by automatically reformatting your code. This won't always produce better code, but a consistent code base is significantly more important than 'better'. To run the formatter, run ruff format. This will automatically fix your code for you.

It is recommended you run the formatter (ruff format) before the linter (ruff check), or you might get warnings about code style from the linter.

For HTML (and inline JS and CSS), we have another linter/formatter called djLint. To run the linter part, run djlint . --lint. To run the formatter, run djlint . --reformat.

Testing

Use of the python-library responses to mock the external systems (wiki and crm).

To run the tests execute pytest from the root-folder of the repository

to only run a specific test-file e.g. the test_end_to_end.py specify the path pytest .\app\tests\test_end_to_end.py -s

to get code coverage analytics run pytest --cov=app

To help debugging the end_to_end-tests it can help to temporarily comment out the headless option in the webdriver.