Installation ============ The following installation guides are for Linux only. Theoretically all the components of this software should be possible to deploy onto other operating systems as well however this is not tested and not supported. You will have to be on your own for that. .. important:: If you plan to host the backend public to the internet: Make sure to only serve the backend over https regardless of your installation method! Without ssl encryption sensitive information like passwords, JWT Tokens for the clients and runners or user data will be transmitted in clear text! .. note:: Prerequisites: Make sure that your DNS and Firewall settings are correct. In particular, the domain that you want to use has to point to the ip address of your server, and your servers firewall needs to allow incoming tcp traffic on ports 80 and 443 (only on port 80 if you go with the reverse proxy setup) Docker ------ We provide a docker image for each of the components of this software (client, backend, runner). The best way to use them is with Docker Compose. In the following we assume that you want to host our backend and client/frontend on the same server, and the runner on a different one. If this assumption doesn't hold for you (e.g. if you want the frontend to be served by a different server than the backends API), then you may have to write your own Dockerfiles and docker-compose.yml or choose a different installation method like NixOS ;). .. note:: For each of the three components there are multiple docker image tags. The examples below will use the 'latest' tag which will pull the latest stable release (recommended). If you want to pull the development version (latest git commit on main branch), then choose the 'main' tag instead. Alternatively you can also pinpoint the docker image to a specific versions. Go to the Packages section of each GitHub repository to find out which tags are available. To use a tag other than 'latest' add it to the end of the 'image:' lines in the docker-compose.yml files below like this: image: /: .. _docker_backend_frontend-label: Backend & Frontend `````````````````` To run the backend you need a config.yml file that configures it. Prepare this file before running the installation steps below. You can start off with the following example (don't forget to replace the !) and modify it to your needs if necessary. Refer to :ref:`description_backend_config-label` for more information about all the configuration options. .. warning:: Please make sure to save 'sessionSecretKey' and 'smtpServer.password' in a secret way on your server! With the 'sessionSecretKey' a bad actor could log in as any user, even as an admin user, and read any current and future user data. With the 'smtpServer.password' a bad actor could authenticate with your mail server and send malicious phishing emails to you users while masquerading as the server admin. In this setup, sessionSecretKey and the smtp password are being read from the environment variables 'PROJECT_W_JWT_SECRET_KEY' and 'PROJECT_W_SMTP_PASSWORD'. If you want you can also choose to set them here directly in the config, but if you do so please take appropriate measures to keep this config file secret! .. code-block:: yaml clientURL: https:///# databasePath: /database loginSecurity: sessionSecretKey: !ENV ${JWT_SECRET_KEY} smtpServer: domain: port: secure: senderEmail: username: password: !ENV ${SMTP_PASSWORD} Choose between the following instructions depending on your setup. If you are unsure which setup to choose then :ref:`standalone_all-label` is probably for you. .. _standalone_all-label: Standalone, All-in-One '''''''''''''''''''''' Additionally to the backend and frontend, the following instructions will also set up certbot to request let's encrypt ssl certificates. If you want to use your own certificates instead, please jump ahead to :ref:`standalone_byo-label`. 1. Install Docker: Refer to your distros package manager / the `Docker documentation `_ for this 2. Create initial directory structure and enter project-w directory: .. code-block:: console mkdir -p project-W/project-W-data/sslCert && mkdir project-W/project-W-data/config && cd project-W 3. Put your config.yml into ./project-W-data/config 4. Put docker-compose.yml in the current directory. Use the following config and make same adjustments if needed (make sure to replace the !): .. code-block:: yaml services: backend: image: ghcr.io/julianfp/project-w_backend volumes: - ./project-W-data/config:/etc/xdg/project-W/ - ./project-W-data/database:/database environment: - JWT_SECRET_KEY=${PROJECT_W_JWT_SECRET_KEY:-} - SMTP_PASSWORD=${PROJECT_W_SMTP_PASSWORD:-} frontend: image: ghcr.io/julianfp/project-w_frontend ports: - 80:80 - 443:443 volumes: - ./project-W-data/sslCert:/ssl:ro - ./acme:/acme environment: - NGINX_CONFIG=initial - SERVER_NAME= certbot: image: certbot/certbot:latest depends_on: - frontend command: >- certonly --reinstall --webroot --webroot-path=/var/www/certbot --email --agree-tos --no-eff-email -d volumes: - ./project-W-data/sslCert:/etc/letsencrypt/live/ - ./acme:/var/www/certbot 5. Generate a JWT_SECRET_KEY that will be used to for generating Session Tokens. If you have python installed you can use the following command for this: .. code-block:: console python -c 'import secrets; print(secrets.token_hex())' 6. Build and run the containers. Replace and with the JWT_SECRET_KEY we generated before and the password of the SMTP Server you want to use respectively: .. code-block:: console PROJECT_W_JWT_SECRET_KEY="" PROJECT_W_SMTP_PASSWORD="" docker compose up -d 7. Check the logs of the certbot container and wait for 'Successfully received certificate.'. Use the following command for this: .. code-block:: console docker logs project-w-certbot-1 If that line appears, then please replace 'initial' in your docker-compose.yml with 'ssl'. After that rerun the command in step 6. If an error is shown instead, then please verify your DNS and Firewall configuration and try again beginning from step 6. In the end, your containers should be up and running and your docker-compose.yml should look like this: .. code-block:: yaml services: backend: image: ghcr.io/julianfp/project-w_backend volumes: - ./project-W-data/config:/etc/xdg/project-W/ - ./project-W-data/database:/database environment: - JWT_SECRET_KEY=${PROJECT_W_JWT_SECRET_KEY:-} - SMTP_PASSWORD=${PROJECT_W_SMTP_PASSWORD:-} frontend: image: ghcr.io/julianfp/project-w_frontend ports: - 80:80 - 443:443 volumes: - ./project-W-data/sslCert:/ssl:ro - ./acme:/acme environment: - NGINX_CONFIG=ssl - SERVER_NAME= certbot: image: certbot/certbot:latest depends_on: - frontend command: >- certonly --reinstall --webroot --webroot-path=/var/www/certbot --email --agree-tos --no-eff-email -d volumes: - ./project-W-data/sslCert:/etc/letsencrypt/live/ - ./acme:/var/www/certbot 8. You may want to setup a systemd service or similar to start the containers automatically. Please be careful with where you store your JWT Secret Key and your SMTP Password, they should always stay secret! 9. You may want to setup a cronjob or a systemd service with systemd timers to periodically restart the certbot container. Let's encrypt certificates are only valid for 90 days, so if you don't your certificate will expire! 10. You may want to set up some kind of backup solution. For this you just need to backup the project-W-data directory (which will include the database, your ssl certificate and your config.yml) and maybe your docker-compose.yml if you made changes to it. .. _standalone_byo-label: Standalone, BYO ''''''''''''''' If you want to bring your own ssl certificate (e.g. self-signed or using some other acme setup), then this is the right setup for you. 1. Install Docker: Refer to your distros package manager / the `Docker documentation `_ for this 2. Create initial directory structure and enter project-w directory: .. code-block:: console mkdir -p project-W/project-W-data/sslCert/ && mkdir project-W/project-W-data/config && cd project-W 3. Put your config.yml into ./project-W-data/config 4. Put your ssl certificate files into ./project-W-data/sslCert. The following files should be in that directory: fullchain.pem (ssl certificate), privkey.pem (ssl certificate private key) and chain.pem (ssl trusted certificate for OCSP stapling). If you use self-signed certificates then chain.pem and fullchain.pem will be the same. 5. Put docker-compose.yml in the current directory. Use the following config and make same adjustments if needed (make sure to replace the !): .. code-block:: yaml services: backend: image: ghcr.io/julianfp/project-w_backend volumes: - ./project-W-data/config:/etc/xdg/project-W/ - ./project-W-data/database:/database environment: - JWT_SECRET_KEY=${PROJECT_W_JWT_SECRET_KEY:-} - SMTP_PASSWORD=${PROJECT_W_SMTP_PASSWORD:-} frontend: image: ghcr.io/julianfp/project-w_frontend ports: - 80:80 - 443:443 volumes: - ./project-W-data/sslCert:/ssl:ro environment: - NGINX_CONFIG=ssl - SERVER_NAME= 6. Generate a JWT_SECRET_KEY that will be used to for generating Session Tokens. If you have python installed you can use the following command for this: .. code-block:: console python -c 'import secrets; print(secrets.token_hex())' 7. Build and run the containers. Replace and with the JWT_SECRET_KEY we generated before and the password of the SMTP Server you want to use respectively: .. code-block:: console PROJECT_W_JWT_SECRET_KEY="" PROJECT_W_SMTP_PASSWORD="" docker compose up -d 8. You may want to setup a systemd service or similar to start the containers automatically. Please be careful with where you store your JWT Secret Key and your SMTP Password, they should always stay secret! 9. You may want to set up some kind of backup solution. For this you just need to backup the project-W-data directory (which will include the database, your ssl certificate and your config.yml) and maybe your docker-compose.yml if you made changes to it. With Reverse Proxy '''''''''''''''''' Follow this guide if you want to run this behind a Reverse Proxy which takes care of SSL. Please really only use this if this is the case since with this setup the webserver of the container will be set up with HTTP only. With a proper Reverse Proxy setup this means that the traffic would stay unencrypted between Project-W backend/frontend server and Reverse Proxy, but then would be encrypted before sending it to the internet. If you were to run the following setup without a Reverse Proxy then all the communication between client and backend as well as possibly backend and runners would be send unencrypted through the internet including passwords, session tokens and user data! .. attention:: Make sure that your reverse proxy is properly configured to handle the upload of large files. The backend can handle files up to a size of 1GB, setting this to anything less in your reverse proxy will hinder the submission of jobs and present the user with possibly confusing error messages! We will not cover the configuration of the reverse proxy here, but for example if you use nginx you will want to set ``client_max_body_size 1g;`` in your config. 1. Install Docker: Refer to your distros package manager / the `Docker documentation `_ for this 2. Create initial directory structure and enter project-w directory: .. code-block:: console mkdir -p project-W/project-W-data/config && cd project-W 3. Put your config.yml into ./project-W-data/config 4. Put docker-compose.yml in the current directory. Use the following config and make same adjustments if needed (make sure to replace the !): .. code-block:: yaml services: backend: image: ghcr.io/julianfp/project-w_backend volumes: - ./project-W-data/config:/etc/xdg/project-W/ - ./project-W-data/database:/database environment: - JWT_SECRET_KEY=${PROJECT_W_JWT_SECRET_KEY:-} - SMTP_PASSWORD=${PROJECT_W_SMTP_PASSWORD:-} frontend: image: ghcr.io/julianfp/project-w_frontend ports: - 80:80 environment: - NGINX_CONFIG=reverseProxy - SERVER_NAME= 5. Generate a JWT_SECRET_KEY that will be used to for generating Session Tokens. If you have python installed you can use the following command for this: .. code-block:: console python -c 'import secrets; print(secrets.token_hex())' 6. Build and run the containers. Replace and with the JWT_SECRET_KEY we generated before and the password of the SMTP Server you want to use respectively: .. code-block:: console PROJECT_W_JWT_SECRET_KEY="" PROJECT_W_SMTP_PASSWORD="" docker compose up -d 7. You may want to setup a systemd service or similar to start the containers automatically. Please be careful with where you store your JWT Secret Key and your SMTP Password, they should always stay secret! 8. You may want to set up some kind of backup solution. For this you just need to backup the project-W-data directory (which will include the database, your ssl certificate and your config.yml) and maybe your docker-compose.yml if you made changes to it. Runner `````` The runner currently doesn't use docker-compose for installation. Instead, you will have to clone the repository and build the docker image manually. .. note:: If you wish to run the container with cuda support, the installation may need some additional steps. Refer to the the user guide for `the NVIDIA container toolkit `_ for more information. 1. Clone the repository and enter it: .. code-block:: bash git clone https://github.com/JulianFP/project-W-runner.git && cd project-W-runner 2. Build the docker image: .. code-block:: bash docker build -t project-w-runner . Note that by default, the runner ``config.yml`` doesn't get copied into the image. Instead, you should mount it as a volume when running the container. If you really want the config as part of the image, remove the relevant line from the ``.dockerignore``. 3. Set the relevant config values: For the runner to work, it needs a config as described in :ref:`description_runner_config-label`. You always need to set the ``backendURL`` and ``runnerToken`` values, otherwise the runner will abort on startup. Please refer to :doc:`connect_runner_backend` for how to do that. .. warning:: The tokens must be unique per runner and must be kept secret. If you accidentally leaked a token, immediately contact an administrator to have the token revoked. If you are the administrator, please refer to :ref:`revoke_a_runner-label` for how to do that. 4. Run the container: .. code-block:: bash docker run --restart unless-stopped -v /path/to/config.yml:/app/config.yml --detach project-w-runner Note that the path to the config file should be an absolute path. The `--restart unless-stopped` option should make sure that the Runner will restart if it should crash and thus always stay online. 5. If you wish to use a custom directory for the Whisper model cache, you should specify it in the ``config.yml`` file: .. code-block:: yaml modelCacheDir: /models ... and mount it as a volume when running the container: .. code-block:: bash docker run --restart unless-stopped -v /path/to/config.yml:/app/config.yml -v /path/to/cache:/models --detach project-w-runner This way, you can remove the container without losing the cache, and you can prepopulate the cache by copying the Whisper models into the directory on the host. NixOS ----- We provide NixOS flakes for the backend, frontend and runner. Each of them include a NixOS module to setup the service, a nix-shell for development purposes as well as a package and overlay for running the service manually if desired. We will focus on the NixOS module here. Backend ``````` First you need to import our flake into your flake containing the NixOS config of your machine. For this add the following to your 'inputs' section of your flake.nix: .. code-block:: Nix inputs = { ... project-W = { url = "github:JulianFP/project-W"; inputs.nixpkgs.follows = "nixpkgs"; }; }; Next you need to pass your inputs as an argument to your outputs, where you then can import the module and apply the overlay: .. code-block:: Nix nixosConfiguration. = nixpkgs.lib.nixosSystem { ... pkgs = import nixpkgs { ... overlays = [ inputs.project-W.overlays.default ]; }; modules = [ inputs.project-W.nixosModules.default ... ]; }; Now you can start using the module. For a full list and description of options go to nix/module.nix in the project-W repository. Also the `settings` attribute set is basically just a copy of the options of the config file (however with different default values), so you can also refer to :ref:`description_backend_config-label` for this part. However the following config should get you started as well: .. warning:: The options 'settings.loginSecurity.sessionSecretKey' and 'settings.smtpServer.password' are available, but they are not very secure since it's contents will be public in the nix store! We strongly recommend to use the envFile option to add the secrets to your config. If you want your secrets to be part of your NixOS config, then please use sops-nix or agenix for that. .. code-block:: Nix services.project-W-backend = { enable = true; hostName = ""; settings = { clientURL = "https:///#"; smtpServer = { domain = ""; port = ; secure = ""; senderEmail = ""; username = config.services.project-W-backend.senderEmail; #probably, if not the same then set something different here }; }; envFile = ""; }; services.nginx.virtualHosts.${config.services.project-W-backend.hostName} = { forceSSL = true; http2 = true; enableACME= true; }; security.acme = { acceptTerms = true; certs = { ${config.services.project-W-backend.hostName}.email = ""; }; }; This setup already enables https and automatic ssl certificate renewal over let's encrypt for you. If you want to run this behind a reverse proxy, then just leave the nginx and acme part away. .. attention:: If you use a Reverse Proxy: Make sure that your reverse proxy is properly configured to handle the upload of large files. The backend can handle files up to a size of 1GB, setting this to anything less in your reverse proxy will hinder the submission of jobs and present the user with possibly confusing error messages! We will not cover the configuration of the reverse proxy here, but for example if you use nginx you will want to set ``client_max_body_size 1g;`` in your config. The envFile should contain the following. Please make sure to keep this secret!!!: .. code-block:: console JWT_SECRET_KEY="" SMTP_PASSWORD="" The JWT_SECRET_KEY can be generated with the following command: .. code-block:: console nix run nixpkgs#python3 -- -c 'import secrets; print(secrets.token_hex())' Rebuild your NixOS config and you are done! The backend now running under the systemd service 'project-W-backend.service' and is being served by nginx (in case you need to check the logs). If you want to do backups, you just need to backup the directory that is set with 'settings.databasePath' (per default: /var/lib/project-W-backend/database) as well as the directory where acme stores the ssl certificates (per default: /var/lib/acme/). Of course you also need to backup your NixOS config, but you probably have that in a git repo anyway ;) Frontend ```````` First you need to import our flake into your flake containing the NixOS config of your machine. For this add the following to your 'inputs' section of your flake.nix: .. code-block:: Nix inputs = { ... project-W-frontend = { url = "github:JulianFP/project-W-frontend"; inputs.nixpkgs.follows = "nixpkgs"; }; }; Next you need to pass your inputs as an argument to your outputs, where you then can import the module (for the frontend no overlay is required): .. code-block:: Nix nixosConfiguration. = nixpkgs.lib.nixosSystem { ... modules = [ inputs.project-W-frontend.nixosModules.default ... ]; }; Now you can start using the module. For a full list and description of options go to nix/module.nix in the project-W-frontend repository. However the following config should get you started as well: .. code-block:: Nix services.project-W-frontend = { enable = true; hostName = ""; backendBaseURL = "https://"; #leave to default if both domains are the same }; services.nginx.virtualHosts.${config.services.project-W-frontend.hostName} = { forceSSL = true; http2 = true; enableACME= true; }; security.acme = { acceptTerms = true; certs = { ${config.services.project-W-frontend.hostName}.email = ""; }; }; This setup already enables https and automatic ssl certificate renewal over let's encrypt for you. If you want to run this behind a reverse proxy, then just leave the nginx and acme part away. Rebuild your NixOS config and you are done! The frontend is now being served by nginx (in case you need to check the logs). Runner `````` First you need to import our flake into your flake containing the NixOS config of your machine. For this add the following to your 'inputs' section of your flake.nix: .. code-block:: Nix inputs = { ... project-W-runner = { url = "github:JulianFP/project-W-runner"; inputs.nixpkgs.follows = "nixpkgs"; }; }; Next you need to pass your inputs as an argument to your outputs, where you then can import the module (for the runner no overlay is required either): .. code-block:: Nix nixosConfiguration. = nixpkgs.lib.nixosSystem { ... modules = [ inputs.project-W-runner.nixosModules.default ... ]; }; Now you can start using the module. For a full list and description of options go to nix/module.nix in the project-W-runner repository. Also the `settings` attribute set is basically just a copy of the options of the runner config file (however with different default values), so you can also refer to :ref:`description_runner_config-label` for this part. However the following config should get you started as well: .. warning:: The option 'settings.runnerToken' is available, but it is not very secure since it's content will be public in the nix store! We strongly recommend to use the envFile option to add the secrets to your config. If you want your secrets to be part of your NixOS config, then please use sops-nix or agenix for that. .. code-block:: Nix services.project-W-runner = { enable = true; settings = { backendURL = ""; #torchDevice = "cuda:0"; #only enable this if you want to tell pytorch explicitly to use the first cuda device of the system }; envFile = ""; }; The envFile should contain the following. Please make sure to keep this secret!!!: .. code-block:: console RUNNER_TOKEN="" Rebuild your NixOS config and you are done! The runner is running under the systemd service 'project-W-runner.service'. By default, whisper models will be cached in the `/var/cache/project-W-runner_whisperCache` directory. Go there if you want to replace them. .. note:: We didn't test if the NixOS module would work with CUDA since we didn't have access to a NixOS machine with NVIDIA GPUs. If additional configuration in the module should be necessary: Contributions welcome! For CUDA support please add the cuda toolkit you want to use to `environment.systemPackages` in your NixOS config. .. _manual_installation-label: Manual installation ------------------- You can also run Project-W barebones. This can be a bit more difficult and the following steps will not be as detailed as the ones with Docker or NixOS. You will have to do stuff like configuring python virtual environments, setting up webservers or compiling the frontend yourself. Backend ``````` 1. Install Python (3.8 - 3.11 are tested to work) and pip 2. Clone this repository and enter it: .. code-block:: console git clone https://github.com/JulianFP/project-W.git & cd project-W 3. Install the package with pip: .. code-block:: console python -m pip install . 4. To run the backend server in production you need a webserver with WSGI support, for example gunicorn. Install gunicorn with pip: .. code-block:: console python -m pip install gunicorn 5. Run the backend server with gunicorn: .. code-block:: console gunicorn --bind :443 --certfile= --keyfile= project_W:create_app() Frontend ```````` The frontend is written in Svelte and needs to be compiled into native Javascript. To do this you will need some build dependencies, however you can remove them after step 4. If you want you can even build it on a different machine and then just move the dist directory to the server between step 4 and 5. 1. Install nodejs 2. Clone the frontend repository and enter it: .. code-block:: console git clone https://github.com/JulianFP/project-W-frontend.git & cd project-W-frontend 3. Install pnpm: .. code-block:: console npm install -g pnpm 4. Install all build dependencies: .. code-block:: console pnpm install 4. Build the frontend (replace with the url to the backend api. Leave empty if the frontend and backend are hosted on the same origin): .. code-block:: console VITE_BACKEND_BASE_URL="" pnpm build 5. You can find the result in the ./dist directory. Setup a webserver (like nginx) to serve all contents of this directory. Optionally you can also setup nginx in a way such that it forwards requests to /api/* routes to the gunicorn webserver (which then should run on a different port without ssl). This way both backend and frontend would run on the same server and origin. Make sure to enable https! Runner `````` 1. Install Python (3.9 or newer), pip, and ffmpeg. 2. Clone this repository and enter it: .. code-block:: bash git clone https://github.com/JulianFP/project-W-runner.git & cd project-W-runner 3. Install the package with pip: .. code-block:: bash python -m pip install . 4. Set the relevant config values: For the runner to work, it needs a config as described in :ref:`description_runner_config-label`. You always need to set the ``backendURL`` and ``runnerToken`` values, otherwise the runner will abort on startup. Please refer to :doc:`connect_runner_backend` for how to do that. .. warning:: The tokens must be unique per runner and must be kept secret. If you accidentally leaked a token, immediately contact an administrator to have the token revoked. If you are the administrator, please refer to :ref:`revoke_a_runner-label` for how to do that. 5. Start up the runner: .. code-block:: bash python -m project_W_runner 6. You may want to make sure that the runner will always restart itself even if it crashes. Currently this might happen in rare cases, so maybe write a script or a systemd service that will always automatically restart the runner in case of a crash.