openworkload/swm-core

Sky Port is a distributed workload management system that orchestrates job execution across cloud and local environments using Erlang/OTP. It provides a user-facing HTTP API for submitting, monitoring, and managing jobs, while handling resource allocation, scheduling, and containerized execution. The system supports secure communication via TLS 1.3 and SSH tunneling, and includes tools for local development and production deployment via Docker containers.

26 changes.

The project rebranded from Sky Workload Manager to Sky Port Core Daemon, introducing a containerized development environment and upgrading to Erlang/OTP 27. Significant architectural changes included migrating C++ serialization to the ei library, refactoring state machines to gen\_statem, and enforcing TLS 1.3 for secure communications. The period also saw the expansion of the user-facing API, the addition of SSH tunneling capabilities, and the standardization of entity accessors across the codebase.

4 changes.

This period focused on establishing production-ready Docker containerization for the SWM service while introducing a dedicated debug environment to streamline local development. Concurrently, the project modernized its configuration management by migrating to the Erlang logger, updating TLS certificate handling, and refactoring systemd environment variables for improved deployment flexibility.

This change introduces a new directory (priv/container/release) containing the necessary artifacts to deploy SWM as a Docker container on production machines. It includes a Dockerfile that builds an Ubuntu-based image with Python 3.10, swm-core, and the swm-cloud-gate, along with supervisord configuration files to manage the swm-core and swm-gate processes. A README provides instructions for building the image, initializing the spool directory, and running the container with exposed ports for internal communication (10001, 10002) and HTTP access (8443, 8444).

priv/container/release · high confidence

The scripts/api directory now provides a comprehensive set of shell utilities for interacting with the Skyport user API. New scripts allow users to cancel, requeue, and submit jobs (including Jupyter notebooks), as well as retrieve details for jobs, flavors, nodes, and remotes, and fetch job stdout/stderr. Existing scripts have been updated to use certificates from the user's home directory (\~/.swm) instead of system paths, and the default container image for job submissions has been updated to Ubuntu 22.04.

scripts/api · high confidence

The networking layer now supports direct SSH tunnel creation and TCP port forwarding via new \wm\_ssh\_client\, \wm\_ssh\_server\, and \wm\_tcp\_proxy\ modules, enabling jobs to securely expose internal services. Additionally, the \wm\_tcp\_client\ and \wm\_tcp\_server\ modules have been refactored to enforce TLS 1.3, update SSL handshake logic, and standardize module naming (e.g., \wm\_tcp\_client\), while \wm\_file\_transfer\ gains a synchronous SFTP upload function for improved reliability.

src/net · high confidence

The scripts directory now includes a comprehensive set of new tools for managing Sky Port containers: \build-debug-container.sh\ and \build-release-container.sh\ handle image creation (targeting Erlang 27.3 for debug builds), while \run-in-dev-container.sh\, \start-debug-container.sh\, \start-release-container.sh\, and \start-skyport-dev-bg.sh\ manage container lifecycle, networking, and background service startup. Additionally, \setup-skyport.sh\ and \setup-swm-core.py\ provide interactive and programmatic initialization for the swm-core environment, and \ping.escript\ offers an SSL-based connectivity check. These scripts replace older development workflows, removing legacy files like \build-dev-container.sh\ (which used Erlang 22.3) and \job.escript\.

scripts · high confidence

A new debug container setup has been introduced in the \priv/container/debug\ directory to streamline local development and testing. This includes a Dockerfile based on Ubuntu 22.04 that pre-installs Erlang/OTP 27.3.3, Python 3.10, and a comprehensive suite of development and debugging tools (such as gdb, htop, and strace). The environment is managed via supervisord configurations (\supervisord.conf\ and \supervisord\_gate.conf\) to automatically start the core service and cloud gate components. Additionally, the \build-package.sh\ script has been relocated to this directory to support the new containerized workflow.

priv/container/debug · high confidence

The \wm\_job\ and \wm\_job\_cli\ modules, which provided the command-line interface for submitting, listing, canceling, and showing job status, have been removed from the source code. This change eliminates the legacy job management functionality from the user-space tools, meaning users can no longer interact with jobs via these specific CLI commands.

src/usr · high confidence

The compute service now supports running jobs locally within the swm-core container for testing purposes, triggered when the scheduled job's node ID matches the current node. This change also introduces a new handler for the 'job\_canceled' event to properly clean up job state and node allocations upon cancellation, and refactors entity attribute access to use the new get/set API. Additionally, job state updates now use a defined constant for the running state, and logging has been consolidated for better readability.

src/srv/compute · high confidence

The \wm\_ctl\ command-line interface has been refactored to support a unified attribute-setting syntax across all entity types (remote, global, grid, cluster, partition, node, user, queue, scheduler, image). Users can now use the pattern \\<entity\> \<name\> set \<attr\> \<value\>\ in addition to the existing \\<entity\> set \<attr\> \<value\>\ form. The interface also removes the legacy \wmjob\ mode and Erlang node simulation commands (\startsim\, \stopsim\, \sim\), simplifying the shell prompt and command structure. Additionally, the \tree\ output format has been improved to display attributes as \Name: Value\ pairs instead of raw tuples, and the \user\ list view no longer displays Groups or Projects columns.

src/ctl · high confidence

The cloud gate interface has been refactored to remove the requirement for explicit credential objects in partition and resource management calls, as the gate now manages its own credentials internally. This change updates the API signatures for operations like creating, deleting, and listing partitions, images, and flavors. Additionally, the system now parses GPU resources from flavor definitions and assigns unique IDs to template nodes, while the virtual resource manager has been migrated from gen\_fsm to gen\_statem to support more robust state handling and improved error reporting during job execution.

src/srv/cloud · high confidence

The user-facing HTTP API now supports additional job management operations: users can requeue existing jobs, purge all jobs for a user, and retrieve job stdout and stderr content via new endpoints. The job submission process has been updated to accept an IP address for submission address configuration and stores the script content directly. New endpoints allow listing flavors, images, and remote sites, and node information now includes roles and resource counts. Job cancellation also cleans up associated entities from the configuration.

src/srv/user · high confidence

Container execution logs are now saved to a dedicated \container.log\ file in each job's directory, allowing users to review output after the job finishes. Additionally, the system now verifies that the specified Docker image and any referenced volume containers exist before attempting to create a new container, preventing failures caused by missing resources.

src/srv/container · high confidence

The C++ library in c\_src/lib has been refactored to replace the erl\_interface ETERM-based serialization with the lighter ei library using byte buffers. This change removes the erl\_interface dependency from the build (Makefile) and updates entity constructors and parsing functions (e.g., SwmAccount, SwmCluster, SwmBootInfo) to decode data from raw character buffers. Additionally, the generated entity code now uses byte buffers for tuple properties instead of ETERM, and the job state constant has been corrected from 'CANCELLED' to 'CANCELED'.

_c\src/lib · high confidence

The monitoring service (wm\_mon) replaces its internal process management with gen\_server, changing the export\_data call from a protected utility wrapper to a direct gen\_server:call. Additionally, standard OTP callback specs (init, handle\_call, handle\_cast, handle\_info, terminate, code\_change) are added to wm\_mon, and specs are added to the Cowboy REST handler (wm\_mon\_rest) for init, content\_types\_provided, and json\_handler. Minor log message formatting fixes are also included.

src/srv/mon · high confidence

The setup configuration has been restructured to support local debugging and release container execution. The legacy \cloud.config\ and \setup-config.linux\ files are removed or renamed, replaced by a new \skyport-openstack.config\ that defines a local partition, a 'localhost' flavor for resource templates, and specific accounts for localhost and Azure operations. The main \skyport.config\ is updated to remove the grid abstraction, simplify the cluster manager to 'node', and add a 'localhost' remote endpoint alongside the Azure remote. Additionally, the systemd service definition is changed from a forking to a simple foreground type to prevent hangs during startup and shutdown, ensuring more reliable service management.

priv/setup · high confidence

The distributed commit module (wm\_commit) has been migrated from the gen\_fsm behavior to gen\_statem, introducing a new state\_functions callback mode and a longer transaction timeout (300000ms) to better handle slow network conditions. Concurrently, the entity API has been standardized: wm\_entity:get/set\_attr() calls are replaced with wm\_entity:get/set(), and wm\_db queries now use the new accessor functions. Configuration synchronization intervals have been increased (60s sync, 120s pull timeout) to prevent timeouts over SSH tunnels, and the wm\_api module now returns {error, timeout} instead of an empty list on call failures. Additionally, a new wm\_resource\_utils module was added to handle ingress port mapping and job networking info, while legacy simulation (wm\_sim) and crash report (wm\_crash\_report) modules were removed.

src/lib · high confidence

The core service now uses the unified wm\_entity:get/2 API instead of the legacy get\_attr/2, standardizing how node and service properties are retrieved across the system. Parent node discovery has been simplified: the find\_my\_parents function no longer accepts or processes short names (sname) for parent resolution, relying solely on host and port arguments, and the concept of a grid management node has been expanded to include cluster management nodes. Additionally, the start\_slave capability has been replaced by start\_peer to reflect the new peer-based node model, and the has\_malfunction simulation feature has been removed.

src/srv/core · high confidence

The admin service no longer supports starting or stopping node simulations, as the corresponding API endpoints and internal logic have been removed. This change also standardizes the entity attribute API by replacing the legacy get\_attr/set\_attr calls with the newer get/set interface across the admin module.

src/srv/admin · high confidence

The scheduler and relocator components have been refactored to replace the legacy gen\_fsm behavior with gen\_statem, introducing new state handling for job cancellation and relocation events. Concurrently, all internal entity attribute access has been standardized from the deprecated get\_attr/set\_attr API to the new get/set functions, ensuring consistent data retrieval and modification across the scheduler's job and node management logic.

src/srv/scheduler · high confidence

The development workflow is updated to support Erlang/OTP 27 and a new containerized setup: the Makefile now distinguishes between debug and release containers (build-debug-container, build-release-container), and a .tmuxinator.yml file provides a pre-configured tmux layout for running core, logs, and services. Documentation is reorganized with a new AGENTS.md guide, a CODE\_OF\_CONDUCT.md, and renamed markdown files (DESIGN.md, STYLE.md). Dependencies are upgraded to gun 2.2.0, cowboy 2.13.0, cowlib 2.15.0, parse\_trans 3.4.2, and sync 0.4.1, with rebar3 lockfile updated to version 1.2.0.

(repo-wide) · high confidence

The application description has been updated from "Sky Workload Manager" to "Sky Port Core Daemon" to reflect the new identity. The startup logic now defaults the parent port to 0 instead of none, and the SSL application has been added to the dependencies list. Additionally, the environment variable SWM\_PARENT\_SNAME is no longer used for the parent sname configuration.

src · high confidence

The system now uses 'R' to represent the running job state instead of 'E' to avoid ambiguity with the error state, and renames the cancelled state label from 'CANCELLED' to 'CANCELED' for US-English spelling consistency. Additionally, the include headers now define default values for the parent API port (10002), SSH daemon and provision ports (10022, 22), data transfer port (31337), certificate directory, remote user directory name ('job'), and credentials file path, establishing standard configuration defaults for the worker manager.

include · high confidence

The example job scripts in priv/examples/jobscripts have been refreshed to reflect current configuration standards and new capabilities. Several legacy scripts (simple-quick.job, simple-sleep.job) were removed and replaced with updated versions (hpc-container.sh, multi-node.sh, netcat.sh) that explicitly define cloud accounts, flavors, container images, and storage settings. Additionally, a new multi-node MPI example (multi-node.job) was added to demonstrate how to run distributed jobs across multiple nodes, including hostfile generation and MPI compilation.

priv/examples · high confidence

The system's connection and synchronization timing has been adjusted: connection retry timeouts increased to 20 seconds, pinger intervals extended to 15 seconds, and configuration sync intervals set to 60 seconds with a 120-second timeout. New configuration options allow limiting concurrent cloud job relocations to one and define specific ports for SSH daemon and parent API communication. The user-facing API documentation has been significantly expanded to include endpoints for listing images, remote sites, nodes, and flavors, while job submission now explicitly supports multipart form data for script content. Additionally, the data schema has been cleaned up by removing obsolete entities like 'malfunction', 'project', and 'group', and job records now include state details and execution paths.

priv · high confidence

The accounting service now uses the simplified \wm\_entity:get/2\ and \wm\_entity:set/2\ functions instead of the previous \get\_attr\/\set\_attr\ variants to retrieve and modify node and resource properties, ensuring consistency with the underlying entity model. Additionally, new public functions \norming/1\, \koef/1\, and \node\_weight/3\ have been added to support pricing calculations based on data volume, price, and network latency, while the module's test suite has been migrated from \erlang:eunit\ to \eunit\.

src/srv/accounting · high confidence

The HTTP server now prioritizes TLS 1.3 for secure connections, upgrading from the previous support of TLS 1.1 and 1.2. This change also simplifies the internal certificate verification logic by leveraging Cowboy's built-in CA file handling, removing the need for manual PEM decoding and custom chain validation code in the application layer.

src/srv/http · high confidence

The porter component has been rewritten to replace the erl\_interface library with the ei library for encoding and decoding Erlang terms. This change updates the C++ build standard from C++14 to C++17 and removes the erl\_interface dependency. For users, this ensures more robust handling of job and user data serialization, improved error reporting for I/O and process state changes, and better alignment with the system's Erlang binary format versioning.

_c\src/porter · high confidence

The configuration files have been renamed from .tmpl to .src, and the system now uses the Erlang logger instead of SASL for logging, directing output to erlang.log. TLS distribution options have been updated to use a CA chain certificate file (ca-chain-cert.pem) instead of a single cert file. Additionally, the systemd unit environment variables have changed from using parent node names (SWM\_SNAME, SWM\_PARENT\_SNAME) to using parent host and port (SWM\_PARENT\_HOST, SWM\_PARENT\_PORT).

config · high confidence

Added C++ unit tests for entity decoding; Expanded and stabilized Common Test suite for SSH tunnels, file transfer, and virtual resources; Updated porter test to use new entity API and script content attribute.

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 60.

• Code Health 84
• Architecture 99
• Maturity 67
• Readiness 59
• Security 63
• Event-Driven 54
• Event Sourcing 100

• 300 commits — 211 feature/other, 89 fixes

• src/srv — 79 commits
• (root) — 65 commits
• src/lib — 28 commits
• c_src/lib — 20 commits
• priv/openapi.yaml — 13 commits
• src/net — 12 commits
• HOWTO/AZURE.md — 11 commits
• priv/container — 11 commits
• scripts/api — 9 commits
• priv/setup — 6 commits
• .github/workflows — 5 commits
• priv/build — 5 commits
• HOWTO/JOB_DIRECTIVES.md — 2 commits
• priv/examples — 2 commits
• python/swmclient — 2 commits
• scripts/setup-skyport-dev.linux — 2 commits
• scripts/start-debug-container.sh — 2 commits
• src/ctl — 2 commits
• src/usr — 2 commits
• test/wm_gate_SUITE.erl — 2 commits

• fix: A set of small fixes and improvements to make cloud main node boot fine again
• fix: Add compile erlang libs to distribution (fix escript)
• fix: Apply temporary fix for the user UID/GID detection
• fix: Attempt to fix github actions
• fix: Exclude wm_cert.erl from formatting because of bug in formatter plugin
• fix: Fix CI
• fix: Fix GPUs in container
• fix: Fix Open Workload web site name
• fix: Fix README markdown syntax
• fix: Fix addresses calculation for pinger
• fix: Fix and enable unit tests in github CI
• fix: Fix cloud gate init test
• fix: Fix common tests
• fix: Fix common tests
• fix: Fix common tests
• fix: Fix common tests
• fix: Fix common tests
• fix: Fix common tests
• fix: Fix compilation issues
• fix: Fix connection to the cloud gate
• …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

openworkload/swm-core was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.