allenai/ScienceWorld

This system is a deterministic simulation environment for interactive agent tasks, featuring a Scala-based simulator and a Python API for interaction. It provides a web-based demo and command-line tools for human and automated agents to navigate environments, perform actions, and access gold path sequences. The platform emphasizes reproducibility through strict deterministic ordering of objects and processes, supported by comprehensive testing and diagnostic utilities.

Property classes in the simulator (including Container, CoolingSource, Device, Edibility, ElectricalConnection, HeatSource, Life, Material, Moveable, and Portal properties) now include a toJSON method, enabling these objects to be serialized into JSON format for external consumption or debugging.

simulator/src/main/scala/scienceworld/properties · high confidence

New utility scripts have been added to the scripts directory to aid in debugging and verifying environment behavior. list\_scienceworld\_task.py prints a formatted table of all available ScienceWorld tasks, their IDs, and variation counts. reproduce\_heat\_transfer\_stochasticity.py runs a specific boiling task multiple times to check for inconsistent steam generation, helping identify stochasticity issues. reproduce\_issue\_82.py lists valid 'open door' actions for task 1-1 to help reproduce and investigate action inconsistency issues.

scripts · high confidence

A new web-based demo (scienceworld-web-server-example.py) is now available, allowing users to interact with ScienceWorld via a browser at localhost:8080 using the pywebio library, with automatic JSON history recording. The command-line examples (human.py, random\_agent.py) have been updated to use the new snake\_case API methods (e.g., get\_task\_names, get\_possible\_actions) and now expose the gold action sequence for debugging. Additionally, the human.py console now supports command-line autocompletion and history via prompt\_toolkit, and the environment step limit in the web demo has been increased to 10,000 steps.

examples · high confidence

The \python-api\ module has removed \loadGoldPaths1.py\, a script for processing gold sequence data, and \python-web-server.py\, which provided a PyWebIO-based web interface and Py4J integration for the Scala simulator. These deletions eliminate the previous Python-side mechanisms for loading gold path examples and serving the interactive web demo, indicating a shift away from these specific legacy interaction patterns.

python-api · high confidence

The simulator now produces deterministic output by sorting object lists (by UUID) when generating JSON representations of object referents, types, and IDs, and by using deterministic permutation generation for action combinations. Additionally, the 'Open Containers' simplification has been corrected to open all closable containers in the environment, rather than only portals, ensuring that items inside containers are accessible by default when this simplification is active.

simulator/src/main/scala/scienceworld/runtime · high confidence

The simulator now ensures deterministic simulation results by ordering object and portal enumerations using UUIDs instead of relying on non-deterministic Set iteration. Additionally, the system now validates referents to prevent errors during gold path generation, specifically checking for empty strings or malformed references (such as missing words in phrases like 'put apple in in inventory'). A typo in the \isDeteted\ method was also corrected to \isDeleted\.

simulator/src/main/scala/scienceworld/struct · high confidence

The simulator now produces deterministic output by enforcing consistent ordering on object collections and action lists. Object enumeration in the input parser uses LinkedHashSet and sorts results by object UUID, ensuring stable iteration order. Action requests are sorted by unique ID and name, and ambiguous match messages are presented in a consistent order. Additionally, portal objects now use canonical referents based on the perspective container, and the system validates generated referents to prevent malformed strings (such as those starting with spaces), exiting with an error if such issues are detected.

simulator/src/main/scala/scienceworld/input · high confidence

The simulator now produces consistent, ordered output when displaying lists of objects (such as in 'look around' descriptions) and when saving or exporting data. Previously, iterating over sets and key-value maps could yield unpredictable orderings; the code now explicitly sorts these collections by name/UUID or key before processing. This ensures that repeated actions or saved files are identical across runs, aiding in debugging and reproducibility.

simulator/src/main/scala/util · high confidence

The simulator now produces deterministic results for processes that previously depended on collection iteration order. In Chemistry, mixing container contents are sorted by UUID before processing, and the 'Sandwhich' class has been renamed to 'Sandwich' with corresponding updates to ingredient logic (e.g., 'cachew' to 'cashew'). In Combustion, water objects are now sorted by UUID before being checked to extinguish fires. Additionally, the 'cannonicalName' parameter in the LifeStage hierarchy (affecting all animal and plant life stages like birds, frogs, and butterflies) has been renamed to 'canonicalName', and various spelling errors in comments and strings throughout the processes package have been corrected.

simulator/src/main/scala/scienceworld/processes · high confidence

This update addresses several stability and correctness issues in the simulator. It prevents StackOverflowError crashes when electrical wires are connected to themselves by adding a recursion depth check in the \connectsToGround\ logic. It also corrects the description generation for pea plants and random genetics plants, ensuring genetic traits are properly prefixed without redundant phrasing. Additionally, the movement and interaction logic for bees and other wandering animals is now deterministic by sorting portals and objects by UUID, and the \Portal\ class now provides a canonical referent based on the user's perspective.

simulator/src/main/scala/scienceworld/objects · high confidence

Corrects spelling errors in comments and room names (e.g., 'green house' to 'greenhouse', 'metallic', 'electrical') and updates the agent spawning logic to sort locations by both name and UUID, ensuring consistent and deterministic placement across runs.

simulator/src/main/scala/scienceworld/environments · high confidence

The gold path example script has been moved from the python-api directory to the goldpaths directory and renamed to exampleLoadGoldPaths.py. The script now reads from a different input file (goldsequences-0-1-2-3.json) and includes logic to collect task metadata (index, variation, fold, description) into a list, although it continues to print action details to the console. Minor formatting fixes and a missing newline at the end of the file were also applied.

goldpaths · high confidence

The Python API now correctly preserves the gold action sequence across environment resets when gold path generation is enabled, fixing a bug where the sequence was previously cleared. Additionally, the \getValidActionObjectCombinations()\ method now returns results in a deterministic, sorted order, and a new \getObjectTree()\ method allows users to retrieve or save the current environment object tree as JSON.

simulator/src/main/scala/scienceworld/runtime/pythonapi · high confidence

The simulator's packaging script (package.sh) now uses a stricter execution mode (set -euo pipefail) and invokes Bash via the PATH to ensure consistent behavior across environments. Additionally, the script dynamically reads the library version from build.sbt instead of hardcoding it, ensuring the packaged JAR always matches the current project version.

simulator · high confidence

The task lookup mechanism in TaskMaker1 now handles variations in task identifiers more gracefully by stripping parentheses and normalizing the 'mendellian' spelling to 'mendelian' before searching the registry. Additionally, the registration process was corrected to use the properly spelled TaskMendelianGenetics2 class, ensuring that tasks with these specific naming patterns are found correctly during simulation setup.

simulator/src/main/scala/scienceworld/tasks · high confidence

The project now extracts its version dynamically from the JAR file metadata instead of hardcoding it in setup.py, and includes new object lookup and task definition files in the package. The README has been significantly updated with badges, a HuggingFace Space demo link, and a corrected EMNLP citation, while the setup configuration adds Python 3.7+ support and new extras for webserver and development dependencies.

(repo-wide) · high confidence

The ScienceWorld Python API has been significantly refactored to improve stability, usability, and maintainability. The Java server launch mechanism has been updated to use \py4j\'s \launch\_gateway\, enabling dynamic port allocation and proper process management, which resolves previous connection and shutdown issues. A new \constants.py\ module centralizes configuration, including a debug mode toggle via the \SCIENCEWORLD\_DEBUG\ environment variable and the loading of task definitions from a new \tasks.json\ file, replacing hardcoded or scattered task lists. Task identification is now handled by a robust \infer\_task\ utility in \utils.py\ that normalizes task names and IDs, while deprecated camelCase methods now trigger explicit deprecation warnings to guide users toward the new snake\_case API. Additionally, internal object references have been corrected (e.g., fixing typos like 'Cachew' to 'Cashew' and 'PrideAndPredjudice' to 'PrideAndPrejudice') to ensure consistency with the underlying simulator.

scienceworld · high confidence

The simulator's execution order for object iteration, taxonomy traversal, and parameter matching is now deterministic. Internal data structures (such as sets and maps in Program, PredicateRunner, and Taxonomy) have been replaced with ordered variants (TreeSet, LinkedHashMap, LinkedHashSet), and array conversions are explicitly sorted by object UUID or name. This ensures that repeated runs with the same input produce identical results, eliminating previous non-deterministic behavior caused by unordered collection iteration.

simulator/src/main/scala/language · high confidence

Task names in the simulator are now generated without parentheses, ensuring consistent formatting across all specific tasks. The location name 'green house' has been corrected to 'greenhouse' throughout the task definitions and pathfinding logic, and the 'Cachew' food item has been renamed to 'Cashew'. Additionally, various typos in comments, descriptions, and error messages (such as 'subtance' to 'substance' and 'mendellian' to 'mendelian') have been fixed, and sorting logic for inclined planes and flower pots has been updated to include UUIDs for deterministic ordering.

simulator/src/main/scala/scienceworld/tasks/specifictasks · high confidence

The simulator no longer crashes with a Java exception when evaluating goals that inspect the life stage of an object lacking a defined life property. In the \GoalFind\ logic, the code now safely checks if the \propLife\ property is defined before attempting to access its \lifeformType\, preventing runtime errors during deterministic enumeration audits. Additionally, the \ObjMonitor\ now returns objects in a deterministic, UUID-ordered set rather than an arbitrary set order, ensuring consistent behavior in goal tracking.

simulator/src/main/scala/scienceworld/tasks/goals · high confidence

The 'look' action now correctly handles Portal objects (such as doors) by passing the agent's current perspective container to the description retrieval logic, preventing errors when viewing these specific objects. This change is accompanied by general code hygiene improvements, including fixing typos in comments (e.g., 'santizied' to 'sanitized', 'poral' to 'portal') and ensuring all modified files end with a newline.

simulator/src/main/scala/scienceworld/actions · high confidence

The gold agent simulator now runs a broader set of tasks by default (0–29, excluding 5) instead of a limited subset, and corrects the JSON sanitization order in RunHistory to properly escape backslashes before quotes. Pathfinding reliability is improved by using stable sorting for object referents to ensure consistent selection of the longest name, and by adding validation checks that terminate execution if invalid referents (too short, leading spaces, or double spaces) are detected during path generation.

simulator/src/main/scala/scienceworld/goldagent · high confidence

Added tests in \tests/test\_scienceworld.py\ to verify that the ScienceWorld environment produces deterministic observations, object trees, and API snapshots across multiple resets and runs. The tests also ensure that object types, referents, and door actions are ordered deterministically, validate that multiple environment instances operate on separate ports without interfering with each other, confirm that task variations are disjoint, and check that the Java process is properly closed upon environment termination.

tests · high confidence

The simulator project has been updated to version 1.3.0, with a corresponding fix to the SBT build configuration for the main class definition. Additionally, a new requirements-dev.txt file has been introduced to manage development dependencies, including build tools, flake8, pre-commit, pytest, tox, and twine, separating them from the core py4j dependency in requirements.txt.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 64.

• Code Health 76
• Architecture 99
• Maturity 49
• Readiness 67
• Security 97

• 242 commits — 205 feature/other, 37 fixes

• (root) — 53 commits
• (repo) — 51 commits
• scienceworld/scienceworld.py — 48 commits
• simulator/src — 27 commits
• .github/workflows — 11 commits
• scienceworld/scienceworld.jar — 9 commits
• examples/human.py — 8 commits
• examples/web_demo_screenshot.png — 5 commits
• examples/scienceworld-web-server-example.py — 4 commits
• scienceworld/scienceworld-1.0.3rc2.jar — 4 commits
• scienceworld/utils.py — 3 commits
• examples/random_agent.py — 2 commits
• scienceworld/object_type_ids.tsv — 2 commits
• scienceworld/scienceworld-1.0.3rc1.jar — 2 commits
• simulator/build.sbt — 2 commits
• simulator/package.sh — 2 commits
• goldpaths/exampleLoadGoldPaths.py — 1 commit
• goldpaths/goldpaths-all.zip — 1 commit
• python-api/python-web-server.py — 1 commit
• scienceworld/constants.py — 1 commit

• fix: Add precommit and fix errors
• fix: Added special case for Portal objects, to fix bug where 'look <door object>' returns an error about needing to use the Poral override
• fix: Bugfix release. 1.1.1
• fix: Bugfix: Added a filter to EnvObject.getReferentsWithContainers() to ensure that the referents that it's returning are not of length zero. Added further checks downstream for gold path generation, that throw an error if the referent meets various issues (short length, starts with a space, or has two spaces in it which is indicative of a missing referent in the middle somewhere, e.g. put apple in in inventory)
• fix: Bump version and fix package.sh
• fix: Fix JAR file version in package.sh
• fix: Fix StackOverflowError when a wire is connected to itself
• fix: Fix errors in mapping of task names to IDs
• fix: Fix flake8
• fix: Fix flake8
• fix: Fix flake8 failures.
• fix: Fix flake8 violations. Facepalm.
• fix: Fix freelook being replaced with a Java exception
• fix: Fix how warnings get displayed for deprecated_api_warning and send them to the regular logger
• fix: Fix issue #77
• fix: Fix open-containers simplification
• fix: Fix order of escaping in RunHistory.sanitizeJSON
• fix: Fix sbt build warnings
• fix: Fix scienceworld-scala version
• fix: Fix syntax error introduced in ca6b8c9ac275dbea82a542edf7d175aafbbede6e
• …and 222 more

• 0 containers · 1 bounded contexts · 0 dependency edges (baseline)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

allenai/ScienceWorld was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.