sparklemotion/mechanize

Mechanize is a Ruby library for automating interaction with websites, handling HTTP requests, form submissions, and file downloads. It provides robust features for web crawling, including navigation history, meta-refresh handling, and streaming large files to disk. The system emphasizes security by preventing credential leakage during cross-origin redirects and supports various authentication schemes like NTLM and Digest. It also includes comprehensive utilities for parsing HTML, managing cookies, and simulating modern browser user agents.

5 changes.

This period focused on addressing critical security vulnerabilities related to credential leakage during cross-origin redirects and meta refreshes. Concurrently, the project modernized its infrastructure by migrating to Bundler, updating user-agent strings for current browsers, and refactoring the error hierarchy to support streaming downloads.

8 changes.

This period focused on restructuring Mechanize's internal architecture by refactoring form fields, page elements, and the HTTP agent into specialized, modular classes. Significant effort was dedicated to expanding test coverage for core functionalities, including security regression tests and robust handling of authentication and relative links. The work also included adding practical example scripts to demonstrate advanced usage patterns like spidering and file uploads.

The examples directory now includes several new scripts: \latest\_user\_agents.rb\ fetches current user agents from WhatIsMyBrowser.com; \spider.rb\ demonstrates web crawling with unlimited history; \wikipedia\_links\_to\_philosophy.rb\ implements the XKCD 903 algorithm to traverse Wikipedia links to the Philosophy article; \flickr\_upload.rb\ shows how to log in and upload to Flickr; \rubygems.rb\ demonstrates logging into RubyGems; \mech-dump.rb\ dumps a page's inspection output; and \proxy\_req.rb\ shows how to use a proxy.

examples · high confidence

Mechanize now strips sensitive headers (Proxy-Authorization, Cookie2, and headers set via request\_headers=) when following a redirect or meta refresh to a different origin (scheme, host, or port). This prevents credential leakage to unintended sites, addressing security advisories [GHSA redacted], [GHSA redacted], and [GHSA redacted].

(repo-wide) · high confidence

The HTTP agent implementation has been restructured into dedicated classes (Agent, AuthStore, AuthChallenge, WWWAuthenticateParser) to improve security and reliability. A key behavioral change is that credentials and sensitive headers (Authorization, Proxy-Authorization, Cookie, Cookie2) are now automatically withheld when a redirect crosses an origin (scheme or port change), preventing accidental password disclosure to third-party servers. The agent now features a robust WWW-Authenticate parser that correctly handles multiple authentication schemes (e.g., 'Negotiate, NTLM') and whitespace delimiters, preventing infinite loops on malformed headers. Additionally, it supports NTLM authentication with domain arguments, parses Content-Disposition headers more strictly (supporting ISO 8601 dates), and introduces configurable timeouts and error code handling for better control over HTTP interactions.

lib/mechanize/http · high confidence

Mechanize now uses a dedicated error class hierarchy under Mechanize::Error (such as ResponseReadError, ContentTypeError, and ChunkedTerminationError) instead of raising generic RuntimeErrors, providing more specific exception types for error handling. Additionally, a new Mechanize::Download class enables streaming large file responses directly to disk, reducing memory usage compared to the previous in-memory Mechanize::File approach, and a Mechanize::DirectorySaver is introduced to save downloaded files into a flat directory structure.

lib/mechanize · high confidence

The generic form field handling has been split into specialized classes (such as Text, Hidden, TextArea, CheckBox, RadioButton, SelectList, and FileUpload) that inherit from a common base. This change improves how different input types are represented and processed, including specific support for keygen tags, image buttons, and multi-select lists, while ensuring fields are sorted by their page appearance before submission.

lib/mechanize/form · high confidence

The page parsing logic has been restructured into dedicated classes (Link, Image, Frame, Label, MetaRefresh, Base) with improved attribute access and resolution. Links now support DOM ID/class queries and lazy text population, while images expose MIME types, captions, and download methods. Meta-refresh parsing now correctly handles decimal delays and escapes special URI characters to prevent errors, and frames can be accessed directly via their content.

lib/mechanize/page · high confidence

The default user-agent strings for Linux Firefox, Mac Firefox, Windows Chrome, Windows Edge, Android, iPad, and iPhone have been updated to current versions (e.g., Firefox 121, Chrome 120, Edge 120, Safari 17.2). This ensures that websites receiving requests from Mechanize will see realistic, up-to-date browser identifiers, which can affect content delivery, caching, and compatibility checks.

lib · high confidence

Added test document for relative link resolution scenarios; Added test fixture for file URLs with embedded spaces; Added test fixtures for HTML parsing and form handling; Expanded test coverage for Mechanize core components; Refactored test infrastructure into modular WEBrick servlets.

The project has switched from the Hoe build system to Bundler, introducing a Gemfile for development dependencies (minitest, rake, rdoc, rubocop, and optional compression gems brotli and zstd-ruby) while moving runtime dependencies into the mechanize.gemspec. This change updates runtime requirements to addressable \~\> 2.8, mime-types \~\> 3.3, net-http-persistent (\>= 2.5.2, \< 5.0.dev), nokogiri (\>= 1.11.2, \~\> 1.11), and webrick \~\> 1.7, and enforces a minimum Ruby version of 2.6.0.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 51 → 65 (+14.4)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 97 → 92 (-5.6)
• Architecture 94 → 87 (-6.9)
• Maturity 57 → 50 (-7.1)
• Readiness 29 → 67 (+38.4)
• Security 72 → 93 (+20.5)

• Coverage not measured — test suite did not build
• Dimension evaluation failed
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• High: security finding (details withheld)
• No exposed public API
• No tests found
• Test reliability not included
• The README is a single-file documentation of the mechanize gem rather than an internal architecture document describing how the library works and why it was built. (README.md)
• The README mentions security vulnerability reporting at GitHub but does not link to SECURITY.md or describe the policy. (README.md)

• Agent.fetch (cognitive 31) (lib/mechanize/http/agent.rb)
• Agent.fetch (cyclomatic 30) (lib/mechanize/http/agent.rb)
• Agent.resolve (cognitive 34) (lib/mechanize/http/agent.rb)
• Agent.resolve (cyclomatic 24) (lib/mechanize/http/agent.rb)
• Agent.response_authenticate (cognitive 19) (lib/mechanize/http/agent.rb)
• Agent.response_read (cognitive 17) (lib/mechanize/http/agent.rb)
• Ambiguous intent: get_file suggests a specific high-level operation (download/save) but takes HTTP GET parameters, while get is the generic HTTP verb. It is unclear if get_file returns a File object, saves to disk, or just performs a GET with different defaults.
• ContentDispositionParser.parse_parameters (cognitive 20) (lib/mechanize/http/content_disposition_parser.rb)
• ContentDispositionParser.rfc_2045_quoted_string (cognitive 18) (lib/mechanize/http/content_disposition_parser.rb)
• CookieJar.load (cognitive 19) (lib/mechanize/cookie_jar.rb)
• CookieJar.save (cognitive 16) (lib/mechanize/cookie_jar.rb)
• Duplicate intent between CookieJar and CookieJarIMethods: It is unclear if these are separate classes or if CookieJarIMethods is a module included in CookieJar. If included, the methods save_as and load_cookiestxt duplicate save and load with different signatures (filename vs io).
• Duplicated block (6 lines × 2) (lib/mechanize/test_case/one_cookie_servlet.rb)
• FileTooLong: http/agent.rb (lib/mechanize/http/agent.rb)
• FileTooLong: lib/mechanize.rb (lib/mechanize.rb)
• FixmeComment (lib/mechanize.rb)
• FixmeComment (lib/mechanize/form.rb)
• Form.parse (cognitive 16) (lib/mechanize/form.rb)
• Form.parse (cyclomatic 24) (lib/mechanize/form.rb)
• HackComment (lib/mechanize/history.rb)
• …and 41 more

• 13 commits — 7 feature/other, 6 fixes

• (root) — 10 commits
• (repo) — 1 commit
• .github/workflows — 1 commit
• lib/mechanize — 1 commit

• fix: fix: a meta refresh to another origin drops sensitive headers
• fix: fix: a redirect crosses an origin when scheme or port changes
• fix: fix: a withheld credential stays withheld for the whole operation
• fix: fix: repair three regressions from the header merge
• fix: fix: request_headers no longer follow a redirect across sites
• fix: fix: strip Proxy-Authorization and Cookie2 on cross-host redirect
• change: Merge pull request #676 from sparklemotion/card-227-redirect-header-leak
• change: ci: include ruby 4.0 in the matrix (#675)
• change: doc: CHANGELOG describes only released-to-released changes
• change: doc: Clean up CHANGELOG
• change: doc: request_headers documents the cross-origin rule
• change: doc: target a patch release
• change: version bump to 2.14.1

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

sparklemotion/mechanize was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.