Backward Compatibility & Migrations

This page outlines breaking changes, purged legacy aliases, and migration steps when upgrading the gem.

0.27.0

Upgrade notes for configs and integrators targeting gem 0.27.0 (current master after 0.26.0):

1. MCP result envelope — Every MCP tool returns { ok, next_step, guidance, payload } in text and structuredContent. Read payload.items, payload.yaml, and payload.rss; scrape text is not a raw item array.
2. MCP AutoFallback — scrape and capture with strategy: "auto" run Faraday → Botasaurus in one call. inspect with auto stays on Faraday (pin botasaurus when you need browser rendering for inspect).
3. Capture output — html2rss capture and MCP capture emit an items selector plus enhance: true only (no per-field title/url/description selector soup). Capture stamps strategy: when AutoFallback selects a concrete transport.
4. CLI --explain — html2rss scrape --explain and html2rss capture --explain print telemetry JSON on stderr; stdout stays the feed or YAML. Status includes admission_drops when Cleanup dropped items.
5. Html2rss.auto_feed_result — URL-only auto discovery returns FeedResult with the same status telemetry as config builds.
6. request.botasaurus OpenAPI 2.0 — Keys must match the scrape-API ScrapeRequest (unknown keys rejected). Use scroll: true (not scroll_to_bottom), window_size: { width:, height: } (not a two-item array), and wait_timeout_seconds in 1..30. See Strategy.
7. Botasaurus wire format — Client expects OpenAPI 2.0 ScrapeSuccess / ScrapeError envelopes from the scrape API.
8. --limit default — Auto discovery default is 25 (AutoSource::DEFAULT_LIMIT).

Run html2rss validate config.yml after editing Botasaurus options.

0.26.0

Upgrade notes for configs and integrators targeting gem 0.26.0:

1. Browserless strategy removal — The browserless strategy, puppeteer-ruby dependency, interaction budgets, and request.browserless.* options have been completely removed.
2. Botasaurus as browser transport — Browser-based scraping and anti-bot bypassing now run through botasaurus (BOTASAURUS_SCRAPER_URL). Use strategy: auto (which tries faraday -> botasaurus) or pin strategy: botasaurus directly.

0.25.0

Upgrade notes for configs and integrators targeting gem 0.25.0:

1. Removed link selector alias — Custom selector configurations must use the canonical url key. The legacy link selector alias and exclusivity checks have been removed.
2. auto article limit (--limit N) — Discovery supports a configurable limit (--limit N CLI flag). Default was 20 in 0.25.0; 0.27.0 default is 25 (Html2rss.auto_source(url, limit: 25)).
3. Channel metadata in auto — auto_source automatically extracts channel-level author and image (favicon, touch-icon, or OpenGraph images) from the host page.
4. Actionable NoFeedItemsExtracted error — auto mode fails loud with diagnostic surface classifications (blocked surface, app-shell, unsupported extraction surface) when zero items are found.

0.24.0

Upgrade notes for configs and integrators targeting gem 0.24.0:

1. auto strategy chain — Default strategy: auto is faraday → botasaurus only. Pin strategy: browserless (or --strategy browserless) when you need Browserless preload/interaction; Browserless is not a fallback tier.
2. RSS enclosures — Configure media with the enclosure selector. RSS does not promote image into <enclosure>; images stay on the description / JSON Feed image.
3. Ruby dual-format / telemetry — Prefer Html2rss.feed_result for one scrape that must render RSS and JSON Feed (or be Marshal-cached). Read result.status.to_h for scrape telemetry (selected_strategy, attempt_count, strategy_attempts under auto).

Removed Legacy Selector Aliases

In previous versions, html2rss accepted legacy selector names with a deprecation warning. These shims have been removed:

Removed Name	Required Name	Description
link	url	Item destination URL
pubDate	published_at	Item publication date
updated	published_at	Item publication date

Migration Example

Update any occurrences of link, pubDate, or updated in your selectors block to url or published_at:

# Deprecated / Removed
selectors:
  link:
    selector: "a.title"
    extractor: "href"
  updated:
    selector: ".date"
# Current & Required
selectors:
  url:
    selector: "a.title"
    extractor: "href"
  published_at:
    selector: ".date"

Removed Channel Attributes

strategy and headers were previously tolerated inside the channel block. These legacy shims have been removed. Both keys must now be placed at the top level of your feed configuration:

# Deprecated / Removed
channel:
  url: "https://example.com/articles"
  strategy: botasaurus
  headers:
    User-Agent: "CustomAgent/1.0"
# Current & Required
strategy: botasaurus
headers:
  User-Agent: "CustomAgent/1.0"
channel:
  url: "https://example.com/articles"

Migration Checklist

When upgrading to 0.27.0 or later:

1. MCP integrators: Read tool results from the envelope payload; configure BOTASAURUS_SCRAPER_URL on the MCP process env.
2. Align request.botasaurus with OpenAPI 2.0: Use scroll, { width, height } for window_size, and wait_timeout_seconds in 1..30 (see Strategy).
3. Validate configurations: Run html2rss validate config.yml.

When upgrading from releases before 0.26.0:

4. Migrate Browserless to Botasaurus: Use strategy: botasaurus or strategy: auto with BOTASAURUS_SCRAPER_URL.
5. Rename URL selectors: Use url rather than link.
6. Rename date selectors: Use published_at rather than updated or pubDate.
7. Move channel-level transport keys: Put strategy and headers at the top level of the YAML file.
8. Use enclosure for RSS media: Select podcast/media URLs with enclosure; image does not become RSS <enclosure>.
9. Prefer feed_result for dual-format / cache: Use Html2rss.feed_result / Html2rss.auto_feed_result and status.to_h.