Skip to content

NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character - #494

Open
nikhil-mittal-165 wants to merge 1 commit into
devfrom
fix/issue-2074-flow-utf8-encoding
Open

NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character#494
nikhil-mittal-165 wants to merge 1 commit into
devfrom
fix/issue-2074-flow-utf8-encoding

Conversation

@nikhil-mittal-165

@nikhil-mittal-165 nikhil-mittal-165 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Root Cause

quick_validate() in FlowScanner/public/parse_utils.py:1418 opens flow files with open(flow_path,'r') and no encoding argument, so Python uses the platform locale encoding. On Windows (cp1252) any flow XML containing a non-cp1252 UTF-8 byte (e.g. 0x9D from U+201D smart quote) raises UnicodeDecodeError. The bare except logs CRITICAL and returns False, dropping the flow; when all flows are dropped the scanner prints 'No flow files found to scan. Exiting...' and never writes flowScannerResultsFile.json, so the Node wrapper FlowScannerCommandWrapper.js fails to read it and surfaces a misleading ENOENT UnexpectedEngineError. Flow XML is authored/declared UTF-8, so locale-based decoding is simply wrong. Confirmed by PYTHONUTF8=1 fixing it, and by the fact that main.py:104 and custom_parser.get_root already read/parse as UTF-8 correctly.

Fix

Read flow files as UTF-8 (with cp1252 fallback) in quick_validate so UTF-8 flows with non-cp1252 bytes are not dropped on Windows locales; added Python unittest coverage and bumped package to 0.40.1-SNAPSHOT

Testing

  • ✅ Unit tests added/updated
  • ✅ All tests passing
  • ✅ Lint checks passing

Functional Testing Evidence

Tested on Dreamhouse project:

  • Test 1: Unit tests - UTF-8 flow validation with cp1252 locale - PASS (3/3 tests passed)
  • Test 2: CLI scan on UTF-8 flow with smart quotes - PASS (flow at 100% completion, no UnicodeDecodeError)
  • Test 3: Fix verification - PASS (commit c3e3ffa correctly reads flows as UTF-8 with cp1252 fallback)

Overall Status: PASS ✅

Files Changed

  • packages/code-analyzer-flow-engine/FlowScanner/public/parse_utils.py
  • packages/code-analyzer-flow-engine/FlowScanner/tests/test_parse_utils.py
  • packages/code-analyzer-flow-engine/package.json

@git2gus

git2gus Bot commented Aug 4, 2026

Copy link
Copy Markdown

Git2Gus App is installed but the .git2gus/config.json doesn't have right values. You should add the required configuration.

@nikhil-mittal-165 nikhil-mittal-165 changed the title fix: flow engine fails on Windows for any flow containing a non-ASCII character (cp1252 UnicodeDecodeError surfaced as ENOENT UnexpectedEngineError) (#2074) NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character Aug 4, 2026
@nikhil-mittal-165
nikhil-mittal-165 marked this pull request as ready for review August 4, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant