Overview
This issue follows from #122, where it was observed that running mothra-text directly produces significantly better text alignment than running the same folios through the mothra landing page. After investigating the integration code, I've identified two mechanisms in the integrated pipeline that have no equivalent in a standalone run_pipeline.py call, and a recommendation for how masking should be handled.
@giannatan — since you own the integration, tagging you here.
Likely Causes
1. Masking is applied via YOLO class-0 bboxes — not via --mothra-json
In the landing page pipeline, text_api.py converts YOLO text-class detections to a mask_json and uses MothraImageMask to pre-mask the image before passing it to run(). This is different from passing mothra_json_path into run() (the code path triggered by --mothra-json in the CLI).
The --mothra-json masking path has been specifically fine-tuned to perform better than plain bbox masking. I have tested this directly: running run_pipeline.py --mothra-json <yolo_json> on the same folios produces substantially better results than the current integration. The integrated pipeline should be changed to pass the YOLO JSON as mothra_json_path to run() rather than pre-masking the image externally.
If a YOLO text-bbox is slightly too large, offset, or covers a line that shouldn't be masked, it gets blacked out before Kraken BLLA ever sees it — causing that line to be silently skipped. This is the most likely explanation for the whole-line skip observed in #122 on folio 013r.
2. Post-BLLA music overlap filter silently drops detected lines
After run() returns, text-service calls filter_lines_over_music(), which drops any text line whose bbox overlaps >30% with a YOLO music region. This filter does not exist anywhere in run_pipeline.py. In manuscripts with interleaved text and music staves, this filter can silently discard valid text lines if YOLO music bboxes extend slightly into text territory.
3. mothra-text submodule version drift
The mothra-text submodule is pinned to a specific commit. If the pinned commit is behind the current main of mothra-text, there may be code-level differences between what the landing page runs and what the standalone pipeline produces. This should be verified and the submodule updated to the latest stable commit.
Recommendation
Masking in the integrated pipeline should be accomplished by passing the YOLO JSON output as mothra_json_path to run(), rather than pre-masking the image externally. This uses the same masking logic as --mothra-json in the CLI, which has been tested and tuned to produce better results. The external pre-masking approach in text_api.py should be removed in favor of this.
The filter_lines_over_music() post-filter should also be reviewed — it may be doing more harm than good on interleaved manuscript pages.
Overview
This issue follows from #122, where it was observed that running mothra-text directly produces significantly better text alignment than running the same folios through the mothra landing page. After investigating the integration code, I've identified two mechanisms in the integrated pipeline that have no equivalent in a standalone
run_pipeline.pycall, and a recommendation for how masking should be handled.@giannatan — since you own the integration, tagging you here.
Likely Causes
1. Masking is applied via YOLO class-0 bboxes — not via
--mothra-jsonIn the landing page pipeline,
text_api.pyconverts YOLO text-class detections to amask_jsonand usesMothraImageMaskto pre-mask the image before passing it torun(). This is different from passingmothra_json_pathintorun()(the code path triggered by--mothra-jsonin the CLI).The
--mothra-jsonmasking path has been specifically fine-tuned to perform better than plain bbox masking. I have tested this directly: runningrun_pipeline.py --mothra-json <yolo_json>on the same folios produces substantially better results than the current integration. The integrated pipeline should be changed to pass the YOLO JSON asmothra_json_pathtorun()rather than pre-masking the image externally.If a YOLO text-bbox is slightly too large, offset, or covers a line that shouldn't be masked, it gets blacked out before Kraken BLLA ever sees it — causing that line to be silently skipped. This is the most likely explanation for the whole-line skip observed in #122 on folio 013r.
2. Post-BLLA music overlap filter silently drops detected lines
After
run()returns,text-servicecallsfilter_lines_over_music(), which drops any text line whose bbox overlaps >30% with a YOLO music region. This filter does not exist anywhere inrun_pipeline.py. In manuscripts with interleaved text and music staves, this filter can silently discard valid text lines if YOLO music bboxes extend slightly into text territory.3. mothra-text submodule version drift
The mothra-text submodule is pinned to a specific commit. If the pinned commit is behind the current
mainof mothra-text, there may be code-level differences between what the landing page runs and what the standalone pipeline produces. This should be verified and the submodule updated to the latest stable commit.Recommendation
Masking in the integrated pipeline should be accomplished by passing the YOLO JSON output as
mothra_json_pathtorun(), rather than pre-masking the image externally. This uses the same masking logic as--mothra-jsonin the CLI, which has been tested and tuned to produce better results. The external pre-masking approach intext_api.pyshould be removed in favor of this.The
filter_lines_over_music()post-filter should also be reviewed — it may be doing more harm than good on interleaved manuscript pages.