MCPcopy
hub / github.com/IPADS-SAI/MobiAgent / frame_texts

Function frame_texts

utils/advanced_ocr.py:618–693  ·  view source on GitHub ↗

从Frame中提取文本列表,优先使用OCR识别 Args: frame: 包含图像路径的Frame字典 Returns: List[str]: 文本列表

(frame: Dict[str, Any])

Source from the content-addressed store, hash-verified

616 callable: 接受Frame参数,返回文本列表的函数
617 """
618 def frame_texts(frame: Dict[str, Any]) -> List[str]:
619 """
620 从Frame中提取文本列表,优先使用OCR识别
621
622 Args:
623 frame: 包含图像路径的Frame字典
624
625 Returns:
626 List[str]: 文本列表
627 """
628 # 获取图像路径
629 image_path = frame.get("image")
630 if not image_path or not os.path.exists(image_path):
631 # 退化到改进的XML提取
632 xml_text = frame.get("xml_text", "")
633 if xml_text:
634 xml_processed = extract_text_from_xml(xml_text)
635 if xml_processed.words:
636 frame['_xml_processed'] = xml_processed
637 # 返回多种格式的文本
638 result_texts = xml_processed.words.copy()
639 if xml_processed.cleaned:
640 result_texts.append(xml_processed.cleaned)
641 if xml_processed.no_spaces:
642 result_texts.append(xml_processed.no_spaces)
643 logger.info(f"图像不可用,使用改进XML提取,得到 {len(result_texts)} 个文本片段")
644 return list(set(result_texts))
645 else:
646 # 降级到传统方式
647 texts = frame.get("xml_texts", [])
648 if not texts and xml_text:
649 texts = [xml_text]
650 logger.warning("XML解析失败,使用原始XML文本")
651 return texts
652 else:
653 texts = frame.get("xml_texts", [])
654 logger.warning("无图像也无XML文本")
655 return texts
656
657 # 使用OCR获取文本
658 ocr_text, backup_text = processor.extract_text_from_image(image_path)
659 words = []
660
661 if ocr_text:
662 processed = processor.process_text(ocr_text)
663 words.extend(processed.words)
664 if processed.cleaned:
665 words.append(processed.cleaned)
666 if processed.no_spaces:
667 words.append(processed.no_spaces)
668
669 # 融合 XML
670 xml_text = frame.get("xml_text", "")
671 if xml_text:
672 xml_processed = extract_text_from_xml(xml_text)
673 if xml_processed.words:
674 frame['_xml_processed'] = xml_processed
675 words.extend(xml_processed.words)

Callers

nothing calls this directly

Calls 6

infoMethod · 0.80
warningMethod · 0.80
process_textMethod · 0.80
debugMethod · 0.80
extract_text_from_xmlFunction · 0.70

Tested by

no test coverage detected