Evaluation Agent: Evaluate code (SVG or mxGraph XML) quality, return score and critique Supports all topic types: paper, survey, blog, textbook Supports all output formats: svg, mxgraphxml
(
code: str,
code_image: Image.Image,
paper_content: str,
reference_figures: List[Image.Image],
iteration: int,
topic: str = 'paper',
output_format: str = None
)
| 1435 | |
| 1436 | |
| 1437 | def evaluate_code( |
| 1438 | code: str, |
| 1439 | code_image: Image.Image, |
| 1440 | paper_content: str, |
| 1441 | reference_figures: List[Image.Image], |
| 1442 | iteration: int, |
| 1443 | topic: str = 'paper', |
| 1444 | output_format: str = None |
| 1445 | ) -> Tuple[float, Optional[Dict]]: |
| 1446 | """ |
| 1447 | Evaluation Agent: Evaluate code (SVG or mxGraph XML) quality, return score and critique |
| 1448 | Supports all topic types: paper, survey, blog, textbook |
| 1449 | Supports all output formats: svg, mxgraphxml |
| 1450 | """ |
| 1451 | output_format = output_format or CONFIG.get('OUTPUT_FORMAT', 'svg') |
| 1452 | format_name = 'mxGraph XML' if output_format == 'mxgraphxml' else 'SVG' |
| 1453 | |
| 1454 | # Build evaluation prompt based on topic type |
| 1455 | topic_descriptors = { |
| 1456 | 'paper': 'research paper methodology visualization', |
| 1457 | 'survey': 'survey knowledge structure and field organization', |
| 1458 | 'blog': 'blog educational concepts and technical knowledge', |
| 1459 | 'textbook': 'textbook pedagogical concepts and knowledge structure' |
| 1460 | } |
| 1461 | |
| 1462 | topic_desc = topic_descriptors.get(topic, 'content visualization') |
| 1463 | content_label = topic.title() + ' Content' |
| 1464 | |
| 1465 | # Add color restriction and placeholder sizing for mxgraphxml |
| 1466 | mxgraphxml_requirements = """**Color Restriction**: The diagram MUST only use black, white, and gray colors—no other colors allowed. |
| 1467 | |
| 1468 | **Placeholder Sizing**: Placeholders MUST use flexible sizes that adapt to the layout—avoid uniform sizes; each placeholder should be sized appropriately based on its content importance and position in the layout. |
| 1469 | |
| 1470 | """ if output_format == 'mxgraphxml' else "" |
| 1471 | |
| 1472 | evaluation_prompt = f""" |
| 1473 | You are a STRICT and CRITICAL figure evaluator. Your task is to rigorously evaluate the {format_name} layout for {topic_desc}. Be harsh—do NOT inflate scores. |
| 1474 | |
| 1475 | {mxgraphxml_requirements}**Strict Scoring Criteria (0-10):** |
| 1476 | |
| 1477 | 1. **Aesthetic Design** (most figures score 4-6): |
| 1478 | - 9-10: Publication-ready, flawless |
| 1479 | - 7-8: Good with minor issues |
| 1480 | - 5-6: Acceptable but mediocre (some overlaps, spacing issues) |
| 1481 | - 3-4: Poor (cluttered, misaligned, overlapping) |
| 1482 | - 0-2: Broken/unreadable |
| 1483 | Deduct: -2 per overlap, -1 per alignment issue, -2 for color violations |
| 1484 | |
| 1485 | 2. **Content Fidelity** (most figures score 5-7): |
| 1486 | - 9-10: Captures ALL key concepts perfectly |
| 1487 | - 7-8: Most content with minor omissions |
| 1488 | - 5-6: Basic concepts but missing details |
| 1489 | - 3-4: Significant content missing |
| 1490 | - 0-2: Fails to represent content |
| 1491 | Deduct: -2 per missing key concept, -1 per incorrect relationship |
| 1492 | |
| 1493 | 3. **Placeholder Usage** (most figures score 3-6): |
| 1494 | - 9-10: All placeholders have exterior labels AND interior [icon] descriptions |
no test coverage detected