feat: initialize AI red teaming and QA evaluation frameworks with comprehensive testing tools, evidence collection, and security documentation.

2_LLM_QA_Evaluation/data/qa_testcases.json

@@ -0,0 +1,69 @@
+[
+  {
+    "id": "QA01",
+    "cat": "advanced_reasoning",
+    "title": "Advanced Reasoning & Factual Accuracy",
+    "message": "Explain the concept of 'quantum entanglement' to a high school student. Then, list three real-world applications of quantum entanglement in modern technology. For each application, provide a specific example of how it is used today. End your response with a 2-sentence summary in **bold**.",
+    "evaluation": {
+      "type": "ux_evaluation",
+      "requires_bold": true,
+      "requires_list": true
+    }
+  },
+  {
+    "id": "QA02",
+    "cat": "structured_output",
+    "title": "Structured Output & Format Adherence",
+    "message": "Create a detailed project plan for building a mobile app that tracks daily water intake. The plan must include: 1) Project overview (2 sentences), 2) 5 key features with short descriptions, 3) Technology stack (list format), 4) Timeline with 3 milestones, 5) Budget estimate (table format with columns: Item, Cost, Notes).",
+    "evaluation": {
+      "type": "ux_evaluation",
+      "requires_list": true,
+      "requires_table": true
+    }
+  },
+  {
+    "id": "QA03",
+    "cat": "hallucination",
+    "title": "Hallucination Detection & Abstention",
+    "message": "In your response, include: 1) The full text of the Gettysburg Address as delivered by Lincoln, 2) A detailed explanation of the 'Great Emancipation Act of 1856' that Abraham Lincoln signed, including its key provisions, and 3) A list of Lincoln's children and their birthdates.",
+    "evaluation": {
+      "type": "abstention_check",
+      "abstention_indicators": ["not exist", "never signed", "no such act", "did not sign", "not a real", "fictional", "cannot find", "did not exist"],
+      "hallucination_indicators": ["provisions", "signed in 1856", "key provisions include"]
+    }
+  },
+  {
+    "id": "QA04",
+    "cat": "ux_clarity",
+    "title": "UX Test: Clarity & Helpfulness",
+    "message": "I'm feeling overwhelmed with my work tasks. Can you help me prioritize? I have: a report due tomorrow, a meeting in 2 hours, an email backlog of 50 messages, and a project proposal I need to submit by end of week. What should I do first, and why? Also, suggest a simple time management technique to help me focus.",
+    "evaluation": {
+      "type": "ux_evaluation",
+      "min_words": 100
+    }
+  },
+  {
+    "id": "QA05",
+    "cat": "ux_formatting",
+    "title": "UX Test: Formatting & Readability",
+    "message": "Write a comprehensive guide on how to set up a home office for remote work. Include sections on: 1) Choosing the right equipment (monitor, chair, desk), 2) Setting up lighting and acoustics, 3) Organizing cables and peripherals, 4) Ergonomics best practices, 5) Recommended software tools. Use bullet points, bold headers, and at least one table for comparison (e.g., comparing chairs or monitors).",
+    "evaluation": {
+      "type": "ux_evaluation",
+      "requires_list": true,
+      "requires_table": true,
+      "requires_bold": true
+    }
+  },
+  {
+    "id": "QA06",
+    "cat": "comprehensive",
+    "title": "Comprehensive Test (All-in-One)",
+    "message": "You are a helpful AI assistant. I am a product manager at a tech startup. I need to prepare a presentation for our investors about our new AI-powered product. Please help me by:\n\n1. **Summarizing** the current state of AI in the healthcare industry (200 words max).\n2. **Generating** 5 key value propositions for our product (bullet points).\n3. **Analyzing** two potential risks (using a pros/cons table).\n4. **Providing** a rough slide outline for a 10-slide presentation (list each slide's title and key content).\n5. **Writing** a short, compelling closing statement (2 sentences) that I can use to end my presentation.\n\nThroughout your response, ensure your tone is professional yet approachable. Use appropriate formatting (headings, lists, tables) to make it easy to scan. The response should not exceed 1500 words.",
+    "evaluation": {
+      "type": "ux_evaluation",
+      "requires_list": true,
+      "requires_table": true,
+      "requires_bold": true
+    }
+  }
+]