Knowledge-Augmented Generation (KAG): A novel approach to AI search

For product managers, the challenge is that AI systems often fall short when faced with the nuanced decision-making required in highly specialized fields such as healthcare, law, and finance. How can you integrate AI, particularly generative text AI like large language models, while ensuring the delivery of accurate information?

A standard solution is to deploy a Retrieval-Augmented Generation (RAG) system. However, RAG struggles with resolving complex issues that demand more than retrieving a single text section.

The solution may lie in Knowledge-Augmented Generation (KAG), an approach that combines Large Language Models (LLMs) with Knowledge Graphs (KGs).

Why Traditional AI Falls Short

1. Logic and Reasoning Gaps

Large language models can seem very smart, but they often make mistakes when they need to do deeper thinking. For example, if you ask about a law, a typical AI might pull the correct sentence from a legal document but then fail to link it to the laws or rules it’s based on. This happens because most AI models don’t understand how ideas fit together; they look for patterns. So, while they can find the correct paragraphs, they might miss key details that connect them logically.

Why RAG Doesn’t Solve This:

Retrieval-augmented generation (RAG) can help AI “remember” facts by pulling relevant text from big databases. But it still doesn’t give AI true “reasoning” skills; if the AI needs to do multi-step thinking—like seeing how one law builds upon another—it often falls short.

2. Domain-Specific Accuracy

Using precise terminology is crucial in the healthcare, finance, and software sectors. Even minor alterations in phrasing can entirely change the meaning of a statement. Consider diabetes: Types 1 and 2 utilize many of the same terms in medical documentation but are different conditions that require distinct treatments. This type of overlap poses challenges for AI systems, which often struggle to differentiate between similar phrases. When AI makes such errors, it can provide incorrect advice that affects the accuracy of decisions.

Why RAG Doesn’t Solve This:

RAG lets the AI look up the proper documents, but that alone won’t teach the minor, crucial differences experts care about. It can copy text about Type 1 and Type 2 diabetes, but it does not always know which rules apply to which type. Whenever a tiny text change matters, plain RAG can drop the ball because the model itself doesn’t understand the content deeply—it just retrieves and rewords it.

3. Complex, Multi-Step Queries

Sometimes, you need the AI to handle a big question involving many smaller parts or rules—like connecting EU data privacy laws across different years and countries. A regular RAG system might find a few pieces of text about these laws, but it won’t always tie them together in a way that forms a complete answer. So you end up with bits and pieces that don’t fully solve your problem.

Example:

A standard RAG system might gather snippets about EU data privacy rules from 2018 and 2023. But it might not tell you clearly how those rules changed over time or how each one applies to a specific product release in 2024. You’re left with half-answers and no clear guidance.

Why RAG Doesn’t Solve This:

While RAG is good at looking up facts, it struggles to break down multi-step questions or combine information from many sources. It cannot reason and piece everything together into one clear picture. That makes it hard to get correct and detailed answers for complex tasks.

What Is KAG and How Does It Solve These Issues?

Knowledge-Augmented Generation (KAG) is a smart way computers use information to answer questions and solve problems. It combines two important tools: large language models (which help computers understand and create text) and knowledge graphs (which organize information in a clear way). By working together, these tools help computers provide accurate and reliable answers, especially in areas like healthcare, law, and finance. Paper

Key Components of KAG

1. Organizing Information Clearly
   • Fixed Information: This is information that is neatly arranged into specific categories, like a library where books are sorted by subjects such as science or history. This helps the computer know exactly where to find certain types of information.
   • Flexible Information: This is information that can be added or changed easily, like notes in a big notebook. It allows the computer to include new facts and ideas without needing to follow strict rules.
2. Better Searching with Mutual Indexing

• Mutual Indexing: Imagine a smart system where every piece of information can quickly link to other related pieces. This makes it easier for the computer to find and connect different bits of information when needed.
• Easy Connections: This means that if the computer finds one fact, it can easily see how it relates to other facts. It’s like having a map that shows how different pieces of information are connected. Smart Thinking with Logical Reasoning
• Breaking Down Questions: When faced with a tough question, the computer can split it into smaller, easier parts. This helps it understand and solve the problem step by step.
• Smart Calculations: The computer can do math and think through several steps to reach the right answer. This is helpful for questions that need detailed thinking.

1. Clear and Consistent Knowledge Alignment

• Bringing Together Different Data: The computer can combine information from different sources to create one clear and organized set of facts. This ensures all the information fits together well.
• Staying Consistent: The computer makes sure that all the answers it gives make sense and agree with each other. This means the information is reliable and trustworthy.

---

Examples of How KAG Works

Healthcare

Example: A doctor uses a computer to find the best treatment for a patient with diabetes. The computer uses KAG to look at medical books (fixed information) and recent research articles (flexible information). It connects related information quickly (mutual indexing) and breaks down the doctor’s question into smaller parts to find the best answer (logical reasoning). This helps the doctor choose the right treatment confidently.

Law

Example: A lawyer needs to find specific laws and past case studies for a new case. KAG helps by organizing all the laws (fixed information) and recent legal news (flexible information). The computer links related laws and cases together (mutual indexing) and breaks down the lawyer’s request to find the most relevant information (logical reasoning). This makes the lawyer’s work faster and more accurate.

Finance

Example: A financial advisor wants to give the best investment advice to a client. KAG helps by organizing financial rules and market data (fixed information) along with the latest financial news (flexible information). It connects related financial information quickly (mutual indexing) and analyzes the client’s needs step by step (logical reasoning). This ensures the advisor provides reliable and tailored advice.

Understanding Knowledge Graphs (KGs)

A Knowledge Graph is a structured representation of information[1] where relationships connect entities (people, places, concepts). Unlike a traditional database that stores isolated facts in rows and columns, a KG forms a network of data points, enabling deeper reasoning and inference.

Why KGs Are Critical for AI

• Contextual Understanding: KGs help AI connect the dots between related entities (e.g., a symptom and a disease).
• Traceable Logic: You can see why an AI reached its conclusion by following the relationships in the graph.
• Scalability: Adding new entities and relationships is typically easier than re-architecting a traditional database schema.

Example: KG in Healthcare

Let’s say a user asks:

“What treatments are available for Type 2 Diabetes, and which are covered by my insurance?”

Entities:

• Disease: Type 2 Diabetes
• Treatments: Metformin, Insulin, Lifestyle Changes
• Insurance Policies: Policy A, Policy B, etc.

Relationships:

• “Type 2 Diabetes” is treated by “Metformin”
• “Metformin” is covered by “Policy A” but not “Policy B”

When these entities and relationships are stored in a KG, your AI can traverse them quickly. It might respond:

“Metformin and insulin are common treatments for Type 2 Diabetes. Metformin is covered under your insurance plan, but insulin may require additional approval.”

This level of structured reasoning is far more powerful than a generic web search that returns a list of websites about diabetes treatments.

How to Build a Knowledge Graph

Building a knowledge graph might sound daunting, but it’s manageable if you break it down into steps:

1. Define the Scope
   • Identify the domain (healthcare, finance, legal, etc.).
   • Outline the key entities (e.g., diseases, treatments, regulations).
2. Gather Data
   • Structured data: Databases, spreadsheets.
   • Unstructured data: Websites, PDF documents, APIs.
3. Build the Graph Structure
   • Represent entities as nodes (e.g., “Type 2 Diabetes,” “Metformin”).
   • Represent relationships as edges (e.g., “treats,” “covered by”).
4. Use Tools to Create and Store the KG
   • Neo4j, Amazon Neptune, and Apache Jena for graph databases.
   • RDF4J (Java) or NetworkX (Python) for graph data modeling.
5. Populate the Graph
   • Information Extraction: Use spaCy or Stanford CoreNLP to identify entities and relationships in text.
   • Manual Curation: Domain experts refine and validate the data.
6. Query and Test
   • SPARQL or Cypher query languages to retrieve information.
   • Test with real-world questions to ensure the KG meets users’ needs.

1. Smarter Customer Support

Use Case: Healthcare and E-Governance

• Scenario: A user asks, “What vaccinations are required for school enrollment, and where can I get them?”
• KAG Advantage: The system retrieves public health guidelines (KG) + local pharmacy data (KG or text) to provide a contextual, accurate answer.

Value:

• Reduced agent load
• Faster response times
• Higher customer satisfaction

2. Domain-Specific Insights

Use Case: Finance, Legal, or Insurance

• Scenario: “How did market trends impact commodity prices last quarter, and how does this affect my portfolio?”
• KAG Advantage: Logical-form reasoning breaks down the query into sub-questions (market trends, commodity data, portfolio holdings). The AI can even do basic numerical computations to highlight profit/loss impacts.

Value:

• Data-backed insights
• Minimized human error
• Real-time, actionable intelligence

3. Enhanced Search and Recommendations

Use Case: Legal Research Platform

• Scenario: “Find cases related to data privacy that reference EU’s GDPR and any subsequent amendments.”
• KAG Advantage: Semantic alignment helps the AI retrieve relevant cases and summarize the historical amendments to the GDPR and how they apply to each Case.

Value:

• Precision results, not just keyword matches
• In-depth recommendations
• Better user experience for professionals

Step-by-Step Guide to Implementing KAG

1. Start Small with Modular Enhancements
   • Add a Knowledge Graph to an existing AI feature. Focus on a specific, high-value use case.
   • Mutual Indexing ensures that both your unstructured text resources and KG data are linked.
2. Refine Through Semantic Knowledge Alignment
   • Continuously update domain-specific terms and relationships in the KG.
   • Incorporate synonyms, translations, and causal relationships to improve accuracy.
3. Use Logical Form Planning
   • Break down user queries into logical steps.
   • Integrate symbolic computations for tasks like cost estimation or risk assessment.

Real-World Results: KAG’s Proven Impact

In the original KAG research paper, KAG outperformed standard RAG methods by 19.6% in F1 scores on HotpotQA. On multi-hop tasks like 2WikiMultiHopQA, it achieved a 33.5% improvement.

Understanding the Tests and What They Measure

When researchers create new ways for computers to understand and answer questions, they use special tests to see how well these methods work. Two important tests are HotpotQA and 2WikiMultiHopQA. Let’s break down what these tests are and what the numbers mean.

---

1. What is HotpotQA?

HotpotQA is a test designed to check how well a computer can answer questions that need information from multiple places. Imagine you ask a question that requires the computer to look at different books or articles to find the answer.

• Example Question: “What is the capital of France, and who is the current president?”

  To answer this, the computer needs to find information about the capital city and the president separately and then combine them.

---

2. What is 2WikiMultiHopQA?

2WikiMultiHopQA is another test that checks the computer’s ability to answer questions by connecting information from two different Wikipedia articles. This means the computer has to “hop” from one piece of information to another to find the answer.

• Example Question: “How does photosynthesis relate to the food chain?”

  The computer needs to understand what photosynthesis is and how it fits into the larger concept of the food chain by looking at two separate topics.

---

3. What is an F1 Score?

The F1 score is a number that helps us understand how good the computer is at answering questions correctly. It considers two things:

• Precision: How many of the answers the computer gave are correct.
• Recall: How many of the correct answers the computer found.

The F1 score combines these two to give a single number between 0% and 100%, where a higher number means better performance.

---

4. What Do the Numbers Mean?

In the research, two methods were compared: KAG (Knowledge-Augmented Generation) and RAG (Retrieval-Augmented Generation). Here’s how they performed:

• HotpotQA:
  • KAG: 85%
  • RAG: 65%

  KAG scored 85 out of 100, while RAG scored 65 out of 100.

• 2WikiMultiHopQA:
  • KAG: 83%
  • RAG: 50%

  KAG scored 83 out of 100, while RAG scored 50 out of 100.

What This Means:

• Higher Scores: KAG scored much higher than RAG in both tests.
• Better Performance: KAG is better at finding and connecting information from different sources to answer complex questions accurately.
• Improvement: KAG improved the scores by 19.6% on HotpotQA and 33.5% on 2WikiMultiHopQA compared to RAG. This means KAG is much more effective in understanding and answering tricky questions.

---

KAG vs. RAG (F1 Scores)
======================
HotpotQA: KAG ~ 85% | RAG ~ 65%
2WikiMulti: KAG ~ 83% | RAG ~ 50%

This translates directly to more reliable and precise outputs for your users.

By automating knowledge alignment and logical reasoning, KAG reduces the overhead of continuously retraining or manually curating AI systems. Product teams can thus iterate and improve faster.

Future Opportunities with KAG

• Open-Source Accessibility: KAG’s underlying technology (like OpenSPG) is open-source, lowering the barrier to adoption.
• Scalability: As more industries and governments adopt AI-driven tools, KAG ensures your product remains flexible and future-proof.
• Multilingual & Cross-Domain: KAG can be extended to handle multiple languages and cross-knowledge domains, opening doors to global applications.

Technologies & Resources

• Graph Databases
• Neo4j, Amazon Neptune, Apache Jena
• Information Extraction
• spaCy, Stanford CoreNLP
• Query Languages
• SPARQL, Cypher
• Open Knowledge Graphs
• Wikidata, ConceptNet
• Further Reading
  • Original KAG Paper
  • Google’s Knowledge Graph
  • Neo4j’s Knowledge Graph 101
  • Full Tutorial on building a knowledge Graph

Why Product Managers Should Care About KAG

KAG represents a quantum leap in AI’s evolution in a world where domain expertise meets intelligent automation. By combining the structured reasoning of Knowledge Graphs with the language capabilities of LLMs, you can:

• Deliver accurate and context-rich answers.
• Improve user satisfaction through logical and trustworthy responses.
• Scale your AI to handle ever more complex queries without ballooning resource costs.