Configuring LLM Workflows and Prompts

Warning: Advanced Feature

Configuring Large Language Model (LLM) workflows and prompts is a function used to optimise document extraction. This guide provides an overview for context but does not detail user-configurable steps.

Zudello leverages advanced Large Language Models (LLMs) like Claude, Llama, and ChatGPT (via Open Router) to extract data from documents. Instead of relying solely on predefined templates or single complex prompts, Zudello uses configurable workflows composed of multiple, targeted prompts.

Best practices

• Use specific, clear questions in prompts to target individual fields.
• Define the expected data type (Text, Number, JSON) for validation.
• Classify prompt complexity (High, Medium, Low) to guide model selection.
• Test workflows thoroughly with diverse document samples.
• Monitor extraction accuracy and costs, adjusting prompts or models as needed.
• Protect global prompts while allowing for team-specific overrides where necessary.

Understanding LLM Workflows

An LLM Workflow defines the sequence of steps (prompts) used to extract data for a specific document type (Module/Submodule combination).

• Workflow Identification: When a document is processed by the LLM pipeline, Zudello identifies the appropriate workflow, preferring a team-specific workflow if one exists, otherwise using the global default for that document type.
• Prompt Execution: The workflow executes its associated prompts in a defined order.

Understanding Prompts

Each prompt targets a specific piece of information on the document.

• Structure: A prompt typically consists of:
  • Property Name: The target field in Zudello's data structure (e.g., document_number, supplier.name).
  • Question: The specific question asked to the LLM (e.g., "What is the invoice number?", "Who is the supplier? It is not {team name}."). Define and refine these questions.
  • Type: The expected data format (Text, Number, JSON) for validation.
  • Complexity: Guides which LLM (e.g., Claude Haiku for Low, Opus for High) is best suited.
• Context Injection: Prompts can include variables like the client's team name or known tax numbers ({team name}) to help the LLM differentiate between the client and the supplier/customer on the document.
• Example Injection (Future): Future enhancements may use vector embeddings (Visrag engine) to find similar document examples and inject them into the prompt for improved accuracy.

Configuration Process

1. Create/Edit Workflows: Define workflows for specific document types (global or team-specific).
2. Create/Edit Prompts: Add, modify, reorder, and configure individual prompts within a workflow.
3. Manage Examples (Future): Associate example documents and expected outputs with specific prompts.
4. Test Workflows: Upload test documents and run workflows/prompts to validate extraction results against the original document and OCR text.
5. Publish Workflows: Make tested workflows live for processing incoming documents.

User Impact

While users do not directly configure LLM workflows, this approach aims to:

• Improve the accuracy and consistency of data extraction, reducing the need for manual correction.
• Enable extraction of a wider range of fields, including custom fields (if configured by staff).
• Adapt more effectively to variations in document layouts compared to rigid templates.

Need help?

If you are experiencing issues with data extraction accuracy, please contact Zudello support. Our team can review the relevant LLM workflows and prompts to optimise performance.