2903 Advanced Workflows
PHASE 10: EXPERT TOPICS - WEEKS 19-20Advanced Workflows
Module Overview: Master the art of designing and implementing sophisticated workflows in Tadabase. Learn how to build multi-stage approval systems, state machines for complex processes, automation chains that orchestrate multiple systems, and optimization strategies for high-volume workflows.
Introduction to Advanced Workflow Design
While basic automation handles simple tasks, advanced workflows manage complex business processes that involve multiple stages, decision points, parallel paths, and sophisticated state management. This is where Tadabase becomes a true business process automation platform.
Expert Principle: Great workflow design is invisible to users. They should experience a seamless, intuitive process without understanding the complex automation working behind the scenes.
State Machine Architecture
Understanding State Machines
A state machine is a system that can be in exactly one of a finite number of states at any given time. State machines are perfect for modeling business processes with clear stages and transitions.
State Machine Components:
1. STATES: Defined conditions (Draft, Pending, Approved, Rejected)
2. TRANSITIONS: Rules for moving between states
3. EVENTS: Triggers that initiate transitions
4. GUARDS: Conditions that must be met for transitions
5. ACTIONS: Activities performed during transitions
Implementing State Machines in Tadabase
Purchase Order Approval State Machine
┌─────────┐
│ DRAFT │ (Created)
└────┬────┘
│ Submit
▼
┌─────────┐
┌──────│ PENDING │
│ └────┬────┘
│ │ Approve/Reject
│ ▼
│ ┌─────────────┐
│ ┌───┤ REQUIRES ├─── Approve ───┐
│ │ │ REVIEW │ │
│ │ └─────────────┘ │
│ │ Reject │
│ ▼ ▼
┌─────────┐ ┌──────────┐
│REJECTED │ │ APPROVED │
└────┬────┘ └────┬─────┘
│ │
│ Reopen │ Process
│ ▼
│ ┌──────────┐
└───────────────────────────────┤PROCESSING│
└────┬─────┘
│ Complete
▼
┌──────────┐
│COMPLETED │
└──────────┘
Implementation Table Structure:
TABLE: Purchase_Orders
- status (dropdown field with states):
* Draft
* Pending Approval
* Requires Finance Review
* Requires Executive Review
* Approved
* Rejected
* Processing
* Completed
* Cancelled
- status_history (long text, JSON or log format)
- current_approver (connection to Users)
- approval_level (number: 1=Manager, 2=Finance, 3=Executive)
- rejection_reason (long text)
- status_changed_at (date/time)
- status_changed_by (connection to Users)
State Transition Logic
Record Rule: Submit for Approval
TRIGGER: When status changes to "Pending Approval"
CONDITIONS:
- Previous status = "Draft"
- Total amount is filled
- Vendor is selected
- All line items are complete
ACTIONS:
1. Set approval_level based on amount:
- $10,000: Level 3 (Executive)
2. Assign current_approver based on level:
- Level 1: Submitter's manager
- Level 2: Finance manager
- Level 3: CFO
3. Send notification to approver:
- Email with PO details
- Include approve/reject action links
- Set notification priority based on amount
4. Log state transition:
- Append to status_history
- Record timestamp and user
- Note: "Submitted for approval"
5. Set SLA timer:
- Create deadline (24/48/72 hours based on level)
- Schedule reminder task
Record Rule: Approve Purchase Order
TRIGGER: When status changes to "Approved"
CONDITIONS:
- Previous status = "Pending Approval" OR "Requires Review"
- Current user = current_approver OR user has override permission
- No holds or flags on vendor
ACTIONS:
1. Check if additional approval needed:
IF approval_level
Record Rule: Reject Purchase Order
TRIGGER: When status changes to "Rejected"
CONDITIONS:
- Previous status = "Pending Approval" OR "Requires Review"
- Current user = current_approver OR user has override permission
- Rejection reason is filled
ACTIONS:
1. Record rejection details:
- Store rejection_reason
- Log who rejected and when
- Append to status_history
2. Notify submitter:
- Email with rejection reason
- Provide option to revise and resubmit
- Include approver comments
3. Clear approval workflow:
- Clear current_approver
- Reset approval_level to 0
- Cancel any pending reminders
4. Archive rejected items (optional):
- Move line items to rejected_items table
- Keep for historical reference
5. Update analytics:
- Increment rejection counter
- Track rejection reasons for reporting
Multi-Stage Approval Systems
Parallel Approval Workflows
Parallel vs. Sequential: Parallel approvals happen simultaneously (faster but requires all approvers), while sequential approvals go one-by-one (slower but allows escalation based on previous decisions).
Parallel Approval Pattern: Document Release
┌──────────┐
│ DRAFT │
└────┬─────┘
│ Submit
▼
┌────────────────────┐
│ PENDING APPROVAL │
└────────┬───────────┘
│
┌──────────────┼──────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ LEGAL │ │TECHNICAL│ │MARKETING│
│ REVIEW │ │ REVIEW │ │ REVIEW │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└──────────────┼──────────────┘
│ All Approve
▼
┌─────────┐
│APPROVED │
└─────────┘
Implementation:
TABLE: Document_Approvals (junction table)
- document_id (connection to Documents)
- approver_id (connection to Users)
- approval_type (dropdown: Legal, Technical, Marketing)
- status (dropdown: Pending, Approved, Rejected)
- comments (long text)
- approved_at (date/time)
RECORD RULE: On Document Submit
1. Create approval records:
- Insert record for Legal reviewer
- Insert record for Technical reviewer
- Insert record for Marketing reviewer
- All start with status = "Pending"
2. Send parallel notifications:
- Email to all three reviewers simultaneously
- Include action links for approve/reject
RECORD RULE: On Approval Record Status Change
1. Check completion:
- Count total approval records for document
- Count approved records
- Count rejected records
2. Update document status:
IF all_approved:
- Set document status to "Approved"
- Send notification to submitter
- Publish document
ELSE IF any_rejected:
- Set document status to "Revisions Needed"
- Compile all rejection comments
- Send to submitter with feedback
ELSE:
- Keep status as "Pending Approval"
- Show progress: "2 of 3 approvals received"
Sequential Approval with Escalation
Expense Report Approval Workflow
Submit → Manager → Director → VP → CFO → Approved
│ │ │ │ │
│ ▼ ▼ ▼ ▼
└────── Rejected at any stage ───────┘
│
▼
Revise & Resubmit
Escalation Logic:
APPROVAL LEVELS:
Level 1: Direct Manager (amounts up to $500)
Level 2: Director (amounts $501-$2,500)
Level 3: VP (amounts $2,501-$10,000)
Level 4: CFO (amounts over $10,000)
RECORD RULE: Calculate Required Approval Level
approval_level_required = CASE
WHEN {total_amount}
Conditional Routing
Dynamic Workflow Routing Based on Criteria
Support Ticket Routing:
RECORD RULE: On Ticket Creation
// Determine routing based on multiple factors
routing_destination = CASE
// VIP customers go to senior support
WHEN {customer.is_vip} = true THEN
"Senior Support Queue"
// Technical issues go to tech team
WHEN {issue_type} IN ("Bug", "Technical Error", "API Issue") THEN
"Technical Support Queue"
// Billing issues go to finance
WHEN {issue_type} IN ("Billing", "Invoice", "Payment") THEN
"Billing Support Queue"
// High priority gets escalated
WHEN {priority} = "Critical" OR {priority} = "High" THEN
"Priority Support Queue"
// Sales-related to sales support
WHEN {issue_category} = "Sales" OR {issue_type} = "Demo Request" THEN
"Sales Support Queue"
// Everything else to general support
ELSE
"General Support Queue"
END
// Assign to next available agent in queue
assigned_agent = GET_NEXT_AVAILABLE_AGENT(routing_destination)
// Set SLA based on priority and customer tier
sla_response_time = CASE
WHEN {customer.is_vip} AND {priority} = "Critical" THEN 30 // 30 min
WHEN {customer.is_vip} THEN 60 // 1 hour
WHEN {priority} = "Critical" THEN 60 // 1 hour
WHEN {priority} = "High" THEN 240 // 4 hours
WHEN {priority} = "Medium" THEN 480 // 8 hours
ELSE 960 // 16 hours
END
sla_due_date = ADD_BUSINESS_MINUTES(NOW(), sla_response_time)
// Send notifications
SEND_NOTIFICATION(assigned_agent, "New ticket assigned")
SEND_AUTO_REPLY(customer, "Ticket received, assigned to " + assigned_agent)
// Schedule SLA monitoring
SCHEDULE_TASK("Check SLA Compliance", sla_due_date)
Complex Automation Chains
Multi-System Orchestration
Orchestration Pattern: Build workflows that coordinate actions across multiple systems and databases, maintaining data consistency and handling failures gracefully.
E-commerce Order Processing Chain
1. ORDER CREATED
↓
2. VALIDATE INVENTORY (Check stock levels)
↓ Success
3. RESERVE INVENTORY (Update stock table)
↓
4. PROCESS PAYMENT (External payment gateway)
↓ Success
5. CREATE SHIPPING LABEL (ShipStation API)
↓
6. UPDATE ACCOUNTING (QuickBooks API)
↓
7. SEND CONFIRMATIONS (Email + SMS)
↓
8. CREATE CUSTOMER RECORD (Salesforce API)
↓
9. SCHEDULE FOLLOW-UP (Task created)
↓
10. ORDER COMPLETE
[At any failure point: Rollback + Error handling]
Implementation with Error Handling:
PIPE: Process_New_Order (triggered on order creation)
STEP 1: Validate Inventory
FOR EACH line_item IN {order.line_items}:
product = GET_PRODUCT(line_item.product_id)
IF product.stock_quantity
Compensation and Rollback Strategies
Handling Failures in Multi-Step Workflows
Rollback Pattern:
PIPE: Rollback_Inventory_Reservation
INPUT: order_id
STEP 1: Find Reservations
reservations = FIND_RECORDS("Inventory_Reservations", {
order_id: {input.order_id},
status: "Reserved"
})
STEP 2: Restore Inventory
FOR EACH reservation IN reservations:
product = GET_PRODUCT(reservation.product_id)
// Return reserved quantity to available stock
product.stock_quantity += reservation.quantity
product.reserved_quantity -= reservation.quantity
UPDATE_PRODUCT(product)
// Mark reservation as rolled back
reservation.status = "Rolled Back"
reservation.rolled_back_at = NOW()
UPDATE_RECORD(reservation)
LOG_INFO("Rolled back " + reservation.quantity + " units of product " + product.name)
END FOR
STEP 3: Update Order
order = GET_ORDER({input.order_id})
order.inventory_status = "Reservation Rolled Back"
UPDATE_ORDER(order)
RETURN {
success: true,
reservations_rolled_back: COUNT(reservations),
message: "Inventory reservation rolled back successfully"
}
Workflow Optimization
Performance Optimization Strategies
Async Processing for Heavy Workflows
Problem: Long-running workflows delay user response and can timeout.
Solution: Use asynchronous processing with status tracking.
Solution: Use asynchronous processing with status tracking.
Pattern: Import Large Dataset
// User uploads CSV file
IMMEDIATE RESPONSE (
Batch Processing Workflows
Efficient Bulk Operations
Pattern: Nightly Customer Segmentation
SCHEDULED TASK: Update_Customer_Segments (runs daily at 2 AM)
// Process in batches to avoid memory issues
batch_size = 1000
offset = 0
total_processed = 0
WHILE true:
// Get batch of customers
customers = FIND_RECORDS("Customers", {
LIMIT: batch_size,
OFFSET: offset,
FILTERS: {
is_active: true
}
})
IF COUNT(customers) = 0:
BREAK // No more customers to process
END IF
// Process batch
FOR EACH customer IN customers:
// Calculate segment based on business rules
total_purchases = SUM(customer.orders.total)
purchase_frequency = COUNT(customer.orders)
avg_order_value = total_purchases / purchase_frequency
days_since_last_order = DAYS_BETWEEN(customer.last_order_date, NOW())
new_segment = CASE
WHEN total_purchases > 10000 AND purchase_frequency > 10 THEN
"VIP"
WHEN total_purchases > 5000 THEN
"High Value"
WHEN purchase_frequency > 5 AND days_since_last_order 180 THEN
"At Risk"
WHEN days_since_last_order > 365 THEN
"Inactive"
ELSE
"Standard"
END
// Only update if segment changed
IF customer.segment != new_segment:
old_segment = customer.segment
customer.segment = new_segment
customer.segment_changed_date = NOW()
UPDATE_RECORD(customer)
// Log segment change for analytics
CREATE_RECORD("Segment_Changes", {
customer_id: customer.id,
old_segment: old_segment,
new_segment: new_segment,
changed_at: NOW()
})
// Trigger segment-specific workflows
IF new_segment = "At Risk":
EXECUTE_PIPE("Create_Winback_Campaign_Task", {customer_id: customer.id})
END IF
END IF
total_processed += 1
END FOR
// Move to next batch
offset += batch_size
// Log progress
LOG_INFO("Processed " + total_processed + " customers...")
END WHILE
LOG_INFO("Customer segmentation complete. Total processed: " + total_processed)
Workflow Monitoring and Debugging
Building Observable Workflows
Workflow Logging Pattern:
TABLE: Workflow_Logs
- workflow_name (text)
- execution_id (text, unique identifier)
- status (dropdown: Started, In Progress, Completed, Failed)
- started_at (date/time)
- completed_at (date/time)
- duration_seconds (number)
- record_id (text, the record being processed)
- step_name (text, current step)
- error_message (long text)
- metadata (long text, JSON)
USAGE IN PIPE:
execution_id = GENERATE_UUID()
// Log start
CREATE_RECORD("Workflow_Logs", {
workflow_name: "Process Order",
execution_id: execution_id,
status: "Started",
started_at: NOW(),
record_id: {order.id},
metadata: JSON.stringify({order_total: {order.total}})
})
// Log each step
UPDATE_LOG(execution_id, {
step_name: "Validate Inventory",
status: "In Progress"
})
// ... perform step ...
UPDATE_LOG(execution_id, {
step_name: "Validate Inventory",
status: "Completed"
})
// Log completion or failure
IF success:
UPDATE_LOG(execution_id, {
status: "Completed",
completed_at: NOW(),
duration_seconds: SECONDS_BETWEEN(started_at, NOW())
})
ELSE:
UPDATE_LOG(execution_id, {
status: "Failed",
completed_at: NOW(),
error_message: error.message,
duration_seconds: SECONDS_BETWEEN(started_at, NOW())
})
END IF
// Create dashboard showing:
// - Success rate by workflow
// - Average execution time
// - Failed workflows needing attention
// - Workflow execution trends
Advanced Workflow Patterns
Event-Driven Workflows
Webhook-Triggered Automation Chain
Scenario: Incoming Stripe Payment Webhook
INCOMING WEBHOOK: /webhooks/stripe/payment-received
PAYLOAD:
{
"event_type": "payment_intent.succeeded",
"customer_id": "cus_123456",
"amount": 5000,
"currency": "usd",
"invoice_id": "in_123456"
}
WEBHOOK RECEIVER PIPE:
1. Validate webhook signature (security)
2. Find related customer order
3. Trigger appropriate workflow based on event
CASE {payload.event_type}:
"payment_intent.succeeded":
EXECUTE_PIPE("Process_Successful_Payment", {
customer_id: {payload.customer_id},
amount: {payload.amount}
})
"payment_intent.payment_failed":
EXECUTE_PIPE("Handle_Failed_Payment", {
customer_id: {payload.customer_id},
reason: {payload.failure_message}
})
"customer.subscription.created":
EXECUTE_PIPE("Setup_New_Subscription", {
customer_id: {payload.customer_id},
plan: {payload.plan_id}
})
"customer.subscription.deleted":
EXECUTE_PIPE("Cancel_Subscription", {
customer_id: {payload.customer_id},
reason: "User cancelled"
})
END CASE
Time-Based Workflow Triggers
Customer Lifecycle Automation
Onboarding Email Sequence:
ON CUSTOMER CREATION:
// Schedule onboarding sequence
CREATE_RECORD("Scheduled_Emails", {
customer_id: {customer.id},
email_type: "Welcome",
scheduled_for: NOW()
})
CREATE_RECORD("Scheduled_Emails", {
customer_id: {customer.id},
email_type: "Getting Started Tips",
scheduled_for: ADD_DAYS(NOW(), 2)
})
CREATE_RECORD("Scheduled_Emails", {
customer_id: {customer.id},
email_type: "Feature Highlight",
scheduled_for: ADD_DAYS(NOW(), 5)
})
CREATE_RECORD("Scheduled_Emails", {
customer_id: {customer.id},
email_type: "Feedback Request",
scheduled_for: ADD_DAYS(NOW(), 14)
})
SCHEDULED TASK: Send_Scheduled_Emails (runs every hour)
emails_to_send = FIND_RECORDS("Scheduled_Emails", {
scheduled_for:
Testing and Debugging Workflows
Testing Strategy: Never test complex workflows in production. Create a staging environment with test data and simulate all scenarios.
Workflow Testing Checklist
- Test happy path (everything works perfectly)
- Test validation failures (missing data, invalid input)
- Test permission scenarios (unauthorized users)
- Test edge cases (null values, extreme amounts)
- Test concurrent executions (race conditions)
- Test external API failures (timeout, error responses)
- Test rollback scenarios (compensation logic)
- Test notification delivery (email, SMS, in-app)
- Performance test with large datasets
- Test with real user accounts and permissions
Summary and Next Steps
You've Mastered:
- State machine architecture for complex processes
- Multi-stage approval systems (parallel and sequential)
- Multi-system orchestration with error handling
- Workflow optimization for performance and scale
- Event-driven and time-based automation
- Monitoring, logging, and debugging workflows
- Real-world workflow patterns for enterprise applications
Next Lesson: Enterprise Features - Learn about HIPAA compliance, SOC 2, enterprise security, and regulatory requirements.
Practice Challenge: Design and implement a complete employee onboarding workflow that includes: multi-stage approvals, account provisioning in multiple systems, scheduled training assignments, automated compliance checks, and 30/60/90 day review triggers. Document state transitions and test all failure scenarios.
We'd love to hear your feedback.