Skip to main content

Playbook Management

Overview​

Playbooks are reusable automation workflows that define how the OWL AI Platform responds to specific AI agent actions. They enable organizations to codify their governance policies into executable rules that automatically approve, deny, escalate, or take custom actions based on configurable trigger conditions.

Key Capabilities​

  • Automated Governance: Execute predefined actions when conditions are met
  • Flexible Triggers: Match actions by risk score, action type, agent, time, and more
  • Action Chaining: Execute multiple actions in sequence (approve, notify, log, etc.)
  • Version Control: Track changes with full version history and rollback
  • Testing Support: Validate playbooks against test data before deployment
  • Metrics & Analytics: Track execution counts, success rates, and cost savings

How It Works​

Playbook Architecture​

┌─────────────────────────────────────────────────────────────────────────────────┐
│                           PLAYBOOK STRUCTURE                                     │
└─────────────────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────────────────┐
│  PLAYBOOK: pb-low-risk-auto-approve                                             │
│                                                                                  │
│  ┌─────────────────────────────────────────────────────────────────────────┐   │
│  │  METADATA                                                                │   │
│  │  - Name: Auto-Approve Low Risk Actions                                   │   │
│  │  - Description: Automatically approve low-risk read operations           │   │
│  │  - Status: active                                                        │   │
│  │  - Risk Level: low                                                       │   │
│  └─────────────────────────────────────────────────────────────────────────┘   │
│                                                                                  │
│  ┌─────────────────────────────────────────────────────────────────────────┐   │
│  │  TRIGGER CONDITIONS (ALL must match)                                     │   │
│  │                                                                          │   │
│  │  ├─ Risk Score: 0-40                                                     │   │
│  │  ├─ Action Types: [database_read, file_read, api_get]                    │   │
│  │  ├─ Environment: [production, staging]                                   │   │
│  │  └─ Business Hours: true (9am-5pm EST, Mon-Fri)                          │   │
│  └─────────────────────────────────────────────────────────────────────────┘   │
│                                                                                  │
│  ┌─────────────────────────────────────────────────────────────────────────┐   │
│  │  ACTIONS (Executed in order when triggered)                              │   │
│  │                                                                          │   │
│  │  1. [APPROVE] Auto-approve the action                                    │   │
│  │  2. [LOG] Create audit log entry                                         │   │
│  │  3. [NOTIFY] Send notification to ops@company.com                        │   │
│  └─────────────────────────────────────────────────────────────────────────┘   │
│                                                                                  │
│  ┌─────────────────────────────────────────────────────────────────────────┐   │
│  │  METRICS                                                                 │   │
│  │  - Execution Count: 1,523                                                │   │
│  │  - Success Rate: 98.5%                                                   │   │
│  │  - Last Executed: 2026-01-20T14:30:00Z                                   │   │
│  │  - Est. Monthly Savings: $2,250                                          │   │
│  └─────────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────────┘

Playbook Matching Flow​

         ┌─────────────────────────────────────────┐
         │          INCOMING ACTION                │
         │  Agent: analytics-agent                 │
         │  Type: database_read                    │
         │  Risk Score: 35                         │
         │  Time: 10:30 AM EST (Tuesday)           │
         └─────────────────┬───────────────────────┘
                           │
                           v
         ┌─────────────────────────────────────────┐
         │      CHECK ACTIVE PLAYBOOKS             │
         │      (Ordered by creation date)         │
         └─────────────────┬───────────────────────┘
                           │
           ┌───────────────┼───────────────┐
           │               │               │
           v               v               v
    ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
    │ Playbook #1  │ │ Playbook #2  │ │ Playbook #3  │
    │ Low Risk     │ │ High Risk    │ │ Weekend      │
    │ Auto-Approve │ │ Escalate     │ │ Block        │
    └──────┬───────┘ └──────┬───────┘ └──────┬───────┘
           │               │               │
           v               v               v
    ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
    │   MATCH      │ │  NO MATCH    │ │  NO MATCH    │
    │ ✓ Risk 0-40  │ │ ✗ Risk 70+   │ │ ✗ Weekend    │
    │ ✓ Read action│ │              │ │   only       │
    │ ✓ Biz hours  │ │              │ │              │
    └──────┬───────┘ └──────────────┘ └──────────────┘
           │
           │ First match wins
           v
    ┌──────────────────────────────────────────────┐
    │           EXECUTE PLAYBOOK ACTIONS           │
    │                                              │
    │  1. ✓ Approve action                         │
    │  2. ✓ Create audit log                       │
    │  3. ✓ Send notification                      │
    └──────────────────────────────────────────────┘

Trigger Conditions​

Available Conditions​

ConditionTypeDescriptionExample
risk_scoreRangeRisk score threshold (0-100){"min": 0, "max": 40}
action_typeListAllowed action types["database_read", "file_read"]
environmentListTarget environments["production", "staging"]
agent_idListSpecific agent IDs["analytics-agent"]
business_hoursBooleanDuring business hours (9-5 EST, Mon-Fri)true
weekendBooleanDuring weekends onlytrue
time_rangeObjectSpecific time window{"start": "09:00", "end": "17:00"}

Condition Logic​

  • ALL specified conditions must match for the playbook to trigger
  • Conditions not specified are ignored (not evaluated)
  • First matching playbook (by creation order) executes
  • Remaining playbooks are skipped once a match is found

Action Types​

Available Actions​

Action TypeCodeDescriptionParameters
ApproveapproveAuto-approve the actionNone
DenydenyBlock the actionreason (optional)
Escalateescalate_approvalEscalate to higher levellevel (L1-L4), reason
NotifynotifySend email notificationrecipients, subject, template
Stakeholder Notifystakeholder_notificationNotify stakeholdersrecipients, subject
Quarantinetemporary_quarantineTemporarily blockduration_minutes, reason
Risk Assessmentrisk_assessmentTrigger deep scandeep_scan, include_context
LoglogCreate audit entryNone
WebhookwebhookCall external endpointurl, method, headers

Action Execution Order​

Actions execute sequentially in the order defined. If any action fails:

  1. Error is logged
  2. Remaining actions continue (fail-forward)
  3. Playbook marked as partial success

Configuration​

Create Playbook​

from owlai import OWLClient

client = OWLClient(api_key="your_api_key")

playbook = client.playbooks.create(
    id="pb-low-risk-auto",
    name="Auto-Approve Low Risk Actions",
    description="Automatically approve low-risk read operations during business hours",
    status="active",
    risk_level="low",
    approval_required=False,
    trigger_conditions={
        "risk_score": {"min": 0, "max": 40},
        "action_type": ["database_read", "file_read", "api_get"],
        "environment": ["production", "staging"],
        "business_hours": True
    },
    actions=[
        {"type": "approve", "parameters": {}, "enabled": True},
        {"type": "log", "parameters": {}, "enabled": True},
        {
            "type": "notify",
            "parameters": {
                "recipients": ["ops@company.com"],
                "subject": "Low-risk action auto-approved"
            },
            "enabled": True
        }
    ]
)

print(f"Created playbook: {playbook.id}")

Create via API​

curl -X POST https://api.owlai.io/v1/playbooks \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "pb-high-risk-escalate",
    "name": "Escalate High Risk Actions",
    "description": "Escalate high-risk actions to L3 approval",
    "status": "active",
    "risk_level": "high",
    "trigger_conditions": {
      "risk_score": {"min": 70, "max": 100},
      "environment": ["production"]
    },
    "actions": [
      {
        "type": "escalate_approval",
        "parameters": {"level": "L3", "reason": "High risk production action"},
        "enabled": true
      },
      {
        "type": "stakeholder_notification",
        "parameters": {
          "recipients": ["security@company.com", "oncall@company.com"],
          "subject": "High-risk action requires approval"
        },
        "enabled": true
      }
    ]
  }'

Update Playbook​

# Update existing playbook
client.playbooks.update(
    playbook_id="pb-low-risk-auto",
    trigger_conditions={
        "risk_score": {"min": 0, "max": 35},  # More conservative
        "action_type": ["database_read", "file_read"],
        "business_hours": True
    }
)

Activate/Deactivate​

# Deactivate playbook
client.playbooks.update(playbook_id="pb-low-risk-auto", status="inactive")

# Reactivate
client.playbooks.update(playbook_id="pb-low-risk-auto", status="active")

Versioning and Rollback​

Version History​

Every playbook change creates a new version:

# Get version history
versions = client.playbooks.list_versions(playbook_id="pb-low-risk-auto")

for version in versions:
    print(f"v{version.version}: {version.created_at} by {version.created_by}")
    print(f"  Changes: {version.change_summary}")

# Output:
# v3: 2026-01-20T10:00:00Z by admin@company.com
#   Changes: Updated risk_score threshold from 40 to 35
# v2: 2026-01-15T14:30:00Z by admin@company.com
#   Changes: Added file_read to action_types
# v1: 2026-01-10T09:00:00Z by admin@company.com
#   Changes: Initial creation

Rollback to Previous Version​

# Rollback to version 2
client.playbooks.rollback(
    playbook_id="pb-low-risk-auto",
    target_version=2,
    reason="Version 3 threshold too restrictive"
)

Compare Versions​

# Compare two versions
diff = client.playbooks.compare_versions(
    playbook_id="pb-low-risk-auto",
    version_a=2,
    version_b=3
)

print(diff)
# {
#   "changes": [
#     {
#       "field": "trigger_conditions.risk_score.max",
#       "before": 40,
#       "after": 35
#     }
#   ]
# }

Testing Playbooks​

Dry-Run Test​

Test a playbook against sample data without executing:

# Test playbook matching
result = client.playbooks.test(
    playbook_id="pb-low-risk-auto",
    test_data={
        "risk_score": 35,
        "action_type": "database_read",
        "agent_id": "analytics-agent",
        "timestamp": "2026-01-20T10:30:00Z"  # Tuesday, business hours
    }
)

print(f"Matches: {result.matches}")
print(f"Matched Conditions: {result.matched_conditions}")
print(f"Failed Conditions: {result.failed_conditions}")
print(f"Would Execute: {result.would_execute_actions}")

# Output:
# Matches: True
# Matched Conditions: ['risk_score', 'action_type', 'business_hours']
# Failed Conditions: []
# Would Execute: ['approve', 'log', 'notify']

Bulk Testing​

# Test multiple scenarios
scenarios = [
    {"risk_score": 25, "action_type": "database_read"},
    {"risk_score": 45, "action_type": "database_write"},
    {"risk_score": 35, "action_type": "file_delete"},
]

results = client.playbooks.bulk_test(
    playbook_id="pb-low-risk-auto",
    scenarios=scenarios
)

for i, result in enumerate(results):
    print(f"Scenario {i+1}: {'MATCH' if result.matches else 'NO MATCH'}")

Metrics and Analytics​

Playbook Metrics​

# Get playbook performance metrics
metrics = client.playbooks.get_metrics(playbook_id="pb-low-risk-auto")

print(f"Execution Count: {metrics.execution_count}")
print(f"Success Rate: {metrics.success_rate}%")
print(f"Last Executed: {metrics.last_executed}")
print(f"Triggers (24h): {metrics.triggers_last_24h}")
print(f"Cost Savings (24h): ${metrics.cost_savings_24h}")
print(f"Avg Response Time: {metrics.avg_response_time_seconds}s")

# Output:
# Execution Count: 1523
# Success Rate: 98.5%
# Last Executed: 2026-01-20T14:30:00Z
# Triggers (24h): 87
# Cost Savings (24h): $1087.50
# Avg Response Time: 2s

Cost Savings Calculation​

Automated playbook approvals save time compared to manual review:

MetricValueCalculation
Time Saved per Approval15 minutesAverage manual review time
Hourly Cost$50/hourFully-loaded engineer cost
Cost per Manual Approval$12.5015 min x $50/hr
Monthly Savings$2,250180 auto-approvals x $12.50

Pre-Built Templates​

Available Templates​

TemplateCategoryUse CaseEst. Savings
template-low-risk-autoProductivityAuto-approve low-risk reads$2,250/mo
template-high-risk-escalateSecurityEscalate high-risk to L3+N/A
template-pii-notifyComplianceNotify on PII accessN/A
template-after-hours-blockSecurityBlock after-hours writesN/A
template-budget-alertCostAlert on budget thresholds$500/mo

Create from Template​

# Create playbook from template
playbook = client.playbooks.create_from_template(
    template_id="template-low-risk-auto",
    playbook_id="pb-my-low-risk",
    customizations={
        "trigger_conditions": {
            "risk_score": {"max": 30}  # More conservative
        }
    }
)

Best Practices​

Playbook Design​

  1. Start Conservative: Begin with strict conditions, loosen as needed
  2. Test Thoroughly: Use dry-run testing before deploying
  3. Monitor Metrics: Track success rates and adjust
  4. Document Purpose: Clear descriptions help future maintenance
  5. Use Versioning: Leverage version control for safe rollbacks

Trigger Conditions​

  1. Be Specific: Narrow conditions reduce false matches
  2. Layer Conditions: Combine multiple conditions for precision
  3. Consider Time: Use business hours for appropriate actions
  4. Environment Awareness: Different rules for prod vs dev

Actions​

  1. Order Matters: Critical actions first, notifications last
  2. Include Logging: Always include audit logging
  3. Notify Appropriately: Don't over-notify, causes alert fatigue
  4. Test Webhooks: Verify external integrations work