Smart Rules Analytics

Overview

Smart Rules Analytics provides comprehensive insights into how your rules are performing. Track trigger frequency, false positive rates, effectiveness ratings, and optimization opportunities to continuously improve your AI governance policies.

Analytics data is collected in real-time as rules are evaluated, providing immediate visibility into rule behavior and enabling data-driven policy refinement.

Key Features

• Performance Scoring: 0-100 score for each rule based on effectiveness
• Trigger Tracking: Count of rule activations over time
• False Positive Detection: Track rules that trigger incorrectly
• Effectiveness Ratings: A-F grade based on combined metrics
• Trend Analysis: Historical performance data
• Optimization Recommendations: AI-powered suggestions for improvement

How It Works

Metrics Collection

Agent Action Evaluated
          |
          v
+-------------------+
| Rule Triggered?   |--No--> No metrics
+-------------------+
          |
         Yes
          v
+-------------------+
| Record Trigger    |
| - timestamp       |
| - action context  |
| - rule id         |
+-------------------+
          |
          v
+-------------------+
| Track Outcome     |
| - approved?       |
| - overridden?     |
| - false positive? |
+-------------------+
          |
          v
+-------------------+
| Update Analytics  |
| - trigger count   |
| - accuracy rate   |
| - performance     |
+-------------------+

Performance Score Calculation

The performance score (0-100) is calculated from multiple factors:

Performance Score = (
  (trigger_effectiveness * 0.3) +
  (accuracy_rate * 0.4) +
  (response_time * 0.1) +
  (admin_feedback * 0.2)
) * 100

Factor	Weight	Description
Trigger Effectiveness	30%	How often triggers lead to correct action
Accuracy Rate	40%	(1 - false_positive_rate)
Response Time	10%	Speed of rule evaluation
Admin Feedback	20%	Manual ratings from reviewers

Configuration

Analytics API Endpoints

Get Rule Analytics:

GET /api/smart-rules/{rule_id}/analytics

Get All Rules with Analytics:

GET /api/smart-rules?include_analytics=true

Get Analytics Summary:

GET /api/smart-rules/analytics/summary

Analytics Response Schema

{
  "rule_id": 15,
  "performance_score": 87,
  "triggers_last_24h": 142,
  "triggers_last_7d": 856,
  "triggers_last_30d": 3421,
  "false_positives": 12,
  "false_positive_rate": 0.014,
  "effectiveness_rating": "A",
  "last_triggered": "2026-01-20T15:30:45Z",
  "average_response_time_ms": 23,
  "compliance_score": 0.95,
  "trend": "stable",
  "recommendations": []
}

Usage Examples

Retrieve Rule Analytics

curl -X GET "https://api.ascend.ai/api/smart-rules/15/analytics" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "rule_id": 15,
  "name": "Block Production File Deletions",
  "performance_score": 92,
  "triggers_last_24h": 8,
  "false_positives": 0,
  "effectiveness_rating": "A",
  "last_triggered": "2026-01-20T14:22:10Z",
  "trend": "improving",
  "recommendations": []
}

Get Rules Sorted by Performance

curl -X GET "https://api.ascend.ai/api/smart-rules?sort=performance_score&order=asc" \
  -H "Authorization: Bearer $TOKEN"

This returns rules with lowest performance first, helping identify rules that need optimization.

Analytics Summary Dashboard Data

curl -X GET "https://api.ascend.ai/api/smart-rules/analytics/summary" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "total_rules": 45,
  "active_rules": 42,
  "total_triggers_24h": 1247,
  "average_performance_score": 78,
  "rules_by_effectiveness": {
    "A": 12,
    "B": 18,
    "C": 8,
    "D": 3,
    "F": 1
  },
  "top_performing_rules": [
    {"id": 15, "name": "Block Prod Deletions", "score": 98},
    {"id": 23, "name": "Monitor External APIs", "score": 95},
    {"id": 8, "name": "Require DB Approval", "score": 93}
  ],
  "needs_attention": [
    {"id": 31, "name": "Legacy Network Rule", "score": 34, "issue": "High false positive rate"}
  ]
}

Effectiveness Ratings

Rules are assigned letter grades based on their overall performance:

Grade	Score Range	Description
A	90-100	Excellent - Rule performs optimally
B	80-89	Good - Minor improvements possible
C	70-79	Average - Consider optimization
D	60-69	Below Average - Needs attention
F	0-59	Failing - Disable or rewrite rule

Optimization

Identifying Problem Rules

Look for rules with:

• High false positive rates (> 10%)
• Low trigger counts (may be too narrow)
• Very high trigger counts (may be too broad)
• D or F effectiveness ratings

Common Optimization Strategies

1. Reduce False Positives:

// Before: Too broad
{
  "condition": "action_type == 'api_call'"
}

// After: More specific
{
  "condition": "action_type == 'api_call' AND target NOT CONTAINS 'internal'"
}

2. Adjust Risk Levels:

// If rule triggers too often with low actual risk
{
  "risk_level": "high"  // Change to "medium"
}

3. Refine Agent Scope:

// Before: All agents
{
  "agent_id": "*"
}

// After: Specific agents that need monitoring
{
  "agent_id": "external-api-bot"
}

Automated Recommendations

The analytics system provides AI-powered recommendations:

{
  "rule_id": 31,
  "recommendations": [
    {
      "type": "condition_refinement",
      "message": "Consider adding environment filter - 85% of false positives are from staging",
      "suggested_change": "AND environment == 'production'"
    },
    {
      "type": "risk_adjustment",
      "message": "Actions blocked by this rule are typically low impact",
      "suggested_change": "Reduce risk_level from 'high' to 'medium'"
    }
  ]
}

Tracking Metrics Over Time

Historical Data

Analytics data is retained for:

• 24-hour granularity: 90 days
• Daily aggregates: 1 year
• Monthly aggregates: Indefinitely

Trend Indicators

Trend	Meaning
improving	Performance score increasing
stable	Performance score consistent
declining	Performance score decreasing
volatile	High variance in performance

Best Practices

1. Review Weekly: Check analytics summary weekly for problem rules
2. Set Alerts: Configure alerts for rules dropping below C grade
3. A/B Test Changes: Use rule versioning to test optimizations
4. Document Changes: Track why rules were modified
5. Archive Poor Performers: Disable rules with sustained F grades

Related

• Smart Rules Overview - Understanding smart rules
• AI Rule Generation - Generate optimized rules
• Manual Rule Creation - Fine-tune rule configurations
• Platform Analytics - Broader platform metrics