meta_api_grabber/RATE_LIMITER_ENHANCEMENTS.md

Meta API Rate Limiter Enhancements

Summary

Enhanced the rate limiter in rate_limiter.py to monitor all Meta API rate limit headers as documented in the official Meta documentation.

New Headers Monitored

1. X-App-Usage (Platform Rate Limits)

Tracks application-level rate limits across all users.

Fields:

• call_count: Percentage of calls made (0-100)
• total_cputime: Percentage of CPU time used (0-100)
• total_time: Percentage of total time used (0-100)

Example:

{
  "call_count": 28,
  "total_time": 25,
  "total_cputime": 25
}

2. X-Ad-Account-Usage (Ad Account Specific)

Tracks rate limits for specific ad accounts.

Fields:

• acc_id_util_pct: Percentage of ad account usage (0-100)
• reset_time_duration: Time in seconds until rate limit resets
• ads_api_access_tier: Access tier (e.g., "standard_access", "development_access")

Example:

{
  "acc_id_util_pct": 9.67,
  "reset_time_duration": 100,
  "ads_api_access_tier": "standard_access"
}

3. X-Business-Use-Case-Usage (Business Use Case Limits)

Tracks rate limits per business use case (ads_insights, ads_management, etc.).

Fields:

• business_id: Business object ID
• type: Type of BUC (ads_insights, ads_management, custom_audience, etc.)
• call_count: Percentage of calls made (0-100)
• total_cputime: Percentage of CPU time (0-100)
• total_time: Percentage of total time (0-100)
• estimated_time_to_regain_access: Time in minutes until access is restored
• ads_api_access_tier: Access tier

Example:

{
  "66782684": [{
    "type": "ads_management",
    "call_count": 95,
    "total_cputime": 20,
    "total_time": 20,
    "estimated_time_to_regain_access": 0,
    "ads_api_access_tier": "development_access"
  }]
}

4. x-fb-ads-insights-throttle (Legacy)

Original header still supported for backward compatibility.

Fields:

• app_id_util_pct: App usage percentage
• acc_id_util_pct: Account usage percentage

Key Enhancements

1. Intelligent Throttling

The rate limiter now uses estimated_time_to_regain_access and reset_time_duration to calculate optimal delays:

# If we have estimated_time_to_regain_access from BUC header
if self.estimated_time_to_regain_access > 0:
    delay = self.estimated_time_to_regain_access * 60  # Convert minutes to seconds

# If we have reset_time_duration from Ad Account header
elif self.reset_time_duration > 0:
    delay = self.reset_time_duration * 0.5  # Use fraction as safety margin

2. Comprehensive Error Code Detection

Expanded error code detection to include all Meta rate limit error codes:

• 4: App rate limit
• 17: User rate limit
• 32: Pages rate limit
• 613: Custom rate limit
• 80000-80014: Business Use Case rate limits (Ads Insights, Ads Management, Custom Audience, Instagram, LeadGen, Messenger, Pages, WhatsApp, Catalog)

3. Debug Logging

All headers are now logged in DEBUG mode with detailed parsing information:

logger.debug(f"X-App-Usage header: {header_value}")
logger.debug(f"Parsed X-App-Usage: {result}")

4. Enhanced Statistics

The get_stats() and print_stats() methods now display comprehensive metrics from all headers:

======================================================================
RATE LIMITER STATISTICS
======================================================================
Total Requests:              0
Throttled Requests:          0
Rate Limit Errors:           0

X-App-Usage (Platform Rate Limits):
  Call Count:                95.0%
  Total CPU Time:            90.0%
  Total Time:                88.0%

X-Ad-Account-Usage:
  Account Usage:             97.5%
  Reset Time Duration:       300s
  API Access Tier:           standard_access

X-Business-Use-Case-Usage:
  Type: ads_insights
    Call Count:              98.0%
    Total CPU Time:          95.0%
    Total Time:              92.0%
    Est. Time to Regain:     15 min

Legacy (x-fb-ads-insights-throttle):
  App Usage:                 93.0%
  Account Usage:             96.0%

Max Usage Across All Metrics: 98.0%
Currently Throttled:          True
======================================================================

Usage

Enable Debug Logging

To see all header parsing in debug mode:

import logging

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

Access New Metrics

All metrics are available through the get_stats() method:

stats = limiter.get_stats()

print(f"App call count: {stats['app_call_count']}%")
print(f"Ad account usage: {stats['ad_account_usage_pct']}%")
print(f"Reset in: {stats['reset_time_duration']}s")
print(f"Regain access in: {stats['estimated_time_to_regain_access']} min")
print(f"API tier: {stats['ads_api_access_tier']}")

# Business use case details
for buc in stats['buc_usage']:
    print(f"BUC {buc['type']}: {buc['call_count']}%")

Testing

Run the test script to see the rate limiter in action:

uv run python test_rate_limiter.py

This will demonstrate:

• Parsing all four header types
• Intelligent throttling based on usage
• Comprehensive statistics display
• Debug logging output

References

• Meta Graph API Rate Limiting
• Meta Marketing API Best Practices