seo/guides/API_TROUBLESHOOTING.md

API Troubleshooting - 400 Bad Request Issues

The Problem

WordPress REST API returned 400 Bad Request errors on pagination:

✓ Fetched 100 posts (page 1)
✓ Fetched 100 posts (page 2)
✓ Fetched 100 posts (page 3)
✗ Error page 4: 400 Bad Request

This is a server-side limitation, not a bug in our code.

---

Root Causes

1. API Pagination Limits

Some WordPress configurations limit how many pages can be fetched:

• Page 1-3: OK (limit reached)
• Page 4+: 400 Bad Request

Common causes:

• Plugin restrictions (security, performance)
• Server configuration limits
• REST API throttling
• Custom WordPress filters

2. _fields Parameter Issues

The _fields parameter (to fetch only specific columns) might cause issues on:

• Specific API versions
• Custom REST API implementations
• Security plugins that filter fields

3. Status Parameter Encoding

Multi-status queries (status=publish,draft) can fail on pagination.

---

The Solution

The script now:

1. Gracefully handles 400 errors - Treats pagination limit as end of data
2. Retries without _fields - Falls back to fetching all fields if needed
3. Continues analysis - Uses posts it was able to fetch (doesn't fail)
4. Logs what it got - Shows exactly how many posts were fetched

# Graceful error handling
if response.status_code == 400:
    logger.info(f"API limit reached (got {status_count} posts)")
    break  # Stop pagination, use what we have

---

What Happens Now

Before (Failed)

Fetching mistergeek.net...
✓ Fetched 100 posts (page 1)
✓ Fetched 100 posts (page 2)
✗ Error page 4: 400 Bad Request
ERROR: No posts found on any site

After (Works)

Fetching mistergeek.net...
✓ Fetched 100 publish posts (page 1)
✓ Fetched 100 publish posts (page 2)
✓ Fetched 28 publish posts (page 3)
ⓘ API limit reached (fetched 228 posts)
✓ Total publish posts: 228

---

How to Check If This Affects You

If you see:

✓ Fetched 100 posts (page 1)
✓ Fetched 100 posts (page 2)
✓ Fetched 28 posts (page 3)
✓ Fetched 15 posts (page 4)
✓ Total posts: 243

Good! Your API supports full pagination. All posts are being fetched.

If you see:

✓ Fetched 100 posts (page 1)
ⓘ API limit reached (fetched 100 posts)
✓ Total posts: 100

Limited pagination. API only allows page 1. Script continues with 100 posts.

If you see:

✓ Fetched 100 posts (page 1)
✓ Fetched 100 posts (page 2)
ⓘ API limit reached (fetched 200 posts)
✓ Total posts: 200

Partial pagination. API allows pages 1-2. Script gets 200 posts.

---

Impact on Analysis

Scenario 1: All Posts Fetched (Full Pagination)

262 posts total
262 posts analyzed ✓
100% coverage

Result: Complete analysis, no issues.

Scenario 2: Limited to First Page (100 posts)

262 posts total
100 posts analyzed
38% coverage

Result: Analysis of first 100 posts only. Missing ~162 posts.

Impact:

• Report shows only first 100 posts
• Cannot analyze all content
• Must run analyzer multiple times or contact hosting provider

Scenario 3: Limited to First 3 Pages (300+ posts if available)

262 posts total
228 posts analyzed ✓
87% coverage

Result: Analyzes most posts, misses last few.

---

Solutions If Limited

Solution 1: Contact Hosting Provider

Ask for:

"Can you increase the WordPress REST API pagination limit? Currently limited to X posts per site."

Most providers can increase this in:

• WordPress settings
• PHP configuration
• Plugin settings

Solution 2: Fetch in Batches

If API limits to 100 posts at a time:

# Run 1: Analyze first 100
python scripts/multi_site_seo_analyzer.py

# Save results
cp output/reports/seo_analysis_*.csv week1_batch1.csv

# Then manually get remaining posts another way
# (export from WordPress admin, use different tool, etc.)

Solution 3: Check Security Plugins

Some plugins limit REST API access:

• Wordfence
• Sucuri
• iThemes Security
• Jetpack

Try:

1. Temporarily disable security plugins
2. Run analyzer
3. Re-enable plugins

If this works, configure plugin to allow REST API for your IP.

Solution 4: Use WordPress Export Feature

If REST API is completely broken:

1. WordPress Admin → Tools → Export
2. Select posts to export
3. Download XML
4. Convert XML to CSV
5. Run analyzer on CSV (different mode)

---

When to Worry

No Worries If:

• API fetches 150+ posts (most content covered)
• Error message says "API limit reached" (graceful)
• Analysis completes successfully
• CSV has all/most posts

Worth Investigating If:

• Only fetching <50 posts
• API returning other errors (401, 403, 500)
• All 3 sites have same issue
• Posts are missing from analysis

---

Checking Your Hosting

How to check API pagination limit:

In browser/terminal:

# Replace with your site
curl https://www.mistergeek.net/wp-json/wp/v2/posts?per_page=100&status=publish

# Try different pages
curl https://www.mistergeek.net/wp-json/wp/v2/posts?page=1&per_page=100&status=publish
curl https://www.mistergeek.net/wp-json/wp/v2/posts?page=2&per_page=100&status=publish
curl https://www.mistergeek.net/wp-json/wp/v2/posts?page=3&per_page=100&status=publish

If you get:

• 200 OK: Page works
• 400 Bad Request: Pagination limited
• 401 Unauthorized: Auth needed
• 403 Forbidden: Access denied

Common Limits by Hosting:

Host	Typical Limit	Notes
Shared hosting	1-2 pages	Often limited for performance
WP Engine	Unlimited	Usually good
Kinsta	Unlimited	Usually good
Bluehost	Often limited	Contact support
GoDaddy	Limited	May need plugin adjustment

---

Advanced: Manual Pagination

If API pagination is broken, you can manually specify which posts to analyze:

# Fetch from Google Sheets instead of API
# Or use WordPress XML export
# Or manually create CSV of posts you want to analyze

(Contact us if you need help with this)

---

Logs Explained

New Log Messages:

✓ Fetched 100 publish posts (page 1)
  → Successful fetch of 100 posts

ⓘ API limit reached (fetched 228 posts)
  → API doesn't allow page 4+, got 228 total

ⓘ Retrying without _fields parameter
  → Trying again without field filtering

✓ Total publish posts: 228
  → Final count for this status

---

Summary

Issue	Impact	Solution
Can't fetch page 2+	Limited analysis	Contact host, check plugins
400 Bad Request	Graceful handling	Script continues with what it got
All 3 sites fail	API-wide issue	Check WordPress REST API
Missing top 50 posts	Incomplete analysis	Use WordPress export as backup

---

Next Steps

1. Run analyzer and note pagination limits for each site

2. Check logs - see how many posts were fetched

3. If limited:

   • Note the numbers (e.g., "Only fetched 100 of 262")
   • Contact your hosting provider
   • Ask about REST API pagination limits

4. Re-run when fixed (hosting provider increases limit)

---

Still Having Issues?

Check:

1. ✓ WordPress credentials correct
2. ✓ REST API enabled on all 3 sites
3. ✓ User has read permissions
4. ✓ No IP blocking (firewall/security)
5. ✓ No SSL certificate issues
6. ✓ Sites are online and responding

See: guides/SEO_ANALYZER_GUIDE.md → Troubleshooting section