Bytespider
ByteDance · training
- name
- Bytespider
- operator
- ByteDance
- purpose
- training
- ua_substring
- Bytespider
- robots_token
- Bytespider
- respects_robots
- no
- verify
- no authoritative published range file; treat unverified Bytespider traffic with suspicion
- notes
- Has a reputation for aggressive crawling and inconsistent robots.txt adherence. Rate-limit at the edge if it causes load.
- canonical_name
- Bytespider
- user_agent_token
- Bytespider
- ua_full
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- bot_type
- training
- bot_type_extension
- —
- opt_out_mechanism
- robots.txt disallow (User-agent: Bytespider) — but adherence is documented as inconsistent; rate-limit / block at the edge if needed
- published_ip_range_url
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- asn
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- reverse_dns_suffix
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- supports_web_bot_auth
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- signature_agent_domain
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- jwks_url
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- verification_methods
user-agent-match- crawl_traffic_share
- 10.25% source
- targeted_content_type
- HTML, text
- documentation_url
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- first_seen_date
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- last_verified_date
- 2026-06-15
- block_vs_allow_recommendation
- block (default) — training crawler with documented aggressive crawling and inconsistent robots.txt adherence and no authoritative IP file to verify; block/rate-limit at the edge.
- citation_referral_value
- low (training; no direct citation or referral)
- cloudflare_verified_category
- — verify-against-primary-at-build ↗ https://radar.cloudflare.com/bots/directory/bytespider
- status
- active
- triples
["Bytespider","operated_by","ByteDance"]["Bytespider","has_bot_type","training"]["Bytespider","has_crawl_share","10.25% (Radar 2026-05)"]- attribute_sources
{"claims":["crawl_traffic_share"],"source":"https://radar.cloudflare.com/bots","last_verified":"2026-06-15"}