Closed Loop

Closed Loop An architecture diagram generated by Archify. LLM Token Generation · (Llama 3 70B, batch size = 1) · Architecture component LLM Token Generation (Llama 3 70B, batch size = 1) Input: 1 token · vector (1 × 8192 dims) ≈ 32 KB of activations · Architecture component Input: 1 token vector (1 × 8192 dims) ≈ 32 KB of activations Arithmetic intensity: · 32 KB data / 140 GB weights = 0.0002 FLOPs/b… · Architecture component Arithmetic intensity: 32 KB data / 140 GB weights = 0.0002 FLOPs/b… Legend Backend Security

Loop

  • • 3 steps, one cycle
  • • Edge labels omitted for clarity