Why data layout matters

Modern CPUs are fast; memory is slow. A cache miss on a modern CPU costs 100–300 ns — hundreds of cycles during which the CPU stalls. Getting your data layout right can matter more than any algorithmic optimisation.

The cache line is the unit of transfer between RAM and CPU cache. On x86 and ARM Cortex-A, a cache line is 64 bytes. Accessing one byte loads 64 bytes. If the next byte you need is in the same cache line — free. If it’s 65 bytes away — another cache miss.

The rule: put data that is accessed together in memory near each other.

Array of Structures (AoS)

The natural C++ layout — an array of struct instances:

1struct Particle {
2    float x, y, z;    // position
3    float vx, vy, vz; // velocity
4    float mass;
5    uint32_t flags;
6};                     // 32 bytes per particle
7
8std::vector<Particle> particles(1000);

Memory layout:

[x0,y0,z0,vx0,vy0,vz0,m0,f0] [x1,y1,z1,vx1,vy1,vz1,m1,f1] ...

When AoS is appropriate

When you process one particle at a time — all fields of one particle in one operation — AoS is good. All fields are in the same (or adjacent) cache lines:

1// Good for AoS — using all fields of one particle
2void integrate(Particle& p, float dt) {
3    p.x += p.vx * dt;
4    p.y += p.vy * dt;
5    p.z += p.vz * dt;
6}

Structure of Arrays (SoA)

Separate arrays for each field:

1struct Particles {
2    std::vector<float> x, y, z;
3    std::vector<float> vx, vy, vz;
4    std::vector<float> mass;
5    std::vector<uint32_t> flags;
6
7    Particles(size_t n)
8        : x(n), y(n), z(n), vx(n), vy(n), vz(n), mass(n), flags(n) {}
9};

Memory layout:

x: [x0, x1, x2, x3, ..., x999]
y: [y0, y1, y2, y3, ..., y999]
...

When SoA is appropriate

When you process one field across all particles — e.g., updating all x positions — SoA loads cache lines densely:

1// Good for SoA — only x and vx accessed, 16 floats per cache line
2void integrateX(Particles& p, float dt, size_t count) {
3    for (size_t i = 0; i < count; ++i) {
4        p.x[i] += p.vx[i] * dt;
5    }
6}

With AoS, this same loop touches every 8th float (stride = 32 bytes) — each cache line holds only 2 useful values, the rest is wasted bandwidth.

With SoA, x and vx are contiguous — each 64-byte cache line holds 16 floats, all used.

Practical measurement

A simple benchmark: update 100,000 positions.

1// AoS loop — stride 8 floats (32 bytes) between x values
2for (size_t i = 0; i < N; ++i)
3    aos[i].x += aos[i].vx * dt;
4
5// SoA loop — stride 1 float (4 bytes), sequential
6for (size_t i = 0; i < N; ++i)
7    soa.x[i] += soa.vx[i] * dt;

Typical results on a desktop CPU (N=100,000):

Layout Time Cache misses
AoS (x update only) ~450 µs high
SoA (x update only) ~120 µs low
AoS (all fields) ~380 µs medium
SoA (all fields) ~490 µs medium
  • SoA wins when accessing one field across many elements
  • AoS wins when accessing all fields of one element at a time

SIMD and SoA

SoA is the prerequisite for SIMD vectorisation. The compiler can auto-vectorise a SoA loop with SSE/NEON; it usually cannot vectorise an AoS loop with stride.

1// Compiler can vectorise this — sequential float array
2for (size_t i = 0; i < N; i += 4) {
3    // Compiler generates 4-wide SIMD: vx[i..i+3] + vy[i..i+3]
4    soa.x[i+0] += soa.vx[i+0] * dt;
5    soa.x[i+1] += soa.vx[i+1] * dt;
6    soa.x[i+2] += soa.vx[i+2] * dt;
7    soa.x[i+3] += soa.vx[i+3] * dt;
8}

Explicitly with ARM NEON intrinsics:

 1#include <arm_neon.h>
 2
 3void integrateX_neon(float* __restrict x, const float* __restrict vx,
 4                     float dt, size_t n) {
 5    float32x4_t vdt = vdupq_n_f32(dt);
 6    for (size_t i = 0; i < n; i += 4) {
 7        float32x4_t xi  = vld1q_f32(x  + i);
 8        float32x4_t vxi = vld1q_f32(vx + i);
 9        xi = vmlaq_f32(xi, vxi, vdt);  // xi += vxi * dt
10        vst1q_f32(x + i, xi);
11    }
12}

Four floats processed per instruction. This is the payoff of SoA layout.

Hybrid: AoSoA

Hot fields extracted into SoA, cold fields left in a separate AoS:

 1struct ParticleHot {       // accessed every frame
 2    float x, y, z;
 3    float vx, vy, vz;
 4};
 5
 6struct ParticleCold {      // accessed occasionally
 7    float mass;
 8    uint32_t flags;
 9    uint32_t id;
10    char name[16];
11};
12
13std::vector<ParticleHot>  hot(N);   // SoA-friendly: pack into separate x/y/z arrays
14std::vector<ParticleCold> cold(N);  // cold data separate — doesn't pollute cache

Or AoSoA — chunks of SoA stored as AoS blocks:

1struct ParticleBlock {
2    float x[8], y[8], z[8];    // 8-wide SIMD block
3    float vx[8], vy[8], vz[8];
4};                               // one cache line per component array
5
6std::vector<ParticleBlock> blocks(N / 8);

Each block fits in a few cache lines. Within a block, SIMD operates on all 8 particles simultaneously. This is common in physics engines and ray tracers.

Applying this to firmware

On Cortex-M, cache is smaller (16–64 kB) and the penalty for a miss is smaller (~10 cycles on M4 vs ~300 on a desktop). But the principle still applies.

DMA buffers and ADC multi-channel: interleaved ADC data is AoS (ch1,ch2,ch3, ch1,ch2,ch3, …). Deinterleaving to SoA before processing improves filter performance:

1// Interleaved ADC: [ch0_0, ch1_0, ch2_0, ch0_1, ch1_1, ch2_1, ...]
2// Deinterleave to SoA
3for (size_t i = 0; i < N; ++i) {
4    ch0[i] = raw[i * 3 + 0];
5    ch1[i] = raw[i * 3 + 1];
6    ch2[i] = raw[i * 3 + 2];
7}
8// Now filter ch0[] sequentially — all in cache

Sensor sample arrays: if your DSP only ever processes one channel, store samples as a flat array (SoA) rather than structs with a channel field.

Checklist

  1. Profile first. Use perf stat -e cache-misses (Linux) or a hardware PMU counter. Don’t optimise data layout by guessing.

  2. Identify the hot loop. Which struct fields does it access? If only 1–2 of N fields, consider SoA.

  3. Separate hot and cold data. Fields rarely accessed in the main loop shouldn’t share a cache line with fields accessed every iteration.

  4. Align to cache line boundaries for independent arrays:

    1alignas(64) float x[N];
2alignas(64) float vx[N];

  5. Use __restrict on pointers in hot loops to allow the compiler to assume no aliasing and generate better SIMD code.

Summary

  • Cache line = 64 bytes = 16 floats — everything between two accesses in that range is “free”
  • AoS: good when all fields of one element are processed together
  • SoA: good when one field across many elements is processed (filters, DSP, physics)
  • SoA enables auto-vectorisation and SIMD — AoS typically doesn’t
  • AoSoA: block-based hybrid — SIMD-friendly blocks, cache-friendly iteration
  • Deinterleave DMA/ADC data to SoA before DSP — one cache line per channel per iteration

What’s next