Classic BPF vs eBPF — The Linux Kernel documentation (2024)

eBPF is designed to be JITed with one to one mapping, which can also open upthe possibility for GCC/LLVM compilers to generate optimized eBPF code throughan eBPF backend that performs almost as fast as natively compiled code.

Some core changes of the eBPF format from classic BPF:

• Number of registers increase from 2 to 10:

  The old format had two registers A and X, and a hidden frame pointer. Thenew layout extends this to be 10 internal registers and a read-only framepointer. Since 64-bit CPUs are passing arguments to functions via registersthe number of args from eBPF program to in-kernel function is restrictedto 5 and one register is used to accept return value from an in-kernelfunction. Natively, x86_64 passes first 6 arguments in registers, aarch64/sparcv9/mips64 have 7 - 8 registers for arguments; x86_64 has 6 callee savedregisters, and aarch64/sparcv9/mips64 have 11 or more callee saved registers.

  Thus, all eBPF registers map one to one to HW registers on x86_64, aarch64,etc, and eBPF calling convention maps directly to ABIs used by the kernel on64-bit architectures.

  On 32-bit architectures JIT may map programs that use only 32-bit arithmeticand may let more complex programs to be interpreted.

  R0 - R5 are scratch registers and eBPF program needs spill/fill them ifnecessary across calls. Note that there is only one eBPF program (== oneeBPF main routine) and it cannot call other eBPF functions, it can onlycall predefined in-kernel functions, though.

• Register width increases from 32-bit to 64-bit:

  Still, the semantics of the original 32-bit ALU operations are preservedvia 32-bit subregisters. All eBPF registers are 64-bit with 32-bit lowersubregisters that zero-extend into 64-bit if they are being written to.That behavior maps directly to x86_64 and arm64 subregister definition, butmakes other JITs more difficult.

  32-bit architectures run 64-bit eBPF programs via interpreter.Their JITs may convert BPF programs that only use 32-bit subregisters intonative instruction set and let the rest being interpreted.

  Operation is 64-bit, because on 64-bit architectures, pointers are also64-bit wide, and we want to pass 64-bit values in/out of kernel functions,so 32-bit eBPF registers would otherwise require to define register-pairABI, thus, there won’t be able to use a direct eBPF register to HW registermapping and JIT would need to do combine/split/move operations for everyregister in and out of the function, which is complex, bug prone and slow.Another reason is the use of atomic 64-bit counters.

• Conditional jt/jf targets replaced with jt/fall-through:

  While the original design has constructs such as if (cond) jump_true;else jump_false;, they are being replaced into alternative constructs likeif (cond) jump_true; /* else fall-through */.

• Introduces bpf_call insn and register passing convention for zero overheadcalls from/to other kernel functions:

  Before an in-kernel function call, the eBPF program needs toplace function arguments into R1 to R5 registers to satisfy callingconvention, then the interpreter will take them from registers and passto in-kernel function. If R1 - R5 registers are mapped to CPU registersthat are used for argument passing on given architecture, the JIT compilerdoesn’t need to emit extra moves. Function arguments will be in the correctregisters and BPF_CALL instruction will be JITed as single ‘call’ HWinstruction. This calling convention was picked to cover common callsituations without performance penalty.

  After an in-kernel function call, R1 - R5 are reset to unreadable and R0 hasa return value of the function. Since R6 - R9 are callee saved, their stateis preserved across the call.

  For example, consider three C functions:

  u64 f1() { return (*_f2)(1); }u64 f2(u64 a) { return f3(a + 1, a); }u64 f3(u64 a, u64 b) { return a - b; }

  GCC can compile f1, f3 into x86_64:

  f1: movl $1, %edi movq _f2(%rip), %rax jmp *%raxf3: movq %rdi, %rax subq %rsi, %rax ret

  See Also

  Casus Belli 21 - PDFCOFFEE.COMDictionnaire de la Bible/Tome 2.2.b ECCLÉSIASTE-ESCLAVE - WikisourceThe Substance (2024) Movie Ending Explained, Themes Analyzed & Body Horror’s Subversion Of Beauty Politics2024 Ghost Mass Outbreak Halloween Event | Pokemon Scarlet and Violet (SV)｜Game8

  Function f2 in eBPF may look like:

  f2: bpf_mov R2, R1 bpf_add R1, 1 bpf_call f3 bpf_exit

  If f2 is JITed and the pointer stored to _f2. The calls f1 -> f2 -> f3 andreturns will be seamless. Without JIT, __bpf_prog_run() interpreter needs tobe used to call into f2.

  For practical reasons all eBPF programs have only one argument ‘ctx’ which isalready placed into R1 (e.g. on __bpf_prog_run() startup) and the programscan call kernel functions with up to 5 arguments. Calls with 6 or more argumentsare currently not supported, but these restrictions can be lifted if necessaryin the future.

  On 64-bit architectures all register map to HW registers one to one. Forexample, x86_64 JIT compiler can map them as ...

  R0 - raxR1 - rdiR2 - rsiR3 - rdxR4 - rcxR5 - r8R6 - rbxR7 - r13R8 - r14R9 - r15R10 - rbp

  ... since x86_64 ABI mandates rdi, rsi, rdx, rcx, r8, r9 for argument passingand rbx, r12 - r15 are callee saved.

  Then the following eBPF pseudo-program:

  bpf_mov R6, R1 /* save ctx */bpf_mov R2, 2bpf_mov R3, 3bpf_mov R4, 4bpf_mov R5, 5bpf_call foobpf_mov R7, R0 /* save foo() return value */bpf_mov R1, R6 /* restore ctx for next call */bpf_mov R2, 6bpf_mov R3, 7bpf_mov R4, 8bpf_mov R5, 9bpf_call barbpf_add R0, R7bpf_exit

  After JIT to x86_64 may look like:

  push %rbpmov %rsp,%rbpsub $0x228,%rspmov %rbx,-0x228(%rbp)mov %r13,-0x220(%rbp)mov %rdi,%rbxmov $0x2,%esimov $0x3,%edxmov $0x4,%ecxmov $0x5,%r8dcallq foomov %rax,%r13mov %rbx,%rdimov $0x6,%esimov $0x7,%edxmov $0x8,%ecxmov $0x9,%r8dcallq baradd %r13,%raxmov -0x228(%rbp),%rbxmov -0x220(%rbp),%r13leaveqretq

  Which is in this example equivalent in C to:

  u64 bpf_filter(u64 ctx){ return foo(ctx, 2, 3, 4, 5) + bar(ctx, 6, 7, 8, 9);}

  In-kernel functions foo() and bar() with prototype: u64 (*)(u64 arg1, u64arg2, u64 arg3, u64 arg4, u64 arg5); will receive arguments in properregisters and place their return value into %rax which is R0 in eBPF.Prologue and epilogue are emitted by JIT and are implicit in theinterpreter. R0-R5 are scratch registers, so eBPF program needs to preservethem across the calls as defined by calling convention.

  For example the following program is invalid:

  bpf_mov R1, 1bpf_call foobpf_mov R0, R1bpf_exit

  After the call the registers R1-R5 contain junk values and cannot be read.An in-kernel eBPF verifier is used to validate eBPF programs.

Also in the new design, eBPF is limited to 4096 insns, which means that anyprogram will terminate quickly and will only call a fixed number of kernelfunctions. Original BPF and eBPF are two operand instructions,which helps to do one-to-one mapping between eBPF insn and x86 insn during JIT.

The input context pointer for invoking the interpreter function is generic,its content is defined by a specific use case. For seccomp register R1 pointsto seccomp_data, for converted BPF filters R1 points to a skb.

A program, that is translated internally consists of the following elements:

op:16, jt:8, jf:8, k:32 ==> op:8, dst_reg:4, src_reg:4, off:16, imm:32

So far 87 eBPF instructions were implemented. 8-bit ‘op’ opcode fieldhas room for new instructions. Some of them may use 16/24/32 byte encoding. Newinstructions must be multiple of 8 bytes to preserve backward compatibility.

eBPF is a general purpose RISC instruction set. Not every register andevery instruction are used during translation from original BPF to eBPF.For example, socket filters are not using exclusive add instruction, buttracing filters may do to maintain counters of events, for example. Register R9is not used by socket filters either, but more complex filters may be runningout of registers and would have to resort to spill/fill to stack.

eBPF can be used as a generic assembler for last step performanceoptimizations, socket filters and seccomp are using it as assembler. Tracingfilters may use it as assembler to generate code from kernel. In kernel usagemay not be bounded by security considerations, since generated eBPF codemay be optimizing internal code path and not being exposed to the user space.Safety of eBPF can come from the eBPF verifier. In such use cases asdescribed, it may be used as safe instruction set.

Just like the original BPF, eBPF runs within a controlled environment,is deterministic and the kernel can easily prove that. The safety of the programcan be determined in two steps: first step does depth-first-search to disallowloops and other CFG validation; second step starts from the first insn anddescends all possible paths. It simulates execution of every insn and observesthe state change of registers and stack.

+----------------+--------+--------------------+| 4 bits | 1 bit | 3 bits || operation code | source | instruction class |+----------------+--------+--------------------+(MSB) (LSB)

BPF_ADD 0x00BPF_SUB 0x10BPF_MUL 0x20BPF_DIV 0x30BPF_OR 0x40BPF_AND 0x50BPF_LSH 0x60BPF_RSH 0x70BPF_NEG 0x80BPF_MOD 0x90BPF_XOR 0xa0BPF_MOV 0xb0 /* eBPF only: mov reg to reg */BPF_ARSH 0xc0 /* eBPF only: sign extending shift right */BPF_END 0xd0 /* eBPF only: endianness conversion */

BPF_JA 0x00 /* BPF_JMP only */BPF_JEQ 0x10BPF_JGT 0x20BPF_JGE 0x30BPF_JSET 0x40BPF_JNE 0x50 /* eBPF only: jump != */BPF_JSGT 0x60 /* eBPF only: signed '>' */BPF_JSGE 0x70 /* eBPF only: signed '>=' */BPF_CALL 0x80 /* eBPF BPF_JMP only: function call */BPF_EXIT 0x90 /* eBPF BPF_JMP only: function return */BPF_JLT 0xa0 /* eBPF only: unsigned '<' */BPF_JLE 0xb0 /* eBPF only: unsigned '<=' */BPF_JSLT 0xc0 /* eBPF only: signed '<' */BPF_JSLE 0xd0 /* eBPF only: signed '<=' */

+--------+--------+-------------------+| 3 bits | 2 bits | 3 bits || mode | size | instruction class |+--------+--------+-------------------+(MSB) (LSB)

BPF_W 0x00 /* word */BPF_H 0x08 /* half word */BPF_B 0x10 /* byte */BPF_DW 0x18 /* eBPF only, double word */

B - 1 byteH - 2 byteW - 4 byteDW - 8 byte (eBPF only)

BPF_IMM 0x00 /* used for 32-bit mov in classic BPF and 64-bit in eBPF */BPF_ABS 0x20BPF_IND 0x40BPF_MEM 0x60BPF_LEN 0x80 /* classic BPF only, reserved in eBPF */BPF_MSH 0xa0 /* classic BPF only, reserved in eBPF */BPF_ATOMIC 0xc0 /* eBPF only, atomic operations */