Cross-compilation creates an easy illusion: if the program built successfully, it should run on the board.
In practice, that often fails. The executable is copied to the device and reports Exec format error. The file exists, but execution says not found. Linking succeeds, but the program crashes when calling a library function. Both targets are ARM, but changing toolchains breaks floating-point arguments. Both targets are RISC-V, but the generated program uses instructions the board does not support.
These are usually not source syntax problems. The binary interface does not match.
The safest first model is this: a compiler emits a binary for a target platform. That platform is not only a CPU instruction set. It includes ELF format, ABI, calling convention, floating-point convention, dynamic linker, C library, kernel interface, and rootfs.
source code
-> compiler target / ISA / extensions
-> ABI / calling convention / libc
-> ELF / dynamic linker / shared libraries
-> kernel / rootfs
-> process starts and runs
So debugging “it builds but does not run” cannot stop at whether the compiler command succeeded.
If the ISA Does Not Match, the CPU Cannot Execute It
The lowest-level problem is instruction-set mismatch.
An x86_64 program cannot run directly on an ARM device. An AArch64 program cannot be dropped into a 32-bit ARM user space. An RV64 program cannot run on a system that only supports RV32. A RISC-V program using unsupported extensions may fault with an illegal instruction at runtime.
Common checks include:
- target is ARM, AArch64, RISC-V, MIPS, or x86
- 32-bit or 64-bit
- RISC-V RV32 or RV64, and which extensions
- ARM AArch32, Thumb, AArch64, or a specific profile
- endianness
- whether the kernel supports this user-space width and ABI
Start with file or readelf -h to inspect ELF machine and class.
Exec format error often points to this layer: the kernel saw an ELF file, but not one it can execute on this system.
ABI Defines How Binaries Work Together
ABI means application binary interface. It is not a source-level API. It is a binary-level contract.
An ABI defines:
- which registers or stack slots carry function arguments
- how return values are passed back
- which registers are caller-saved or callee-saved
- how the stack is aligned
- sizes and alignment for structs, enums,
long, pointers, and other types - system-call numbers and argument passing
- ELF relocation, symbols, thread-local storage, and related rules
- floating-point argument convention
When ABI does not match, source code may look fine while binaries misunderstand each other.
For example, an application may pass floating-point arguments according to one convention while a library reads them according to another. An application may assume long is 64-bit while another boundary interprets it differently. Handwritten assembly may fail to preserve callee-saved registers. Stack alignment may be wrong, causing SIMD or floating-point code to crash inside a library.
These bugs are more dangerous than compile failures because they may only crash at a specific call boundary after running for a while.
Calling Convention Decides Whether Function Boundaries Match
Calling convention is one of the most visible parts of an ABI.
It answers:
- which registers hold the first arguments
- where additional arguments go
- which register holds the return value
- who adjusts the stack
- which registers must survive a call
- how many bytes the stack must be aligned to
- which extra state an interrupt or exception entry must preserve
Normal C-to-C calls are usually handled by the compiler, so this boundary is easy to miss. But it becomes hard contract when you write assembly, startup code, syscall wrappers, FFI, JIT code, context switching, or ISR trampolines.
For example, an RTOS context switch that does not save floating-point registers may work until one task uses floating point. A syscall wrapper that puts arguments in the wrong registers sends nonsense to the kernel. Handwritten assembly that breaks stack alignment may crash later inside a C library function.
Calling convention is not style. It is the contract that makes binaries callable.
Floating-Point ABI Is a Common Trap
Floating-point ABI is a common failure point in embedded Linux and MCU toolchains.
Differences include:
- whether hardware FPU exists
- whether the compiler emits hardware floating-point instructions
- whether floating-point arguments are passed in FP registers
- whether libraries are built with the same floating-point ABI
- whether the RTOS context switch saves floating-point registers
Some platforms have an FPU but use a soft-float ABI. Some toolchains emit hard-float instructions that the target chip or kernel configuration does not support. Some applications and libraries share the same ISA but not the same floating-point ABI, causing link or runtime failures.
So options such as arm-linux-gnueabi, arm-linux-gnueabihf, -mfloat-abi, and -mfpu are not mere performance switches. They affect the binary interface.
Missing Dynamic Linker Can Look Like not found
In embedded Linux, one confusing symptom is that a file exists, but running it says not found.
The missing file may not be the executable. It may be the dynamic linker requested by the ELF.
Dynamically linked programs usually have an INTERP segment, such as /lib/ld-linux-*.so.*. When the kernel runs such an ELF, it first loads this interpreter. If the rootfs does not contain that path, the error can look like a missing executable.
Inspect it with:
readelf -l ./app
Look for Requesting program interpreter.
This commonly happens when:
- the app was built with a glibc toolchain but the rootfs uses musl
- the cross-compiler came from another SDK or distro
- dynamic-linker path differs
- the rootfs lacks the matching C library and loader
- shared-library dependencies were not copied
The question is not only whether the file exists. The runtime requested by the ELF must exist too.
sysroot and rootfs Must Belong to the Same World
Cross-compilation usually uses a sysroot. It contains target headers, libraries, and linker paths. The rootfs is what the device actually uses at runtime.
If build-time sysroot and runtime rootfs come from different worlds, failures include:
- compile-time headers do not match runtime libraries
- runtime symbol versions are missing
- library paths differ
- C libraries such as glibc, musl, and uClibc are mixed
- dependencies such as libstdc++, libgcc_s, pthread, rt, or dl are missing
- kernel headers used for build do not match actual kernel capabilities
Architecture name cannot fix this. Two systems may both be ARM Linux and still be incompatible because C library, dynamic linker, ABI, and library versions differ.
A safer engineering practice is to build applications with the SDK or toolchain that belongs to the target rootfs, or at least ensure sysroot and rootfs come from the same release or build system.
Toolchain Triples Are Clues, Not the Whole Truth
Cross-toolchain names often contain triples or quadruples, such as:
aarch64-linux-gnu
arm-linux-gnueabihf
riscv64-linux-gnu
arm-none-eabi
They provide important clues:
- target architecture
- whether the target is Linux
- likely C library or ABI convention
- whether it targets bare metal
- possible floating-point ABI
But the name does not tell everything.
The same riscv64-linux-gnu label may have different default ISA extensions, code model, libc version, or dynamic-linker path. The same arm-none-eabi may be used for Cortex-M bare metal but still require explicit CPU, FPU, float ABI, linker script, and startup files.
So a toolchain name is the first clue. The final answer is in compiler flags, ELF attributes, link result, and target runtime.
Static Linking Is Not a Universal Escape
When dynamic-library problems appear, static linking is tempting.
Static linking can reduce dependency on the dynamic linker and shared libraries, but it does not remove every ABI issue:
- ISA still must match
- system-call ABI still must match
- kernel features still must exist
- DNS, NSS, time zones, certificates, device files, and runtime resources may still be needed
- statically linked glibc may still depend on runtime configuration in some cases
- C++, threads, localization, and plugin mechanisms may add constraints
Static linking is useful for some deployment models, but it is not a pass to run across arbitrary root filesystems.
Debugging Order for “Builds but Does Not Run”
When a binary does not run on the target, check in this order:
file app: architecture, bit width, dynamic or static linkreadelf -h app: ELF class, machine, endiannessreadelf -A appor related tools: architecture attributes, floating-point ABI, extensionsreadelf -l app: dynamic-linker pathlddor target equivalent: whether shared libraries exist- sysroot used for build and rootfs on target come from the same source
- kernel supports the target ABI, bit width, and required system calls
- if assembly, FFI, plugins, or RTOS context switching are involved, check calling convention and saved registers
- if the program later hits illegal instruction, check ISA extensions and target CPU capability
- if it crashes randomly, check ABI, stack alignment, floating-point register saving, and library versions
This path turns “cross-compilation is broken” into more specific layers.
What to Remember
A successful cross-compile only means the compiler and linker accepted the source and link inputs. It does not guarantee the target device can load, link, start, and correctly execute the binary.
Whether a program can run depends on a longer contract: ISA, extensions, bit width, endianness, ABI, calling convention, floating-point ABI, ELF interpreter, C library, shared libraries, rootfs, kernel support, and runtime resources.
Do not stop at “is it ARM/RISC-V?” or “did compilation pass?” Use file, readelf, dynamic-linker path, sysroot/rootfs origin, and ABI attributes to confirm that the binary and target system belong to the same world.