GGML - AI at the edge

A tensor library called ggml is used in machine learning to support big models and great performance on low-cost hardware. Both llama.cpp and whisper.cpp make use of it.
16-bit float support in C code

Support for integer quantization (e.g., 4-bit, 5-bit, and 8-bit)
automated distinction
Built-in optimisation techniques, such as ADAM and L-BFGS
Apple Silicon-optimized
uses AVX/AVX2 intrinsics on x86 architectures
Web support with WASM SIMD and WebAssembly
No reliance on other sources
No memory is allocated while it is running.
Support for guided language output