French AI Startup ZML Launches Free Tool to Accelerate Inference Across Multiple AI Processors

ZML, a Paris-based open-source project led by serial entrepreneur Steeve Morin, released version 2.0 on March 24, 2026, delivering a hardware-agnostic inference engine that compiles large language models directly onto NVIDIA, AMD, Google TPU, and AWS Trainium chips from a single codebase. The release marks a significant shift in how AI workloads can be deployed across competing hardware platforms without proprietary lock-in. ZML’s approach bypasses the Python and PyTorch dependencies that have dominated AI infrastructure, instead using a Zig-based architecture to generate standalone native binaries optimized for each processor type.

A New Path to Hardware-Agnostic Inference

ZML v2 represents a fundamentally different engineering approach to AI inference than existing solutions. The project is written 92.7% in Zig, combined with MLIR and OpenXLA technologies, enabling the compiler to transform model computation graphs into native binaries tailored to specific accelerators. This architecture eliminates the traditional dependency on Python wrappers around CUDA kernels, a pattern that dominates competing inference servers like vLLM and Ollama.

The technical distinction carries practical implications. By compiling models directly to hardware rather than relying on Python-based Intermediaries, ZML achieves lower-level control over inference optimization and reduces the software layers between model and processor. Currently in alpha status, the LLMD inference server operates as a technical preview, supporting single-GPU deployment with a maximum batch size of 16 and compatibility with Llama and Qwen model architectures. While these limitations reflect early-stage development, they underscore ZML’s focus on production-grade performance for specific, high-demand use cases rather than universal coverage.

Why Inference Matters Now More Than Ever

The timing of ZML’s release aligns with a dramatic shift in AI compute spending. Deloitte projects that inference workloads will account for approximately two-thirds of all AI compute by 2026, a sharp increase from just one-third in 2023. This transition from training-dominated spending to inference-dominated operations has forced enterprises and developers to reconsider how they deploy and optimize models in production environments.

ZML’s hardware-agnostic design directly addresses this inflection point. As inference becomes the dominant AI workload, organizations face pressure to avoid vendor lock-in with single-chip providers. The ability to compile a model once and deploy it across NVIDIA GPUs, AMD accelerators, Google TPUs, and AWS Trainium chips reduces switching costs and enables workload flexibility. For enterprises managing diverse infrastructure or planning multi-cloud strategies, this capability represents a meaningful departure from the CUDA-centric ecosystem that has defined AI infrastructure for the past decade.

Steeve Morin’s Track Record Shapes Project Ambitions

Steeve Morin brings substantial experience to ZML’s leadership. As a serial entrepreneur with four Startup exits and previous leadership of Synergetics AI—a company focused on autonomous and agentic AI systems—Morin has demonstrated expertise in scaling AI systems to production maturity. His involvement signals that ZML is positioned as a production-grade inference stack rather than a research prototype, even during its alpha phase.

The open-source, free distribution model reflects ZML’s core mission: decoupling AI workloads from proprietary hardware ecosystems. Unlike commercial inference platforms that monetize through licensing or managed services, ZML operates as a community-driven project aimed at establishing “any model, many hardwares, one codebase” as a viable standard. This approach creates potential friction with hardware vendors but aligns with growing developer demand for portability and independence in AI infrastructure choices.

Context Within the Broader French AI Ecosystem

ZML emerges within a European AI landscape increasingly defined by ambitious independent projects and well-funded startups. While ZML itself remains an open-source initiative without disclosed venture funding, France has established itself as a center for AI innovation. Mistral AI, a separate French AI startup, raised €1.7 billion in Series C funding led by ASML at an €11.7 billion post-money valuation, making it Europe’s most valuable AI company. Notably, ZML supports inference on Mistral’s model architectures, though the two organizations operate independently.

The distinction between ZML and commercial AI startups reflects a broader diversification in how AI infrastructure develops. While companies like Mistral focus on model development and commercial deployment, projects like ZML address foundational infrastructure challenges around hardware portability and inference efficiency. This division of labor allows specialized projects to tackle narrow but critical problems without the overhead of venture-backed company structures.

Technical Limitations and Realistic Deployment Scope

ZML’s current capabilities reflect honest assessment of alpha-stage development. The inference server does not yet support multi-GPU sharding, a critical feature for large-scale deployments requiring distributed computation across multiple accelerators. Prefix caching—an optimization technique for repeated prompt processing—remains unimplemented. Architecture support extends only to Llama and Qwen model families, excluding other popular open-source and proprietary models.

These constraints position ZML squarely in the early-adopter phase. Organizations running single-GPU inference workloads with compatible model architectures can begin testing immediately. Enterprises requiring multi-GPU sharding, broader model support, or advanced caching strategies will need to monitor ZML’s development roadmap or continue relying on more mature platforms like vLLM, which has evolved significantly since its UC Berkeley origins in mid-2023. Red Hat’s AI Inference Server, built on vLLM, represents the maturity curve that competing projects must eventually reach to claim enterprise readiness.

What Comes Next for Hardware-Agnostic Inference

ZML’s trajectory will depend on community adoption and the project’s ability to address current limitations. Multi-GPU sharding support and expanded model architecture compatibility represent the immediate technical priorities. The open-source nature of the project means that adoption velocity will signal whether hardware-agnostic inference can challenge the NVIDIA-dominated inference ecosystem or remains a niche alternative for specific use cases.

The broader significance lies in establishing a credible alternative path for AI inference deployment. As inference workloads grow to dominate AI compute spending, tools that prevent vendor lock-in and enable workload portability will attract increasing attention from enterprises concerned about long-term infrastructure flexibility. ZML’s March 2026 release marks the beginning of this conversation, not its conclusion.