How Lokit works

Format-specific parsers produce either a materialized BaseStructure or a one-shot StreamingStructure. Both carry the same Data units, target shapes, plurals, status, inline tags, comments, context, metadata, and extensions. Writers, regeneration, matching, and database ingestion consume that model instead of depending on a TMX, XLIFF, PO, or Office object tree.

Use lokit.parse when you need repeated access, mutation, or random lookup across the complete document. Use lokit.stream when units can be consumed once in order. Streaming parsers and async bridges move units through bounded iterators rather than retaining a full XML DOM, which makes large localization files practical without tying memory growth to the complete input size. Availability is format-specific; the API-selection and format pages state where a synchronous document stream exists.

Lokit continues to read and write legacy XML interchange such as TMX and XLIFF, but applications do not have to carry those XML trees through their own code. The native .lokit format is a sparse, versioned interchange that round-trips every field in Lokit's documented model within explicit v1 bounds, including the distinction between absent values and present empty values. A conversion to a different format adopts the destination format's semantics; use regeneration when the original file envelope must be retained.

The high-volume .lokit, TMX, and XLIFF paths use native Rust parsers. Rust provides a memory-safe native core for parsing, validation, spans, and canonical .lokit formatting, while the public Python boundary stays strictly typed. Release wheels also compile the typed Python layer with mypyc. Bounded batches and detached Python data units keep the native implementation from leaking format-specific objects into application code.

DOCX and PPTX extraction and reinsertion use Lokit's bundled native C# Office runtime and do not require Microsoft Office to be installed. The Office APIs still return the same Python model as every other parser, while export and regeneration use the source package when document relationships and layout must be preserved.

Lokit combines event-driven parsing, bounded streaming, parallel TMX parsing, direct archive rewriting, and PostgreSQL COPY staging. That architecture targets both throughput and predictable memory use; it is more important than a single headline benchmark. Benchmark values are being refreshed, so the benchmark guide focuses on corpus, semantic output, timing scope, throughput, and peak cold RSS.

document.export creates a canonical file from the common model. document.regen instead applies translated units to an original source file so the result follows the source package, document, catalog, or XML envelope. It is the normal path for producing localized customer files quickly while leaving unrelated source content in place.

The translation-memory API can ingest BaseStructure or StreamingStructure directly. It serializes units, locale targets, plurals, inline tags, segment parts, context, comments, metadata, and extensions into a versioned PostgreSQL schema, then supports exact, contextual, tag-aware, and trigram matching. Continue with the database workflow, Alembic schema guide, or typed row-model guide.