TMX, XLIFF & file splitting

Turn a multilingual materialized or streaming document into independent per-locale documents and physical output files without losing target-specific integrity.

What splitting means in Lokit

split_targets() splits by target locale, not by byte size or arbitrary unit count. Because all inputs become the same model, it works on a multilingual TMX, XLIFF, .lokit, spreadsheet, or any other materialized BaseStructure. Each result has one target_locale, one-entry target_locales, selected values in Data.target, and no remaining Data.targets mapping.

Split any detected file into physical outputs

Use lokit.parse.file() when format defaults are enough, then choose any supported exporter for each locale. Explicit locales select a subset; an empty tuple discovers them from document metadata and unit target mappings in encounter order.

split_file.py

from pathlib import Path
import lokit
source = Path("multilingual.tmx")output_dir = Path("localized")output_dir.mkdir(parents=True, exist_ok=True)
document = lokit.parse.file(str(source))for locale, localized in document.split_targets().items():    localized.export.lokit(output_dir / f"{source.stem}.{locale}.lokit")

Split a translation memory

A multilingual TMX stores targets in Data.targets. Splitting promotes each locale's text, status, tags, plural metadata, comments, metadata, and extensions into an independent document. Stable application IDs and original TMX IDs in extensions are retained; collision-safe generated IDs remain deterministic when the source repeats or omits tuid values.

split_tmx.py

from pathlib import Path
import lokit
memory = lokit.parse.tmx("memory.tmx", progress=False)for locale, localized in memory.split_targets(include_missing=False).items():    localized.export.tmx(Path("tm") / f"memory.{locale}.tmx")

Split XLIFF targets and preserve resources

XLIFF units retain the original resource in extensions["resource"], their original unit ID, and segment ID where present. group_by_resource=True writes separate XLIFF <file> groups inside each locale's physical output; it does not create extra filesystem files. Duplicate IDs from different source resources use collision-safe internal keys and recover their original ID on export.

split_xliff.py

from pathlib import Path
import lokit
catalog = lokit.parse.xliff("catalog.xliff")for locale, localized in catalog.split_targets(("fr-FR", "de-DE")).items():    localized.export.xliff(        Path("xliff") / f"catalog.{locale}.xliff",        group_by_resource=True,    )

Choose whether missing units remain

include_missing=True keeps source units that do not have the selected locale and sets their selected target to None; this is useful for generating complete translation jobs. include_missing=False omits them, which is useful for delivering only populated translations. A legacy Data.target participates only when the document identifies a single matching target locale.

Materialized splits are independent copies

BaseStructure.split_targets() returns dict[str, BaseStructure]. It deep-copies metadata, comments and nested origins, contexts, plurals, tags and their attributes, document extensions, and unit/target extensions. Editing one locale's document therefore cannot mutate the source document or another locale's result.

Stream TMX or XLIFF once with bounded memory

StreamingStructure.split_targets() returns a StreamingTargetSplit context manager. Entering it consumes the source exactly once and writes bounded temporary native .lokit spools. The per-locale StreamingStructure values are valid only inside the with block; leaving it closes active iterators and removes the temporary directory. The split context itself is one-shot and cannot be reopened.

split_stream.py

from pathlib import Path
import lokit
stream = lokit.stream.xliff("large-catalog.xliff")with stream.split_targets(include_missing=False) as localized_streams:    for locale, localized in localized_streams.items():        localized.export.xliff(            Path("out") / f"catalog.{locale}.xliff",            group_by_resource=True,        )

Locale validation and practical limits

Requested locale tuples cannot contain empty or whitespace-only values, duplicates are removed while preserving order, and one split supports at most 256 locales. Passing explicit locales avoids a discovery pass over late target mappings and is preferable when the desired set is already known. Synchronous streaming split is available wherever a StreamingStructure exists—currently .lokit, TMX, XLIFF, DOCX, and PPTX—while lokit.parse.file() provides the general materialized path for other formats.

Target integrity during promotion

The selected target becomes the legacy/single-target view without flattening away its facts: known target status wins, target plural wins when present, target metadata merges field-by-field over base metadata, non-empty target comments replace base comments, target extensions merge over base extensions, and target inline parts/maps become Data.tags.target_*. Source text, source tags, base context, and document provenance remain attached.