Tags & inline content

Keep formatting, placeholders, links, and native inline payloads aligned while moving content between TMX, XLIFF, HTML, IDML, Office, and .lokit files.

Text and structure are deliberately separate

Lokit stores readable text in Data.source, Data.target, and TargetData.text. Inline structure is stored beside those strings as an ordered sequence of TextPart and CodePart values plus a code map. This lets applications translate plain text without carrying an XML tree while retaining enough identity, pairing, attributes, and native payload to write inline content again.

Inline model

Data├── source: "Click here"└── tags: Tags    ├── source_parts    │   ├── TextPart("Click ")    │   ├── CodePart("open-link")    │   ├── TextPart("here")    │   └── CodePart("close-link")    └── source_tag_map        ├── "open-link" → TieData(..., pair_id="link-1")        └── "close-link" → TieData(..., pair_id="link-1")

Tags, TargetTags, parts, and maps

Type	Fields	Role
`Tags`	`source_tag_map`, `target_tag_map`, `source_parts`, `target_parts`	Source plus selected/legacy target inline content on `Data`.
`TargetTags`	`tag_map`, `parts`	Inline content for one locale inside `TargetData`.
`TextPart`	`value`	Literal text in sequence order.
`CodePart`	`ref`	Reference to exactly one code in the corresponding map.
`TieData`	code identity and source-format details	Portable legacy representation used by parsers, writers, splitting, .lokit, and the database.
`Segment`	`parts`, `codes`, derived `plain_text`	Validated canonical view used by the public rendering helpers.
`InlineCode`	kind, semantic, pair/alignment IDs, offset, order, native payload	Normalized code used while rendering from one syntax to another.

TieData field reference

Field	Meaning
`id`	Code identifier referenced by `CodePart.ref`.
`type`	`TieType`: open, close, or standalone structural classification.
`attributes`	Parsed source attributes as strings; namespace-qualified names may be retained.
`attribute_data`	Opaque auxiliary data, including namespace mappings needed for raw reconstruction.
`position`	Plain-text offset captured during extraction.
`order`	Stable encounter order used when rebuilding code sequences and pair numbers.
`pair_id`	Shared identity connecting an opening and closing code.
`original_name`	Native element/tag name, including a namespace prefix when available.
`original_text`	Native inline payload, such as escaped TMX/XLIFF code content.

Normalized conversion types

Type	Complete value/field map
`TagAttribute`	Optional `namespace`, plus `name` and `value`.
`NativeCode`	`syntax`, `name`, optional `namespace`, ordered `attributes`, optional native `payload`, and optional `equivalent_text`.
`InlineCode`	`id`, `kind`, portable `semantic`, `pair_id`, `alignment_id`, `text_offset`, `order`, and its `native` code.
`InlineCodeKind`	`open`, `close`, `standalone`, `isolated-open`, `isolated-close`, `annotation-open`, and `annotation-close`. The legacy `TieData` adapter currently produces open, close, or standalone.
`InlineSemantic`	`generic`, `emphasis`, `strong`, `link`, `line-break`, `image`, `variable`, or `annotation`.
`ConversionOutcome`	`exact`, `placeholder`, or `dropped`.
`ConversionDiagnostic`	Code ID, source/destination syntax, outcome, and message.
`ConversionReport`	Tuple of diagnostics plus derived `is_exact`. It is a public result container for integrations; `render_segment()` itself returns a string and enforces `UnsupportedTagPolicy`.
`TagIntegrityError`	`ValueError` subclass raised for inconsistent references or unsafe conversion.

Every TieType family

Paired HTML-derived types are available for a, abbr, b, bdi, bdo, cite, code, data, dfn, em, i, kbd, mark, q, rp, rt, ruby, s, samp, small, span, strong, sub, sup, time, u, and var, each with .open and .close variants. br, img, and wbr are standalone. Unknown or format-specific codes use custom.open, custom.close, or custom.standalone, so an unfamiliar source tag does not need to be mislabeled as a known HTML element.

Parse for round-trip integrity or rendered strings

The default include_tags=False keeps source and target strings plain while retaining the structural maps and parts for writers. This is the safest parse-edit-export path. Set include_tags=True only when the consumer wants inline markup embedded in the returned strings; tag_syntax selects that rendered syntax and the structural objects remain attached to the projected copy.

tag_modes.py

import lokitfrom lokit.types import TagSyntax, UnsupportedTagPolicy
# Lossless editing and conversion: keep plain text plus structural tags.round_trip = lokit.parse.tmx("rich.tmx")round_trip.export.xliff("rich.xliff")
# Presentation/integration view: embed safe HTML in returned strings.rendered = lokit.parse.tmx(    "rich.tmx",    include_tags=True,    tag_syntax=TagSyntax.HTML,    unsupported_tags=UnsupportedTagPolicy.ERROR,)

Cross-format syntax conversion

`TagSyntax`	Processing
`NATIVE`	Reuses the parser's native syntax and payload.
`HTML`	Maps portable semantics to safe HTML elements and filters unsafe attributes/URLs.
`TMX_14`	Renders opening, closing, and standalone codes as TMX `bpt`, `ept`, and `ph`.
`XLIFF_12`	Renders paired and standalone codes as XLIFF 1.2 `bpt`, `ept`, and `ph`.
`XLIFF_20`, `XLIFF_21`	Renders codes as `sc`, `ec`, and `ph`.
`IDML`	Maps supported ranges to `CharacterStyleRange`.
`DOCX`, `PPTX`	Identify native Office syntax; general cross-rendering is not promised. Office export/regeneration performs package-aware reinsertion.

Portable semantics recognize strong/bold, emphasis/italic, link, line break, image, and variable codes; everything else remains generic. Writers also consume TieData directly—for example, the XLIFF writer emits paired codes as bx/ex and standalone codes as x, while the TMX writer preserves supported native elements or emits safe TMX placeholders.

Integrity validation and stale-part protection

Segment.validate() rejects a code referenced twice, a dangling CodePart, an unreferenced code, or an incomplete open/close pair. Destination writers may add stricter rules; the TMX writer checks proper nesting for structural hi and sub pairs. legacy_parts_match_text() requires the concatenated TextPart values to equal the current plain string. If an application changes text without updating its parts, segment_from_legacy() and the standard writers intentionally fall back to plain text instead of reusing stale tags around the wrong words.

validate_tags.py

from lokit.types import TagSyntax, segment_from_legacy
unit = document.data["welcome"]if unit.tags is not None:    segment = segment_from_legacy(        unit.source,        unit.tags.source_parts,        unit.tags.source_tag_map,        syntax=TagSyntax.TMX_14,    )    segment.validate()

Unsupported tags and HTML safety

The default UnsupportedTagPolicy.ERROR raises TagIntegrityError when a destination cannot safely represent a code. PLACEHOLDER emits an escaped lokit-code marker; DROP removes the code but keeps surrounding text. HTML rendering uses a safe element set, strips event-handler, style, and namespace declaration attributes, and rejects javascript:, vbscript:, and HTML data URLs in href or src.

Sanitized and raw dictionary projections

StringMode.SANITIZED returns plain strings. StringMode.RAW reconstructs native inline XML from current parts, original names, attributes, namespace data, and payloads for .lokit, TMX, and XLIFF interchange projections. Raw projection raises TagIntegrityError rather than silently inventing markup when parts are stale, a reference is dangling, or the native name is unavailable.

raw_inline_xml.py

import lokitfrom lokit.types import DictField, StringMode
rows = lokit.parse.to_dict(    "rich.xliff",    fields=(DictField.UNIT_ID, DictField.SOURCE, DictField.TARGET),    strings=StringMode.RAW,)