Skip to content

OFM Database Modifications for OFMX 1.0.0

Created by: svoop

⚠ See related issue ofmx#1 (moved) as well.

This draft describes how contributors exchange data by use of OFMX documents as discussed with @philhug.

Terms

  • importer – External source of OFMX (e.g. the "aip2aixm" tool)
  • editor – The OFM Editor
  • database – The OFM database
  • consumer – External consumer of OFMX
  • core feature – See OFMX wiki. Core features in this draft include all of their auxiliary features as well.

Import

An external data source is translated to OFMX by an importer and then uploaded to the editor. The editor aggregates and validates the data which is then written to the database. Workflow:

  1. OFMX is imported to the editor and translated into a DOM.
  2. All @mid attributes are removed from the DOM.
  3. @mid attributes are generated on every *Uid element.
  4. The editor diffs @mid lists to detect additions and deletions.
  5. A core feature is...  – not flagged as protected: The editor removes all deletions from the database.  – flagged as protected: The editor logs all skipped deletions.
  6. The editor generates a payload digest of every core feature and builds an ordered map.
  7. The editor diffs the payload hash digest maps to detect modifications.
  8. A core feature is...  – not flagged as protected: The editor updates all modifications on the database.  – flagged as protected: The editor logs all skipped modifications.
  9. The editor displays the log to the importer.

Export

The editor reads the data from the database, assembles it into an OFMX file, validates it and delivers it to the consumer. Workflow:

  1. The editor reads the data from the database and assembles it to OFMX.
  2. The @mid remain in the markup (to facilitate consuming the data). It is important to inform the consumers that @mid may only be used to match features contained in this OFMX document since @mid may change across different exports as the data and therefore *Uid evolve.
  3. The OFMX is validated and on...  – pass: The OFMX is delivered.  – fail: An error is logged, displayed and an admin is notified.

Moving Parts

Namespaces

Every OFMX document contains a UUID in OFMX-Snapshot@namespace. The database stores the data in such a fashion as to guarantee, imports to one namespace cannot alter the data from another namespace in any way. The first upload with a new UUID creates the corresponding namespace. All subsequent imports have access to the data stored within this namespace only.

*Uid and @mid

OFMX identifies and associates features by use of *Uid tags. To facilitate the matching of *Uid, the editor generates @mid when the OFMX is imported:

The editor reads the OFMX into a DOM. In a first pass, it removes all @mid attributes already present anywhere in the DOM. In a second pass, it generates @mid attributes for every *Uid element by hashing the *Uid payload into a UUID. From this point forward, the editor can associate features easily with *Uid@mid. The @mid remain in the OFMX document.

⚠ *Uid elements are usually six characters long (e.g. AhpUid), however, there is one exception: AseUidSameExtent found as part of Adg. To find all *Uid element nodes, the DOM therefore has to be scanned e.g. with /^[A-Z][a-z]{2}Uid.*$/.

Payload Digest

The editor builds a digest over the entire payload of a core feature (including its auxiliary features) by use of a hash function such as MD5. Payload digests are written to the database, but are not part of the OFMX document!

⚠ Payload digests may be an overkill if it's possible to otherwise reliably diff the complete payload of a core feature.

Detection of Additions and Deletions

Additions and deletions are detected by diffing the *Uid@mid of all core features present in the database versus present in the import.

Build two ordered lists of all core feature @mid e.g. as "feature#mid":

Ase#19027994-d89d-475b-989d-f137be2ed6f5
Ase#54ca8ef9-0dbe-4cf1-9a09-a9f28c2e6a13
Org#8f570e02-4052-43e9-9123-8a64c3a7c141
(...)

Diffing the two lists disclose additions and deletions.

Detection of Modifications

Detection of modifications are detected by diffing the payload digests of all core features present in the database versus present in the import.

Build two ordered maps of all payload digests to core features e.g. as "digest -> feature#mid":

814df75121ec307a61dd05994016352e -> Ase#19027994-d89d-475b-989d-f137be2ed6f5
86c19cac55ef2e2a4d953504d45d1d4c -> Org#8f570e02-4052-43e9-9123-8a64c3a7c141
e152476ea80f6fe49dcdb8da9c6b582c -> Ase#54ca8ef9-0dbe-4cf1-9a09-a9f28c2e6a13
(...)

Diffing the two lists of map keys disclose modifications and point to the modified core feature.

Protection of Manual Edits

Whenever a core feature is manually edited with the editor (e.g. fixing typos in the remarks, fixing geometry etc), a protection flag is set on the database. When this flag is set, the core feature must not be overwritten by subsequent imports anymore. A message should be logged instead instructing the uploader that this particular core feature has to be modified manually.

However, protected core features may be deleted if they no longer appear in a subsequent import.

Edge Cases

Some core features have to be dealt with manually at all times. Say, a TMA airspace of an airport in region A extends into another region B. This TMA may present in only one or both relevant AIP. Or it's split in two parts present on the two relevant AIP.

OFM defines rules as to which region takes ownership over such edge cases which is why they usually have to be entered manually by use of the editor. The importer must assure not to include edge cases such as a partial TMA in the above example.

Edited by Sven Schwyn