Model Layer

Understand MobX

TerriaJS's model layer is based on MobX, so it is essential to understand MobX. In particular, the MobX Concepts and Principles is a short and enlightening read.

It is also helpful to understand what MobX reacts to. We recommend giving it a read before you start working on TerriaJS's model layer, and then read it again any time MobX isn't behaving the way you expect it to.

Types of classes in the Model layer

• Traits: Define the configurable properties of a each stratum of a model object. Traits classes have no logic or behavior, only properties. Note that traits classes are often built by mixing in several sets of traits. Example: WebMapServiceCatalogItemTraits.
• Models: Concrete, usable model objects representing things like map items and groups in the catalog. They are composed of mix-ins plus some model-specific computed properties and logic. Example: WebMapServiceCatalogItem.
• Mixins: Provide computed properties and behavior intended to be mixed into model objects to add capabilities to them. Example: GetCapabilitiesMixin.
• Load Strata: (singular: Load Stratum) Strata that provide values for a subset of the properties defined on model's traits class, usually by loading them from an external source such as WMS GetCapabilities service. Example: GetCapabilitiesStratum.

Reactivity

The TerriaJS model layer generally looks and feels - at least on the surface - like a traditional object-oriented design. An item or group in the catalog, for example, is an instance of a class, such as WebMapServiceCatalogItem. Model objects are mutable, meaning we can modify them in-place without the need to copy them first or anything of that sort. This is different from the approach used in Redux, which requires that the application's state be represented with immutable objects and that transitions to new states happen in a very controlled fashion.

Unlike most OO applications, though, the TerriaJS model layer is reactive. Virtually all properties in the model layer are declared @observable. An observable property is just like a normal property; you can get and set its value as normal. On top of that, however, an observable property raises an event when it changes.

Many classes also define @computed properties. Computed properties have a getter and optionally a setter. The getter is invoked once the first time the property is accessed and the value is memoized (cached). Further accesses of the property will get the memoized value.

The interesting part is that MobX automatically keeps track of the set of observables (including other computed observables) that were accessed during the execution of the getter. If any of those properties change, the computed observable's getter will be invoked again and the new value memoized. This new invocation may access different observable properties than it did the first time around, and it is those new properties that will trigger any future re-evaluations of the computed property. We can subscribe to change notification on computed properties in the same way we subscribe to regular observable properties.

Rather than directly subscribing to property changes, MobX applications typically use reactions. A reaction, such as one created with the autorun function, is a bit of code that, when it runs, accesses some observable and computed properties and does something, such as modifying the non-reactive Cesium and Leaflet mapping layers. If any of the properties accessed change in the future, the reaction will run again.

Avoid reactions and other side effects

A reaction, as described above, is a type of side-effect. When a property changes, "something" happens. That "something" may be surprising to whoever wrote the code that modified the property, and so this is an extremely common source of bugs.

A good rule of thumb is this: all properties should either be simple data properties (i.e. no getter or setter), or they should have only a getter and that getter should be a pure function of other properties. A pure function is one that can be called any number of times with no observable side effects, and that returns the same value for the same inputs every time it is called.

Computed properties with setters should be avoided. In our old architecture, we frequently used computed properties with a setter to model properties that are configurable by the user, but that have a default value computed from other properties when there is no user-specified value. In the current architecture, this is better modeled by:

• Defining the configurable property on the Traits class.
• Defining a computed property on the derived Model class that accesses super.propertyName (if mixins should be allowed to define the property) or this.flattened.propertyName (if only a directly-configured property value should be used) and, if that is undefined, computes and returns the default value from other properties.

If a property does have a setter, it should obey these laws:

1. You get back what you put in: If you set the value of the property and then get it, you should get back the same value you set.
2. Putting back what you got doesn't change anything: If you get property's value, and then set it with that same value, it doesn't change anything.
3. Setting twice is the same as setting once: There should be no observable effect from setting a property to the same value twice.

Note: These are the Lens Laws from functional programming.

We need to be more precise about what we mean by "same value" in the laws above. For primitive types (strings, numbers, booleans, null, and undefined), "the same" means that comparing the values with === returns true. === should also return true for references to model objects and other mutable, reactive objects. Arrays should be MobX reactive arrays where each element in the array follows the rules stated here.

For other types of objects, such as Cesium's mutable JulianDate, Cartographic, and Cartesian3 types, "same value" means that the instances should conceptually represent the same value (e.g. using the equals function on the two instances should return true), but the instance may be different. In particular, properties of these types should declare their type as Readonly<JulianDate>, Readonly<Cartographic>, or Readonly<Cartesian3>. The objects should be copied on set. Returning a frozen object (e.g. Object.freeze) from get may help to prevent bugs, particular if some clients of the property are expected to not be using TypeScript.

Scenarios for avoiding side effects

Value comes from loading metadata (e.g. GetCapabilities), but users should be able to override the loaded value by specifying it explicitly in the catalog file or UI.

Create a load stratum for the values loaded from metadata. Example: the rectangle property on WebMapServiceCatalogItem's GetCapabilitiesStratum.

Value can be specified in the catalog file or UI, but if it's not, use a default.

The default can be applied by overriding the property in a mixin or model class, accessing super.propertyName, and returning the default value if it is undefined. The default value may be computed from other properties if desired. Example: legendUrls property of WebMapServiceCatalogItem.

Single source of truth

Another good rule of thumb is that there should always be a single source of truth for any piece of information. We run into trouble when the same information is stored in multiple places, even if it's stored in slightly different ways. When one is updated, we'll either need to remember to update the other, or we'll need to set up a reaction or other side-effect to automatically keep the other in sync.

A much less error-prone approach is to make one of the two places the information is needed a @computed property, computed from the other one. This is usually pretty straightforward, but it gets more difficult when one of the two places is part of Cesium, Leaflet, D3, or some other non-reactive part of our application. The solution here is to adapt it to be reactive. There are two broad approaches for this, depending on the direction of information flow:

MobX observable is the source of truth, a non-reactive part of the application must be kept in sync

This is perhaps the only place it is justified to use a MobX autorun. The autorun should scoop up all the information it needs from the reactive world and push it into Cesium, Leaflet, D3, etc. This should be done as close to the non-reactive world as possible, and it must consider performance. For example, an easy way to keep Cesium's list of imagery layers in sync with the TerriaJS workbench is to create an autorun that removes all the layers and then re-adds the imagery layers for all catalog items on the workbench. This would "work", but even small changes would cause the globe's imagery layers to vanish and then reload.

Instead, Cesium's current state should be modified to match the desired state derived from the workbench. A useful illustration of an extreme version of this principle can be found in how React interacts with the browser DOM. React components produce a desired state of the DOM each time their render method is called. The magic of React is that it efficiently updates the DOM to match that desired state. Fortunately, we are rarely trying to update state that is as complicated as the browser DOM, so we don't need to use techniques as sophisticated as React's. The result, though, should be similar.

Non-reactive property is the source of truth, reactive (MobX) world must be kept in sync

MobX provides Atoms, which can be used to create a reactive view of non-reactive properties. With this we could, for example, create a property that retrieves the current Cesium clock time and that will notify any interested parties when that time changes. Behind the scenes, we use an Atom and a subscription to Cesium's clock tick event to trigger reactions when the clock time changes.

In this scenario, consider whether you could instead let the reactive world take ownership of this property and be the source of truth. Events in the non-reactive world may still trigger changes in this property's value, but all clients - including the non-reactive ones - would query the reactive property whenever they need this piece of information.

Put as much logic as possible in the model layer

We want to keep the UI as small and simple as possible, because:

• The UI is the hardest part of TerriaJS to test with automated tests. If it has lots of complicated logic, we will need to spend a lot of time writing tests for it to ensure that it behaves correctly. A very simple UI is easier to test, and is perhaps so simple that automated tests for it are not needed at all.
• We would like everything that can be done by a user interacting with the UI to be doable programmatically as well. If the UI encodes complex rules or has its own states, this may not be possible, or doing so may require duplicating complicated "business logic" that already exists in the UI.

Therefore, whenever possible, TerriaJS logic should be in the Model layer instead of in the UI. The UI should be a pure function from Model state to React components, and actions that simply execute functions or change a small number of properties in the Model layer.

Evaluate observable properties as late as possible. In particular, avoid getting the value of an observable before starting an async operation and using it when it completes.

Defining Properties

A few notes on defining properties in Model classes:

• Set only traits: All settable properties of a Model should be in its Traits. Because Traits are the only properties that are serialized/deserialized for catalog configuration and for sharing, settable properties that are not part of Traits prevent us from being able to completely recover application state. The only exception to this rule is for highly transient properties, such as whether a load from a remote source is currently in progress.
• Covariance: If you override a gettable property in a derived class, its type must be covariant with the base class type. That is, it is fine of the derived class property returns string while the base class property returns string | undefined. And it is fine if the derived class returns Dog while the base class returns Animal. But it is not ok if this relationship is reversed. You shouldn't really have any settable properties, but if you do, the types of such properties must be identical in base and derived classes.
• Equals: Pay attention to the comparer/equals to use with observables to determine if a new value is equal to an old one. The default equals is usually fine for primitive types (e.g. string, number, boolean), observable arrays, and objects whose properties are themselves observable (e.g. Traits). But for other types, especially Cesium types like JulianDate, Cartographic, and Cartesian3, it is essential to specify an equals. Typically this looks like this: @computed({ equals: JulianDate.equals }).

Traits

Creating new Traits

Correct:

export class SomeNewTraits extends ModelTraits {
  ...
}

Extending existing Traits classes

Never extend existing Traits classes directly - instead use the mixTraits() function.

For example

Correct:

export class SomeExtendingTraits extends mixTraits(DimensionTraits) {
  ...
}

Incorrect:

export class SomeExtendingTraits extends DimensionTraits {
  ...
}

If Traits are extended directly, trait properties may leak into the super class. For more details see traits in depth.

Time

Time-varying Models have their own currentTime, startTime, stopTime, etc. properties. The currentTime is the property that the dataset should use to determine what to display (i.e. what time to show).

Datasets that are synchronized to the timeline clock are listed in terria.timelineStack.

Note: In our old architecture there was a `useOwnClock` property, which no longer exists in the new architecture. Instead, datasets that would have been `useOwnClock=false` should now be added to the `timelineStack`.

On tick of the timeline clock, the TimelineStack will the current time and paused state to all datasets it contains. It will copy changes to startTime, stopTime, and multiplier to the "active" (top) dataset in the timeline stack.

When a dataset becomes the top of the timeline stack, or the top dataset's time-related properties change, the currentTime, startTime, stopTime, and multiplier properties are copied to the timeline's clock.

ReferenceMixin

ReferenceMixin is used to create models that are references to other models. For example, a MagdaReference is a model that points to a particular record in a Magda catalog. The type of catalog item we need to use to access it is not known until we load that record from the Magda registry and see what kind of record it is (e.g. a WMS, GeoJSON, etc.). In fact, it may not even be a catalog item, it might be a group.

Our roughly object-oriented approach makes it difficult for a MagdaReference to become another type after the Magda registry is read. But even if that were possible, we need to preserve access to the original traits and behavior that told us how to access the registry, too. For one thing, we'll need this to reload from the registry later.

So, instead of trying to make a model change type upon load, ReferenceMixin creates/references a separate model, called the target, that is not resolved until the reference is loaded.

There are two types of references

• Strong references
• Weak references

Strong references

These are a one-to-one mapping from a reference to it's target. The target can only have one reference, and it is aware of it's reference (through the sourceReference property).

Some rules

• The target model of a ReferenceMixin may be obtained from the target property, but it may be undefined until the promise returned by loadReference resolves. It may also be stale if a relevant trait of the reference has changed but loadReference hasn't yet been called or hasn't yet finished.
• For simplicity, a particular model may only be the target of a single reference. Multiple references cannot point to the same target. The reference that points to a particular model may be obtained from the model's sourceReference property.
• The uniqueId of the target model must be the same as the uniqueId of the model with ReferenceMixin.
• The model with ReferenceMixin may be in terria.models.
• The target model must not be in terria.models.
• The instance referred to by the target property should remain stable (the same instance) whenever possible. But if something drastic changes (e.g. we need an instance of a different model class), it's possible for the target property to switch to pointing at an entirely new instance. So it's important to only hold on to references to the ReferenceMixin model and access the target as needed, rather than holding a reference to the target directly.

Weak References

These are treated more like a shortcut to a model.
Here it is possible to have many-to-one mapping from references to a target. The target is not aware of it's references (sourceReference is not set).

Weak references are determined by weakReference = true.

The main use-case for weak references is the CatalogIndex - which provides for the complete tree of the catalog with a subset of model properties (eg id, name, memberKnownContainerUniqueIds...). This can be used to search and locate catalog members which may be nested in multiple references/groups (for example MagdaReference -> WebMapServiceCatalogGroup -> WebMapServiceCatalogItem).

AsyncLoader

The AsyncLoader class provides a way to memoize (of sorts) async requests.

AsyncLoader accepts an async function which can be use to do the following:

1. load data from an asynchronous service
2. transform the data into something that can be stored in 1 or multiple observables
3. set those observables.

It works by calling an async function forceLoadX() in a @computed called loadKeepAlive. This @computed will update if observables change that were used in forceLoadX(). Because we are using a @computed in this way - it is very important that no changes to observables are made before an async call.

This means that we can call asyncLoader.load() many times without worrying about

forceLoadX() shouldn't be called directly - instead you should use asyncLoader.load() method - for example in CatalogMemberMixin we have

• the abstract method forceLoadMetadata
• loadMetadata() which wraps asyncLoader.load()
• loadMetadata() can be called as many times as needed
• See CatalogMemberMixin example

A correct example:

async function forceLoadX() {
    const url = this.someObservableUrl;
    const someData = await loadText(url);
    runInAction(() => (this.someOtherObservable = someData));
}

This function will only be called again when someObservableUrl changes.

---

If there is any synchronous processing present it should be pulled out of forceLoadX and placed into 1 or multiple computeds.

An incorrect example:

async function forceLoadX() {
    const arg = this.someObservable;
    const someData = someSynchronousFn(arg);
    runInAction(() => (this.someOtherObservable = someData));
}

Instead this should be in a @computed:

@computed
get newComputed {
  return someSynchronousFn(this.someObservable);
}

---

Other tips:

• You should not nest together AsyncLoaders. Eg. ts async function forceLoadX() { await this.forceLoadY(); }

For more info, see lib\Core\AsyncLoader.ts

CatalogMemberMixin example

CatalogMemberMixin contains an AsyncLoader, which uses the followingfunction which must be implemented by a class:

/** Calls AsyncLoader to load metadata. It is safe to call this as often as necessary.
 * If metadata is already loaded or already loading, it will
 * return the existing promise.
 *
 * This returns a Result object, it will contain errors if they occur - they will not be thrown.
 * To throw errors, use `(await loadMetadata()).throwIfError()`
 *
 * {@see AsyncLoader}
 */
async loadMetadata(): Promise<Result<void>> {
  return (await this._metadataLoader.load()).clone(
    `Failed to load \`${getName(this)}\` metadata`
  );
}

/**
 * Forces load of the metadata. This method does _not_ need to consider
 * whether the metadata is already loaded.
 *
 * You **can not** make changes to observables until **after** an asynchronous call {@see AsyncLoader}.
 *
 * Errors can be thrown here.
 *
 * {@see AsyncLoader}
 */
protected async forceLoadMetadata() {}