Learning the National Household Model

Table of Contents

If you're reading this, maybe you are interested in learning to use the National Household Model (NHM). This guide should supply a fairly complete understanding of what the NHM is and how best to use it; however, it isn't exhaustive, and there are some references you will need to grasp if you want to fully follow how the model works.

This document starts with an overview that describes roughly how the model works, without getting into the specifics of the language, and then a section on reading the language and using the manual before you get to actually doing anything. Many people find it a bit difficult to learn something without diving in immediately and wildly pushing some buttons; if you think this describes you, you may want to skim-read the first two parts and go the first simple scenario. However, understanding how to read scenario language is ultimately essential to being able to use the model well, and skipping the required learning is likely to prove a false economy.

Many of the sections below introduce features of the model which are quite abstract when taken on their own, but are useful when combined to achieve complex effects. We have tried to include motivating examples as you go along, but in some cases a realistic example would involve the use of several different features in concert, and we have tried to avoid producing examples which depend on understanding material not yet discussed. If as a result you find that the examples in some sections are too abstract, have a look in the 22 section near the end. It contains several realistic examples which bring together the primitive features which are each explained in their own section.

1 Overview

The NHM is a programming language for a discrete event simulator that simulates energy-related behaviour for domestic dwellings using a SAP-like energy calculation. The dwelling data is mostly derived from the EHS (although other data may be used). Let's look at the italicised parts in a bit more detail:

1.1 A programming language

The NHM is programmable; it might be better to think of it as a tool for making models rather than a particular model, as most of the things you will want it to simulate you will have to explain to it in your program. It does not simulate very much apart at all from what your program instructs. It is designed to make certain kinds of simulation program easy to write, so they can have a short description, but you will still have to write something.

For those readers who have used programmable tools before, the NHM will have some degree of familiarity; you write a scenario, which is just some text obeying a special syntax, and then ask the model to run your scenario. It simulates what it's been instructed to simulate, and produces any outputs (typically big tables about what happened) that it's been instructed to emit. There is more about the syntax in the section on reading scenario language

You can then take the outputs and use them to inform policy decisions.

1.2 Discrete-event simulation

A discrete-event simulation is one where time passes in fits and starts between events, and nothing happens otherwise. An event is an instantaneous change in the state of affairs that happens at a particular time. Other types of models might use differential equations to represent values which change continuously in time, but in a discrete event model everything stays the same until an event comes along, at which moment there is a corresponding change to how things are. Then everything stays in its new arrangement until the next event, and so on.

The model uses the syntax in your scenario to set up the sequence of events which are going to happen in the simulation. This sequence (the event queue) is a list of 'things to do', sorted by the date on which they should be done. The events that a scenario defines usually describe a command for changing something, like "take a random 10% of semi-detached houses in the south-west, and offer each one some loft insulation", or a command to make some output, like "record how many semi-detached houses in the south-west have loft insulation".

Once the model has read your scenario and set up its future events, it simply goes through the list and does them in order until it's either reached the maximum date you want to simulate, or there are no events left to think about. An important thing to understand here is that at a particular point in simulated time, the model has an idea of how the world is. When an event happens, it can change this representation of the world, so that the next event is presented with a different state of affairs. This is the analog of the arrow of time which seems to exist in reality - things which happen in the past may affect the future, but not vice-versa.


Figure 1: The sequence of events. Each circle represents a state of affairs, and each arrow the execution of an event. S0 is how things are to start with; then we insulate some 10% of the houses in S0 (E1). S1 is how things are once some insulation has been handed out. Next, from S1 E2 takes place: we hand out new boilers for some other 10% of the houses, producing S2. Note that we would expect 1% of the population to get lucky and get both measures, if the two samples are independent. The size and cost of a boiler can depend on how well-insulated the house is, so the fact that E1 has affected some house may affect the outcome of E2 for that house, and if we simulated E2 followed by E1 the result may be different.

1.3 Dwellings and the stock

The state of affairs mentioned in the previous section is mostly defined by the condition of a collection of dwellings; a dwelling is a place where one or more households reside, typically a flat or a house (as distinct from a building, which may contain several dwellings, or a household, of which a dwelling may contain several).

The initial state (S0 in 1) is defined by a stock, which is a file containing descriptions of the dwellings that should exist to start with, which we will call cases. The stock which the model will use is specified in the scenario, so the same scenario can be run against different initial conditions by referring to a different stock.

Each case in the stock has a weight associated with it, which indicates how many dwellings in the real world correspond with that description. For example, there might be 5,000 dwellings which are to be represented by a description of a medium-sized flat in London, whereas there may only be 500 dwellings represented by a description of a detached house in the North of Scotland. The total weight of all the cases in the stock should equal the number of dwellings in regions which the stock covers.

1.3.1 Weighting

Although a particular population of dwellings may be the same from the point of view of the stock (and so be represented by a single case with the appropriate weight), these dwellings might have different experiences during the simulation. For example, if a case represents 1000 dwellings, and they all have a 10 year old boiler, we might expect each of those dwellings' boilers to have a 10% chance of needing replacement in the next year. At the end of the year, some subset of the 1000 dwellings1 will have had to get a new boiler, and will no longer be in the same condition as their original cohort.

In an ideal world, the model would handle this by simulating each dwelling independently: the original case of 1000 would be modelled as 1000 distinct dwellings, each able to take a different course. Unfortunately with current computers the NHM cannot efficiently simulate the ~25 million dwellings that would make up a national stock. Instead, the model takes a position somewhere between the two extremes of simulating cases (which are of varying weights, some quite large) and simulating individual dwellings (of which there would be too many). Each case is divided up into several simulated dwellings, using a number called the quantum; a simulated dwelling (afterwards just referred to as a dwelling, except clarification is required) will still represent multiple real dwellings, but (a) most simulated dwellings will have roughly the same weight, and (b) the weight can be smaller than the large case weights in the stock.

For a case of weight \(w\), using a quantum \(q\), \(\lfloor w/q \rfloor\) dwellings will be created with weight \(q\), and one extra dwelling will be produced with weight \(w - q * \lfloor w/q \rfloor\), where this is not zero (\(\lfloor x \rfloor\) is the greatest integer below \(x\)).

To give an example of how this works, imagine a stock summarised by the following table; each row is a case, with a unique identifier (and all the other information stored about a a case omitted, on which more later):

Case ID Weight
\(A\) 1000
\(B\) 100
\(C\) 500
\(D\) 700

Now let us see what dwellings would be simulated with different quanta2:

  • With a quantum of 1000, the 4 dwellings produced are:

    \(A_1: 1000, B_1:100, C_1:500, D_1:700\)

    This is the extreme where cases and simulated dwellings have a 1:1 relationship.

  • With a quantum of 500, the 6 dwellings produced are:

    \(A_{1,2}: 500, B_1:100, C_1:500, D_1:500, D_2:200\)

    Wholly divisible cases \(A\) and \(C\) produce simulated dwellings with a weight of \(500\); \(B\) and \(D\) each produce a remainder with weight less than the quantum.

  • With a quantum of 100, the 23 dwellings produced are:

    \(A_{1..10}:100, B_1:100, C_{1..5}:100, D_{1..7}:100\)

    In this case all the simulated dwellings are of equal weight, as all cases are neatly divisible, and each simulated dwelling represents 100 real dwellings. Consequently, the smallest 'unit of decision' is 100 identical houses - changes can only happen to 100 houses at a time, all at once.

Note that decreasing the quantum increases the number of simulated dwellings, but decreases their weights so that the gross weight is maintained. As the number of simulated dwellings increases, the running time and memory requirements of the model increase. Since the main impact of quantisation is on random processes, for some kinds of simulation it is sensible to use a large quantum so that the model will run more quickly; for example, investigating 'technical potential' may be fine using 1 dwelling per case, whereas a policy which is expected to affect very small populations might require a correspondingly small quantum3.

1.3.2 What is in a stock; creating stocks

We have said that a stock contains descriptions of dwellings, but without any more detail. Each dwelling is considered as a hierarchical structure, composed of various different parts:

  • Some basic attributes, like region, and built form
  • Information about the technologies in the house
    • This is a description of the heating, lighting and water heating systems
  • Information about the structure of the house
    • This describes the physical condition of the house, in terms of :
      • a series of storeys, each of which has some walls, which have u-values, and so on.
      • four elevations, each describing the doors and windows in that elevation
  • Information about the people in the house, and finally
  • Some additional properties, which are miscellaneous extra data that the model doesn't really understand

Creating these hierarchical descriptions is itself quite an involved task; there are some parts of the NHM collectively called the Stock Importer which convert some different sorts of input data into the form required for a stock. These input data are either the various national surveys (the English Housing Survey and so on), or an intermediate set of tables called "DTO" tables. The format of these tables and a full description of the process are out of scope for this document; they are documented in the manual.

1.4 Exercise: thinking about weights

Imagine a stock which contains cases as described in table 1.

Table 1: A summary of an imaginary stock containing 5 cases.
Case ID Weight Floor area
C1 100 1000
C2 400 1500
C3 300 800
C4 500 1200
C5 100 300
  1. What simulated dwellings and what gross weight would be produced using a quantum of:
    1. 50
    2. 100
    3. 150
    4. 500
  2. Here is a procedure for sampling from the set of simulated dwellings:

    To sample dwellings of weight \(w\), shuffle the dwellings into a random order, and then take as many from the beginning of the order up to but not exceeding total weight \(w\).

    If we take a sample of weight 500, what is the expected floor area, and the variance in expected floor area, for a quantum of:

    1. 1
    2. 50
    3. 100
    4. 150 An analytical answer may be unfeasible here, and for 500, but you can simulate this with Excel quite easily
    5. 500

1.4.1 Solutions

  1. Writing \(n\) dwellings from case \(C\) with weight \(w\) as \(n \times C(w)\):
    1. \(100 \times C1(1), 400 \times C2(1), 300 \times C3(1), 500 \times C4(1), 100 \times C5(1)\), with a gross weight of 1400
    2. \(2 \times C1(50), 8 \times C2(50), 6 \times C3(50), 10 \times C4(50), 2 \times C5(50)\), with a gross weight of 1400
    3. \(1 \times C1(100), 4 \times C2(100), 3 \times C3(100), 5 \times C4(100), 1 \times C5(100)\), with a gross weight of 1400
    4. \(1 \times C1(100), 2 \times C2(150), 1 \times C2(100), 2 \times C3(150), 3 \times C4(150), 1 \times C4(50), 1 \times C5(100)\), with a gross weight of 1400
    5. \(1 \times C1(100), 1 \times C2(400), 1 \times C3(300), 1 \times C4(500), 1 \times C5(100)\), with a gross weight of 1400
  2. For the expectation of the equally divisible situations, we can do this analytically

    Consider the random variable produced for the mean floor area of drawing 700 dwellings when the quantum is 1; it is well-known that the sample mean (which is what we are computing) is an unbiased estimator of population mean, and that shuffling produces an unbiased sample, so this is just the expected floor area of the population (about 1121).

    Now consider the case where the weight is wholly divisible for all the cases (say a weight of 50); rather than taking the sample mean we are taking a weighted sample mean, but by the linearity of expectations this is the same thing (since every sampled dwelling has weight 50).

    For the unequally divisible situations, the analysis is too long to write out here. However, it is easy to approximate computationally. A quick simulation produces the following table for the data above:

    Quantum Mean sample mean Deviation in sample mean
    1 1121 9.0553851
    50 1120 62.633857
    100 1123 95.739229
    150 1118 110.45361
    200 1115 126.01587
    250 1107 144.81367
    300 1118 158.23716
    350 1105 188.58685
    400 1091 207.32583
    450 1095 219.25328
    500 1083 236.59248

    The key points to note are: (a) the mean is reasonable4, and (b) the deviation in the mean grows

1.5 SAP-like energy calculation

The NHM is a domestic energy model. It uses the dwelling descriptions in the stock to predict dwellings' energy uses using the BREDEM model, which is the basis for SAP. Many of the odd results which can be produced by the NHM are attributable to features of the SAP method, in which the meaning or use of certain terms can be quite counterintuitive. We would recommend that you read the SAP document before using the NHM in anger, or at least look through the SAP worksheet5. We also provide a brief overview here, which also explains a little bit of building physics for those new to the entire area.

A SAP calculation breaks down into a few parts, which are (in rough order of signifiance):

  • Space heating
  • Water heating
  • Lighting and appliance use

1.5.1 Heat demand; Newton's law, mean temperatures, u-values

Heat demand is mostly calculated using a version of Newton's law of cooling, which states that the rate at which heat flows from one body to another is proportional to the difference in temperature between them. Writing this as an equation

\[ H = k \cdot (T_1 - T_2) \]

So if the two bodies are at the same temperature, there will be no net flow of heat, at 1 degree there will be \(k\) Watts, and at \(x\) degrees difference there will be \(k\cdot x\) Watts. The constant of proportionality \(k\) is known as the heat loss coefficient or just heat loss; a higher heat loss means that more energy is needed to keep your house warm. In SAP, \(k\) is composed of two parts:

  1. The fabric heat loss, which is to do with heat flow through the building's surfaces
  2. The ventilation heat loss, which is to do with heat lost due to air circulating between the inside and outside of the house

    The fabric heat loss is defined to be

    \[ \sum_s A_s \cdot u_s \]

    where for each external surface of the building \(A_s\) is the area of that surface and \(u_s\) is the u-value. From this you can see that if you double the u-value of a surface or its area you double its contribution to the heat loss. The ventilation heat loss is defined by a more complex rule, but is less likely to be significant; if you are interested, see the SAP worksheet.

    Once we know \(k\) for a dwelling, we are on our way to working out how much energy is needed to keep it warm; if the temperature inside is \(T_1\) and outside it is \(T_2\), then heat will be flowing out of the dwelling into the environment at \(H = k \cdot (T_1 - T_2)\). In an unheated dwelling this would reduce \(T_1\) and increase \(T_2\), and this process would continue until eventually their temperatures equalized6. In SAP, we presume that the dwelling's occupants operate their heating system to counteract this tendency, maintaining the internal temperature at \(T_2\). To do so, they must replace the loss \(H\) with an equal heat input to the building supplied by two sources:

    1. Converting primary energy into heat in the space heater (burning gas, or heating up a storage heater), and
    2. Incidental heat gains from other sources of heat like lights and appliances, and solar gains from the sun's radiation.

A little more is written about u-values and their relationship to insulation and r-values below, in How does the model think about insulation.

In this sketch we have not explained where \(T_1\), \(T_2\) or u-values come from; these are all important factors affecting energy use, so we will describe them a little:

  1. Internal and external temperatures

    SAP is a monthly energy calculation, so external temperatures are typically regional monthly averages; these values can be changed away from the SAP defaults in your NHM scenario.

    Internal temperature is more complicated, but because it is one of the significant factors for overall energy use it is worth understanding. The calculation is affected by several different parameters; looking only at the parameters might lead you to imagine a more sophisticated technique than is really used, but all of these parameters reduce to some adjustment of \(T_1\) in the equation above. The list of parameters is:

    1. Living area temperature

      This represents the intended temperature in the main part of the house when the heating is on.

    2. Rest-of-dwelling temperature / temperature difference

      The temperature in the rest of the dwelling can be defined as another temperature, or as a temperature difference from the living area temperature. In the second case, the difference is adjusted before being added.

    3. Living area fraction

      This is simply a number indicating what fraction of the floor area is the 'living area'; this value is computed in the stock import and stored by each house.

    4. Heating schedule

      The heating schedule is a pattern of on/off times for the heating system, by day of the week. Typically this is set to morning/evening heating on weekdays, and all-day heating at the weekend.

    5. Responsiveness of main heating system

      SAP defines heating systems to have a 'responsiveness'; as far as we know this is not a quantity with a measurable definition, but represents something about how the heating system responds to the required output, so a heating system with a thermostat is more responsive than one without.

    6. Type of heating system

      Some heating systems and combinations of heating controls will lead to an adjustment in the living area temperature, or the heating schedule. These kinds of adjustments are mostly detailed in SAP tables 4a,b,c etc.

    7. Thermal mass parameter

      The thermal mass parameter of the house is related to its construction and size, and used to determine the rate of cooling of the house; this affects the slope of the downward segments in figure 2.


    Figure 2 illustrates how all of these factors essentially boil down to a single mean value; this is why although the energy calculator refers to heating patterns, living area fractions and so on, it is not actually able to produce useful in-day load profiles, nor does it understand the difference between night and day temperatures, and so on.

1.5.2 Internal and solar gains

Some heat is produced within the dwelling but not by the heating system. For example, lights, electrical appliances, cooking, people's bodies and the hot water system all produce some heat as a side-effect of their operation. As well as this, some of the sun's heat is captured within the house, mostly through the windows.

In SAP, these gains are quantified by a variety of different rules, and all contribute to meeting some of the need for heat; as a result, the heat output required from a heating system is usually a bit less than the raw heat output required to maintain a sensible internal temperature. The amount of demand met by gains is modulated by a "gains utilisation factor", which appears to have the effect that increases in gains have a diminishing usefulness for meeting the need for heat.

In rough summary, the determinants for gains are:

Lights and appliances
Both of these are electrical in SAP, and all of the electricity they consume is taken to produce heat within the dwelling. The requirement is determined by a polynomial in the floor area of the dwelling. For lights, the demand is also modulated by a model of the sunlight coming in the windows.
Metabolic gains
Each person in the house is presumed to contribute a certain fixed wattage in gains.
Hot water gains
The production, use, and storage of hot water can provide a lot of gains. The main components of this are:
  1. Heat lost at the tap
  2. Heat lost in distribution to the tap
  3. Heat lost in the primary circuit, which is affected by whether the pipes are insulated or not
  4. Heat lost from any heat storage; this can be a large term if the house has a large tank with poor insulation
Cooking gains
The SAP score does not include cooking, but BREDEM does; the NHM includes a cooking model, which can be excluded from energy use figures if required (the documentation for house.energy-use explains how). Cooking demands are all converted into gains, and are modelled by a simple regression.

1.5.3 Hot water

The hot water calculation is quite complex, and for a full description you should read BREDEM or SAP. The demand for heat for hot water is calculated in a few steps:

  1. The base demand for hot water is calculated, which is a function of the number of occupants in the house
  2. The heat content of this water is calculated using the specific heat of water and an assumed monthly temperature increase from the mains water to the hot water that's produced
  3. To this base demand various additional demands are added to account for losses; the kinds of losses depend on the type of hot water system, but include
    • combi boiler losses, perhaps associated with the repeated heating of a combi boiler
    • distribution losses in the pipework of the house
    • primary circuit losses, in houses which have a hot water tank and hence a primary circuit
    • tank losses, in houses which have a hot water tank
  4. Finally demand met by any solar thermal system is netted off, using a complex rule which computes heat available from the sun, and meets some amount of the demand less than that, modulated by a utilisation factor. As the demand for hot water shrinks, so the effectiveness of solar hot water to meet a fraction of that demand shrinks, in this model.

1.5.4 Meeting demand; mysterious adjustments

Once all of these factors have been considered, a calculation of the fuel requirement can be made; for many heating systems which involve a boiler, this is complicated by the determination of an effective efficiency, which is a function of the winter and summer efficiencies of the boiler and the total heat output for space and water heating. This adjustment reflects the fact that a boiler may be more efficient if it is used more, perhaps because when the boiler is only producing hot water in the summertime it is more likely to be starting from cold.

This overview should give you a feel for the important factors in a BREDEM or SAP type calculation, but it does not go into any of the many quirks and details which can confound any direct understanding of its ways. Most of the key values we have mentioned (efficiencies, demand temperatures, responsiveness, hot water losses and so on) are adjusted by any of a number of footnotes, tables, subheadings and so on in the SAP document.

For example, when determining the hot water efficiency of a heat pump, if the heat pump is supplying all of the hot water its efficiency is reduced. Similarly, if a dwelling has the right combination of heating controls, for some heating systems the efficiency of the system will be changed, for some the demand temperature will be changed, and for some the final consumption will be multiplied by a scaling factor.

If you wish to have a good feel for this undeniably important part of the model, we can only recommend that you read the SAP document in order to know the ins and outs of these various adjustments and correction factors.

2 Reading scenario language

Language is often divided into syntax and semantics. Roughly speaking, syntax is the study of what forms are structurally valid or correct, with reference only to the parts of speech involved, and semantics is about the meaning associated with those forms.

For example, in the English language you might say that the words

In in in flask hath bear qualitative. Of! Of!. fruitsome noisesome sponge quadrilateral ambition

are syntactically invalid; no matter what interpretation you put on the nouns and verbs, so long as they remain nouns and verbs the words cannot be taken to form a valid sentence. There are some things we can see here that typically never happen in English; two occurrences of the word in normally do not follow one another, an exclamation point is never followed by a full stop and so on.

The (famous) sentence

Colourless green ideas sleep furiously

has perfectly acceptable syntax, but its semantics are all over the place; it is nonsensical to apply the adjectives colourless and green to the same referent, and the noun in this case (idea) is not the kind of thing that has a colour anyway, nor is it an actor which can sleep, and finally one might wonder about whether it is possible to sleep furiously. Syntax is often expressed with some sort of diagram giving a hierarchy which relates the different syntactic elements together. In this case, we could say something like ([colourless green] ideas) (sleep furiously), to indicate that colourless green is an adjective phrase affecting the noun ideas to give a noun-phrase, and furiously is an adverb used with sleep to give a verb-phrase, which combines with the noun-phrase to make a sentence.

Anyhow, English syntax is not the subject of this document, but hopefully this analogy helps, and it should illustrate how natural language has quite a formal structure that is made easy by familiarity rather than anything innate.

In this document we will first cover the NHM syntax in detail; then we will go into the semantics. Dividing syntax from semantics is, for many people, a mentally gruelling and unnatural thing to do, and there is every possibility that in reading this bit of the manual you will be thinking some of the following:

  • Why must I learn all these technicalities? Surely I could just get on with something?
  • Why does the author keep going on about the distinction between meaning and syntax? Surely it's all academic anyway?
  • Why do so many of these examples use nonsense words? What do they have to do with my work?
  • What's the point of all these parentheses and brackets and so on?
  • If octopuses could live on land, would we have them as pets like we do cats and dogs?

To answer at least some of these questions, it is helpful to try the mental exercise of 'playing computer': When a you read a human language, especially one you are unfamiliar with, you probably infer the syntax from what you already know about the meaning. You have enormous tacit knowledge about what kinds of words go where, and what a sentence is likely to be trying to say from its context; you can understand from the top down. Because the NHM language is for a computer to read rather than a human, none of this is helpful; instead it is a hindrance. The computer has no tacit knowledge, and no understanding of context; in fact is has no knowledge or understanding. Instead, it can only process the language by following a precisely defined set of rules, and a lot of those rules are the syntax; it can only form its model of the code from the bottom up. Where you can use your contextual understanding to resolve ambiguity, the computer must rely only on these rules. Learning the syntax is an exercise in trying to do this bottom-up reading yourself, so as to imagine how the computer is 'reading' the scenario. The better you are able to do this, the easier you will find it to be productive with the tool in the end. The syntax section of the manual tries to help with this by presenting questions of syntax without the context that you would normally reach for to form a top-down understanding.

2.1 Syntax

We now come onto the syntax for NHM scenarios; this defines what scenarios the model can understand, and how to read their structure.

The scenario language is a kind of special-purpose programming language. A lot of languages work by describing a series of steps to do1. Some parts of an NHM scenario form a list of steps, but as a rule it does not proceed by processing one line at a time in sequence.

Instead, the statements in a scenario should to be interpreted as instructions that have meaning given to them by the context in which they are written.

Every statement in a scenario one of four things:

  1. A literal, which is a word, number or other sequence of characters except spaces, or any sequence of characters between double quotation marks (including spaces)
    • hello is a literal
    • hello world is two literals, hello and world
    • 103.4 is a literal
    • 90% is a literal
    • 01/01/2014 is a literal
    • *my-long.thing!* is a literal
  2. An expression (or s-expression or symbolic expression), which is exactly one literal followed by zero or more other statements enclosed in parentheses ( and ). Expressions are usually commands to the model, and the first literal gives the name of the command. The rest of the things are arguments to the command; they belong to the command and affect what it does in some way. Different commands accept different arguments.
    • (hello) is an expression, which is invoking a command called hello
    • (+ 1 2 3) is an expression, which is invoking the command + with three literals 1, 2 and 3 as inputs. In this case + does what you would think as do numbers, and this is an instruction to sum the values 1, 2 and 3.
    • (measure.wall-insulation type: Cavity) is an expression invoking the command measure.wall-insulation, and giving it the arguments type: and Cavity. Arguments which end with a colon (:) tell the command something about the intended meaning of the following argument, so here Cavity is the type of measure.wall-insulation
    • (+ (* 1 2) 3) is an expression invoking + with two arguments; the first argument is another expression (* 1 2), and the second is the literal 3. Again, this means what you might expect: multiply 1 by 2, and then add 3 to the result.
    • (1 + 2) is an expression, but it does not mean "add 1 and 2", or "supply 1 and 2 to +". If it meant anything it would be supply the literals + and 2 to a command called 1; however the model has no command called 1.
    • ((1 + 3) * (4 + 5)) is not a legal expression, because the head is not a literal.
    • () is not a legal expression, because it has no head
  3. A list, which is a series of other statements enclosed in brackets [ and ]
    • [1 2 3 4] is a list of the four literals 1, 2, 3, and 4.
    • [(+ 1 2) 8] is a list with two things in; the expression (+ 1 2) and the literal 8
    • [] is a list with nothing in it
  4. An infix expression, which is a normal mathematical expression enclosed in braces { and }. Infix expressions must be space separated, because the names of NHM literals can contain characters that look like mathematical operators. Infix expressions are translated automatically into standard expressions when they are read. You don't have to use infix expressions anywhere if you don't want to, but in a few places they may make things easier to read.

    • {1 + 2} is an infix expression which is read as the expression (+ 1 2)
    • {1+2} is an infix expression which is read as the expression (1+2), which is not + applied to 1 and 2.
    • {9 * x + 33 / 7} is an infix expression which is read as the expression (+ (* 9 x) (/ 33 7)).

    When you read an expression, it can be helpful to mentally translate it into a description of an instruction; here are some examples: (we will cover what the terms written actually mean, and how to use them later)

     capex:(* 1.1 (size.m2)))

    This can be read as

    Use the command measure.wall-insulation, giving it a resistance of 0.01, a thickness of 50, and a capex determined by using the command * with 1.1 and the result of the command size.m2.

     to: (filter (house.region-is London)))

    Reads as

    Use the apply command, giving it the command measure.standard-boiler, and telling it that to is the result of using the filter command with the house.region-is command given London.

    These translations are ignorant of the meaning of all the terms. We have tried (and sometimes succeeded) to make it so that the names of commands and of their named arguments have a guessable meaning. In the examples given, you can hopefully determine something about what the commands written will actually do. Every command has a page in the manual which should describe what arguments it takes, and what it does with them.

    It can be helpful to think of s-expressions as hierarchies, a bit like an org-chart. For example, the expression

    (a (b c d)
       (e f g (h (i j) k)))

    could be illustrated as in figure 2. Just as in a real org-chart, entities which are higher up (like managers) are more important than those lower down (human resources); the managers direct the actions of their subordinates but because they are ideas people they do not have to understand the implied detail. They need only be concerned with the requirements they and the form of the outputs they will produce. In the examples above, you might say that measure.wall-insulation delegates the task of computing a number for the argument capex: to the expression (* 1.1 (size.m2)); in this case * does not 'know' that its output is to be used for capex, and measure.wall-insulation does not 'know' how to multiply two numbers together.


    Figure 2: A hierarchy describing the expression (a (b c d) (e f g (h (i j) k))). Arrows point to nodes which are more general, or important, or controlling. The node a controls b and e, and may determine their environment and use both their outputs; e can only see the effect of b insofar as a relates them.

2.1.1 Summary: basic syntax

  • Everything on a line following a ; is ignored as a comment
  • The text is broken up into parts according to
    • brackets and parentheses
    • white space (spaces or line breaks)
    • the colon character :
  • NHM syntax describes a hierarchy
    • The hierarchy is spelled out by parentheses ( and )
  • The first word after a ( names a command
    • In this a word is any sequence of characters except the ones mentioned above, e.g.:
      • this-is-a-word
      • this.is.a.word
      • this/is-a-word
    • Everything after the first word up to the corresponding ) is an argument to that command
  • Commands that are higher up in the hierarchy use or control lower down commands
    • Commands are usually not "aware" of their siblings at equal level in the hierarchy
  • So:
    • (this and that) is the command this applied to the arguments and, and that.
    • (example [1 2 3] hello) is the command example applied to the arguments [1 2 3] and hello
      • [1 2 3] is a list containing the three words 1, 2 and 3
    • (first (second x y) z) is the command first applied to two arguments:
      1. the command second applied to the words x and y, and
      2. the word z

2.1.2 Examples by comparison with prefix notation

If you find the syntax hard to read, you may find it helps to look at some equivalent syntax written as infix expressions versus the prefix form used by the NHM. Excel (for example) typically uses an infix syntax for formulae.

To make it easier to follow, these examples use some syntax which has the same meaning in Excel as it does in the NHM (once written in prefix form). However, most syntax does not have a directly translatable meaning like this, and you will need to learn the meanings of the commands used from the manual in order to read or write it.

Infix (Excel) Prefix (NHM)
1 1
1+2 (+ 1 2)
3*4 (* 3 4)
1+2+3*4 (+ 1 2 (* 3 4))
1/2 + 9 (+ (/ 1 2) 9)
MAX(1,2,3) (max 1 2 3)
MAX(2*2, 2+2) (max (* 2 2) (+ 2 2))
SIN(X)/X (/ (sin x) x)
SIN(COS(X)) (sin (cos x))
3>2 (> 3 2)
-sin(x) (- (sin x))

Things you might have noticed include:

  1. All functions must have parentheses, even the common +, -, * and /
  2. As a result, there is no precedence rule apart from the parentheses. There is no need to think about whether a*b+c*d is the product of a and b summed with the product of c and d or whether it is the product of a, the sum of b and c, and d.
  3. The functions which don't have special treatment in infix world like SIN and COS are written in the same way, except that:
    • The function's name is after the opening (, and
    • The function's arguments are separated by white space not commas (this unambiguous because of 1. and 2. above)

2.1.3 Named or keyword arguments

At the start of this section we saw some examples like (measure.wall-insulation type:cavity). This is the command measure.wall-insulation given two words, type: and cavity (remember that : is a character that splits up the input, like white space is). These two words supply a single argument called a keyword argument, or a named argument. The first word type: tells the command how it is to use the second argument, in this case cavity. This is in contrast to unnamed or positional arguments, in which the command determines how to use the argument by where it is. For example, when dividing two numbers in prefix notation we write (/ a b). This means to divide a by b; the / command knows that the first argument a is the numerator and the second argument b the denominator. If we swapped a and b around to get (/ b a) the / command would take b as the numerator and a as the denominator. This syntax is easy to remember and read only because we were all taught about / at school; it is a familiar function and we know it expects two parameters, and so on. Other model commands (like measure.wall-insulation) may also take many parameters, some of which may be optional; it would be difficult to remember which arguments were which if their meaning was given by their position. Named arguments solve this problem by letting us forget about the order of the arguments, and giving us a hint about meaning when we read a command (although the ultimate meaning is still best found from the manual). For example, returning to wall insulation we might see any of the following:

  • This

        type: cavity
        thickness: 50
        resistance: 0.01)
  • or

        resistance: 0.01
        thickness: 50
        type: cavity)
  • or

    (measure.wall-insulation type: cavity resistance: 0.01 thickness: 50)
  • or

    (measure.wall-insulation type: cavity resistance: 0.01 thickness: 50
                             capex: 500)

In each case, we know that measure.wall-insulation is being given:

  • a type: of cavity
  • a resistance: of 0.01
  • a thickness: of 50

In addition, in the final example there is another argument capex: being given the value 500.

In summary, if you see a word ending with a : within a command, you know it is a keyword, and that the thing following it is being given to the command as a keyword argument. That means that you can look up that keyword argument in the manual page for the command to know how the model will use it. The command is responsible for using its arguments, and will use them in the order it needs to - the order they have been written is not important except when it has meaning for the command (like for /). For keyword arguments, the order never matters.

2.1.4 Mixing keyword arguments and positional arguments

Some commands may accept both arguments which need a keyword and arguments which do not need a keyword. These arguments can be mixed up; for example

    hat: fedora
    swindle: no
    900 700
    fire-station: kensington)

should be read as the following:

  • a use of the command a-made-up-command, given the arguments
    • by position (these have no keyword, and swapping them around would change the meaning):
      1. 103
      2. 900
      3. 700
    • by keyword (the order is not important):

Any of the following writings should be read the same!

    hat: fedora
    swindle: no
    fire-station: kensington
    103 900 700)
    103 900 700
    hat: fedora
    swindle: no
    fire-station: kensington)
(a-made-up-command 103 900 700
    hat: fedora swindle: no
    fire-station: kensington)
    103 900
    hat: fedora swindle: no fire-station: kensington

However, as a convention we would suggest using the form

    hat: fedora
    swindle: no
    fire-station: kensington
    103 900 700)

where the keyword arguments are written out first and the positional arguments follow. It is all the same to the model, but for the human reader this consistency makes it less likely that the eye skips over something by accident. For the same reason, it is often a good idea to avoid putting several keyword arguments on the same line, and so on. However, the question of how to layout your code is really a matter of judgement, and you should make a decision based on the particular circumstance rather than trying to adopt a universal principle.

2.1.5 List arguments

Sometimes a command may accept more than one value for one of its arguments. For example, the model contains a command aggregate which produces tables of summary statistics, and this command can accept a keyword argument over: which tells it about which populations you want summary statistics for. Since you might want summaries for several populations, you might want to be able to put in several values for over:. The way to express this to the model is by putting the multiple values in brackets [], so for aggregate (which we describe in detail later), you would write something like:

    over: [first second third]

This should be read as:

  • use the command aggregate, with the arguments
    several values supplied, to wit:
    1. first
    2. second
    3. third

This is quite different from

    over: first second third

which (as explained above) would be read as

  • use the command aggregate with the arguments
    • by position:
      1. second
      2. third
    • by keyword:

and different still from

    over: (first second third)

which would be read as

  • use the command aggregate with the arguments
    having the single value:
    • a use of the command first, with the positional arguments
      1. second, and
      2. third

The order of values in a list can be significant, so rearranging a list may change the meaning of the command using it. Lists may be used as values with some keyword arguments and some positional arguments. If an argument is allowed to be a list, it is equivalent to write [the-value] and just the-value, i.e. a single item will be taken as a list containing only that one item.

2.1.6 Syntax errors; things you should never write

Here is a quick list of things which should always/never happen in any valid scenario. When you are getting started, you will probably write some syntactic mistakes as you get used to the general shape of things. Checking this list will help you see some common mistakes.

  • every opening bracket [, parenthesis (, quote ="= or brace { should always have a matching closing symbol
  • a parenthesis should never follow another parenthesis immediately, like ((
  • a parenthesis should never close immediately, like ()
  • a parenthesis should always have a word immediately after it, like (scenario...
  • a parenthesis should never have a keyword immediately after it, like (capex: ...
  • a keyword should never have another keyword immediately after it, like first: second:
  • a keyword should always have an expression after it, like first: something or first: (something else) or first:[]
  • a keyword should never be before a closing symbol, like first: ) or first: ]
    • a keyword may appear after a closing symbol, like b: in (Q a: (W) b: (E))
  • a keyword should never be written directly within a list argument, like [first: x]
    • they may appear within expressions that are in a list, like [blah (something first:x) qwer]
  • an open bracket should never follow an open parenthesis, like ([ ...

As an exercise, try explaining why each of these is true.

2.2 Exercises: read and write some syntax

  1. For each of these bits of syntax, which sentence gives a description consistent with this section of the manual? Note that the syntax written here may not have any semantics assigned to it in the NHM, but it is syntactically OK, so you could define some terms in a scenario that would give it a meaning. Further note if any of them seem like they look like a mistake.
    1. (house.energy-use)
      1. calculate house.energy minus use
      2. invoke the command house.energy with the unnamed argument use
      3. invoke the command house.energy-use
    2. (house.energy-use by-fuel:MainsGas)
      1. invoke the command house.energy-use with the unnamed argument by-fuel:MainsGas
      2. invoke the command house.energy-use with MainsGas for the named argument by-fuel:
    3. (+ 1 house.energy-use by-fuel: mainsgas)
      1. invoke the command + with two unnamed arguments, 1 and house.energy-use, and the keyword argument by-fuel: being mainsgas
      2. invoke the command + with the arguments 1, and
        • the command house.energy-use given a keyword argument by-fuel: being mainsgas
    4. (aggregate.count name:how-many)
      1. invoke the command aggregate with the unnamed argument count and and the keyword argument name: being how-many
      2. invoke the command aggregate.count with the keyword argument name: being how-many
    5. (+ 10 (* 4 (house.total-floor-area)) 3)
      1. invoke the command + with the arguments * 4 and house.total-floor-area and 3
      2. invoke the command + with the arguments 10, and:
        • the command * with the arguments 4 and
          • the command house.total-floor-area, and
          • 3
      3. invoke the command + with the arguments 10, and:
        • the command * with the arguments 4 and
          • the command house.total-floor-area
        • 3
    6. (/ 8 (+ 3 4))
      1. invoke the command + with the arguments / 8 and 3 and 4
      2. invoke the command / with two arguments, 8 and
        • the command + itself having two arguments, 3 and 4
    7. (8 / 9 + 3)
      1. invoke the command / with 8 and 9 and then invoke the command + with 3
      2. invoke the command 8 with the arguments /, 9, + and 3
    8. (1+2)
      1. invoke the command 1 with + and 2 as arguments
      2. invoke the command + with 1 and 2 as arguments
      3. invoke the command 1+2 with no arguments
  2. For each of these bits of syntax, write your own outline description
    1. (/ 1 2 3 4 5)
    2. (+ + + + plus)
    3. (+ (1 2) (/ 3) 4)
    4. (+ (1 2) (/ 3 4))
    5. (+ 1 2 (/ 3 4))
    6. (+ 1 2 / 3 4)
    7. (log base:9 9)
    8. (log 9 base: 9)
    9. (log (log 9) base: (log base:9 9))
    10. (measure.standard-boiler fuel:mainsgas capex: (+ 500 (* 40 size.kw)))
    11. (qwer [123 (qwer 303)] pip: [pop])
  3. For each of these outline descriptions, write equivalent syntax
    1. The word fish
    2. A use of the command fish
    3. The command fish applied to the single unnamed argument 4
    4. The command + applied to two unnamed arguments:
      • the command house.energy-use, given the named argument:
      • the command house.energy-use, given the named argument:
  4. For each of these bits of syntax, indicate why it cannot be valid syntax
    1. (+ 1 2 3
    2. (+ ((* 3 4)))
    3. [(house.energy-use by-fuel:) mainsgas]
    4. ([house.region] [house.built-form])

2.2.1 Answers

  1. Consistent descriptions are:
    1. 3. - there is one word in parentheses, which is the use of a command
    2. 2. - there are three words, the first being the command, the second ending with : and so a keyword
    3. 1. - all the arguments are to +, because it has the parentheses
    4. 2. - as for question 2
    5. 3. - the parentheses mean several levels of commands, and the 3 belongs to the + not the *
    6. 2. - command always after the opening parenthesis
    7. 2. - command still always after the opening parenthesis
    8. 3. - command after the opening parenthesis; there are no spaces so it is one word, 1+2.
  2. Our descriptions are:
    1. / applied to 1, 2, 3, 4, and 5
    2. + applied to +, +, +, and plus
    3. + applied to three arguments:
      1. 1 applied to 2
      2. / applied to 3
      3. the word 4
    4. + applied to two arguments:
      1. 1 applied to 2
      2. / applied to 3 and 4
    5. + applied to three arguments
      1. 1
      2. 2
      3. / applied to two arguments
        1. 3
        2. 4
    6. + applied to 5 arguments:
      1. 1
      2. 2
      3. /
      4. 3
      5. 4
    7. log applied to two arguments:
      1. the keyword argument base: being 9
      2. a positional argument 9
    8. exactly the same as 7.
    9. log applied to two arguments
      1. the keyword argument base: being:
        1. log applied to two arguments
          1. the keyword argument base: being 9
          2. a positional argument 9
      2. a positional argument:
        1. the command log applied to the single argument 9
    10. the command measure.standard-boiler, with two named arguments:
      the word mainsgas
      itself a command, +, applied to:
      1. 500
      2. the command * applied to two arguments
        1. 40
        2. size.kw
    11. the command qwer applied to two arguments:
      1. the keyword argument pip:, being:
        1. a list with one value in, the word pop
      2. a positional argument, being alist of two values:
        1. 123
        2. the command qwer applied to the single argument 303
  3. Each description can be written as:
    1. fish
    2. (fish)
    3. (fish 4)
    4. (+ (house.energy-use by-fuel:mainsgas) (house.energy-use by-fuel:housecoal))
  4. The errors are:
    1. There is no closing parenthesis for the opening one
    2. The use of the command * is itself being used as a command (double parentheses)
    3. The keyword by-fuel: has no value; it is immediately followed by )
    4. The list [house.region] cannot be after a parenthesis, as a list cannot be the name of a command.

2.3 Semantics

By now we have covered the NHM syntax, but little to nothing about the semantics. If you have a well-formed scenario the semantics say what simulation will if it runshappen. However, not all syntactically correct scenarios can be made into a simulation, because not every command can be used in every place. For example, you cannot add a house's region to its floor area, because region is not something that can go within + (ultimately because it's not a number). There is an exhaustive list of what commands can be used where in the language reference.

In truth most of the rest of this document is about the semantics, going into depth about the behaviour of different parts of the language, and what commands can be used where.

However, for familiarity here is a shallow overview of the semantics associated with common bits of syntax.

  • Top level elements There are two things you will see at the top level of a scenario
    • scenario is the command declaring a single scenario. This will instruct the model to load some houses from a stock file, and then simulate changes to the houses and trigger the production of reports
    • batch is the command declaring a suite of related scenarios A batch contains a scenario: argument, which is a scenario, and an inputs: argument, which generates a table of parameters. The parameters are substituted into 'placeholders' in the scenario: argument to produce many variants, which are run and then summarised together. This is mostly good for sensitivity analysis.
  • Things which go inside a scenario
    • Commands to make things happen; on.dates, on.change

      Mostly you will use on.dates within the top level of a scenario. It sets up the simulation to do things when the model gets to particular dates. The things inside it are the things that will get done. The order in which things get done is determined by the date, and then by the line number.

    • Commands to change things about how the model works

      Along with on.dates, you may write a few things mostly starting with context. which change 'global' settings in the model. For example, context.tariffs sets up fuel prices, context.weather changes the weather, context.carbon-factors changes the carbon factors and so on.

    • Definitions

      Finally you may see definitions of variables and actions. The command def defines a variable, which will either be stored against each modelled house (like number of measures house has received) or as a single global counter (like number of measures given out in total) depending on the on: argument.

      There are also commands def-action, def-test, def-function and def-report which merely serve to define things for later reuse; they have no direct effect on their own.

  • Things which happen on particular dates We mentioned on.dates which makes things happen; here are some of the things it can make happen:
    • apply is one of the most basic things you can make happen. When it happens, it finds a population of houses defined by its to: argument, and then goes through the actions it has to take applying each action to all the houses in turn.
    • aggregate is a command to produce a summary table about a population of houses. When it happens, rows are added to that table, so it lets you see the state of the stock at a moment in time.
    • repeat is a command that tries doing something to the stock again and again until a condition is met
  • Measures and actions apply is what you use to change houses, using an action. A measure is a kind of action that represents a technology being put into a house. The NHM has lots of actions; the important ones are:
    • action.case, which lets you pick an action to take depending on how the house is
    • do, which lets you combine a series of actions into one
    • choice, which lets you pick an action to take based on how the house will be afterwards
    • action.do-nothing and action.fail, which are explained later
  • Modules and templates Along with everything described so far, you are likely to encounter, modules, templates and macros. These are ways to make your scenario shorter to read and write by avoiding writing the same or similar things again and again. They are explained later, but basically template and any command starting with ~ are special, and transform the scenario in some way before it gets run (and so before any stock is loaded or energy calculations are done or anything).
  • Includes Finally if your scenario gets big enough or has parts that need reusing, it may be broken up into separate files. Any of the commands starting with include is akin to a mechanism for copy-pasting another file into your scenario when it is run.

3 The NHM desktop application

You are most likely to use the NHM by running the NHM desktop application7, which is a tool for editing scenarios with access to the help, and running them and looking at the results. If you are working with other people, you can use the git version control system to collaborate on scenarios, but this is by no means obligatory. This section of the manual will cover installing the application on your computer, using it to edit and run scenarios, and will direct you to other resources where you can learn more about it.

3.1 Downloading the application

The latest version of the NHM is always available from http://deccnhm.org.uk/standalone/. This presents four different builds for different operating systems. For the windows version, you will need to download cse.nhm.ide.application-win32.win32.x8664.zip. This requires you have:

  • A 64-bit processor and associated Windows operating system
  • Sufficient memory; this is probably 4GiB (gigabytes) at a minimum

It includes a version of the Java Runtime Environment (JRE) which it needs, so you do not need to install this. However, the applications offered for other platforms do not include this, so if you want to use the Mac OS or Linux versions you will need to install your own JRE.

3.2 Installing the application

The NHM requires very little installation - all that you need to do is unzip the zip file you downloaded in the previous step somewhere on your computer. In windows you can usually do this by right-clicking on the zip file, and choosing Extract all...


If you are installing the application in a multi-user environment, where several users of the same machine may run it at the same time, you may need to do further setup. The Eclipse documentation explains more about this for system administrators.

3.3 Using the application

Now that you have the application installed, let's have a quick walk-through of the major features. Start the application double-clicking on the National Household Model.exe program.


If everything is working correctly, you should see a loading window like this:


Once the application has started up you will be presented with a blank slate for working on NHM scenarios. The window which appears will look something like this, except without all the arrows and boxes drawn on.


You can see in this window the main way the NHM interface works. There is one big window, which has the usual menu bar at the, then a tool bar (the row of icons below the menu bar)8. Below that the window is divided up into several non-overlapping rectangular panels. Each panel may house some "views", which show information. In the picture above, there are three panels, one on the left, one at the bottom right and one at the top right. You can see that the left panel contains only the Project Explorer view, and the panel at the bottom contains these four views:

  • Problems
  • Git Repositories
  • Model runs
  • Model run log

In the bottom panel, the Problems view is currently displayed. Clicking on the name of one of the other views would switch change to that view.

Each panel also has some little controls in the top right, a thin horizontal bar and a horizontal bar with a rectangle underneath it. The thin horizontal bar will shrink (minimize) the panel down to a narrow bar on the side of the window. The fat horizontal bar will expand (maximize) the panel to fill the whole window. The problems view and the project explorer view both have a cross (X) at the right of their name. Clicking this will close the view. They also both have a view menu, which is the little downward pointing arrow near the minimize and maximize buttons. This shows a menu with options relevant to the view. You can also move views around by dragging and dropping the tab with their name in on the screen.

Try clicking on any of these buttons to see what they do, to get a feel for the user interface.

If you have closed a view, or got confused about how things are, you can put things back to how they started by right clicking on the "NHM" button in the top right (by the house icon), and choosing Reset:


You can also bring back individual views if you have closed them by looking in the Window menu:


Finally in the middle of the window is the editor area. This is where your scenario text will be displayed. At the moment it is empty, because there is no scenario open. We will remedy this by creating a scenario to edit.

3.3.1 Creating a scenario

All work in the NHM application must be contained within a project, which will be shown in the Project Explorer on the left. To make a new project with a scenario in it, right click in the white space in the project explorer and choose New -> Scenario Project:


This will display a wizard, which is a series of steps to go through shown in a window on the screen. The first step of the new scenario project wizard looks like this:


Since we are making a new project, we need to have a name for the project. This can be anything you like, but it is easiest if it does not contain spaces. A project is really just a folder on your computer. By default, it will go inside a folder which the NHM calls its workspace. You can see this in the wizard as well. Clicking Next > to move on we get to the next page of the wizard:


This lets us pick a version of the model that we want to use. The version you want to use is associated with the project, so every scenario in that project will be validated and run using the chosen version of the model. The NHM desktop application can contain several versions of the model at once. These can either be release versions, which should be reliable and will not change, or test versions which are still undergoing development. When you (or your systems administrator) updates the NHM desktop application, it will get any newer versions of the model that are available. If you have chosen one of the top two options here (latest release or latest version, including test versions) then if you update the NHM desktop application and a new release or new test version is available, that version will be used. The NHM desktop application does not update itself unless you ask it to, so this need not be an intimidating prospect. Once you have chosen a version of the model you can go on to the next page (or you can click Finish to skip it, if you like):


This page is just for making a simple scenario to get started with. It is not intelligent in any way, it just puts in some pro-forma text that is built into the application. We will choose to have a quick-start scenario, and press Finish.

Once the wizard is completed, an icon will appear in the Project Explorer for your new project. If you click on the + next to this icon the project will "expand" to show you the scenario file that it contains. Double click on that scenario file to open it. You should see something like this:


The window looks like it did before, except that the blank panel now shows the scenario editor. There is also an error in the new scenario - there is no stock with the given name. This is shown in four places:

  1. In the scenario editor, in the left hand margin, there is a red X. If you point the mouse at this X, a "tool tip" will appear describing the error.
  2. In the scenario editor, the opening bracket of the scenario has a little red underline. This is because the error is to do with the scenario command in particular. The bracket which is underlined is for the command which is confusing the model.
  3. In the right hand margin, there is a little red rectangle. This shows all the errors in the whole scenario, so if the scenario did not all fit on the screen, the right margin would show if there was an error off the bottom that you could not see on the left margin. Clicking on the error marks in the right margin will scroll the editor to the error.
  4. In the Problems view at the bottom, there is a list of errors. Each error is shown as a line in the list. Double-clicking on an error will find it in the editor.

Let's fix the error by importing a stock.

3.3.2 Importing a stock

To import a stock you must use another wizard. Right click on the project icon in the Project Explorer, and choose Import...


This will show a list of the import wizards that are known. At the moment the stock import wizard is found under Other:


Press Next > and select a stock file to import by clicking Browse


Pressing Next > again will show you a page where you need to select a version of the model to use.


The stock will be imported using the version you select. Note that if a new version is released with a different stock importer, you will have to do the import again - the application will not change your stocks once they have been created. Pressing Next > will run the importer on the file, showing you a page of progress:


In the middle is a log of output that the stock importer produces. This may help explain if there are problems in the stock. In this example, there are no problems so we can press Finish. This will put the new stock file in our project, so we can use it in a scenario. You should see it in the Project Explorer view. You can amend your scenario to use the new stock - one way to do this is to write out the name of the stock, but you can ask the application to type this for you. Open the scenario, delete any existing text after the stock-id:, and press Control-Space (i.e. depress the control key with one finger, then keeping it held down press and release the space bar, then release control). You should see a suggestion box appear above the scenario editor, like this:


The suggestion box has two parts - on the left, a list of suggestions, and on the right help about each suggestion. You can go up and down in the list with the up and down arrow keys, and you can choose a suggestion by pressing the return key. Press return to select the stock you want to use, and the NHM will fill in the stock name for you. Your scenario is now valid, but the application will not show this yet. You have to save the scenario to have it re-validated. You can tell that the scenario is not saved because there is a star (*) in the title in the editor. Press Control-S to save, and your screen should look like this:


Now the scenario has no errors, we could run it and inspect the results.

3.3.3 Running a scenario and viewing results

To run a scenario, you can either press the shortcut key Control-R, or click on the run button in the toolbar. For now, let's use the toolbar:


In this screenshot there is only one option for running, Run on this computer. You may have more options if your NHM knows about servers for running scenarios on. The user interface works the same way in either case. Once you have selected where to run your scenario, a new row will appear in the runs view:


This tells you how your scenario is getting along. If you are using a remote server to run your scenarios, you can switch off the application and open it up again later to find out what happened. If you are running the scenario locally, you must leave the application open to wait for the results. When your scenario has finished, the State column will show =Complete:


At this point, you can download the results of the run. To do this, either double-click on the run, or click on the button in the top left of the runs view which is like an arrow pointing into a tray. A dialog will open asking you where to save the results


The results will be placed in a folder contained in the folder you select, and will appear in the Project Explorer:


The folder that is created has been named according to (a) the name of the scenario that was run and (b) the date when it was run. You can see the output files that have been created - in this picture there is just counts.tab, from the aggregate report in the scenario. You can open these tab files within the nhm directly by double clicking, if you just want to inspect them:


They will open up in a new tab the editor panel. Here you can see that the scenario is also still open. However, you may want to open them up in another program, or send them in an email or something. To open them using another program, you can right click on them, and select Open With, Other:


In the dialog that comes up, you can choose any editor built into the NHM or a program on your computer. For example, here is how to open the file using Excel:


If you want to open all tab files in Excel when double clicking in the NHM, tick both Use this editor for all 'counts.tab' files and then Use it for all '*.tab' files as well. Once this is done you can enjoy your tables in the familiar and comforting environment of Microsoft's premier table editor:


The result files and scenario files are not stored in any special way - they are just files in a folder on your computer. This folder is called the workspace folder. You can locate these files in Windows by right-clicking on any file in the Project Explorer and choosing StartExplorer, Show in File Manager:


You can see that the Project Explorer view is just showing you some files, which you can copy, email, or otherwise manipulate in whatever way you are used to:


To find out where the workspace is, you can either open a project in Windows explorer like this and navigate up a level, or you can choose Switch Workspace, Other... from the File menu. The window which comes up will contain the current workspace as the default:


You can have more than one workspace if you like, for different working contexts, and switch between them with this menu.

Now that we know how to run a scenario and look at the outputs, let's look at how to make slightly more complicated scenarios.

3.3.4 Creating more scenario files and including them

Usually your scenarios will be big enough to warrant being divided into several parts. To make a new scenario file, either to include from another file or because you are writing a new scenario, you need to make a new file with the .nhm file extension. The easiest way to do this is to right click in the project explorer on the folder you want to put your new file in, and select New, File:


This will open a window where you can put in the file name. Make sure it ends with .nhm so that the application treats it as a scenario.


The newly created file will appear in the Project Explorer, and you can double-click on it to open it. It will open in another tab in the editor area:


Now we have our tabula rasa, we can begin typing some things. In our example, let's say that we wish to define an NHM module. We could type this all out, but let's save time and have the typing done for us. Press Control-Space and the suggestions window will reappear. In the list of suggestions, you should see a suggestion for ~module. Press return and it will type most of a module for you, selecting the name to fill in. You can fill in a name, and then press tab to move to the next logical place to type something. This suggestion has also put in an init template for you, as this is NHM convention. Click the cursor after the closing bracket for init, but before the closing bracket for =module, and press Return to put in a new line. Note that the editor has indented the line for you. Now press Control-Space again, and choose the suggestion for entering a template definition:


Type the name of your template in the first placeholder (the name), and press tab to go to the next (the arguments). Let's make a template with no arguments - just backspace to delete them. Press tab again to place the cursor inside the template definition. In our example we have called the template count-of-houses:


We want the template to produce the count of the houses, so we are going to insert aggregate.count9. Type agg to get started, and then press Control-Space again. Some suggestions appear:


Press down until you get to aggregate.count, and then press Return to insert it. The cursor moves to the end of the command, before its closing parenthesis. We want to give our aggregate a name, so press space to insert a space and then press Control-Space again:


The suggestions include name:, which we want to supply. Note that the help also tells us what name: means in this context. Press return and fill in any old name. Now save the file. You will notice that the model produces a validation error, saying that the file does not contain a scenario. It is customary to put a scenario at the top of a file defining a module, which shows how the module works, and ideally helps you test its output independently of other modules. We have typed one in which you can see below - try typing this yourself, pressing Control-Space whenever you have done a bit to see what the model suggests.


In this screenshot you can see that the auto-complete suggestions include user-defined templates which are available in your scenario. Here we are filling in my-module/count-of-houses into our test scenario, to show people how it is meant to be used. After we have chosen it, note that hovering the mouse over it will show a tool-tip which tells you the definition of the template and where it came from:


Saving the new module file shows now validation errors, so we are ready to use it somewhere else. In an ideal world we would also have checked that our module did what we had intended it to do.


Switching back to my-project.nhm, we can now make use of our module. First we have to include it10; fortunately the auto-complete can help us with this. At the top of my-project.nhm (before the scenario), press Control-Space. Note that one of the suggestions is include-modules:


Choose this, press space to go to the first argument, and press Control-Space again to see suggestions for includes:


We can select the new module and press return to fill in its name Now we are using some modules we should use ~init-modules. It doesn't actually matter in this example but it is good practice.


And now we can use our new template - in the existing aggregate command, delete the old aggregate.count command and press Control-Space. In the list of suggestions, you can now see my-module/count-of-houses. This is the new template we just created! Press return to insert it.


Note again that you can hover over the template to show its definition and where it came from:


Now we can run our main scenario again, to see if the results are different. Clicking run, notice that a window comes up asking for a bit of text to disambiguate the new run from existing runs. We type in 'with template', just so we know


In the runs window you can see a new run, which has our note after the Scenario in brackets.


When it has finished you can download the results like before:


3.4 Editing scenarios efficiently

An NHM scenario is just a text file, so you could edit all your scenarios in Notepad11. However, the NHM application knows about the syntax for scenarios, and so can help you with your editing.

3.4.1 Auto-completion

We have already seen how pressing Control-Space will display a menu of suggestions that you can select from. These suggestions are relevant to the context in which you have pressed Control-Space; that is to say, the model will try only to suggest things which can legally be typed into the place where the insertion point is.

To make this a bit clearer here are some examples; in these examples, the vertical bar symbol | denotes the insertion point where you pressed Control-Space.

  • agg| The model will show anything that can be written starting with agg at this point.
  • (agg| The model will show any commands that can be written that start with agg at this point. Inserting a command will leave the cursor just before its closing parenthesis, so you can press space to start typing its arguments.
  • (house.energy-use | The model will show anything that can go within house.energy-use. This will be:
    • Other commands or values which can be used for the first unnamed argument to house.energy-use
    • The keyword part of keyword arguments which house.energy-use understands, like name: or by-fuel:
  • (house.energy-use by-fuel: | The model will show values that can be supplied for the by-fuel: argument of house.energy-use

The suggestions that the model displays are categorised. If you press Control-Space repeatedly, the suggestion window will cycle through the categories which it knows. You can see the displayed category in the bottom right of the suggestion window. For example, this list of suggestions is only showing keywords for keyword arguments; you can see this in the bottom right where it says "keywords":


3.4.2 Context help

The editor will show you help for anything that is written in a scenario. If you hover the mouse over a word which you want help on, a tooltip will pop up describing that thing. For example, here is what pops up when you hover over the aggregate command:


If you hover the mouse over a keyword argument, the tooltip will tell you about what it means. If you hover over the left parenthesis of an unnamed argument, the tooltip will tell you about how that argument will be used.

3.4.3 Colouring in

The scenario editor colours in the text on the screen for you. This should make it easier to read. Different parts of the text are given different colours, which you can change if you like. Open the preferences:


And choose General, Appearance, Colors and Fonts; there is a section in there called NHM Scenarios, which lets you change the colours and shows you what they mean:


We find that the distinction between built-in commands and macros or templates is helpful; by default this is blue versus brown.

3.4.4 Automatic typing and useful keyboard shortcuts

Sometimes you cannot write everything you want by pressing Control-Space many times. In this case, the editor will try and help you to keep your parentheses correct, by typing or deleting some parentheses for you when you press certain keys. This can be confusing, and you can turn it off in the preferences if you want:


However, it is worth trying using the automatic typing for a little while as you may grow to like it. Here is a description of what the keys do, and how we would imagine you might use them:

  • Adding and deleting parentheses When you typing a parenthesis or delete a parenthesis, if the scenario is already balanced the editor will try and keep it balanced. If the scenario is not balanced, the editor will not try and do anything clever. In addition, if you use the selection to change more than one character at a time, say by selecting a region and deleting it or overwriting it, the editor will not do anything clever.
    • When you type an opening parenthesis, the editor will insert the closing parenthesis for you, leaving the cursor in between the two, e.g: given blah blah |, typing ( gives blah blah (|)
    • When you type a closing parenthesis, the editor will move the cursor to the next closing parenthesis, rather than inserting one. This means that you can still type () to get (). For example, given (blah | blah), typing ) will give (blah blah)|
      • If there is only blank space between the insertion point and the next closing parenthesis, the blank space is deleted: given (blah| ), typing ) will give (blah)|. If this is causing particular confusion, consider the next part on moving parentheses around.
    • When you delete a parenthesis, its partner is deleted wherever it is in the text, e.g. given, (blah blah)|, pressing backspace will give blah blah|. Because the insertion point is in the natural place you can still backspace over a lot of text.
    • If you select some text and type an opening parenthesis, a closing parenthesis will be added at the end of the selection, e.g. given something |blah blah| something, typing ( gives something (blah blah)| something.
  • Moving parentheses around things you have already written; often if you are deleting or adding parentheses, what you are really wanting to do is to move something 'inside' an existing bracket or to put something out of a bracket. The editor has commands to do these two things for you
    • To move the next thing to be within the current bracket, you can press Control-Left Bracket (the key with [ written on it, usually to the right of the p key). For example: given (|) one (two three) four, sequential presses of Control-Left Bracket will produce:

      1. (|one) (two three) four
      2. (|one (two three)) four
      3. (|one (two three) four)

      This action 'absorbs' subsequent expressions into the current expression.

    • The opposite action is to eject the last expression from the current expression. This is on the key Control-Right Bracket. For example: given (one (two three) four|), sequential presses of Control-Right Bracket will produce:
      1. (|one (two three)) four (note the cursor is moved to the beginning of the current expression)
      2. (|one) (two three) four
      3. (|) one (two three) four
  • Moving the cursor around quickly Often you want to move the cursor around the brackets quickly. Alt and the arrow keys are set up to do this.
    • Alt-Up Arrow will move to the start of the current expression, or if already there to the start of containing expression, e.g. given (some (expression with things|)), repeated presses will give:
      1. (some (|expression with things))
      2. (|some (expression with things))
    • Alt-Down Arrow will move to the start of the next expression within the current expression, e.g.: given =(an | (expression (with) (sub-expressions))), repeated presses give:
      1. (an (|expression (with) (sub-expressions)))
      2. (an (expression (|with) (sub-expressions))).
      3. (an (expression (|with) (sub-expressions))) - note because there is nothing within (with), nothing happens. We cannot go any further 'down'.
    • Alt-Left Arrow will skip to the previous previous expression start or end at this level, or go up a level, e.g. given (a) (b) (c (d e)) (f)|, repeated presses give:
      1. (a) (b) (c (d e)) |(f)
      2. (a) (b) (c (d e))| (f)
      3. (a) (b) |(c (d e)) (f)
      4. (a) (b)| (c (d e)) (f)
      5. (a) |(b) (c (d e)) (f)
      6. (a)| (b) (c (d e)) (f)
      7. |(a) (b) (c (d e)) (f)
    • Alt-Right Arrow will skip the next expression start or end at this level, or go up a level. This is simply the reverse of Alt-Left Arrow.

You can get a summary of all these keys do by pressing F1 when in a scenario editor. With a little learning, using combinations of these keys should let you do many kinds of edits much more quickly and without making errors. For example, imagine you have some scenario code calculating a number, but you realize that you need to divide it by 1000. Here is how things are to start with; the expression is as shown, and the cursor (|) is before the expression.

(scenario ...
    (blah blah
    (* 3.1415 (+ x y) in-use-factor))

Rather than worrying about precise cursor positioning, we can simply do the following:

  1. Type a left parenthesis - we need more parentheses because we know we want another level in our expression. The editor types the right parenthesis for us:

    (scenario ...
        (blah blah
        (* 3.1415 (+ x y) in-use-factor))
  2. Type the division symbol, as we are going to divide something:

    (scenario ...
        (blah blah
        (* 3.1415 (+ x y) in-use-factor))
  3. Absorb the next expression into the current expression, by typing Control-Left Bracket

    (scenario ...
        (blah blah
           (* 3.1415 (+ x y) in-use-factor)))
  4. Skip over the entire expression, so we can type the denominator, by typing Alt+Right Arrow

    (scenario ...
        (blah blah
           (* 3.1415 (+ x y) in-use-factor)|))
  5. Press return, to make things nice and clear

    (scenario ...
        (blah blah
           (* 3.1415 (+ x y) in-use-factor)
  6. Type in the denominator
(scenario ...
    (blah blah
       (* 3.1415 (+ x y) in-use-factor)

With practice, you may find that these commands allow you accurately edit scenario code with convenience and ease, and that many parenthesis-related woes are diminished. However, this may take some time to learn, and it is not for everyone.

3.4.5 Quick outline

In a long scenario you may find that scrolling up and down to find things becomes hard work. Try pressing Control-o (Control and the o key at the same time), to bring up the quick outline. This shows the hierarchy of the scenario in a temporary window. You can filter the quick outline by typing, select particular lines by pressing up and down, and you can press return to rapidly scroll to the selected line.

3.5 Navigating between files and definitions

If you have a lot of scenario files in your workspace, or a lot of template definitions, getting between them can be tiresome. The application offers a key to jump quickly to a particular file, Control-Shift-R (all three keys at once). To find a specific template, you can press Control-Shift-T.

3.6 Viewing files side-by-side

You have already seen how to open multiple editors, and switch between them by clicking on their tabs. You can also look at two (or more) editors at once if your screen is big enough, by dragging their tabs around on the screen. Try opening two editors, and then dragging the tab for one of them to the right hand side of the editor area. You should see a black rectangle showing where the editor will go if you let go, and when dragging to the right this should take up half of the editor area on the right. Let go, and you will have two editors open next to each other.

3.7 Viewing the top and bottom of a long file simultaneously

The above lets you open two files side-by-side or above each other. Sometimes you want to open one file, but see two parts of it at once. You can do this by typing Control-Shift-Minus (Control-Underscore), or Control-Left Brace (Control-Shift-Left Bracket), or looking in the Window, Editor menu.

3.8 Finding other actions

The application has all sorts of other useful functions, like search and replace across multiple files, local file history, team-working using git, comparisons between two files and so on. These functions can be hard to find, just like in any other program. Fortunately there is a quick way to search all the things the application knows how to do - the quick access box. This is the text box shown on the right hand side of the toolbar near the top of the window. Typing in here will pop up a list of all the commands, settings, help topics and so on that the application knows; for example, here is what happens if you type in "Run".


You can see a list of views that could be displayed on the screen, commands that can be taken in the current context, and preferences that you could change. You can focus the quick access box by pressing Control-3, so it is a quick way to do things for which you have forgotten the commands.

3.9 Accessing the manual

As well as context sensitive help within scenarios, the NHM application also provides the scenario language manual. You can get to this by choosing Help Contents in the Help menu. This will either open the help in a new window, or within the application, depending on your operating system. In either case, the help will look something like this:


The "book" icons on the left are documentation for different parts of the application. You may have more such books than are shown in the screenshot, but you should have one entitled National Household Model. Clicking on this will show you the full manual for the latest version of the model that your application contains. This is where you can find the changelog, which will tell you about any recent changes to the model that you should be aware of.

The other documentation categories may tell you about

  • Using EGit, the git client built into the NHM application
  • The general principles for the application interface

3.10 Glossary

The NHM application is based on some software called Eclipse, which is what does the work of drawing windows on the screen, managing projects and so on. It has its own help section and terminology, and many of the menus and dialogs will refer to things using these words. Here is a quick list of a few of these words and what they mean:

The workspace is a folder on your computer. The application looks in this folder for scenarios and projects. You can have more than one workspace, but you can only use one workspace at a time. It also holds special files containing any settings that you have changed in the application while using that workspace (for example, if you have changed the editor colours).
The workspace contains projects. These are folders within the workspace folder which contain other files and folders. Every file in the application must belong to a project. They are the organising unit of work for the NHM application.
Sometimes the application will refer to "resources". This is its generic term for files, folders and projects.
The main application window is called a "workbench window". You can open more than one workbench window by choosing Window, New Window. Every workbench window has a menu bar at the top tool bar, and an editor area in the middle which may be surrounded by views.
An editor is a part of the interface responsible for opening and maybe changing a file. The application contains different editors for different sorts of file, like the scenario editor for .nhm files. Editors appear in the editor area of a workbench window.
A view is a part of the interface which displays some information but is not an editor. The Project Explorer is a view. Views are arranged into panels which surround the editor area.
A perspective is an arrangement of views and other icons in the interface. You will normally use the NHM perspective. This is indicated by the NHM button on the right hand end of the tool bar. You can use this to reset all the views to their default arrangement by right-clicking.
Content Assist
Content assist is the pop-up suggestions that appear when you press Control-Space.
The "platform" refers to the underlying machinery which makes the application work.
Plugin / Feature
A plugin is a bit of code which adds some behaviour to the application. A feature is a collection of plugins. For example, all the NHM related behaviour in the application is provided by a feature which is added to the bare minimum Eclipse needs to work. EGit is provided by another feature.

4 A simple scenario, by example

At this point you may be feeling like you have read a lot of background material, without getting on to anything concrete (or not, if you chose the option to skip the beginning so you could immediately do something). Without further ado, here is a complete, simple scenario which works and does something:

;; Text following a semicolon is a comment, which the model will ignore.
;; All scenarios start with the scenario command
 ;; these next are keyword arguments for the scenario
 ;; The scenario should start in 2015
 start-date: 2015
 ;; It should end in 2020
 end-date:   2020
 ;; This names a stock - the model will use this to find
 ;; the housing stock to use. Note that stocks are not built-in
 ;; to the model, nor is there any central repository of stocks
 ;; that it will know to refer to. In the standalone NHM, this just
 ;; refers to a stock file called my-houses.stock in the same folder
 ;; as the scenario file.
 stock-id:   my-houses.stock

 ;; After the named arguments are the scenario's other arguments.
 ;; These usually either
 ;; a) Define things like variables or measures.
 ;;    For example here we are defining a kind of wall insulation:
   ;; the first argument for def-action is a name
   ;; the second argument is the definition of the measure, in this case
   ;; the measure we are using is measure.wall-insulation
    ;; capex: specifies the capital cost - here 100 + 50 * area
    capex: (+ 100 (* 50 (size.m2)))
    ;; the thickness of installed insulation
    thickness: 50
    ;; the type of insulation to put in
    type: cavity
    ;; the thermal resistance
    resistance: 0.01))

 ;; This definition didn't cause anything to happen - it's just telling the model
 ;; about the "my-insulation" measure we may want to use later.

 ;; b) set up things in the simulation which are "global", e.g. fuel costs
 ;; Here we set up a default tariff (one every house will be on at the start)
  ;; this is the tariff
   ;; a tariff may cover several fuels, but this one
   ;; covers only one fuel
    type: MainsGas
    (charge (* 0.05 (house.meter-reading)))))

 ;; c) cause things to happen at particular points in time.
 ;; on.dates is a command to do some things on particular dates
  ;; its first argument defines the dates - regularly here stands in
  ;; for every year from scenario start date to end date.
  ;; and then we tell it what we want to do on the dates:
  ;; here we are going to use the measure we defined on 5%
  ;; of detached houses
   to: (sample 5% (filter (house.built-form-is Detached)
   ;; the # refers to the my-insulation defined above

  ;; and after that we will report on energy use
  (aggregate name:report
             (aggregate.mean name:energy (house.energy-use)))

 ;; d) instruct the model to turn on or off certain standard reports
 ;; here we turn on the transactions report

We suggest you take this scenario, plug in a stock that you like, run it, and see what outputs you get!

4.1 Exercise: describe the scenario

Describe in detail what you think the example scenario above does when the model runs. Do not do this by guessing from what's written down; instead, look in the manual or use the context-sensitive help to read the documentation for each term that is used. Remember that the model is stupid, and only understands the scenario insofar as it is programmed to blindly perform certain calculations in response to it. Compare your answer to our description below.

4.2 Our description

This scenario simulates the following sequence of events:

  1. Loading the stock
    1. The stock is loaded from my-houses.stock, starting in 2015. This means that whatever houses whose build year is before or equal to 2015 will be considered to exist at the start of the run; houses built in subsequent years will be constructed at the start of their build year.
    2. The scenario has no quantum: or weighting: specified, which means it will take the defaults of 400 and Uniform. As a result, the cases in the stock will be represented by simulated houses with weights of around 400, but most cases will probably produce houses with a slightly lower weight. Any cases whose weight is less than 400 will just be one house, with that weight.
    3. There is a default tariff defined by context.tariffs; whenever a house is constructed it will get put on that tariff. Since nothing else speaks about tariffs, all the houses will be put on the tariff.
    4. All other details are unspecified, and so fall to defaults; houses will be given a standard heating schedule, weather conditions, and so on. These defaults are described in an appendix to this document.
  2. Now we get to things which are going to be simulated. All of these things happen each year, from 2015 to 2020.
    1. Firstly, we have the events caused by the on.dates rule; these will happen at the start of the year because:

      • The scenario dates are specified as years; these are shorthands for the first of January in each year
      • The on.dates happens regularly, which defaults to the anniversaries from the start date until the end date.

      This rule says to do two things, an apply and an aggregate. The apply will always happen before the aggregate in each year, as they are written in that order.

      1. The apply will first compute the set of houses defined by its to: argument. This is the job of sample, which will compute a subset of its second argument thus:

        • sample asks filter to produce a set of houses, which in turn asks all-houses.
        • all-houses produces all the houses that currently exist, and gives them to filter;
        • filter looks at each one of those houses and asks (house.built-form-is Detached) about them, keeping the ones where that is true. It gives these detached houses back to sample.
        • now sample has all the detached houses, it picks a random subset having 5% of their weight, and gives this back to apply

        Now apply has a set of houses to work on. It shuffles them into a random order (this doesn't matter for this scenario, but it could), and goes through them one at a time. For each house, it asks the measure named #my-insulation to operate on the house if possible. The measure is defined up above the apply, by def-action; you could just as well write the measure directly in the apply

        The measure, when presented with a house, does the following:

        • Checks if the house is suitable for the measure, according to the rules detailed in the manual. In this case, the measure is cavity wall insulation, so the house must have a wall without cavity insulation whose construction type indicates a cavity is present.
        • If the house is not suitable, the measure fails, and the house is unchanged. apply does not care about this, and moves on to the next house.
        • If the house is suitable, the measure computes its size, and makes that available as size.m2; this is the sum of the area of all the suitable walls.
        • Then the measure computes the capex: rule; in this case fifty times size.m2, plus 100. Once the capex: rule has produced a result, the measure causes a transaction from the house to the market of that amount, so the house has paid for the measure.
          • Because we have report.transactions turns on at the bottom, this transaction will be recorded in the transaction log.
        • Finally the measure changes the description of the house in some way; in this case, as it is an insulation measure, it goes to each suitable wall and:
          • Marks the wall as having 50mm of cavity insulation
          • Computes it a new u-value as \(u' = 1/(1/u + (0.01 \times 50))\); this is the resistance of the insulation (its unit resistance times thickness), plus the resistance of the existing wall (\(1/u\)), converted back to a u-value.

        Once the measure has happened, the house's energy calculation result and so on may be different, as its thermal properties may have changed. Important things to note here are:

        • The to: argument of apply did not know about the measure; it has blindly picked some houses, so whether or not they are suitable or have already had a measure before does not come into it unless you ask for it to. So, if you ask to apply a measure to a sample containing 10,000 houses, unless you have already made sure they are suitable, you may not install 10,000 measures.
        • The way the sample is written is important; 5% of detached houses is not the same as those houses which happen to be detached from a 5% sample of all the houses.
      2. Now that apply is finished and probably changed some houses, on.dates will tell the aggregate to do its thing. When the on.dates sets the aggregate off, it will go through each of its over: sets and ask them to compute themselves. In this case, it will always be all-houses (since we are using the default), which is just all the houses that exist at the point.

        Next, the aggregate will ask each of the aggregation rules it contains to compute a value for each of these sets. In this case, we have aggregate.mean; aggregate will present aggregate.mean with the set of houses produced by all-houses. aggregate.mean will then go to each house in that set and present it to house.energy-use, which will produce the energy calculator's estimate of energy use. aggregate.mean takes all of these values and computes a weighted average of them, which is what it gives back to the aggregate.

        Finally the aggregate will write down that value against the simulation's current date and the name of the set, so in this case a row will be produced in a report for group all-houses containing the current mean energy use.

    2. Along with the explicit on.dates rule written in the scenario, there are two automatic rules which happen at the end of every year: houses pay their opex costs for particular technologies they have installed, and they pay their fuel bills according to their current tariffs. In our scenario there are no opex costs set up, so we can ignore that. However, we do have a tariff, so:

      For each house, at the end of the year, the tariffs are computed. All our houses are on a single tariff, which charges 5p per kWh of mains gas used. When the tariff is computed, all the charges it contains are calculated, and these produce transactions according to the relevant amounts.

      In our scenario, these will be visible only in the transaction log report, which we have enabled.

  3. The simulation finishes on 01/01/2020; as a result we will see the final year's on.dates, but not the transactions for the tariffs which happen at the end of the year.

This is a detailed description of what happens, but it does not say much about what we would expect to see. Let us use our imaginations to guess about the effects we will observe in the outputs. For this we need to make up some statistics about our stock; let's say that there are 6 million detached houses, of which 3 million have uninsulated cavity walls. Let's also say that 80% of these houses are on gas and use gas as their heating fuel.

In the first year, we will sample 5% of the detached houses. This will give us 5% of 6 million, which is 300,000. These houses are as they were in the stock. Of these 300,000 we would expect roughly half to have an uninsulated cavity, by coincidence. However, we could get anything from 100% to 0% uninsulated. The actual distribution can be had with some elementary statistics, but the important point here is that the model didn't do anything clever to guess our intention. It just did the sampling as stated.

Anyhow, taking the expectation, we will go through these houses and about 150,000 of them will get cavity walls; they will spend some money, which we will see in the transaction log. The remaining houses will be unaffected.

Next the model will make our first year aggregate; this will just contain a single number for overall mean energy use. Again, we get what we asked for; the model does not guess that we wanted change in energy use and produce a baseline, so we cannot see the impact of our first year's efforts. Later chapters will explain more about how to report on the effects of measures in detail, and how to schedule things to happen at different times.

Now we come to the second year; again, we compute a sample of 5% of detached houses. This will be (with very high probability) a different sample to the previous year, but it may well intersect. We would expect about 5% of 5% to be in the intersection in this first year. As a result of this, the proportion which are uninsulated will have gone down a bit - to put it another way we are sampling 5% from a population which now contains (3 million - 150,000) uninsulated cavities. Note that our sample can still in-principle have any proportion of uninsulated houses, as there are more than 300,000 of each sort left to take.

Later chapters will explain how to avoid re-sampling by marking houses or targetting only suitable houses. Anyhow, in our second year we will in expectation insulate slightly fewer houses, and so we will see slightly fewer transactions.

The second year report will show some reduction in energy use; however, it will be quite small, as we are looking at the populatin total. Later chapters will explain how this report could summarise particular subpopulations so as to be more informative.

The end-of-year fuel cost transactions in the second year will also be slightly lower, for those houses which (a) have been insulated and (b) were gas heated.

In the third year, we will have insulated some more houses (although not quite as many in the second year), so our sample will be expected to contain yet fewer uninsulated houses; as a result the expected reduction in energy use and bills in the third year will be a bit lower than in the second.

This process will continue for subsequent years, with the energy use and expenditure asymptotically approaching the values expected for full insulation of cavities in detached properties. The expected shape of the curve is an exponential decay, as each time we shrink the population we are sampling from by a fixed proportion.

4.3 Exercise: amend the scenario

As you have hopefully discovered, this scenario does some things you might not want to do:

  • It doesn't report on the direct effects of the measure being installed
  • It doesn't avoid re-sampling houses that have been affected already

Try making some changes to get a better understanding of this (you may want to return to these questions after reading later chapters):

  1. Amend the scenario to report on more details
    • Population level details about:
      • How many houses are suitable for the measure (house.is-suitable-for)
      • How many houses are likely to be in the target set for the apply (aggregate.sum, aggregate over:, house.is-suitable-for, filter, house.built-form-is)
      • How fuel costs are affected for these houses as well as energy use (house.fuel-cost)
    • House level details about the houses targetted (probe)
    • Summaries of the specific target set (probe.aggregate)
  2. Amend the scenario to avoid re-sampling; there are two different ways to do this which mean different things:
    • Using house.flags-match and update-flags: to avoid affected houses
    • Using house.is-suitable-for to target only suitable houses in the apply

5 Making things happen

Now that you (hopefully) have a feeling for what the model is, this part will show you how to make it do simple tasks like producing reports and making basic changes to the state of the stock.

5.1 When do things happen; what dates, how ties are broken

As mentioned above, when the model runs your scenario it does the following:

  1. Sets the date to the start-date: of the scenario
  2. Schedules things that are going to happen in the future
  3. Does the scheduled things in date order, until the end-date: of the scenario is reached

A small number of things are scheduled automatically:

  1. Loading the stock; this is always the first thing that happens
  2. Paying fuel bills and operational costs; this always happens annually, at the very end of the year. There is more information about how bills and operational costs are determined in the sections on Money and Tariffs.

Everything else that happens must be scheduled to happen by something written in your scenario. The simplest tool for scheduling is a command called on.dates. If you look it up in the manual, you will see that it belongs at the top level in your scenario (i.e. as an argument to the scenario command), and that it expects:

  1. An argument containing a list of dates, which say when something is to happen
  2. A series of arguments which define things that should happen

You can think of on.dates as a device for attaching some free-floating things to do to some specific dates, so that the things will be done on those dates.

For example, you might write:

 start-date: 01/01/2014
 end-date:   01/01/2020
 stock-id:   my-stock

  [01/01/2014 01/06/2014 01/01/2018]


This scenario will schedule the following:

  1. On 01/01/2014
    • Load the stock
    • Do the command insulate-some-houses (this is not a real command)
    • Do the command make-a-report (also not real)
  2. On 01/06/2014
    • Do the command insulate-some-houses again
    • Do the command make-a-report again
  3. On 31/12/2014, 31/12/2015, 31/12/2016, 31/12/2017
    • Pay fuel bills and operational costs; however, the default behaviour is that fuel and operational costs are zero, so effectively this does nothing
  4. On 01/01/2018
    • Do the command insulate-some-houses for a third time
    • Do the command make-a-report for a third time
  5. On 31/12/2018, 31/12/2019
    • Pay fuel bills and operational costs
  6. On 01/01/2020 the simulation finishes

If you want different things to happen on particular dates, you can put more than one on.dates in a scenario. The scheduled dates are interleaved. Any ties are broken by the order in the scenario. For example,

 start-date: 01/01/2014
 end-date:   01/01/2020
 stock-id:   my-stock

  [01/01/2015 01/01/2010]

  [01/01/2014 01/01/2010]

Sets up the following schedule (ignoring fuel bills etc.):

  1. On 01/01/2014
    • do do-thing-B (a made-up command)
  2. On 01/01/2015
    • do do-thing-A (also made-up)
  3. On 01/01/2010
    • do do-thing-A again
    • do do-thing-B again; this comes second because it is second in the scenario

The reason questions of scheduling are important is the same as why scheduling is important in the real world: the things which have already happened determine how things are now, so doing A then B may not have the same result as doing B then A. If B is producing information in a report and A installs some measures, A then B will show you the outcome of doing A, whereas B then A will show you how things were before doing A, and then install the measures, without telling you anything about it (doing B then A then B again will show you before and after, of course).

5.1.1 Other sorts of dates

Alone with writing literal dates in on.dates, you can also use the commands scenario-start, scenario-end, and regularly to produce the scenario start date, end date, or a repeating series of dates. For example, if you write



in your scenario, you will schedule do-some-thing to happen annually from the start date of the scenario; you can provide arguments from:, every: and until: to regularly to change the dates it produces. For example, (regularly from:01/01/2020 every:"3 months" until:01/01/2030) will produce quarterly dates from 01/01/2020 until 01/01/2030 (inclusive).

You can also write just the year, like 2015 as a shorthand for 01/01/2015, and you can write two dates joined by two dots to represent an annual sequence starting on the first up to the second. In this form, 2015..2020 is a shorthand for (regularly from:01/01/2015 until:01/01/2020 every:"1 year").

5.2 Making reports; making changes to the housing stock

Now we come to what sort of thing can you put in on.dates to make something happen; there are a few options, but the most useful one is probably apply. The manual for apply will tell you that it expects two arguments:

  1. An action to apply as the first argument
  2. An optional named argument to: which is a command to select a set of houses

apply can be thought of as a device for gluing a free-floating thing that you want to do to a population of houses. Putting this together with on.dates you can create an instruction to do a given thing, to a given population, on some particular dates:

(scenario ; start date etc. omitted
   to: (some-houses))))

A lot more detail is given on what kind of thing you can use in place of (some-action) or (some-houses) later on. For now, let's omit to: (some-houses) entirely (the manual explains that the default for to: is (all-houses), which unsuprisingly represents all the houses in the model), and focus on (some-action).

An action is the broad term for something in the model which does something to a house; this includes

  • measures, which are actions that represent an installation of some technology in a house. Examples include:

    • measure.loft-insulation and measure.wall-insulation, which would insulate the loft or walls respectively
    • measure.standard-boiler and measure.heat-pump, which would install a new boiler or a heat pump

    As you can see, most measure commands begin with measure.; this has no special meaning to the model, it's just to make the names distinctive.

  • Actions which put other actions together; for example you might want to install a package of several measures together, or choose between some alternatives
  • Actions which output information about the condition of a house, which are covered in the next section.

5.3 Targetting who things happen to; selecting houses with tests or at random

As mentioned, by default apply will operate on all the houses in the model. You can also use the commands filter, union, sample, and bernoulli to select subsets of the population.

filter is a command which will produce a set of houses by using another logical test (true-or-false) command on each house, and only keeping those houses which pass the test. For example

 to: (filter (house.has-loft) (all-houses)))

will apply some-action only to those houses which are (a) in all-houses (i.e. all the houses) and (b) for which house.has-loft is true. house.has-loft is an example of a boolean function or test. You can think of what happens here as:

  1. something triggers the apply (probably on.dates)
  2. apply asks (filter ...) to produce a set of houses to act on
    1. (filter ...) asks (all-houses) for a set of houses, getting back all the houses
    2. filter goes through each house in turn, and asks house.has-loft about that house
    3. filter responds to apply with the houses which passed the test
  3. apply shuffles the houses that were chosen into a random order
  4. apply goes through each chosen house in turn, and asks (some-action) to operate on that house

If you are looking for a particular number of houses, but you don't mind exactly which ones, you can use sample; it will sample randomly from a set of houses to produce a result of a certain size. For example

 to: (sample 15% (all-houses)))

will take the houses in all-houses, pick a random subset of them whose size is 15% of the total, and then apply some-action to them in a random order. sample will interpret samples of size less than 1 as proportions (15% is equivalent to 0.15, as far as the model is concerned), and samples of size 1 or more as counts, so

 to: (sample 1000000 (all-houses)))

will take the houses in all-houses, pick a random subset of them whose size is 1 million, and then apply some-action them in a random order.

You can combine sample with filter by using one as an argument to the other; for example

(sample 10% (filter (house.has-loft) (all-houses)))

produces a random 10% of those houses which have a loft. This is not the same as

(filter (house.has-loft) (sample 10% (all-houses)))

which instead takes a random 10% of all the houses, and then retains only those which happened to have a loft; the size of the second of these will probably be smaller than the first. When reading this, you can see that the command with the deepest number of brackets is worked out first; this is always true for the sampling commands (their precedence is as though for arithmetic).

If you want to sample houses at random using a per-house probability, you can use the bernoulli command. It is like a probabilistic version of filter. Where filter expects a logical function about a house and uses it to deterministically rule the house in or out, bernoulli takes a numerical function about a house, and uses it to compute a probability that the house will be included or excluded. For example

 to: (bernoulli 10% (all-houses)))

does the following:

  1. again, on.dates will trigger the apply on some dates
  2. on.dates asks bernoulli for some houses
    1. bernoulli asks all-houses for some houses, getting all the houses
    2. for each of these houses in turn, bernoulli produces a uniform random number from 0 to 1; if the number is less than or equal to 0.1, the house is included, otherwise it is excluded
    3. bernoulli returns all included houses to the apply
  3. apply takes each included house in turn, in a random order, and asks some-action to act on the house

The difference between bernoulli with 10% and sample with 10% is that bernoulli may pick more or less than 10% - it could pick 100% or 0%, if the random numbers came up right. In contrast, sample will always pick 10%3.

5.4 Things which happen by themselves; carbon factors, weather, opex and bills

Along with things scheduled using on.dates, there are a few things which are scheduled 'automatically' by the model. We have already touched on opex and bills, which are processed at the very end of each year. The scenario can also contain 'global' carbon factors and weather, which can be functions of time. If one of these is a function of time, the simulation will contain some automatically included events when the carbon factors or weather would change.

5.5 Exercises about scheduling events

  1. Referring to the manual and using your knowledge of energy modelling, what are the significant differences likely to be between the following measure applications?

    1. Naive:

        size: (size (house.peak-load))
        capex: (* 100 (size.kw)))
        capex: 500       type: cavity
        resistance: 0.01 thickness: 50))
    2. Less naive:

        capex: 500       type: cavity
        resistance: 0.01 thickness: 50)
        size: (size (house.peak-load))
        capex: (* 100 (size.kw))))
    3. Package:

       (do all:true
            capex: 500       type: cavity
            resistance: 0.01 thickness: 50)
            size: (size (house.peak-load))
            capex: (* 100 (size.kw)))))


    • Higher up things tend to happen first
    • Things which happen first can affect what comes after, but not vice-versa
    • Packages affect suitability
  2. Consider the following partial scenario, in which A, B, C, D, E, F, and G represent things that are to happen (e.g. application of measures, running reports etc.)

     start-date: 2005
     end-date:   2011
     (on.dates scenario-start A)
     (on.dates scenario-end B)
     (on.dates scenario-start C)
     (on.dates repeatedly E)
     (on.dates 1990..2015 F)
     (on.dates [2005 01/05/2010 02/01/2011] G))

    In what order and on what dates are A to F performed; note that some of them may happen more than once and some may happen on the same date but still have an order.

5.5.1 Answers:

  1. There are two things going on here:

    • Whether the insulation or boiler measure happens first (1 versus 2 and 3)
    • Whether the measures are combined into a package or not (3 versus 1 and 2)

    Installing the insulation first in this case will probably make the boilers cost less. This is because:

    • The boiler's capex depends on size.kw (it's £100/kW)
    • The boiler's size depends on house.peak-load. Reading the manual for house.peak-load:

      This is determined using Newton's law of cooling, so peak load = temperature difference * specific heat loss.

      Our knowledge of energy modelling tells us that if we install insulation we will reduce the specific heat loss, so if insulation goes in first the boiler will be smaller. On the other hand, the cost of insulation is only affected by the insulated area, so if the boiler goes in first, it is sized for the bigger heat loss, and then the insulation goes in exactly as before.

    So, if the same houses were exposed to 1, 2, and 3 in separate scenarios, the results for 2 and 3 would have lower boiler costs.

    The second difference is between 1 and 2, and 3; 1 and 2 are supplying two separate measures to the apply command, one after another. The documentation for apply says about its unnamed arguments:

    the actions to apply; the first action will be applied to each house in turn, then the second will be applied to each house in turn, and so on.

    So in the first two cases, if the houses are house A, B, … Z, we will be doing the following:

    For case 1: go to house A and try to install a boiler, then go to house B and try to install a boiler, … then go to house Z and try to install a boiler, then go back to house A and try to install insulation, then go to house B and try to install insulation, and so on.

    For case 2: go to house A and try to install insulation, then go to house B and try to install insulation, … then go to house Z and try to install insulation, then go back to house A and try to install a boiler, then go to house B and try to install a boiler, and so on.

    Case 3 is quite different; we are using the do action, with the two measures inside it to create a package.

  1. The commands happen as follows:

    Date Command
    01/01/2005 A,B,C,E,F,G
    01/01/2006 E,F
    01/01/2007 E,F
    01/01/2008 E,F
    01/01/2009 E,F
    01/01/2010 E,F
    01/05/2010 G
    01/01/2011 E, F

    The commands happen left-to-right top-to-bottom in the table (so it goes A,B,C,E,F,G,E,F,E,F…)

    Points to note:

    • Nothing which lies outside the start and end dates happens
    • Things which are on different dates happen in order
    • Things on the same date happen according to their order in the scenario.

5.6 Exercises about sampling houses

  1. For each of these sampling commands, give a description of the sample that is produced which states the range of sizes of the sample, and anything else you may know about the houses in the sample.
    1. (sample 50%)
    2. (bernoulli 50%)
    3. (sample 50% (filter (house.region-is London)))
    4. (filter (house.region-is London) (sample 50%))
    5. (filter (house.built-form-is Detached) (filter (house.region-is London)))
    6. (union (filter (house.region-is London)) (filter (house.built-form-is Detached)))
  2. Write a sampling command to produce:
    1. The set of all the houses
    2. A random 5% of all the houses
    3. A random 100,000 houses
    4. All the houses in the South West
    5. 20% of the houses in the South West
    6. 20% of the houses in the South West and 10% of the houses in London
    7. 30% of the houses in the South West and in London
    8. A set in which each house has a 5% chance of membership
    9. A set in which a house's chance of membership is given by its floor area, clamped to 100.
    10. All the detached houses
    11. All the detached houses in the south west
    12. A set of detached houses, taken from a 10% sample of all the houses
    13. 10% of the detached hosues

5.6.1 Answers

  1. These commands produce:
    1. A sample containing 50% of all the houses that exist. Its size will (to within 1 quantum) be exactly 1/2 of the total weighted count.
    2. A sample containing somewhere between zero and all of the houses. Its size will, in expectation, be 1/2 of the total weighted count. Each case has a 50/50 chance of being in the set.
    3. A sample containing 50% of all the houses that are in London. Its size will (to within 1 quantum) be exactly 1/2 of the total weighted count in London. All the houses in it will be in London.
    4. A sample containing only the houses which turned out to be in London when a sample of 50% of all the houses was taken. All the houses in it will be in London. Its size will range from the size of London minus half the total population (or zero, if London is smaller than half the total population), up to whichever is less between the size of London and half the total population.
    5. A set containing all the houses which are both in London and are Detached.
    6. A set containing all the houses which are in London and all the houses which are Detached. It will contain houses which are not detached, and houses which are not in London, but every house in it will be either in London, detached, or both.
  2. You could produce these outputs with
    1. (all-houses)
    2. (sample 5% (all-houses)), or equivalently just (sample 5%)
    3. (sample 100000 (all-houses)) or (sample 100000)
    4. (filter (house.region-is SouthWest) (all-houses)) or just (filter (house.region-is SouthWest))
    5. (sample 20% (filter (house.region-is SouthWest))), reads as "sample 20% of [filter [houses in the south west] from all the houses]"
    6. (union (sample 20% (filter (house.region-is SouthWest))) (sample 10% (filter (house.region-is London))))
    7. (sample 30% (union (filter (house.region-is SouthWest)) (filter (house.region-is London))))
    8. (bernoulli 5%)
    9. (bernoulli (min 100% (* 100% house.floor-area)))
    10. (filter (house.built-form-is detached))
    11. (filter (house.built-form-is detached) (filter (house.region-is southwest))), or (filter (all (house.built-form-is detached) (house.region-is southwest)))
    12. (filter (house.built-form-is detached) (sample 10%))
    13. (sample 10% (house.built-form-is detached))

6 Getting information out

We have now covered the idea that the simulation contains a house-centric model of the world, and that you can apply different actions and measures to sets of houses to change that We have not however said much about how to find out the consequences of the changes which we are simulating. This is a significant omission, as all programs without some kind of output are equivalent; the whole reason for having a program is to get some output!

Outputs from the NHM are usually referred to as reports; normally they are tables, which the model stores in 'tab-separated values' format. These are contained in files ending with the .tab extension, and are simply plain text files in which each line represents a row of the table, and within each row the columns are delimited by 'tab' characters12. These tab-separated files can be read using just about any tool, including Microsoft's Excel, R, SPSS, SAS and many others.

To persuade the NHM to produce a report, you need to include some commands in your scenario which instruct the simulator accordingly. It is not possible to extract any more information from a simulation than you requested after the fact without changing the scenario and re-running it, so if your scenario takes a long time to run take a moment to be sure you have the outputs you need before setting it in motion.

6.1 An overview of the reporting commands

There are four commands for producing reports which you are most likely to use:

  1. aggregate, a command which outputs summary information about populations of houses. This is useful for keeping track of high-level trends in your scenario.
  2. probe, a command which outputs information about individual houses. This is useful for generating detailed data which you can process using other tools, or for digging into what is happening when the model produces confusing results.
  3. probe.aggregate, a command which produces summary information about the impact of a specific action on a population of houses. It is the summarising analog for probe.
  4. def-report and the report: keyword, which are relatively new commands that combine the features of aggregate and probe.

We will consider these in order

6.2 Using aggregate to produce summaries of the stock

The aggregate command is analogous to apply in that it should be used within on.dates. Where apply uses a sampling command to find a population of houses, and then does a measure to them whenever the surrounding on.dates tells it to, aggregate uses some sampling commands to find some populations and then uses some aggregation commands to work out summary statistics on those populations when the surrounding on.dates tells it to.

The general form of aggregate is:

    name: <<the name of the report file to produce>>
    over: [ <<a list of sampling commands>> ]
    divide-by: [ <<a list of cutting variables>> ]

    << a series of aggregations, like aggregate.count, aggregate.mean, etc. >>

When an aggregate is triggered, it does the following:

  1. Compute the populations indicated by over:
  2. For each population, for each dwelling in the population, work out the divide-by: values
  3. For each sub-population induced by divide-by: values:
    • For each aggregation contained in the aggregate, work out that aggregation and record it in the result against the current date, the over: population and the divide-by: values.

For example, here is a very simple aggregate:

    over: [(all-houses)]
    divide-by: []

    (aggregate.count name:count))

Let us say this is triggered at the start of the run on 01/01/2020; what happens is:

  1. The set (all-houses) is computed, which is all the houses
  2. The divide-by: values are used to cut up all-houses; there are none, so it is not divided
  3. The resulting set, which is just all houses, is presented to aggregate.count, which produces the total weight
  4. This value is written down as a row in the report example-1, as follows:

    Date Group Count
    01/01/2020 (all-houses) 24000000

If the on.dates was set to trigger on several dates, each date would add a row to the table, producing something like:

Date Group Count
01/01/2020 (all-houses) 24000000
01/01/2021 (all-houses) 25000000

6.2.1 Adding more groups

Let us add another group to the aggregate:

    over: [(all-houses) (filter (house.region-is London))]
    divide-by: []

    (aggregate.count name:count))

The result is:

  1. The set (all-houses) is computed, which is all the houses
    1. The divide-by: values are used to cut up all-houses; there are none, so it is not divided
    2. The resulting set, which is just all houses, is presented to aggregate.count, which produces the total weight
    3. This value is written down as a row in the report example-1
  2. The set (filter (house.region-is London)) is computed, and the same process repeated; the final table now has two rows:

    Date Group Count
    01/01/2020 (all-houses) 24000000
    01/01/2020 (filter (house.region-is London)) 10000000

Note here that (all-houses) and (filter (house.region-is London)) are intersecting, so the same houses are being represented in multiple rows. The model will not guess if you mean to say "houses in london versus other houses"; you would have to define those two sets or use divide-by:.

6.2.2 Using divide-by:

The divide-by: argument effectively produces more groups, but in a convenient syntax. It breaks each of the groups from the over: argument down by a series of values about each house. For example, we could amend the first example above to be divided up by built form:

    over: [(all-houses)]
    divide-by: [(house.built-form)]

    (aggregate.count name:count))

Now when triggered the aggregate does this:

  1. Compute (all-houses), producing a set of all the houses
  2. For each set produced (here just all-houses), work out (house.built-form), and break the set into subsets according to its distinct values. Built form is a categorical variable with the values MidTerrace, EndTerrace, SemiDetached, Detached, Bungalow, ConvertedFlat, PurposeBuiltLowRiseFlat and PurposeBuiltHighRiseFlat, so we will have a subset for each category that has some representatives.
  3. For each of these subsets, each aggregation is computed. In this case, that is just the count; we end up with the following table:

    Date Group house.built-form count
    01/01/2020 (all-houses) MidTerrace
    01/01/2020 (all-houses) EndTerrace
    01/01/2020 (all-houses) SemiDetached
    01/01/2020 (all-houses) PurposeBuiltLowRiseFlat

If we add more than one divide-by: variable, we will have a further breakdown. For example, we could add the region as well:

    over: [(all-houses)]
    divide-by: [(house.built-form) (house.region)]

    (aggregate.count name:count))

Now (all-houses) will be divided up according to all the extant combinations of (house.built-form) and (house.region), and a count produced for each distinct combination. Unlike in the second example where we specified two groups in the over: argument, for a given group the different divide-by: rows are always mutually exclusive, because they vary in at least one of the dividing values.

6.2.3 Different kinds of aggregation

In the examples so far we have seen aggregate.count. This is a command which knows how to produce a number when another part of the model, like aggregate, presents it with a set of houses; it produces the weighted count. The sets of houses used are provided implicitly by the context in which aggregate.count is used. There are other commands which produce a number from a set of houses:

  • aggregate.mean, which works out the weighted mean of another function For example:
    • (aggregate.mean (house.total-floor-area)) works out the mean of the floor area when it is used on a set of houses
  • aggregate.sum, which works out the weighted sum of another function For example:
    • (aggregate.sum 1) is the same as aggregate.count. It goes to each house in the set, calculates the number 1, multiplies it by the weight, and sums the values.
    • (aggregate.sum (house.energy-use)) adds up the weighted total energy use for all the houses in the set.
  • aggregate.n-tile, which works out a weighted n-tile (like the median) of another function For example
    • (aggregate.n-tile n: 0.5 (house.energy-use)) works out the median energy use for all the houses in the set
  • aggregate.where, which works out another aggregate but only on houses which pass a given test For example
    • (aggregate.where (house.region-is London) (aggregate.mean (house.energy-use))) will work out the mean energy use for all the houses in the set which are in London. You may have noticed how this looks a bit like using one of the sampling commands on another one - aggregate.where contains an aggregate.mean, and modifies its input.

6.2.4 Using aggregate effectively

Remember the rule for scheduling that we learned above! If an aggregate command is triggered on the same date as an apply command which changes some houses, in the rows for that date it will either (a) see the results, because it is written after the apply, or (b) not see the results, because it is written before the apply.

If you want your aggregate to always see the results of changes that are happening every year, you can trigger it before and after the changes in each year.

For example, in some code like this:

    [ 01/01/2015..01/01/2020
      02/01/2015..02/01/2020 ]
    (aggregate name: report-which-sees-all

    [2015 2016 2018]

    (apply ...))

The aggregate will run on the first and second of January every year, and the apply on the first of January 2015, 2016 and 2018. Since the events on the first are tied, the tie will be broken by position and the aggregate will happen first. As a result, the rows on the first will show how things started out in the year, and the rows on the second how they were affected.

However, if you really want to determine the effect of your actions, there are better ways - aggregate is most useful for producing simple counts or sums that you can look at, for example to count up how many measures were installed in a given year.

6.3 Using probe and probe.aggregate to see what has happened

As we just saw, using aggregate to determine exactly what is going on in a scenario can be quite difficult, and it is not intended for producing results that go down to house level13. To help with this, the model offers the probe command. We have already seen the pattern of putting a command around another command of the same kind to change its behaviour with sample, filter and aggregate.where; probe follows this pattern, but for an action that you are applying to a house. Putting a probe around another action changes the action's behaviour so that it also makes a report which describes what happened whenever the action gets used.

For example, imagine we are installing a new boiler in some houses; the command for this is measure.standard-boiler, so our scenario might have something like this in it:

(on.dates ...
    (apply to: (sample 100000)
           (measure.standard-boiler fuel:MainsGas winter-efficiency: 85%
                                    update-flags: affected))
    (aggregate name: heating-systems
           divide-by: [house.main-heating-system-type house.flags]
           (aggregate.sum (house.energy-use))))

This example uses flags, which are described later on; here it suffices to say that when a house get a boiler it will also be "marked" as affected, and that mark will appear in house.flags which is being used to divide up the population in the aggregate.

We are hoping to install 100,000 boilers every year, and expecting that the overall energy use will go down by some amount. However, when we run the scenario, we have two problems:

  1. We don't seem to be installing 100,000 boilers every year, and
  2. The effect on energy use is not as much as we thought.

It seems that what we have told the model to do is not quite what we intended!

One way to diagnose this kind of problem is to use probe; as described it goes around an existing measure or action. In this scenario there is only one measure, which is the boiler, so we amend the example to include a probe; probes have the following form:

(probe name:<<name of report>>
       capture: [ << list of values to record>> ]
       <<another measure or action>>

so adding the probe around our existing measure we get:

(on.dates ...
    (apply to: (sample 100000)
           (probe name:boiler-investigation
                  capture: [
                      (house.main-heating-system-type name:h)
                      (house.flags name:f)
                      (house.heating-efficiency measurement: winter
                                                of: PrimarySpaceHeating
                                                name: e)
                  (measure.standard-boiler fuel:MainsGas winter-efficiency: 85%
                                           update-flags: affected)))
    (aggregate name: heating-systems
           divide-by: [house.main-heating-system-type house.flags]
           (aggregate.sum (house.energy-use))))

You can see that the probe surrounds the measure, occupying the space it used to occupy; from the point of view of the other elements of the scenario, the probe is kind-of "transparent", in that the meaning of the simulation itself is unchanged. However, from our point of view the simulation is changed, in that we will have a new output file called dwelling/probe-boiler-investigation.tab, which will help us in our diagnosis. If you write a scenario like this now and run it, you will see that file contains some output of this form (some columns omitted for space reasons):

Date dwelling-id weight h (before) f (before) e (before) suitable h (after) f (after) e (after)
2015 1234 500 StandardBoiler   0.8 true Condensing affected 0.85
2015 5678 400 StorageHeater   1 false n/a n/a n/a
2016 1234 500 Condensing affected 0.85 true Condensing affected 0.85

Each row in this table corresponds to a moment in the simulation when a particular simulated dwelling was sent to the probe by the apply as a result of the on.dates triggering it. You can see that every row contains the date at which the change is happening, a sequence number, the unique ID of the dwelling which is being considered, and the weight of that dwelling so that you can make weighted calculations if you need to. As well as these values, each row also contains (1) the values given to capture: about the house before the measure within the probe got applied, (2) an indication of whether the measure succeeded or not, and (3) when the measure did succeed columns providing the capture: values after the measure's application. In the example table above, you can see one dwelling was presented and got a measure (dwelling 1234), and another was presented and was not suitable (5678). This report can therefore show us exactly which houses were considered, and how (if) they were affected.

In our hypothetical conundrum we might analyse the table and notice the following:

  1. On each chosen date, around 100,000 dwellings were presented to the probe; the sum of the weight column would be 100,000 when grouped by the date column
  2. However, under 100,000 dwellings had a successful measure installation. This explains why the aggregate contained a lesser effect than we expected.
  3. We might then filter the results to show only the cases where the measure did not install, and look for regularities. We might see that most of these houses have electric heaters of some sort; perhaps they are off the gas grid, and so not eligible for a gas boiler?
  4. Finally, we would notice that in later years we are operating on some houses which already have an 85% efficient mains gas boiler, and for which house.flags includes affected. This would remind us that the sample command does not automatically understand that we want non-overlapping samples between years.

6.3.1 probe.aggregate

There is an analog to probe which probes the aggregate effect of an action; it is like the aggregate command combined with the probe command.

For example, you can write

    (probe.aggregate name:aggregate-effect
       capture: [(aggregate.mean house.energy-use)]
       (measure.wall-insulation ...)

To see the before-and-after effect on mean energy use resulting from applying the wall insulation measure. The results will be broken down according to whether the measure succeeded.

As with the aggregate command, you can also provide probe.aggregate with a divide-by: argument to cut the results according to different aspects of the house.

6.4 Using def-report and the report: argument

probe and aggregate were both added fairly early in the development of the model, and are a quick and efficient way to inspect bits of a scenario. However, if you find that you want to probe a lot of actions, and also want some summary statistics produced for the same outputs, they can become quite tiresome to write down. The def-report, which was added more recently, gives an alternative way to report on houses. You use it by defining a report in one place (in the top level of the scenario), and then by sending houses to that report from other places where you want to know what is going on.

Defining a report is a matter of using the def-report command, which has the general form:

(def-report <<name of report>>
    (column ...)
    (column ...)
    (cut ...)
    (cut ...))

So for example, if we wanted a report which told us about the built form and energy use of houses we might write

(def-report example-report
            (column name: built-form value: house.built-form)
            (column name: energy-use value: house.energy-use))

Each column here has a name:, which is the text used to name the column, and then a value which is a command used to compute the value for that column when a house is sent to the report. The easiest way to send a house to the report is to use the send-to-report action, for example:

(on.dates (scenario-start)
            to: (all-houses)
            (send-to-report from:this-place #example-report)))

This is all similar to what you have seen before, except for the octothorpe (#) symbol written before example-report. It is used to indicate that the word example-report is a cross-reference to the report defined by the def-report we wrote above. In the NHM application's scenario editor, these cross-references will be coloured in differently to make them stand out. Since send-to-report is an action, we can put it inside apply like any other action, and it gets used on all the houses in the to: set; in this example, all the houses that exist. Each time a house is sent to a report, two things happen:

  1. It is treated as though it were observed by a probe command; in this case, a row will be entered into a file called dwelling/probe-example-report.tab which contains the built form and energy use for the house. In addition it will contain the from: argument for send-to-report, so that you can send houses to a report from several places and determine which rows came from where.
  2. The values observed are summarised, and when the simulation date advances they may be contributed an aggregate summary of all the houses which were sent to the report since the last time the date changed. In our example, the summary will contain only the date, the from: column and a weighted count. However, you can add more summary statistics and aggregations by using the summary: argument of column and by adding the cut command into the def-report

6.4.1 Column summaries

Each column can have multiple summary values computed for it using the summary: argument. Let us amend our example report to use this facility

(def-report example-report
            (column name: built-form value: house.built-form summary: [(in detached semidetached)])
            (column name: energy-use value: house.energy-use summary: [min max mean]))

This report will now generate the following summary statistics:

  • the count of houses where built form is either detached or semidetached
  • the minimum, maximum and mean values for energy use

Available summaries are sum, mean, variance, min, max and n-tile (for numbers), count (for logical values), and in (for categorical values). The language reference describes these in more detail.

6.4.2 Cuts

Cuts are a way of replicating the divide-by: argument in the aggregate command. Each cut in a def-report produces another aggregate output, in which the results will be broken down by the columns named in the cut. These columns must be in the report. For our example, let us add a break down by built form:

(def-report example-report
            (cut name:my-first-cut built-form)
            (column name: built-form value: house.built-form summary: [(in detached semidetached)])
            (column name: energy-use value: house.energy-use summary: [min max mean]))

The word built-form in the cut refers to the column name:built-form below it. An aggregate report called my-first-cut-of-example-report will be produced, in which the summary: values are shown for all houses sent to the report, broken down by date of sending and the built form column.

6.4.3 Sending from other actions

One of the really useful things about def-report is that you can send houses to it from many places. This does not have to involve send-to-report; every action in the model has the named argument report:, which accepts a list of reports to send the house to. The house will be sent to the report, before and after the action is taken, just as though it had been placed inside a probe command.

When the report: argument is used, the name: of the sending command is used where the from: argument is used in send-to-report.

For example, if using the action (measure.standard-boiler name:my-boiler-measure report:#example-report), the example report would get rows with my-boiler-measure in the from column.

6.4.4 A whole scenario

We could re-write our example probe scenario above using def-report like this:

(def-report heating-systems
            (column name:h value: house.main-heating-system-type)
            (column name:f value: house.flags)
            (column name:e value: (house.heating-efficiency measurement:winter of:primaryspaceheating)
                           summary: [mean min max])
            (column name:energy value:house.energy-use summary: sum)
            (cut name:cut-heating-and-flags h f outcome))

(on.dates ...
      (apply to: (sample 100000)
                    (measure.standard-boiler fuel:MainsGas winter-efficiency: 85%
                                             report: #heating-systems
                                             update-flags: affected)))

As written this is not much shorter, but you can see how if we were installing several measures in different places, sending them to the report would be easier.

6.4.5 Special columns

In the example above, we have shown one of the special columns that can be included in a cut; these are:

  • outcome; this column is true if the measure succeeds and false otherwise. A measure will fail if it is unsuitable for the house. This is explained in more detail later.
  • sent-from; this column is the from: attribute of the originating send-to-report, or the name: attribute of a measure used with report:
  • selected; this column is true if the condition being reported on ends up become part of the model's view of how things really are. This is relevant for hypotheses and choices, in which measures may be installed in a house hypothetically but where the hypothesis may never be realised. This is explained more in the later section on hypotheses and choices.

6.5 Other reports; what measures have gone into which houses; who spent what money

As well as the user-defined reports you can produce with probe, aggregate, probe.aggregate, and def-report with report:, there are several built-in reports which produce other kinds of outputs that can help you analyse the results of a simulation.

6.5.1 report.aggregate-measure-costs

Produces a summary table detailing the number, cost, and size of measures which have been installed

6.5.2 report.fuel-costs

Reports annually on the total fuel cost paid by each house.

6.5.3 report.global-accounts

Reports on the balance of the model's global accounts; whenever a house receives or spends money, this is transferred from one of a number of global accounts. This report shows changes in the global accounts' balances as they change.

6.5.4 report.measure-installations

Produces a dwelling-level log of measure installations. When a measure is installed in a dwelling, a row is added to this file containing the type of measure, the cost, the size, the date, and the dwelling ID for which the measure was installed.

6.5.5 report.sequence

Produces a report detailing the sequence of events that was simulated. This can be useful to determine what order things are ending up happening in. It also contains information about the random number generator state, which allows you to determine at what point two simulations containing stochastic behaviour have diverged.

6.5.6 report.transactions

Produces a log of every transaction that has happened in the model. This is described more fully in the section on 11, but in summary each row describes the transfer of some money between a house and a global account, or in some cases between two global accounts. Such a transfer happens whenever a house receives a measure, or a subsidy, or pays its fuel bill, or similar.

6.6 Exercises about reports

  1. Amend an existing scenario to include a report which allows you to answer these questions:
    1. How many houses are there in each region?
    2. How many houses are there of each built form?
    3. How many houses are there whose floor areas are

      • 0-10m
      • 10-100m
      • 100-1000m
      • 1000m-10000m and so on.

      Hint: you could use the log and round commands for this.

    4. For each of these categories, what is the mean energy use and emissions
  2. Amend a scenario to emit a report from which you could scatter plot the relationship between floor area and modelled energy use by some different types of fuel.
  3. Amend a scenario in which you install a measure to emit a report which tells you exactly which houses were affected by the measure, and what the measure's impact on their carbon emissions (house.emissions) was.
  4. Amend a scenario in which you install a measure to emit a report which tells you how many houses were affected by the measure, and what the measure's summed impact on their carbon emissions was, broken down by built form.

6.6.1 Answers

  1. These are all things which you can do with the aggregate command, or cut within def-report. We give an example of each:

    (aggregate name:exercise
        divide-by: (house.region)
        (aggregate.count name:count)
        (aggregate.mean name: mean-energy-use (house.energy-use))
        (aggregate.mean name: mean-emissions (house.emissions)))

    In this, the divide-by part could be replaced with (house.built-form) to answer question 2., or with (round (log (house.total-floor-area))) for question 3. Alternatively all dividing factors could be included in the same report within square brackets, in which case the report would detail each possible combination of factors. Using def-report instead:

    (def-report exercise
        (column name: energy-use value: (house.energy-use) summary: mean)
        (column name: emissions value: (house.emissions) summary: mean)
        (column name: house-region value: (house.region))
        (column name: house-form value: (house.built-form))
        (column name: house-area value: (round (log (house.total-floor-area))))
        (cut name: by-region house-region)
        (cut name: by-form house-form)
        (cut name: by-area house-area))
    (on.dates (scenario-start) (apply (send-to-report #exercise)))
  2. This is a candidate for a probe report, or def-report, as for a scatter plot we want to see information about each case. Using a probe:

    (on.dates (scenario-start)
       (apply (probe name: p
                     capture: [
                       (house.total-floor-area name: tfa)
                       (house.energy-use name: total)
                       (house.energy-use by-fuel: mainsgas name: gas)
                       (house.energy-use by-fuel: electricity name: elec)

    The def-report equivalent would be

    (def-report example
       (column name: tfa value: (house.total-floor-area))
       (column name: total value: (house.energy-use))
       (column name: gas value: (house.energy-use by-fuel: mainsgas))
       (column name: elec value: (house.energy-use by-fuel: electricity)))
    (on.dates (scenario-start)
       (send-to-report #example))
  3. Again, this is a probe or def-report; if our existing scenario has something like this in it for the measure:

    (on.dates ...
        (apply ...
               (measure.wall-insulation ...)

    when using a probe we would wrap the measure in the probe so:

    (on.dates ...
             (apply ...
                    (probe name:effects
                           capture: [
                           (measure.wall-insulation ...))

    alternatively with def-report

    (def-report effects
        (column name:emissions value: (house.emissions)))
    (on.dates ...
             (apply ...
                    (measure.wall-insulation report: #effects ...)
  4. This would be a job for probe.aggregate or def-report; again using the fragment above, we would have either

    (on.dates ...
             (apply ...
                    (probe.aggregate name:effects
                       capture: [(aggregate.sum (house.emissions))
                       divide-by: (house.built-form)
                       (measure.wall-insulation ...)


    (def-report effects
        (column name: emissions value: (house.emissions) summary: sum)
        (column name: form value: (house.built-form))
        (cut name: by-form form outcome))
    (on.dates ...
       (apply ...
              (measure.wall-insulation report: #effects...)

7 Insulation measures

There are a few measures which perform insulation in the model; measure.wall-insulation, measure.loft-insulation, measure.floor-insulation, measure.glazing and so on. All of these work in fairly similar ways.

7.1 How does the model think about insulation

Before we come to the details of each measure, let's revisit how the energy calculator thinks about insulation. Insulation is important to the energy calculator because it affects the specific heat loss of the building; roughly speaking this is worked out as

\[ H = V + \sum_s A_s \cdot U_s \]

Where the sum is over external surfaces \(s\), and \(A_s\) is the surface's exposed area and \(U_s\) the u-value of the surface. \(V\) is an additional term covering heat losses due to ventilation, which is not so interesting here.

This value \(H\) is important because it is (roughly speaking) proportional to the heat required in the house, and so the fuel that must be burned to meet that requirement; halving \(H\) will roughly halve the fuel inputs for space heating.

In each modelled house, there are various external surfaces which contribute to the sum above:

  • Walls
  • Windows
  • Doors
  • Floors
  • Roofs

For each of these, the model has a stored u-value and area5, and some information about what kind of insulation is present. When an insulation measure is applied to a house, it will change the u-value and the information about what kind of insulation is present.

You may be used to thinking of insulation in terms of a final u-value; in the interests of verisimilitude, the model prefers to use r-values (resistances). An r-value is simply the reciprocal of a u-value, but the important thing is that r-values can be added together; imagine you are applying the same insulation to two different walls. The first wall has very poor thermal performance (let us say a u-value of 10), whereas the second has good thermal performance (\(u = 0.5\)). Let us also say that the insulation material on its own has a u-value of 1 (neither excellent nor terrible).

The u-value of the wall after insulation should clearly not be the same in both cases; instead it is determined by the combined performance of the wall with the new material. The best-case value for this is the sum of the two layers' thermal resistances, so for wall 1:

\[ u' = \frac{1}{1/10 + 1/1} = 0.91 \]

whereas for wall 2:

\[ u' = \frac{1}{1/0.5 + 1/1} = 0.33 \]

Those measures in the model which add insulation to an existing surface are usually controlled by a (per-unit-thickness) resistance: and thickness: arguments, which are multiplied together to get a total resistance, which is then used to modify the existing u-value of the surface. In these cases you can specify a final u-value function if you prefer, in which case the impact of the insulation is independent of the initial condition of the wall.

Those measures which replace an existing insulated element (like glazing) are controlled by a simpler u-value, as the old material is being entirely replaced and so should not affect the end result.

Insulation measures also have some measure-specific rules around their suitability and the parts of a house which they affect:

7.1.1 Wall insulation

In the NHM, a building is composed of a series of storeys, and each storey has a floor plan which is a polygon in which each line is a wall. Each of these walls has a construction type, which indicates what the wall is made of, and a record of how much of three different kinds of insulation (internal, external and filled cavity) are found on the wall. Walls with different construction types can accept different sorts of insulation, summarised in this table:

Construction Internal External Filled Cavity
Granite or Whinstone Yes Yes No
Sandstone Yes Yes No
Solid Brick Yes Yes No
Cob Yes Yes No
Cavity Yes Yes Yes
Timber Frame Yes Yes Yes
System Build Yes Yes Yes
Metal Frame Yes Yes Yes
(Party walls) No No No

Normally, a wall cannot have a form of insulation if it already has some insulation of that form, although this can be overridden if desired.

A house then is suitable for a given form of insulation if, for at least one of its walls:

  1. The wall's construction type will accept that type of insulation, per the table above
  2. The wall does not have any insulation of the given form so far.

Upon installation, the wall's u-value will be changed, and the wall will be marked as having some of the given form of insulation.

7.1.2 Loft insulation

Each storey in an NHM house may have some exposed surface on its top. All of these exposed surfaces are considered to form the insulatable roof area. The existing insulation thickness and u-value are stored, as well as information about the presence of a loft.

Unlike wall insulation, loft insulation can be installed when there is already some insulation present, when the measure allows it; this is controlled by the top-up: argument.

Loft insulation is suitable when:

  1. There is some roof area
  2. The roof's construction type is not pitched slate or tiles
  3. The measure is a top-up measure and the existing thickness is less than the target thickness, or the measure is not a top-up and there is no insulation present
  4. The house has a loft

When top-up insulation is added, the resistance which is added is determined using the thickness topped up, so a top-up from 200mm to 300mm with a resistance of 0.01 will add a resistance of 1.

7.1.3 Glazing

Each of the four elevations (front, back, left and right) of a house in the NHM have a 'glazed proportion' (this is how the data is recorded in the EHS), and that fraction of each wall belonging to a given elevation is taken to be glazed. The glazed area may be broken down into different kinds of glazing, as some houses will have mixed types of glazing.

NHM glazing measures are simpler than the other insulation measures; installing glazing simply replaces all the glazing already in the house with the new sort, and this is always considered to be a suitable action to take.

Note that this means that you can replace double glazing with single glazing, if you so wish; the suitability rules are intended to be as basic as possible, preventing things which are infeasible rather than things which are undesirable.

7.1.4 Actions to set insulation (as opposed to measures)

As well as offering insulation measures, the model has some corresponding actions which will change the insulation state of parts of the house without respect to suitability and without charging any money.

These are action.set-loft-insulation, action.set-wall-insulation, action.set-glazing and so on. They are not intended for modelling the installation of measures, but instead are there as a blunt instrument for changing a house into a given condition.

Even blunter instruments include action.reset-walls and others, which will change the thermal properties of all the walls in a house in one go.

7.2 Capital cost of insulation, and size.m2

When an insulation measure is installed, it temporarily records the area which was insulated and makes it available through the size.m2 command. The value of size.m2 is particular to the measure which contains it in the scenario; it is not a record of the total area insulated so far or anything, but is an ephemeral value which should be used during the installation of a measure or package.

To make this more explicit, if you write

(measure.wall-insulation ... capex: (* 100 size.m2))

When the capex: rule is worked out, size.m2 will produce the area insulated by this application of this measure to the current house.

Similarly, if you write:

(apply (measure.wall-insulation ... capex: (* 100 size.m2))
       (measure.loft-insulation ... capex: (* 200 size.m2)))

The capex: rule for wall-insulation will be 100 pounds per square meter of wall insulation installed by the wall insulation measure, and the capex: rule for loft insulation will be 200 pounds per square meter of roof area that was insulated.

This hopefully illustrates how the value of size.m2 depends on where it is written. A side-effect of this is that if you write something like this:

(aggregate name: total-area
           (aggregate.sum size.m2))

To report on "total insulated area", this will not work; as far as the model is concerned, this means something like "report on the total area insulated by doing this report", which unsuprisingly is zero. If you wish to report on ephemeral values like size.m2, you must do one of two things:

  1. Report on them using a probe in the right context:

    (probe capture: [size.m2]
           (measure.wall-insulation ... ))


    (probe.aggregate capture: [(sum size.m2)]
                     (measure.wall-insulation ...))
  2. Store the ephemeral values into a variable from the right context, and then refer to them later:

    (do (measure.wall-insulation ...)
        (increase #insulated-area size.m2))

    The do and increase commands and variables have not been met yet, so you will have to skip ahead for more on these.

The cost produced by the capex: rule is available using the command capital-cost, which has the same behaviour; capital-cost is the capital cost of the command containing it, so reporting on capital-cost is asking for the capital cost of the report.

There is more about this in the section on money.

7.3 How to put in some insulation

Offering insulation measures is no different to using any other action: you use the apply command to try and put the measure into a population of houses. Each measure has slightly different parameters; here are some examples:

  1. Wall insulation:

    (apply (measure.wall-insulation
                type: cavity
                thickness: 50
                resistance: 0.002
                capex: (* 10 size.m2)))

    In this example we are offering wall insulation any construction that has a cavity. If any house has a wall with a cavity which is not already insulated, the wall will be marked as having 50mm of insulation, and its u-value changed by adding a resistance of \(50 \times 0.002\). The house will incur a cost of ten pounds per square metre of wall that was changed.

    You can directly specify a u-value instead, if you do not want the initial condition of the wall to matter:

    (apply (measure.wall-insulation
                  type: cavity
                  thickness: 50
                  u-value: 0.2
                  capex: (* 10 size.m2)))

    The u-value, resistance and thickness arguments can all be given formulae instead of constants, which will be worked out each time the measure is used. Here is an example using a table of u-values which depend on the build year of the house:

    (measure.wall-insulation ...
            u-value: (~lookup-table
                        row-keys: house.sap-age-band
                        [age  u-value]
                        [A    2]
                        [B    1]
                        [C    0.5]
                        [D    0.25]
                        [E    0.1]
                        [*    0.05]
  2. Loft insulation:

    (apply (measure.loft-insulation
                thickness:  300
                resistance: 0.02
                capex: 500
                top-up: false

    In this example, we will install 300mm of loft insulation on any suitable roof which currently has no insulation at all (the top-up: argument determines this). The u-value will be updated by adding a resistance of 300 * 0.02, and the house will pay a flat fee of 500 pounds.

    When using top-up: true, the u-value will be updated by adding the resistance (300 - existing thickness) * 0.02.

  3. Glazing:

    (apply (measure.install-glazing
                capex: (+ 100 (* 50 size.m2))
                frame-type: metal
                gains-transmittance: 0.9
                light-transmittance: 0.9
                glazing-type: triple
                insulation-type: lowehardcoat
                u-value : 0.2

    As you can see, glazing has more parameters than other forms of insulation; this is because of its effect on internal gains and lighting usage in the BREDEM calculation. You can find sensible values for these parameters in SAP; the defaults used by the model if you do not specify values do not appear to be sensible as of this writing, so leaving them out is not a good idea. In this example, we will install some triple glazing with a u-value of 0.2, at a cost of 100 pounds plus 50 pounds per square metre of windows installed.

    The measure will replace all the windows in the dwelling with windows of this standard.

    To give a quick summary:

    frame factor
    the amount of the window area not taken up by the frame (and so passing light and heat)
    frame type
    the material the frame is made of; this has no direct effect, but needs to be recorded as you may want it later if you wish to impose different u-values on a house and these depend on the frame type
    gains transmittance
    a factor indicating what proportion of heat from the sun is included into the calculation of solar gains
    light transmittance
    a factor indicating how much light from the sun passes into the house, which has a small effect on the lighting demand
    glazing type
    the kind of glazing; as with frame type, this has no direct effect
    insulation type
    akin to glazing type and frame type
    the thermal property of the window, which is significant

7.4 How to find out about insulation on houses

There are various commands which you can use to find out about insulation state, either for reporting purposes or to use to change scenario behaviour. These are all documented in more detail in the manual, but here is a summary list:

produces the proportion of a house's glazing which is double glazed or better.
produces the thickness of any floor insulation
produces the thickness of any loft insulation
produces the predominant wall type; this is the wall type which has the greatest external area
produces the type of floor construction for the ground floor
produces the type of roof construction
tests whether a house has a loft or not

a complex test which lets you determine whether any of the walls of a house satisfy conditions you are interested in. For example,

(house.any-wall has-cavity-insulation: true)

Will tell you whether the house has any walls for which there is some filled cavity insulation installed. Similarly

(house.any-wall has-cavity-insulation: false)

Will tell you whether the house has any walls which do not have cavity insulation (including those walls which cannot have cavity insulation).

(house.any-wall has-construction: anycavity)

Will tell you whether any of the walls have a cavity; these can be combined, so

(house.any-wall has-construction:anycavity has-cavity-insulation:false)

Will tell you whether a house has at least one wall which currently has no cavity insulation, and has one of the cavity construction types.

an analog of house.any-wall which only passes if every external wall a house has satisfies the conditions that you are interested in.

7.5 Exercises about insulation

Here is a drawing of a normal terraced house; the front elevation is on the left of the drawing.


  1. Work out its heat loss coefficient, as it is now (in this exercise we are disregarding some details; the extra height on storeys for the floor joists, ventilation losses etc; these factors are included in the model).
  2. Based on this, work out the amount of power required from a heat source to maintain a 7 degree temperature difference with the outside
  3. For each of the following measures, note which parts of the house change, and recalculate the heat loss coefficient and power to maintain a 7 degree temperature difference
    1. (measure.wall-insulation type: internal thickness: 50 resistance: 0.01)
    2. (measure.wall-insulation type: cavity thickness: 50 resistance: 0.02)
    3. Both of the above measures
    4. (measure.glazing type:double u-value: 1)
    5. (measure.glazing type:triple u-value: 0.2)
    6. (measure.loft-insulation thickness: 200 resistance: 0.02)
    7. All of the above measures

7.5.1 Answers

  1. The heat loss coefficient is the sum of these elements

    Floor Element Area u heat loss
    G front wall 4 0.5 2.
    G front glazing 4 1 4
    G left wall 4 2 8
    G rear wall 2 2 4
    G rear glazing 2 1.5 3.
    G rear wall (inset) 4 2 8
    G right wall 4 2 8
    G floor 20 0.8 16.
    G roof 8 2 16
    1 front wall 4 2 8
    1 front glazing 4 1 4
    1 rear wall 4 0.5 2.
    1 rear glazing 4 1.5 6.
    1 roof 12 2 24
      total     113.


    • The areas of the front and back walls are reduced by the glazing in them
    • The party walls contribute no heat loss
    • The roof area of the ground floor is its area less the floor area of the first floor
  2. The power requirement is the heat loss multipled with the temperature difference, so \(113\times 7=791\text{watts}\)
    1. This will change all the walls except the rear wall of the first floor, which already has internal insulation. The amended values will be

      Floor Element Area u r u' delta heat loss
      G front wall 4 0.5 0.5 0.4 -0.4
      G front glazing 4 1 0 1 0
      G left wall 4 2 0.5 1. -4.
      G rear wall 2 2 0.5 1. -2.
      G rear glazing 2 1.5 0 1.5 0.
      G rear wall (inset) 4 2 0.5 1. -4.
      G right wall 4 2 0.5 1. -4.
      G floor 20 0.8 0 0.8 0.
      G roof 8 2 0 2. 0.
      1 front wall 4 2 0.5 1. -4.
      1 front glazing 4 1 0 1 0
      1 rear wall 4 0.5 0 0.5 0.
      1 rear glazing 4 1.5 0 1.5 0.
      1 roof 12 2 0 2. 0.
        total         -18.4

      The final heat loss will then be 113 - 18.4 = 94.6, with a power requirement of 662 watts

    2. This will change all the walls except the front wall of the ground floor, which already has a filled cavity. The amended values will be:

      Floor Element Area u r u' delta heat loss
      G front wall 4 0.5 0 0.5 0.
      G front glazing 4 1 0 1 0
      G left wall 4 2 1 0.66666667 -5.3333333
      G rear wall 2 2 1 0.66666667 -2.6666667
      G rear glazing 2 1.5 0 1.5 0.
      G rear wall (inset) 4 2 1 0.66666667 -5.3333333
      G right wall 4 2 1 0.66666667 -5.3333333
      G floor 20 0.8 0 0.8 0.
      G roof 8 2 0 2. 0.
      1 front wall 4 2 1 0.66666667 -5.3333333
      1 front glazing 4 1 0 1 0
      1 rear wall 4 0.5 1 0.33333333 -0.66666668
      1 rear glazing 4 1.5 0 1.5 0.
      1 roof 12 2 0 2. 0.
        total         -24.666667

      The reduction in power requirement is then 171 watts.

    3. Applying both the measures involves summing the r-values which we have applied in each case:

      Floor Element Area u r u' delta heat loss
      G front wall 4 0.5 0.5 0.4 -0.4
      G front glazing 4 1 0 1 0
      G left wall 4 2 1.5 0.5 -6.
      G rear wall 2 2 1.5 0.5 -3.
      G rear glazing 2 1.5 0 1.5 0.
      G rear wall (inset) 4 2 1.5 0.5 -6.
      G right wall 4 2 1.5 0.5 -6.
      G floor 20 0.8 0 0.8 0.
      G roof 8 2 0 2. 0.
      1 front wall 4 2 1.5 0.5 -6.
      1 front glazing 4 1 0 1 0
      1 rear wall 4 0.5 1 0.33333333 -0.66666668
      1 rear glazing 4 1.5 0 1.5 0.
      1 roof 12 2 0 2. 0.
        total         -28.066667

      This nicely illustrates how insulation has diminishing returns - the effect of applying both forms of insulation is much less than the sum of their effects when taken independently.

    4. The glazing measure is not computed as a resistance but simply as a replacement of the basic u-value:

      Floor Element Area u heat loss
      G front wall 4 0.5 2.
      G front glazing 4 1 4
      G left wall 4 2 8
      G rear wall 2 2 4
      G rear glazing 2 1 2
      G rear wall (inset) 4 2 8
      G right wall 4 2 8
      G floor 20 0.8 16.
      G roof 8 2 16
      1 front wall 4 2 8
      1 front glazing 4 1 4
      1 rear wall 4 0.5 2.
      1 rear glazing 4 1 4
      1 roof 12 2 24
        total     110.
    5. The only difference between the two kinds of glazing for our purposes is the u-value:

      Floor Element Area u heat loss
      G front wall 4 0.5 2.
      G front glazing 4 0.2 0.8
      G left wall 4 2 8
      G rear wall 2 2 4
      G rear glazing 2 0.2 0.4
      G rear wall (inset) 4 2 8
      G right wall 4 2 8
      G floor 20 0.8 16.
      G roof 8 2 16
      1 front wall 4 2 8
      1 front glazing 4 0.2 0.8
      1 rear wall 4 0.5 2.
      1 rear glazing 4 0.2 0.8
      1 roof 12 2 24
        total     98.8
    6. Applying loft insulation will affect both the roof parts only.

      Floor Element Area u r u' delta heat loss
      G front wall 4 0.5 0 0.5 0.
      G front glazing 4 1 0 1 0
      G left wall 4 2 0 2. 0.
      G rear wall 2 2 0 2. 0.
      G rear glazing 2 1.5 0 1.5 0.
      G rear wall (inset) 4 2 0 2. 0.
      G right wall 4 2 0 2. 0.
      G floor 20 0.8 0 0.8 0.
      G roof 8 2 4 0.22222222 -14.222222
      1 front wall 4 2 0 2. 0.
      1 front glazing 4 1 0 1 0
      1 rear wall 4 0.5 0 0.5 0.
      1 rear glazing 4 1.5 0 1.5 0.
      1 roof 12 2 4 0.22222222 -21.333333
        total         -35.555555

      Because the roof was a large surface with a bad u-value, the change is pretty significant.

    7. Let us combine all these effects:

      Floor Element Area u r u' delta heat loss
      G front wall 4 0.5 0.5 0.4 -0.4
      G front glazing 4 1 4 0.2 -3.2
      G left wall 4 2 1.5 0.5 -6.
      G rear wall 2 2 1.5 0.5 -3.
      G rear glazing 2 1.5 4.333 0.20001333 -2.5999733
      G rear wall (inset) 4 2 1.5 0.5 -6.
      G right wall 4 2 1.5 0.5 -6.
      G floor 20 0.8 0 0.8 0.
      G roof 8 2 4 0.22222222 -14.222222
      1 front wall 4 2 1.5 0.5 -6.
      1 front glazing 4 1 4 0.2 -3.2
      1 rear wall 4 0.5 1 0.33333333 -0.66666668
      1 rear glazing 4 1.5 4.333 0.20001333 -5.1999467
      1 roof 12 2 4 0.22222222 -21.333333
        total         -77.822142

      Here we have derived the effective r-value for the replacement of the windows, as though we were insulating them rather than replacing them. The combined effect is a power reduction of about 540 watts.

8 TODO Heating measures

8.1 How does the model think about heating measures

Heating measures are harder to describe than insulation measures. You might think of a heating system as a thing which meets a need for some amount of heat by converting some fuel at some efficiency. In this view, installing a new heating system might change a house's heating fuel and efficiency, and so convert the same demand for heat into a different demand for heating fuel.

Whilst this is superficially correct, in a BREDEM or SAP type calculation it is complicated by two considerations:

  1. The choice of heating system may affect the demand for heat, for various reasons:
    • Moving from point-of-use hot water to central hot water will introduce distribution losses
    • Moving from an instantaneous combi boiler providing hot water to a standard boiler with a tank will introduce tank losses and primary circuit losses
    • Some combinations of heating system and heating controls will change the demand temperature in the house; SAP table 4e lists these kinds of adjustments, and SAP table 9 also makes such an adjustment based on the control type column.
    • The heating system responsiveness will change the demand temperature in the house; this is a dimensionless number between 0 and 1 which reflects the time a heating system takes to respond to demand for heat. Systems with a lower responsiveness are assumed to result in a higher average internal temperature. SAP table 4d gives the responsiveness for wet heating systems, and table 4a the responsiveness for other types of heating system.
  2. The efficiency of the heating system may be adjusted according to one of a number of rules; SAP table 4c lists the adjustments that are used. In addition the efficiency used for boilers is usually a function of their winter and summer efficiencies, which are combined in the harmonic ratio of the space heating and water heating heat demands.

In the model, when you install a heating measure, some other changes will be made to ensure that the result makes sense:

  • If installing something which requires a central heating system, and the house does not have a central heating system, a central heating system will also be installed. This reflects the cost of fitting pipework, not installing a boiler. Measures which may have this effect have an additional named argument system-capex:, which is used to determine the capital cost of installation.
  • If installing something which requires a hot water tank, a new tank will be installed (even if there is an existing tank). The tank's insulation thickness and capacity are given by cylinder-insulation-thickness: and cylinder-volume: respectively.

Most heating systems have a notion of size; you can control this using the size: argument of the measure to enter a sizing rule. A sizing rule will typically look something like this:

(size min: 5 max:50

The size used is calculated from the first unnamed argument (in the example, this is house.peak-load, which is a reasonable choice. The peak load is the heat output required to keep the house warm on a cold day; by default to produce a 19 degree internal temperature against a -5 degree external temperature). The min: and max: arguments are optional, and restrict the measure's suitability.

The sizing rule is used in two ways:

  1. It is calculated before calculating the capital cost or operational cost, and the result is made available via the size.kw command. This allows you to compute a size for a heating system, and then use it in the pricing rule to affect the price
  2. As mentioned, the min: and max: arguments restrict the suitability. If the sizing function produces a value below the min: or above the max:, the measure will be unsuitable. This is intended to reflect the fact that some technologies do not work well or are not available in some sizes.

You can use any function for the sizing function, so you could combine house.peak-load with round to represent a technology typically available in 10kW increments:

(size max:50
   (round (house.peak-load) to: upper 10))

This indicates that we should take the peak load, but if it is 1kW we should round it up to 10, if it is 11kW we should round it to 20 and so on.

8.2 How to put in new heating systems

The commands for putting in new heating systems are:

  • measure.standard-boiler This installs a standard condensing boiler, with a hot water tank and central heating system. The boiler will supply space and water heat.
  • measure.combi-boiler This measure installs a condensing combi boiler, with a central heating system. You can specify a storage combi boiler using the storage-volume: argument. The boiler will supply space and water heat.
  • measure.heat-pump This measure installs a heat pump, with a central heating system. The type of heat pump is given by the type: argument, and the efficiency by the cop: argument. The heat pump will supply space and water heat.
  • measure.storage-heater This measure installs storage heaters for space heating.
  • measure.room-heater This measure installs a secondary room heater.
  • measure.warm-air-system This measure installs a warm air system; it can only ever replace an existing warm air system, so you can never increase the number of warm air systems in the stock.
  • measure.district-heating This measure installs a district heating system; as far as the model is concerned, this is a bit like a boiler whose fuel input is CommunityHeat. The SAP rules about primary fuel inputs to community heat or CHP are not modelled, as there is no information about the kind of heat network a house would be connected to.
  • measure.solar-dhw This measure installs a solar water heating system. The SAP / BREDEM model for solar water heating is complicated, and the output from the heater depends on the demand for hot water in a complex, nonlinear way. Roughly speaking, the useful output of the solar water heater is greater in houses with a geater demand for hot water. This presumably represents the fact that the heat output is of a relatively low grade, and so will be of most benefit if the tank is cold.

8.3 TODO How to find out about heating systems and fuels

There are several commands which will tell you about the kind of heating system in a house.

8.4 TODO Exercises about heating

9 Flags

We have already briefly seen flags. Flags are one of the ways to store information on houses in the simulation. A useful metaphor for flags is that they are sticky labels with a single word written on them. You can stick as many labels to each house as you like, and you can test whether a house has the right combination of labels when filtering the population or applying an action.

9.1 Marking houses as different with flags

At the start of the run, there are never any flags on any of the houses. Naturally our first question is: how do we put flags on some houses? The simplest way to do this is with the action house.flag:

(on.dates (scenario-start)
          (apply to:(sample 20%)
                 (house.flag flag-A flag-B)))

This will:

  • Produce a random sample of 20% of all the houses (if this is unfamiliar, read the preceding sections)
  • Go through that 20% sample and use the house.flag action on all of them
  • This will mark each house with two flags, flag-A and flag-B. You can use any word (without spaces) for a flag, so flag-A and flag-B are indicative here. It could be house-of-interest, or boiler-has-failed or whatever you like.

Once a house has been given a flag, it will keep it until you remove it again. The model does not automatically do anything to the flags on houses, and it has no understanding of their names, so they are entirely your responsibility to manage. In this example, we have marked 1/5 of the houses, and that particular set will remain so marked until we change it. Because of this, if you flag some houses because they are in a certain state (for example, having a high energy use), the presence of the flag later in the run tells you that the houses used to have high energy use. This is useful, when you want to see how a population has changed, but potentially confusing if you don't bear in mind this is what's happening.

Now that you know how to put some flags on a house, we can look at how to find houses which have particular flags.

9.2 Using flags to select a set of houses

A house either has a flag or it doesn't, so the thing we are looking for is a logical test which will distinguish these cases. The test for this is house.flags-match, which will tell you whether a house has a certain combination of flags.

For example, to report on the set of houses having flag-A, we might write

(aggregate name:houses-with-A
           over: [
                 (filter (house.flags-match flag-A))

Here we are aggregating over: the set (filter (house.flags-match flag-A)), in which (house.flags-match flag-A) will be true for any house that has flag-A on it.

9.3 Using flags to affect suitability, and making actions also do flags

If you look in the manual pages for most actions and measures you will see that they accept the two named arguments test-flags: and update-flags:.

These let you do two things:

  1. test-flags: tells the action that, before it is applied, it should check something about the flags on the house, and fail if they are not how you need them to be
  2. update-flags: tells the action that, after it has succeeded (if it succeeds), it should modify the flags on the house so that they are a certain way.

For example, you could restrict a measure to work only houses with the flag flag-A like this:

(apply to: (all-houses)
            test-flags: flag-A

Note that this does not make the measure suitable for all houses with flag-A; it makes the measure unsuitable for all houses without flag-A, but if a house has flag-A but is already unsuitable it does not become suitable.

This argument accepts a list, so you can also write

(apply to: (all-houses)
            test-flags: [flag-A flag-B]

to restrict the measure to houses with both flag-A and flag-B.

The dual to this is update-flags:; say you wanted to mark a house when it had received some insulation from a particular measure, so that you could keep track of it later or report on it.

(apply to: (all-houses)
            update-flags: house-got-some-wall-insulation

In this you could refer to the set (house.flags-match house-got-some-wall-insulation), and it would contain any house to which the above measure was applied successfully. These houses would be guaranteed to have had some wall insulation.

If you wanted to split the population into houses which did and did not, out of the target population, you could write

(apply to:(sample 50%)
       (house.flag house-considered)
       (measure.wall-insulation ... update-flags: house-affected))

Now all the houses which were offered the measure (here a random 50%) will be flagged as house-considered, but only those which got insulation will be flagged as house-affected.

9.4 Patterns for flags, and the negation shorthand

All of the commands which work with flags actually work with patterns for flags. A pattern is like a very short rule which lets you match a whole set of flags all at once. The simplest patterns are plain words, like we have already written; the pattern flag-A just matches the flag flag-A.

More complicated patterns can include some special symbols:

  • the star symbol * will match any text. For example, the pattern flag-* matches any flag which starts with flag-, so if you say (filter (house.flags-match flag-*)) you would get houses flagged with flag-A and flag-B and flag-some-other-flag and so on.

    (filter (house.flags-match *)) gives the set of all houses which have got any flag on them at all.

  • The question mark ? will match any single letter. For example, flag-? matches flag-A and flag-B, but does not match flag-XY because XY is two letters.
  • You can write several alternatives like <X,Y,Z>. This pattern matches X or Y or Z.

These parts can be combined, so for example <X-,Y->* will match the flags X-, Y-, X-abcdef, Y-qwert, and so on and so forth.

You cannot use patterns when adding a flag to a house; for example if you were to say (house.flag *) to try and add all possible flags to a house, the model will not validate your scenario.

There is one other special bit of flag syntax; prepending an exclamation point ! to any pattern negates it. When testing a flag this means that no flags should match the pattern, rather than that at least one should. When changing a flag, this means that all flags matching the pattern should be removed.

Here are some examples:

  • (house.flag !flag-A) - this will remove just the flag flag-A from any house it's applied to.
  • (house.flag !flag-*) - this will remove all flags which start with flag- from any house it's applied to.
  • (house.flag !*) - this will remove all the flags from any house it's applied to this is because * matches all the flags that are there, and ! means to remove them, because we are changing flags, rather than testing them.
  • (house.flags-match !flag-A) - this will be true for any house which does not have the flag flag-A on it.
  • (house.flags-match !flag-A flag-B) - this will match any house which does not have flag-A AND does have flag-B.
  • (house.flags-match !flag-A !flag-B) - this will match any house which does not have flag-A AND does NOT have flag-B.
  • (house.flags-match !flag-*) - this will be true for any house on which no flag starts with flag-
  • (measure.standard-boiler ... test-flags:!got-boiler update-flags:got-boiler) - this will install a boiler in any house which can have a boiler and does not have the flag got-boiler. If it succeeds, and installs a boiler, then it will add the flag got-boiler to the house. Notice this means that this measure (in the absence of other commands to remove the got-boiler flag) can only be applied successfully once to each house.

9.5 Using flags to change a number, like a cost

Because you can check a flag with a logical test, you can use it in other places to do things like controlling the capital cost of a measure.

For example, imagine you were modelling the distinction between easy to treat and hard to treat cavities in the stock; the model does not know about this to start with. Firstly, you would write some code to decide about which houses are hard to treat:

(on.dates (scenario-start)
          ; set up HTT flag;
          (apply to: (sample 10% (...)) ; say a random 10% of some kinds of houses are HTT
                 (house.flag HTT)))

;; then later:

(apply to: (all-houses)
       (measure.wall-insulation ...
            capex: (function.case
                       (when (house.flags-match HTT)
                       default: 500)))

In this example we have defined the capex with the command function.case, which uses logical tests to switch between different values. Here it will be 1000 when the house has the HTT flag on it, but only 500 otherwise.

Note that for this specific case you might instead want to do something different in a real scenario, like having two variants of the measure defined:

(def-action htt-cwi
            (measure.wall-insulation ...
                    capex: 1000
                    test-flags: htt
                    update-flags: got-htt-cwi))

(def-action normal-cwi
            (measure.wall-insulation ...
                    capex: 500
                    test-flags: !htt
                    update-flags: got-normal-cwi))

This uses the def-action command which is explained later; this is just an example of something which you can model more than one way, and the 'right' way depends on what you are trying to do. You could either say that insulation is a measure, and in HTT houses it costs more, or you could say that HTT insulation is a different measure to normal insulation, which costs more and for which only HTT houses are eligible.

9.6 Reporting on what flags a house has

You can use (house.flags-match ...) as a value in any report, for example in the capture: argument of a probe, or as a test in aggregate.where, or as divide-by: in aggregate. There is another useful command, house.flags which produces all the flags on a house, separated by commas. This also accepts a pattern, to print all the flags which match the pattern. For example, if you wanted to see all flags starting with got- on a house you could output (house.flags got-*) and you would see a value like got-boiler,got-insulation if the house had those two flags on it. Similarly you could write (house.flags !got-*) to see all flags which do not start with got-

This is a useful thing to use in divide-by: or capture: if you have set up your measures to put a consistent pattern of flags on houses when applied.

9.7 Exercises about flags

Here are some patterns together with sets of flags. In each case, write down the following

  1. Whether house.flags-match would be true, given the list of flags
  2. What the value of house.flags would be in each case, if given the pattern
  3. What the new set of flags would be if the patterns were used with update-flags: (if legal)
  4. Whether the measure (action.do-nothing test-flags: [...]) would be suitable, if the patterns were filled in the space.
Patterns Flags
* A B C
!* A B C
got-* got-boiler got-insulation gottfried
!htt htt-house large-house
!suitable unsuitable
suitable eligible suitable large-house
? A house B
<?,??> aa a aaa b bbb

9.7.1 Answers

  1. *
    1. false (must have a flag)
    2. empty (no text output)
    3. Not a legal value for update-flags:
    4. not suitable (must have a flag)
  2. *
    1. true (has at least one matching flag)
    2. The text A,B,C
    3. Not a legal value for update-flags:
    4. suitable
  3. !*
    1. false (must not have any flags)
    2. empty (no flags match !*)
    3. no flags left afterwards (removes them all)
    4. not suitable
  4. got-*
    1. true (both got-boiler and got-insulation are matching=
    2. got-boiler,got-insulation
    3. gottfried
    4. suitable
  5. !htt
    1. false (although there is htt-house, it is not exactly htt)
    2. htt-house,large-house, as they both match the pattern of not being htt
    3. htt-house and large-house (unchanged, it does not match any to remove)
    4. unsuitable
  6. !suitable
    1. true (the house does not have the flag suitable, so it's OK). The unsuitable flag is irrelevant. This shows how the model does not understand the meaning of what you write.
    2. unsuitable, as this matches the rule of not being the word suitable
    3. unsuitable, as the flag suitable is not there to be removed.
    4. the measure would be suitable
  7. suitable eligible
    1. false, as the flag eligible is not present and would be required
    2. suitable, as this matches the pattern for suitable
    3. suitable large-house and eligible
    4. unsuitable
  8. ?
    1. true, as A and B both match ?
    2. A,B
    3. Not a legal pattern for update-flags:
    4. suitable
  9. <?,??>
    1. true, as the flags aa, a, b all amtch
    2. a,aa,b
    3. not a legal pattern for update-flags:
    4. suitable

10 Variables

Flags are one way to store information on a house. They can store boolean values, as they are either there (true) or not there (false). Whilst you can do anything you like with boolean values and a supply of patience, it is often useful to be able to store other kinds of information. In the NHM, you can store numbers as well as flags14. These numbers can be stored against individual houses so that each house may have its own value, or globally for all the houses in the simulation.

10.1 Defining and accessing variables; variable scope

Whilst flags are defined just by being used, variables must be defined before you can use them. This is done using the def command, which always belongs directly within a scenario. def has the following form:

(def <<variable name here>> default: <<default value here>> on: <<where the variable lives>>)

Let's address the arguments in turn:

  1. The variable name is the first positional argument, and it is required. It can be any word (including punctuation), and is what you will use to refer to the variable when changing or looking at its value
  2. The default: argument defines a starting value for the variable. If you do not give a default:, accessing the variable before storing a value in it will cause the simulation to halt with an error (see 23.4).
  3. The on: argument defines where the variable "lives"; there are three options:
    • house. A variable defined to be on: house may take a different value for each house; in this sense it is like a flag, except it stores a number rather than just being present or missing. This could be used to store something like the number of times a house has been considered by a policy, or the year when it was last given a subsidy, or similar.
    • simulation. A variable defined as on: simulation is the same for all houses; it is more like a counter to keep track of something shared by all the houses. For example, you might use a simulation variable to record how many of a certain measure are available. This is shared by all the houses, as there should be one supply chain for everyone rather than one for each house.
    • event. A variable defined to be on: event is like one defined on:house except it is ephemeral; each time an apply or aggregate or similar command completes, the value is forgotten and goes back to its default (if any).

Now that we know how to define a variable, we come on to how to use them. There are two ways to use a variable:

  1. By accessing its value in a calculation or a report
  2. By changing its value with an action

To access a variable, you merely refer to its name; if you like, you can prefix this with an octothorpe (# symbol) to make it clear that this is what you are doing.

For example, in this scenario:

    (def my-variable on:house default: 10)

    (on.dates (scenario-start)
              (aggregate name: a-report
                         (aggregate.sum (+ 3 #my-variable))
                         (aggregate.mean #my-variable))))

Will produce two columns in the report:

  1. The sum, which will be 13 times the number of houses modelled; this is because:
    • The default value of the variable is 10, and we do not change it anywhere
    • aggregate.sum is summing the value of the variable plus 3 over all the houses
    • ten plus three is thirteen.
  2. The mean value of the variable, which is just 10, because again we have not yet changed it.

The key point here is that anywhere where we would write a number, or a function that computes a number, we can refer to a variable instead.

If the variable is defined as on: house, the value will be that for the house or houses being looked at, just as for other house-specific values like house.energy-use.

Dual to accessing a variable is changing it. There are three commands for changing variables, which are all actions

  1. set For example, applying the action (set #x 3) will store 3 in the variable x
  2. increase Applying (increase #x 10) is equivalent to (set #x (+ 10 #x)); it changes the stored value to be ten more than it is already.
  3. decrease Decrease is the opposite of increase; (decrease #x 10) will reduce the stored value of the variable by ten.

Plugging some of these into our previous example we can make some changes to the variable before we report on it:

    (def my-variable on:house default: 10)

    (on.dates (scenario-start)

              (apply to: (sample 5%)
                     (increase #my-variable 10))

              (apply to: (sample 10%)
                     (set #my-variable (* my-variable 1.1)))

              (apply to: (filter (house.region-is london))
                     (decrease #my-variable 1))

              (aggregate name: a-report
                         (aggregate.sum (+ 3 #my-variable))
                         (aggregate.mean #my-variable))))

Now there will be eight different states for the value of #my-variable' across the population, depending on membership of the two samples or the filter to London:

  1. Unchanged, if not in London or either random sample, i.e. 10
  2. 9, if in London but neither random sample
  3. 11, if in the middle sample only
  4. 10, if in the middle sample and in London
  5. 20, if in the first sample only
  6. 19, if in the first sample and in London
  7. 22, if in the first and second samples
  8. 21, if in the first asn second samples and in London

Estimating the values produced in the report is left as an exercise for the reader.

The values stored in variables can be taken from other calculations, and the set, increase and decrease commands can be used in any place where other actions might be used.

For example, to store the initial energy use for every house in a scenario, before any changes are made, you could write

    (def original-energy-use on:house)
    (on.dates scenario-start
              (apply (set #original-energy-use (house.energy-use))))

    ;; later on

    (on.dates ...
       (aggregate name: impacts
          (aggregate.mean (- #original-energy-use (house.energy-use))))))

This illustrates how variables hold the values put in them at particular moments in simulated time. In this case, after being set, the #original-energy-use variable will not change. The house.energy-use command on the other hand will always reflect the current energy use, so by reporting on the difference we can see the change in energy use that has resulted for each house by whatever changes the scenario has so far wrought.

In principle you now know everything there is to know about variables in the NHM; in practise you may wish to refer to a few examples of how variables can be used. Before we come to that you should read the subsequent sections, in particular those about choices and packages and suitability and failure.

Examples of how to use variables to model particular effects are given in 22

10.2 Exercises about variables

For each of the following apply commands, describe the value or values of the variable #x after the apply is finished:

  1. First

    (def x on:house)
    (apply (set #x (house.energy-use)))
  2. Changing to simulation

    (def x on:simulation)
    (apply (set #x (house.energy-use)))
  3. Using increase

    (def x on:house)
    (apply (increase #x (house.energy-use)))
  4. Adding a default:

    (def x on:simulation default:0)
    (apply (increase #x (house.energy-use)))
  5. With a function

    (def x on:simulation default:0)
    (apply (increase #x (* (house.weight) (house.energy-use))))
  6. Referring to own value

    (def x on:house default:10)
    (apply (increase #x (* #x #x)))

10.2.1 Answers

The answers are:

  1. #x will take several values stored against each house, containing the energy use of that house at the time the apply happened.
  2. #x will be a single value for the whole simulation, which will contain the energy use of one house at random (whichever one comes through the apply last).
  3. The scenario will fail with errors, as #x has no default value, but we are trying to increase it, i.e. add a new value to its existing value, without ever providing an existing value.
  4. With the default, and on: simulation, there will be one value which will have each house's energy use added to it. The sum that is produced will be unweighted.
  5. Like 4, but weighted, as the global x will be increased by the weight-energy use product for each house.
  6. Each house will have #x equalling 110, as it has a default of 10, and (* #x #x) is 100.

11 Money

The NHM includes a simple model of money, in which each house has a ledger of transactions it has participated in. This ledger can be manipulated and reported on to find out about how much money has been spent, and on what things.

11.1 How does the model think about money

Every house has a ledger of transactions. A transaction is a record of the transfer of an amount of money between a house and a named pool of money (representing something like the market, or a policy's spending) on a certain date. The ledger never contains transactions which have yet to happen; however, these can be predicted within the model if you wish to consider the future costs of an immediate action.

Transactions are caused by a few different things in the model:

Installing measures
Measures typically have a capex: argument, which accepts any number-valued command. When a measure is installed, if it succeeds, a transaction is made from the account of the house which received the measure to a ledger representing the market for all measures.
Operational costs
Some measures have an opex: argument, which takes a number. When the measure is installed, the this value will be calculated and stored in the house. Until the thing put in by the measure is removed by another measure (say if a boiler is replaced), the house will make a transaction for this operational cost once a year at the very end of the year. The house is said to have an obligation to pay its operational costs.
Fuel costs
Every year, at the end of the year (when the operational costs are also paid), the house has an obligation to settle for its energy use. This is done by evaluating the house's tariffs, which are discussed in more detail in the next section. A tariff has the opportunity to create a few different transactions for different parts of the bill.
Financial actions
The above are all of the built-in financial considerations in the model; however, if you wish to model other movements of money, there are various commands to do this:
  • finance.with-subsidy, which encloses another measure and meets some or all of its cost with an immediate subsidy
  • finance.with-loan, which encloses another measure and meets some or all of its cost with an immediate subsidy to be repaid over time as a loan
  • finance.with-obligation, which encloses another measure and sets up an arbitrary obligation to pay some amounts in the future
  • measure.with-additional-cost, which encloses another measure and adds an arbitrary extra transaction to it
  • pay, which allows for the transfer of money between non-house entities.

11.1.1 Inflation, income, investment and so on

The NHM does not model inflation, income, return from investments or anything like that. All transactions are assumed to be denominated in units of equivalent value. If you wish to account for inflation, we suggest modelling the prices of things in the NHM using real values, and adding the inflation-adjustment off-model.

11.2 How to find out the costs of things

The best way to make this subject concrete is to take an existing scenario (hopefully you have one by this point in the guide!) and add report.transactions inside the scenario command. This will produce a report containing the model's entire transaction log. Each row of this report shows everything the model knows about a transaction that has happened; the columns are:

this is typically the ID of a dwelling, although money can be moved between policies
this is the weight of the payer, which is usually the dwelling weight
the date when the transaction occurred
this is the name of the entity which received the payment
this is the weighted amount of money transferred
this is a slash-separated path in the scenario of the command which caused the transaction
each transaction may have several tags on it which indicate something about the type of transaction; this is a comma-separated list of these tags. For example, any capital expenditure will have :CAPEX as a tag, any loan related transaction will have :loan, and so on.

For most questions about money whose results are not needed on-model, it should be possible to produce the answer by processing the transactions log using another tool.

Now we will look into the different commands which produce transactions, and the ways to ask about the costs of particular operations within the model, for specific reporting or for making decisions.

11.3 Costs for measures

As mentioned, measures generate transactions to pay for their installation. All measures have a capex: argument; when a measure is installed, if it is suitable, it will compute its capex: after the installation is finished and immediately produce a transaction which causes the payment of that amount from the house to a payee called :market. The transaction will have the tag :CAPEX. Measures with an opex: argument compute its value after installation, and cause the future payment of that amount each year at the end of the year, tagged :OPEX. If the measure is later removed or replaced, the associated opex payments will stop.

11.4 Costs for fuel

As mentioned, fuel costs are incurred by every house at the end of the year. The pricing structure is determined by the tariffs for the house; these are described later in the 13 section. Tariff-related transactions are always tagged with their fuel and with the :fuel tag. You can add other tags on the charge command if you want to, for disambiguation of the transaction log.

11.5 Examining costs on-model

For on-model questions, you can use several different commands to look at the costs associated with particular types of transactions on houses or accounts.

11.5.1 Immediate costs

For inspecting the immediate cost of commands which are currently happening, for example within a probe report, or to compute the subsidy required for a newly installed measure, the model has the commands net-cost and capital-cost.

These commands add up the costs due to transactions which have happened to the current house as a result of the action which is computing the value; normally this is the action which contains the net-cost or capital-cost command (this is explained in more detail later, in the section on ephemeral values). capital-cost sums the cost of all transactions which are tagged :CAPEX, whereas net-cost sums the cost of all transactions.

You can also parameterise net-cost to sum a particular subset of transactions; this is documented in more detail in the manual, but you can for example write

(net-cost tagged:"!:CAPEX")

To determine the net cost of the current action excluding any :CAPEX transactions. The matching rule supplied to tagged: follows the same form as the rules for matching flags a house, except it matches tags on the house's transactions instead.

11.5.2 Historical costs

Where net-cost and capital-cost allow you to examine the costs associated with what the model is currently doing, the commands house.sum-transactions, house.balance and account.balance allow you to ask the model about the total expenditure that has happened so far.

The first of these, house.sum-transactions, is like a version of net-cost which includes all the transactions that have happened to a house; with no arguments it is a synonym for house.balance, and will produce the net effect of all the money transferred in the simulation so far on a house's bank balance.

Remember that the model does not simulate receipt of income, so this will usually be negative and become moreso over time as the house pays for fuel and maintenance, and for any measures it receives (less any subsidies).

To find out about the total impact of a subset of transactions you can write, for example:

(house.sum-transactions tags: ":CAPEX")

This is equivalent to capital-cost, but gives the capital expenditure of the current house over the whole simulation so far.

Finally, account.balance is the reverse of house.balance, telling you how much money was transferred from or to a particular "global account".

For example, writing

(account.balance ":market")

Will tell you the total amount spent on measures in the simulation.

11.6 Paying for measures with subsidies or loans

So far we have discussed only things which cost a house money, like installing measures or paying bills. Sometimes you may want to install measures but to pay for some part of their cost on behalf of the house.

There are two forms of subsidy available:

  1. Immediate subsidies which are not recouped, using finance.with-subsidy and finance.fully, or
  2. Immediate subsidies which are recouped later by a loan, using finance.with-loan

All of these three commands work in the same way: you write them around another action, and they do the action but also provide the subsidy and set up any loan.

For example:

   (measure.standard-boiler fuel:mainsgas capex: 1000))

will perform the inner action (installing a new boiler, and hence costing 1000 pounds), and then (if the inner action is suitable), it will provide a subsidy whose value equals the net cost of performing the inner action (in this case, 1000 pounds).

As a result, if you were to write:

    capture: [(net-cost)]
    (finance.fully X))

the net-cost captured would always be zero, no matter what X was, because finance.fully always provides a subsidy which cancels out the net cost of the thing it's financing.

In a less-generous policy, we might use finance.with-subsidy, in which we have the option to calculate the value of the subsidy using a rule:

    subsidy: (min 500 (net-cost))
    (measure.standard-boiler fuel:mainsgas capex:1000))

In this example we use the rule that the subsidy is whatever is least of the net cost of the measure and 500, effectively clamping the upper end of the subsidy to 500 pounds. In this case if we were to add a probe and capture the net-cost for the entire action, it would be 500, as the house would incur the capital cost of 1000 and then receive the subsidy of 500.

Finally, if we were approaching the nadir of policy generosity we might be looking to saddle the house with a loan, which naturally the householders would only accept if it were economically rational. To model this you can use finance.with-loan:

    principal: (net-cost)
    rate: 5%
    term: 10

    (measure.standard-boiler fuel:mainsgas capex:1000))

In this example, a house will install a boiler, incurring a capital cost of 1000 pounds, but then it will be given a 1000 pound immediate subsidy (the principal:, which in this case is the net cost of 1000 pounds). So far the house is doing well - it has a free boiler - but it also incurs the obligation to repay the loan, with interest, which happens on the anniversaries of the date of installation. The loan payments are of equal size, and the interest on the remaining principal is compounded annually.

These finance commands can be combined; for example, you might want to pay for some of a measure using a subsidy, and then settle the remainder with a loan; here is a more complex example:

    principal: (net-cost)
    rate: 6%
    term: 10

        subsidy: (min 500 (* 2 (fall-in (house.emissions))))

        (do (measure.standard-boiler fuel:mainsgas capex:1000)
            (measure.wall-insulation type:cavity capex: (+ 400 (* size.m2 10))))

Although the loan is written at the top, it is the outermost thing and so it is considered last; you should read this as something like:

  1. Finance doing the following with a loan, where the principal of the loan should equal the net cost of the thing we are financing
    1. Finance doing the following with a subsidy, where the subsidy amount is whichever is least of 500 pounds, or double the fall in emissions
      1. Install as a package, the following:
        1. a boiler, at a cost of 1000 pounds
        2. wall insulation, at a cost of 400 pounds and 10 pounds per square meter insulated

Filling in some example figures we might have:

  1. Before we can do the loan, we must do the subsidy:
    1. Before we can do the subsidy, we must install the measures
      1. Installing the measures, we have
      2. boiler, at a cost of 1000 pounds
      3. insulation, at a cost of (say) 1500 pounds
      4. the net cost of all this is now 3500
    2. Now we can consider the subsidy amount; say we reduce emissions by 300 kgCO2/year, we will offer a subsidy of 500 as this is less than 600.
    3. finance.with-subsidy gives the house 500 pounds, so the net cost of the subsidised package as a whole is now 3000
  2. Now we can work out the principal for the loan, which is the net-cost of the subsidised package; this is 3000
  3. The house is given 3000 pounds now, and made to repay this over 10 years at 6% compounded annually.

This example also illustrates how commands like net-cost refer to the net cost of the action which contains them, rather than the net cost of everything that has happened in the scenario.

11.7 Adding costs rather than subsidies

The finance.with-subsidy cannot produce costs for a household; if the subsidy: argument is negative (which would represent a cost), no subsidy is offered.

The command measure.with-cost is an equivalent which allows the addition of further costs to another action. For example, if you wished to model a service where an advisor visits a house and charges them fifty pounds for the suggestion to decrease their thermostat setting, you could write

    cost: 50

    (action.set-heating-temperatures living-area-temperature: 19))

This would reduce the house's heating temperature, and charge the house 50 pounds for doing so.

11.8 Setting up arbritrary payments in the future (obligations)

The previous section shows how we can set up future payments by offering a loan. There is one more finance command, finance.with-obligation, which is more complicated than the other sorts. This command allows you to commit a house to an arbitrary sequence of future payments.

It has two important arguments:


the amount: argument gives a command for working out the amount that the house must pay (or if negative, receive). This command is computed each time the house has to make a payment, so it can change over time.

As a consequence, you may want to store the amount a house should pay or receive in a variable, if the value of the command for computing it would change over time, or if it contains an ephemeral value like net-cost.

the schedule: argument determines when the obligation is due, and so when the amount: argument will be used to compute a value for a transaction.

For example, to set up a house to receive 100 pounds every year for a decade, you could write

    amount: -100 ; negative as it is a cost
    schedule: (periodic-payment interval: "1 year" lifetime: "10 years")

    ;; we need to finance something!

The amount is computed each year, so we might write:

    amount: (sim.year)
    schedule: (periodic-payment interval: "1 year" lifetime: "10 years")

    ;; we need to finance something!

This would make the house pay the value of the current year (so 2016 pounds in 2016, 2017 in 2017 and so on) each year for ten years. This is because the sim.year command when used produces the simulation's current year, and the future payments are computed in the year when they are due.

11.9 Distinguishing particular costs

As mentioned, all transactions have a set of tags on them; most finance-related commands can either filter the transactions they include by including or excluding transactions which have certain tags, or can put specific tags on the transactions they generate.

Actions which generate transactions will typically have the arguments:

a list of tags to write on every transaction that the action is responsible for, and
the name of the global account which is on the other side of the transaction from a house.

11.10 Computing present values, prediction, ways of discounting things

As mentioned, the model makes no attempt to represent things like cost of capital, inflation, or any other financial matters. All the model captures is the movement of particular amounts of money from place to place; you can set the prices for these things, and it is up to you whether you wish to use real (inflation-adjusted) prices in the future parts of the simulation or not. Similarly, if you wish to compute the discounted value of running a particular policy for the wider world, it is up to you to work this out off-model, bearing in mind the decisions you have made about what prices to use and what they ought to mean15.

However, there are circumstances in which a scenario needs to compute a discounted value on model. In the NHM this is usually a matter of prediction, rather than accounting; whilst the model does not represent discounted or inflated figures as the ground truth, it does allow you to model the prediction of future values or costs.

For example, you might want to represent a situation in which a house makes a decision between some alternatives which have an immediate cost in exchange for a future benefit based on their expected (possibly discounted) value over some horizon.

There are a couple of simple commands which produce predictions for the current year:

  • house.fuel-cost predicts the costs generated by the obligation to pay the fuel bill at the end of the current year, assuming that the house continues to use energy at the current rate and remains on the same tariff
  • house.annual-cost predicts all of the costs the house is expecting to incur in the next year, or a subset which have certain tags

Together with these is the command future-value, which computes the value of another command each year for some number of years into the future, insofar as this can be predicted, and adds up those values.

For example, you can write

(future-value horizon: 3 (house.annual-cost))

to predict the annual cost over the next three years. This will compute the annual cost thrice, each time in a hypothesis, one which looks like the current date, then 1 year hence, and finally two years hence. The sum of these values will be produced.

Because the future-value command computes its argument multiple times in a new hypothesis for each year up to the horizon, it can also produce discounted sums by multiplying the value produced in each future year by an appropriate discount factor.

You can write whatever discount function you like, but the model includes exponential-discount and hyperbolic-discount which probably cover most needs.

For example, if you wished to calculate the future costs of a house's obligations including an exponential discount, you could write

    horizon: 10
    (exponential-discount rate:5%

The exponential-discount command determines the discount factor in each year and multiplies its first argument by it, in this case the house's annual cost. Putting this all together, you can calculate a "net present value" for doing something by writing

(+  (capital-cost) ;; the immediate capex
        (exponential-discount rate:5%

Note here how the capital-cost is not inside the future-value but is added to it; this is because the future-value of capital-cost is zero, since in the future hypotheses being produced no further measures are being installed.

The astute reader may be wondering how exponential-discount works; after all, to work out a discount factor for each future year it needs to know how far into the future the prediction has got when it is worked out.

The answer is that the hypotheses generated by future-value do provide access to the simulation's true current year as well as to the hypothetical year using the notion of different levels of foresight. When you use future-value, you can specify the argument predict: to specify a the foresight levels which should be granted to the house. These determine which things should change as the prediction moves into the future, and which things should be opaque. For example, if you wished to do the present value calculation above, but allowing the house perfect foresight about future fuel prices but not future carbon factors or weather, you could write:

(+  (capital-cost) ;; the immediate capex
        predict: [tariffs]
        (exponential-discount rate:5%

In this, the model would compute the discounted annual cost using future fuel prices for the house's current tariffs, but without future carbon factors, weather, or anything else time-sensitive. If the house's tariff included a charge which depended on a house's emissions, whose unit price changed over time, the future-value above would reflect the changes in the rate, but would not reflect any changes in emissions factors included in the scenario.

If you wish your own scenario functions to have different degrees of predictability, you can use the foresight: argument supplied on time-sensitive functions; for example, if you had a function which changed based on the year:

    (when (= (sim.year) 2020) 10)
    default: 9)

You can control the degree of predictability of this function by providing a foresight: argument to sim.year. Say any future-value which is capable of predicting fuel prices ought to be able to predict the value of this function, you could write

    (when (= (sim.year foresight: tariffs) 2020) 10)
    default: 9)

If you were then to predict the future value of this function for 2 years from 2019 with future-value, you could get one of two outcomes:

    horizon: 2
    predict: []
      (when (= (sim.year foresight: tariffs) 2020) 10)
      default: 9))

In this case, where we have said that nothing can be predicted, the function would be computed twice:

  1. Once in hypothetical 2019, where the function would produce 9
  2. Once in hypothetical 2020, but where sim.year would be blinded to the hypothetical year, and so would still appear to be 2019, so the function would produce 9

The value would be 18. If we change to say predict: [tariffs], the result will instead be 19, as we have

  1. An evaluation in hypothetical 2019, producing 9
  2. Another in hypothetical 2020, including for the sim.year, producing 10.

This also answers the question above of how exponential-discount can work; you can always compute the number of years into the future a function is being predicted by the following expression:

(- (sim.year foresight:always) (sim.year foresight:never))

The first term in the difference is always the year being predicted, and the second is always the true year; these levels of foresight are special and can never be prevented or overridden, so this always evaluates to zero in the first hypothetical year, one in the second and so on.

An exponential discount factor of (say) 5% is then just

(/ 1
   (pow 105%
        (- (sim.year foresight:always) (sim.year foresight:never))))

Indeed, this is exactly the value computed by exponential-discount given a rate of five percent.

11.11 TODO Exercises about money

12 Ephemeral values like net-cost, capital-cost, size.m2

Most model commands' meanings depend on the context where they are used. The commands net-cost, capital-cost, size.m2 and size.kw all refer to the closest action containing them, which you might call the "current command", so their meaning is "the net cost of the current command so far", and so on.

To make this clearer, here are some examples:

    subsidy: (net-cost)
        (measure.standard-boiler capex: 500 ...)
        (measure.wall-insulation capex: 200 ...)))

In this example, net-cost refers to the net cost of the finance.with-subsidy so far; since this is before the subsidy has been given (because we're working out the amount), this means that it is the net cost of the inner do command, which is 500 + 200 = 700.

If we were to add a further do command around the finance.with-subsidy like so:

(do (finance.with-subsidy
     subsidy: (net-cost)
         (measure.standard-boiler capex: 500 ...)
         (measure.wall-insulation capex: 200 ...)))

    (set #variable (net-cost)))

The second use of net-cost's nearest containing action is the outermost do, and the net cost of that at the time when the set is worked out is zero. This is because by the time we get to the set the actions which are part of the do have done the following:

  1. Spent 500 pounds on a boiler (net cost of the outer do is now 500)
  2. Spent 200 pounds on insulation (net cost of the outer do is now 700)
  3. Offer a subsidy equal to the net cost of the inner do, i.e. 700; at this point the net cost of the whole action is 700-700 = 0

As a result the variable #variable will be set to zero.

The capital-cost command is similar, except it only includes capital costs, so if we were to say

(do (finance.with-subsidy
     subsidy: (net-cost)
         (measure.standard-boiler capex: 500 ...)
         (measure.wall-insulation capex: 200 ...)))

    (set #variable-1 (net-cost))
    (set #variable-2 (capital-cost)))

The outcome would be that #variable-1 is zero but #variable-2 is 700 (as it is just the capital expenditure, not the subsidy).

The sizing commands size.m2 and size.kw are normally used in the capex: arguments for insulation or heating measures, in which context their closest containing action is the measure itself, so they refer to its size:

    size: (size min:0 max:1000 (log (house.total-floor-area)))
    capex: (+ 100 (* 500 (size.kw))))

In this example, the size for the boiler is determined to be the log of the house's floor area, and the capital cost is 100 plus 500 times that size (as explained in the section on heating measures, the size has no significance apart from its effect on size.kw and things defined in terms of it).

The same rules apply to the propagation of size.kw as to capital-cost and net-cost; for example, if you write:

(do (measure.loft-insulation ...)
    (measure.wall-insulation ...)
    (set #variable-1 (size.m2)))

Then the variable #variable-1 will contain the area in square metres insulated by the closest containing action, which is the do command; this is the area insulated by the loft insulation measure plus the area insulated by the wall insulation measure. If you wanted to record each one separately you could write something like:

    (do name:A (measure.loft-insulation ...)
        (set #variable-1 (size.m2)))
    (do name:B (measure.wall-insulation)
        (set #variable-2 (size.m2))))

In this case, the first size.m2's closest containing action is the do which we have given the name A, and the second's is the do named B, so #variable-1 is the area of loft insulation and #variable-2 the area of wall insulation.

The closest containing action can equally well be something like a probe report:

(probe name:p
       capture: [(capital-cost)]

In this example, the capital-cost command will be computed twice, and both times it will give the capital cost of the closest containing action, which is the probe. The first time, the capital cost will be zero, because the application of probe has incurred no capital costs. The second, it will be the capital cost incurred by applying the probe, which is the capital cost of doing the action inside the probe.

12.1 Places where ephemeral values will not work, or will be confusing

The context-sensitive nature of ephemeral values creates some potential for confusion:

  1. Ephemeral values will not work in aggregate reports; this is because there is no containing action when the value is worked out in the report. Asking an aggregate report to compute capital-cost is like asking for the capital cost of the report; the model has no way of knowing that you mean to say the capital cost of the measures you put in in the last year, or over the simulation so far, or similar.
  2. Ephemeral values will not work as expected in 16 produced by the under command or set under:; this is the value will be the value for the actions applied in the hypothesis. The model has no way of knowing if you mean to ask for the value outside the hypothesis, and always produces the value inside, even if it happens to be zero (as it will be for many commands).

13 Tariffs

The previous section mentions the fuel bill and tariffs, without going into detail about how they work. This part of the guide explains how to define and use tariffs. Before getting into the language, let's go through how tariffs are represented in the NHM.

An NHM tariff is a package of rules which define how to calculate the bill a house should pay for a subset of fuels. Each house can be on several different tariffs at once, but they can only have one tariff for each fuel, and if they take a tariff they must use it for all the fuels it covers. For example, imagine you have the following tariffs available:

  • Dual fuel

    Fuel Unit Price
    Electricity 10p
    Gas 5p
  • Cheap electricity

    Fuel Unit Price
    Electricity 8p
  • Expensive gas

    Fuel Unit Price
    Gas 10p

A given house could either take the Dual fuel tariff, or the Cheap electricity and Expensive gas tariffs. However, they could not have Cheap electricity and Dual fuel, to get the best of both worlds and pay 8p/5p.

The rules in a tariff may be almost arbitrarily complex; each fuel covered by the tariff has a rule, and the rule may contain multiple charges, where each charge has a function to compute the payment a house should make; this function may depend on the amount of energy the house has used, or it may be a constant value, or any other function that you can write.

A house's fuel bill is worked out and paid at the end of the year, every year; by default the model puts all the houses on an 'empty' tariff for each fuel, which makes no charges, so unless you write down some tariffs nobody will pay any bills.

13.1 Tariff syntax

Tariffs are normally defined in the context.tariffs element in the scenario. This has two named arguments, defaults: and others:, each of which can accept a list of tariff definitions.

The tariffs in defaults: will be added to all the houses at the start of the run. Because of this, only one of these tariffs may cover each fuel. The tariffs in others: can be any collection of tariffs, which houses may be switched onto later.

For example:

(scenario ...
       defaults: [A B C]
       others: [X Y Z]))

The tariffs A, B and C will be added to each house when it is created; X, Y and Z will have no effect unless they are explicitly used by action.set-tariffs (of which more later).

A tariff definition is either tariff or tariff.simple. Let's look at an example of tariff first:

    name: dual-fuel
    (fuel type: peakelectricity (charge (* 0.1 house.meter-reading)))
    (fuel type: offpeakelectricity (charge (* 0.1 house.meter-reading)))
    (fuel type: mainsgas (charge (* 0.05 house.meter-reading))))

This is a tariff implementing the Dual fuel example above; the house will be charge 10p/unit for peak and off peak electricity, and 5p for mains gas.

The house.meter-reading command has some special behaviour:

  • At any point in time, it evaluates to the total number of units of fuel that have not yet been paid for.
  • Within a fuel command in a tariff, it refers to the unpaid for units of that specific fuel.

This also illustrates another point of interest: the Electricity fuel type has no consumption in tariffs, in order to prevent accidental double-counting. It is always broken down into PeakElectricity and OffPeakElectricity, whether or not the house is on a tariff that has different charges for these two categories.

To complete the example above, we could write something like this:

    defaults: [
        name: dual-fuel
        (fuel type: peakelectricity (charge (* 0.1 house.meter-reading)))
        (fuel type: offpeakelectricity (charge (* 0.1 house.meter-reading)))
        (fuel type: mainsgas (charge (* 0.05 house.meter-reading))))
    others: [
        name: cheap-electricity
        (fuel type: peakelectricity (charge (* 0.08 house.meter-reading)))
        (fuel type: offpeakelectricity (charge (* 0.08 house.meter-reading))))
        name: expensive-gas
        (fuel type: mainsgas (charge (* 0.1 house.meter-reading))))

We could either write the dual-fuel tariff in defaults, or cheap-electricity and/or expensive-gas, but we could not write dual-fuel with either of the others, as dual-fuel is incompatible with both of the others.

Now we can look at a few different kinds of tariff that we could write:

13.2 Examples of different tariffs you can write

13.2.1 Simple unit rates, and standing charges

Unit rates can be written using tariff and charge, as seen above; additionally the command tariff.simple gives a slightly simpler syntax for unit price + standing charge type tariffs:

(tariff.simple name: dual-fuel
    (function.simple-pricing fuel: mainsgas unit-price: 0.05)
    (function.simple-pricing fuel: peakelectricity unit-price: 0.1)
    (function.simple-pricing fuel: offpeakelectricity unit-price: 0.1))

This has a slightly flatter structure and may be easier to read, but is otherwise interchangeable.

Along with the price, function.simple-pricing accepts a standing-charge: argument, but there is a subtlety to using it: the standing charge is applied to anyone who is on the tariff, whether or not they actually use any of the given type of fuel.

This means that you can put someone on a tariff like this:

(tariff.simple name:biomass
   (function.simple-pricing fuel: biomasswood unit-price: 0.06 standing-charge:100))

and they will be charged 100 pounds per year even if they use no biomasswood fuel.

To write a standing charge which does not get applied if you use no units, you can write it out using the more complex tariff structure:

(tariff name: biomass
   (fuel type: biomasswood
         (charge (* house.meter-reading 0.06)) ;6p per unit
         (charge (function.case (when (> house.meter-reading 0) 100)
                                default: 0)) ;; 100 pounds if we have used more than 0 units

Here we have used function.case to define the standing charge part - the value of the charge is 100 only when the meter reading exceeds 0.

13.2.2 Time-series in tariffs

The examples we have shown so far have simple functions for prices. However, each tariff is computed again each year, so you can use more complex price functions, including ones which change over time. There are several ways to write a time-series in the NHM; the most readable is probably using the ~lookup-table macro command, which translates into a use of the lookup command. Here is an example:

(tariff name: gas-changing-over-time
    (* house.meter-reading
          row-keys: sim.year
          [YEAR       PRICE]
          [<2000        0.1]
          [2000..2020  0.15]
          [2021        0.14]
          [>2021       0.13]))))

In this example, the charge will be calculated as the meter reading times the value drawn from the table; the table is keyed on the current year, so the cost will change when the tariff is computed in different years.

13.2.3 Rising-block tariffs

Another kind of tariff you could model is a rising-block tariff (indeed, a tariff with a standing charge is a simple form of rising-block tariff). For example, say we wanted to charge 2p/unit for the first 500kWh of gas used, and 8p/unit after that; we could write:

(tariff name:rising-block
   (fuel type:mainsgas
         (charge (* 0.02 (min 500 house.meter-reading))) ; first 500 units
         (charge (* 0.08 (max 0 (- house.meter-reading 500)))) ; after 500 units

13.2.4 Tariffs with a straight tax

Within a tariff you can use the net-cost command to find the net cost of computing the cost for the current fuel so far. For example, we could amend our rising block tariff to include a 20% tax:

(tariff name:rising-block
   (fuel type:mainsgas
         (charge (* 0.02 (min 500 house.meter-reading))) ; first 500 units
         (charge (* 0.08 (max 0 (- house.meter-reading 500)))) ; after 500 units
         (charge (* 20% net-cost)) ;; 20% of whatever the previous charges have cost for this fuel

Note that the net-cost is not for the tariff as a whole but just for the fuel, so this would not affect other fuel commands in the same tariff. To do this, you would need to introduce the same charge into each fuel.

13.2.5 Tariffs with an emissions tax

Tariffs do not have to be based on house.meter-reading; you can introduce other components if you like. For example, here we charge 1 pound per unit of emissions associated with each fuel:

          name: dual-fuel
          (fuel type: peakelectricity
              (charge (* 0.1 house.meter-reading))
              (charge (* 1 (house.emissions by-fuel: peakelectricity))))
          (fuel type: offpeakelectricity
              (charge (* 0.1 house.meter-reading))
              (charge (* 1 (house.emissions by-fuel: offpeakelectricity))))
          (fuel type: mainsgas
              (charge (* 0.05 house.meter-reading))
              (charge (* 1 (house.emissions by-fuel: mainsgas)))))

The charge has been added into each fuel; we have had to specify the fuel twice, because house.emissions is not like house.meter-reading, and does not have special behaviour based on being inside a fuel; only house.meter-reading works like this.

13.3 Switching people's tariffs

Now we have seen how to define tariffs, and how to set the default tariffs for a scenario, we should look at how to change people's tariffs.

When using these commands, remember that the model will not switch people back off particular tariffs unless you tell it to. For example, if you put someone on an economy 7 type tariff because they have a storage heater, and then you replace their storage heater with a boiler, the model will not switch them on to another tariff for you. They will continue to be on their original tariff until such time as the action.set-tariffs command is used to change them onto another tariff.

13.4 Policies that modify your bill whatever tariff you are on (like the Warm Home Discount)

Along with tariffs, a house's bill may include so-called extra charges. An extra charge is what it sounds like - an additional charge (or subsidy) added on to your bill after the tariffs have been computed. Unlike tariffs, extra charges are not exclusively responsible for particular fuels. They can be added to a house with action.extra-fuel-charge, and removed with action.remove-fuel-charge.

For example, if you wanted to offer all houses a subsidy of 100 pounds toward their fuel bill you could write

        (extra-charge -100)))

The value is negative because it is a subsidy. In this example, even houses with a bill of less than 100 pounds would receive a 100 pound subsidy. If you wanted the subsidy to be up to 100 pounds you can refer to the cost of the other tariffs using net-cost again:

       (extra-charge (- (min 100 net-cost)))))

Here we have offered a subsidy of 100 pounds or the net-cost of the other tariffs, whichever is least.

If you wanted the net cost of a specific fuel, you can associate a fuel type with the extra charge, which makes net-cost aware only of the costs for that fuel.

          fuel: mainsgas
          (- (min 100 net-cost)))))

Here we have offered a subsidy of up to 100 pounds off the gas bill. Unfortunately the division of electricity into peak and off-peak applies here as well, making writing an equivalent rule for two fuels a little more difficult:

      (extra-charge (- (min 100 (net-cost tagged: ":<peak,offpeak>electricity"))))))

Rather than associating the charge with a fuel, we have used the tagged: argument of net-cost to refer only to transactions tagged with either :peakelectricity or :offpeakelectricity.

To remove a charge, you will need to refer to it by its name:. For example, if you had offered all houses a mains gas subsidy as above, except with a name

         name: mains-gas-subsidy
         fuel: mainsgas
         (- (min 100 net-cost)))))

But then you wanted to find some houses and remove the subsidy from them, you would elsewhere write something like:

(apply to:(sample 10%)
    (action.remove-fuel-charge #mains-gas-subsidy))

In this, #mains-gas-subsidy refers to the extra-charge above whose name: argument is mains-gas-subsidy.

You can also remove all extra charges from a house just by applying (action.remove-fuel-charge). This is useful if you are computing a hypothetical fuel cost, of which more later.

13.5 TODO Exercises about tariffs

14 Choices and packages of measures

So far we have mostly looked at how to apply individual measures. You can put several measures directly inside an apply command, but they are not applied to the houses one at a time; instead the first measure is applied (potentially failing) to all the houses, then the second, …

To apply a series of measures to a specific house, the model offers some other commands which compose different actions together. These produce new, compound actions which do things like:

  • Install both A and B, or neither
  • Install whichever is cheapest of A and B
  • Install A in rural houses and B in urban houses
  • Install whichever is cheaper of A and B in rural houses, or whichever is cheaper of C and D in urban houses

Compositions like this can be repeated ad infinitum, as each combination is itself an action which can be further combined. In any case, all the commands to combine actions have a common "shape"; they each take some n actions and put them together into one single action.

14.1 Picking between alternatives

First let us consider two ways to turn n candidate actions into one by selecting one as the one to do, and disregarding the rest. The first way makes the selection based on information which is independent of the effects of any of the candidates. The action to take is chosen before we do it, and so even before we know whether it will work. The second way is to make the selection after the fact, with knowledge of the effects that each candidate will have.

14.1.1 Before the fact (if statements)

To make a choice before the fact, the model provides the action.case command. It ties each candidate together with a logical test; when the action is applied to a house, the first candidate whose associated test passes for that house is used, and the action.case behaves as that candidate.

For example, if you wanted to install measure A in rural houses and measure B in urban houses, you might write:

    (when (house.morphology-is rural) A)
    (when (house.morphology-is urban) B))

Each of the when commands in the action.case has two arguments:

  1. The test, and
  2. The candidate action

Upon being applied action.case goes through the when conditions in the order they are written down, working out each test in turn. Once it encounters a test that passes (produces a true value), it behaves as the associated action. It does not consider any subsequent when commands after the action has been performed, so it will only ever use one of the candidates.

If your conditions are not exhaustive, you can use the default: argument to specify a 'fallback' action that will be applied if and only if none of the when commands' tests pass. This is one way to write an "if" type action:

    (when (> 1000 (house.total-floor-area))
          (measure.floor-insulation ...))
    default: (action.do-nothing))

In this example, we will apply measure.floor-insulation if (> 1000 (house.total-floor-area)), and we will (successfuly) do nothing otherwise. In truth, action.do-nothing is redundant here, as it is used as the default: even if you do not write it down; hence writing (action.case) is equivalent to writing (action.do-nothing).

The suitability and success of action.case (of which more later) is equal to the suitability of the action which is chosen, so the when that is selected need not be a suitable option. If you wish to restrict yourself to suitable alternatives, you can include (house.is-suitable-for ...) in the when's test.

14.1.2 After the fact (least-cost, etc.)

action.case is all very well when we can decide on our candidate without thinking about what the candidate will do. However, we often want to make a choice between alternatives based on something about their outcome; for example, we might want to choose the cheapest option, or the most cost-effective. The command for this is called choice, and has the following form:

    select: <<rule for picking the winner>>
    <<option A>>
    <<option B>>
    <<option C>>

Rather than using a simple logical test, as in action.case, choice uses a so-called selector for picking the winning candidate. These selection commands are provided so that you can decide between the multiple alternatives based on information about all of their outcomes. For example, there is no logical test in the NHM which can tell you which option has the least cost, as the tests are only defined to know about a single house. Instead, you can use the selection command select.minimum, which will choose the candidate for which a certain value is minimum, after the fact.

Let us look at minimising emissions by installing a measure:

    select: (select.minimum house.emissions)
    <<option A>>
    <<option B>>
    <<option C>>)

This choice command will do the following;

  1. From the current state of affairs, it will produce three hypotheses (of which more later)
    • A hypothesis in which only option A has taken place
    • A second in which only option B has taken place
    • A third in which only option C has taken place
  2. It will offer these hypotheses to the selection rule, which will pick one of them to come true. If any of option A, B or C prove to be unsuitable, their hypotheses will not be offered to the selection rule. In the case of select.minimum, this means:
    1. Calculate house.emissions according to hypothesis A; this is the emissions that will result from doing option A
    2. Calculate the equivalent for hypotheses B and C
    3. Compare these three values, and select whichever hypothesis has the minimum value.

So this new combined action does whichever of the three options is suitable and has minimum emissions after the fact.

There are several other selection commands:

  • select.maximum; this is hopefully self-explanatory by contrast with =select.minimum.
  • select.weighted; this performs a weighted random selection; for example, if we write

        select: (select.weighted 1)

    The choice will produce three hypotheses for A, B and C, and then ask select.weighted to pick one. select.weighted will in turn compute the value of its argument (here, just 1) in each hypothesis (similarly to how select.minimum does), but rather than taking the minimum it will:

    1. Compute the total weight of all suitable options; in this case the argument is always 1, so the total weight is 3
    2. Randomly select one of the alternatives using its weight normalised by the total as a probability. In this case, that means that each option has a 1/3 chance of being selected. If you were to apply such a choice to 1000 houses, you would expect about 333 to get option A, 333 to get option B, and 333 to get option C (assuming they are all suitable for all the alternatives). Note how the weight is a bit different to a probability, as a result of the normalisation.
  • select.filter; this reduces the range of candidates using a logical test, and then uses another selection command to pick from the reduced list of candidates. For example, if you wished to select the option with minimum emissions whose net cost of application was not greater than 1000 pounds, you could write:

        select: (select.filter
                    test: (< net-cost 1000)
                    selector: (select.minimum house.emissions))

    This will:

    1. Produce the three hypotheses for A, B and C (again, allowing for suitability)
    2. Ask select.filter to pick one; it will compute the test: argument in each case, and collect up those for which the test passes. For example this might just be B and C, if A were too expensive.
    3. select.filter will then ask select.minimum to pick an alternative from this restricted subset; the result will then be one of B or C.
  • select.fallback; this is a slightly complicated rule. It allows you to try one way of selecting between alternatives, but "fall back" to another way if the first way does not admit any of the alternatives; this is only really relevant in combination with select.filter. For example, you might want to say "if any of the options reduces energy use to less than 10000 units, do the cheapest; otherwise, do that which has the maximum reduction".

        (select.filter test:(< house.energy-use 10000)
                       selector: (select.minimum net-cost))
        (select.minimum house.energy-use))

    In this example the first selector will be applied, but if it admits none of the alternatives (because they all fail the test:), it will use the second selector instead.

14.1.3 Choices to do with before-to-after differences

All of the choices shown above are a pure function of the state of affairs after the alternative has been considered. It does not matter what the house was like beforehand; in any situation where the outcomes are the same, the same choice will be made. However, some models must involve a function of the change resulting from an alternative; if you are considering cost-effectiveness, this is to do with the ratio of effect to cost, and effect is typically a change. In recent versions of the model there are two ways to refer to these kinds of changes. One involves storing the quantity of interest before the change in a variable, and is explained in the section on variables, and one involves referring to the original value of the quantity of interest, which is explained in the section on hypotheses. To summarise quickly here, one may either write:

  (set #initial-value (house.emissions))
    select: (select.maximum (/ (- #initial-value house.emissions) net-cost))
    A B C

or (but only in recent model versions)

  select: (select.maximum (/ (fall-in house.emissions) net-cost))
  A B C)

For more about these see the respective sections below.

14.1.4 action.do-nothing in choices

We have mentioned several times that choices offer only the suitable alternatives to their selection rule. Suitability rules are discussed in more detail below; however, we mention here that choice will only ever select a suitable alternative, and if there are no suitable alternatives it will itself be unsuitable and will fail. If you wish to ensure that a choice is always suitable, you can include action.do-nothing as one of the alternatives; it is always suitable, and so (so long as it is not ruled out by a use of select.filter) it is always available for selection.

14.1.5 Ties in choices

In all cases, ties are resolved by order of writing in the scenario; for example if using (select.minimum x), if two candidates both produce minimal x the first one written will be selected.

14.2 How to combine several options together; making packages

The other way to assemble n actions into one is to do them in a defined order; the model provides the do command for this. Its syntax is quite simple:


When applied to a house, this command will first apply measure A, and then apply measure B to the house (in the condition left by applying A), then apply measure C to the house (in the condition left by B), and so on.

The do command has special behaviour in two aspects: first, it uses 16 to ensure that only suitable packages can have any effect, and second it allows you to see and record "ephemeral" values like net-cost and size.m2 whose values depend where they are being evaluated in the scenario.

Remember that any action applied to a house can go one of two ways:

  1. The action succeeds, because it is suitable for the house, and it may have some effect on the house like installing a boiler or changing some flags (this last part is not obligatory; there are two basic commands which are suitable and succeed but have no effect, namely action.do-nothing and an empty (do) statement).
  2. The action fails, because it is unsuitable for the house, in which case it is forbidden for the action to have any effect on the house, and if it does it's a bug in the model.

Bearing this in mind there is a natural question for do: what happens if one of the measures that we are doing is unsuitable, and fails?

The answer depends on the all: argument to do:

14.2.1 All-or-nothing packages

When do is used with all: true, or with no value given for all: as the default is true, the do action is only successful when every action inside it completes successfully.

If all but the last action in a do command succeed, the whole do fails, and it follows from the rules about suitability mentioned above that the do as a whole must have no effect; that is to say, all of the effects of the successful actions in the do are reversed if action fails.

For example, the action:

(do (measure.wall-insulation type:cavity ...)
    (measure.standard-boiler fuel:mainsgas ...))

Is a package giving the all-or-nothing proposition of cavity wall insulation and a new boiler together, or neither one. Only a house on the gas grid with an uninsulated cavity-type wall could be be affected by this action, as for any other house one of the measures would fail precluding the other from having an effect.

14.2.2 Try-everything packages

If the all: argument is false, the do command will resolve the problem differently; it will ignore the success or failure of the actions within it and simply try to do them one after another, and it will always be considered to succeed. The effect of the do will be the compounded effect of whichever actions where successful, since none of the unsuccessful actions will have had any effects (as this is forbidden).

Changing our previous example:

(do all:false
    (measure.wall-insulation type:cavity ...)
    (measure.standard-boiler fuel:mainsgas ...))

We have an action that can have four outcomes:

A house on gas, with an uninsulated wall
Wall insulation is installed and then a boiler is, successfully
A house off gas with an uninsulated wall
Wall insulation is installed, successfully
A house on gas, without an uninsulated wall
A boiler is installed, successfully
A house off gas without an uninsulated wall
Nothing happens at all, successfully

This form of the do command just throws some measures at a house without caring what sticks.

14.3 Access to ephemeral values

As mentioned, the do command has special handling of ephemeral values; when using the following commands within a do

  • set
  • increase
  • decrease
  • consume
  • fail-unless

The values of net-cost, capital-cost, size.m2, size.kw, original, rise-in and fall-in are all computed with respect to the effect of the do command so far. For example, if we amend the example above:

(do (set #x-1 (net-cost))
    (measure.wall-insulation capex: 100 type:cavity ...)
    (set #x-2 (net-cost))
    (measure.standard-boiler capex: 200 fuel:mainsgas ...)
    (set #x-3 (net-cost)))

The variables #x-1, #x-2 and #x-3 will take the values 0, 100, and 300 respectively; this is because when the first set happens the net cost of the do is zero (as it has done nothing to spend money), when the second happens the net cost is 100 as the do has bought some insulation for 100 pounds, and when the third happens it is 300 as the do has bought both 100 pounds of insulation and 200 pounds of boiler.

The same rules about suitability apply here, so the outcome would either be these values of the variables and the two measures installed, or the variables unchanged and neither measure installed.

The choice command has a similar rule for its select: argument; when values in the select: are computed they are computed with respect to a particular alternative, and so net-cost is the net cost of doing the alternative, capital-cost the capital cost, (rise-in x) the increase in x caused by doing that alternative and so on.

14.4 Exercises about choices and packages

  1. For each of these situations, which outcome or outcomes would you expect to see (you may want to read the section above on flags first):
    1. Applying the action

          (when (house.region-is London)
                (house.flag london))
          (when (house.built-form-is Detached)
                (house.flag detached))
          (when (> house.total-floor-area 500)
                (house.flag large)))

      to each of these houses, all having no flags

      1. A detached house in london with a floor area of 400
      2. A terraced house in london with a floor area of 600
      3. A terraced house in wales with a floor area of 500
      4. A terraced house in wales with a floor area of 600
      5. A detached house in wales with a floor area of 600
    2. Applying the action

          (house.flag A)
          (house.flag B)
          (house.flag C))

      to a house with no flags

    3. Applying the action

          select: (select.minimum house.total-floor-area)
          (house.flag A)
          (house.flag B)
          (house.flag C))

      to a house with no flags

    4. Applying the action

          select: (select.weighted 5)
          (house.flag A)
          (house.flag B)
          (house.flag C))

      to a house with no flags

    5. Applying the action

      (do (house.flag A)
          (action.do-nothing test-flags: !A)
          (house.flag B))

      to a house with no flags

    6. Applying the action

      (do all:false
          (house.flag A)
          (action.do-nothing test-flags: !A update-flags: C)
          (house.flag B))

      to a house with no flags

    7. Applying the action

      (do (action.do-nothing test-flags: !A update-flags: C)
          (house.flag A)
          (house.flag B))

      to a house with no flags

    8. Applying the action

          select: (select.minimum 0)
          (do (house.flag A)
          (house.flag B))

      To a house with no flags

    9. Applying the action

        select: (select.minimum #x)
        (do (house.flag A)
            (set #x 1))
        (do (house.flag B)
            (set #x 2))
        (do (house.flag C)
            (set #x 3)))

      To a house with no flags or variables set

    10. Applying the action

        select: (select.maximum #x)
        (do (house.flag A)
            (set #x 1))
        (do (house.flag B)
            (set #x 2))
        (do (house.flag C)
            (set #x 3)))

      To a house with no flags or variables set

    11. Applying the action

        select: (select.weighted #x)
        (do (house.flag A)
            (set #x 1))
        (do (house.flag B)
            (set #x 2))
        (do (house.flag C)
            (set #x 3)))

      To a house with no flags or variables set

    12. Applying the action

        select: (select.filter test: (> #x 1)
                               selector: (select.minimum #x))
        (do (house.flag A)
            (set #x 1))
        (do (house.flag B)
            (set #x 2))
        (do (house.flag C)
            (set #x 3)))

      To a house with no flags or variables set

  2. Write some code to do each of the following actions
    1. Install three measures

There are some more exercises related to choices and packages in the section on suitability and failure.

14.4.1 Answers

  1. The effects will be
    1. By type of house:

      1. The flag london will be added
      2. The flag london will be added
      3. No change, but the action will be a success; this is because of the default: being action.do-nothing unless otherwise specified
      4. The flag large will be added
      5. The flag detached will be added

      If you have added more than one flag in any case, remember that action.case will only ever do one of its alternatives

    2. The flags A, B and C will all be added
    3. The flag A will be added; this is because:
      • choice only ever adds one flag
      • house.total-floor-area is the same for all the alternatives, so there is a tie
      • ties are resolved by order of writing and A is at the top
    4. One third of the time, the flag A will be applied, one third of the time B will be applied, and one third of the time C will be applied.
    5. The action will always fail, and hence have no effect. This is because:
      • The first action in the do will always succed, and leave the house having flag A
      • The second action will fail whenever the house has flag A, i.e. always
      • The third action will never get a look-in, because the second action will always fail
      • The effect of the first action will always be reversed because a failed action may never have effects on the house, and the do has failed
    6. The house will always have just the flags A and B added. This is because:
      • all: is false, meaning that do will plod along trying each of its actions and ignoring whether they fail or not
      • (house.flag A) will succeed, adding A
      • (action.do-nothing test-flags:!A update-flags:C) will fail, because of the presence of A; however, do will keep going because all:false
      • (house.flag B) will succeed, adding B
    7. The house will always end up with the flags A, B and C. This is because although the action.do-nothing test-flags: !A would fail if the flag A were present, the do is performing its actions in sequence and we have not got to (house.flag A) by that point yet. These last three examples illustrate how the effects of actions feed-through to each other.
    8. The house will always end up with the flag B. This is because choice can never select an unsuitable/failed alternative, and the first alternative will always fail.
    9. The house will always end up with the flag A and the variable x being 1. The three alternatives will produce the following three hypothetical situations:

      • flag A is present and x is 1
      • flag B is present and x is 2
      • flag C is present and x is 3

      Of these, the first minimises x, so this is the outcome that will be selected.

    10. The converse of the previous example; flag C and x equals 3 will happen.
    11. The three hypotheses will be selected with differing probabilities:
      • flag A is present and x is 1, with probability 1/6
      • flag B is present and x is 2, with probability 2/6, or 1/3
      • flag C is present and x is 3, with probability 3/6, or 1/2
    12. The house will always end up with flag B and x being 2, because: The three hypotheses will be produced as before:

      • flag A is present and x is 1
      • flag B is present and x is 2
      • flag C is present and x is 3

      First select.filter is used, and knocks out the first alternative because x is not greater than 1:

      • flag A is present and x is 1
      • flag B is present and x is 2
      • flag C is present and x is 3

      Then select.minimum is used, and chooses the first of the remaining alternatives, because x is least in that case.

15 Measure suitability and failure

In the NHM an action is something that you use to change a house. Actions may have one of two outcomes:

  1. The action may succeed, and can then change the state of affairs in some way, for example
    • installing insulation in a house, and spending some money to do so
    • changing a user-defined variable on a house
    • putting in a boiler AND putting in some insulation together
    • writing in a report and installing double glazing
    • doing nothing, but with a merry sense of success
  2. The action can fail, in which case it definitely does not change how things are

Whether an action succeeds or fails is determined by its suitability. An action which fails is unsuitable (and any unsuitable measure will fail), whereas one which succeeds is suitable (and any suitable measure will succeed).

Measures have certain built-in suitability criteria, which are defined within the model code which enacts them.

For example:

  • a gas boiler measure will not be suitable for a house which is not on the gas grid.
  • cavity wall insulation will not be suitable for a house which has only solid walls

These suitability criteria are documented in the language reference for each measure.

15.1 Suitability of compound actions

All other actions also have suitability rules, which are also documented in the language reference. These rules are more complex for the compound actions like do, choice and action.case. It is important to understand these rules if you use the compound actions, because they can (a) cause your actions to fail when you might expect them to succeed, and also (b) you can use them to model complicated behaviour like supply chains, post-hoc constraints, random outcomes and all sorts of other effects. The sections on these commands describe their suitability rules a little, but this section covers them in more detail, explaining how they can be put together.

15.1.1 Actions using test-flags:

In the section on flags we saw that most actions in the model have the test-flags: argument, which makes the action check the presence or absence of certain flags as a way of changing its suitability.

Here are some examples:

  • (measure.wall-insulation type:cavity test-flags: A ...); this measure will only be suitable if (a) it would be suitable anyway (i.e. the wall insulation rules are satisfied), AND (b) the house has the flag A on it.
  • (measure.wall-insulation type:cavity test-flags: [A B] ...); this measure will only be suitable if it would be suitable anyway AND the house has BOTH the flags A and B on it.
  • (measure.wall-insulation type:cavity test-flags: [A !B] ...); this measure will only be suitable if it would be suitable anyway AND the house has the flag A on it AND the house DOES NOT have the flag B on it.
  • (measure.wall-insulation type:cavity test-flags: * ...); this measure will only be suitable if it would be suitable anyway AND the house has at least one flag on it
  • (measure.wall-insulation type:cavity test-flags: !* ...); this measure will only be suitable if it would be suitable anyway AND the house has NO flags on it
  • (measure.wall-insulation type:cavity test-flags: !B* ...); this measure will only be suitable if it would be suitable anyway AND the house has NO flags on it that start with B

The test-flags: argument can only make an action less suitable for houses; it will not remove any existing suitability rules. It is checked before the other suitability rules are checked, as a precondition; if it does not pass, the measure immediately stops and produces a failure.

15.1.2 action.do-nothing and action.fail

The special action action.do-nothing always succeeds (UNLESS you use test-flags: on it, in which case it can fail). The special action action.fail always fails, no matter what.

These two are useful in two ways:

  • action.do-nothing is useful in a choice, where you want to be sure that there is always a feasible option.
  • action.fail is useful within action.case, when you want to trigger a failure under certain circumstances.

15.1.3 action.case

action.case has the suitability/success behaviour of whichever of the actions in it end up happening. Recall that action.case may contain several different alternatives, thus:

   (when A X)
   (when B Y)
   default: Z)

In this case, if X is suitable, but the test A does not pass for a house, and B does pass but Y is not suitable, action.case will pick Y and also be unsuitable. The choice of alternative is taken before any consideration of suitability, so having a suitable option does not mean it will be chosen. If you wanted this, you could augment the tests to check suitability as well:

    (when (all A (house.is-suitable-for X))
    (when (all B (house.is-suitable-for Y))

In this example, each when also includes the test for suitability, so the action.case will never attempt an unsuitable alternative.

15.1.4 do

The do command is quite complicated; its suitability behaviour depends on the value of the all: argument it is supplied:

  • If all: is true (or not supplied), do is only suitable if each action inside it is suitable, at the point when they are applied. Thus if any action inside the do fails, the entire do will fail.
  • If all: is false, do is always suitable (unless it also has test-flags:, which are checked as a precondition). Even if all the actions inside the do fail, it will succeed, and the effects of any successful changes will be kept (unless the do is inside a further do which turns out to fail).

When we say that the do is suitable if the actions inside are suitable at the point when they are applied, we mean this; consider the following do statement:

    (action.do-nothing test-flags: !A update-flags: A)
    (action.do-nothing test-flags: !A))

Imagine a house which does not have the flag A; if we consider each of the actions inside the do in turn, they both appear to be suitable for the house. After all, it does not have the flag A. However, a successful application of the first action will add the flag A to the house, and as a result the second action will be unsuitable once the first has happened. Hence this do statement is always unsuitable and will always fail.

When a do command fails, its effects are 'rolled back' - in the example above, not only is the do statement perpetually unsuitable, it also has no effect. This is because although the first action will put the A flag on the house, the failure of the second causes the failure of the do, and a failed action may not have any effects.

More realistic examples have the same all or nothing behaviour:

(do (measure.wall-insulation ...)
    (measure.standard-boiler fuel:mainsgas))

A house exposed to this action will receive either the wall insulation and the boiler or neither; it must be suitable for both, and it must be suitable for the boiler after its walls have been insulated.

One useful effect of this is to allow action.case to 'undo' things after the fact. For example, imagine we want to install a boiler, but only if the fuel bill is sufficiently reduced:

(do (measure.standard-boiler ...)
        (when (< (fall-in house.fuel-cost) 500)

We can only know the fall in fuel bill by installing the boiler (as we have to do an energy calculation to figure it out). In this example we install the boiler, and then use action.case to trigger action.fail only if after the boiler is installed the fuel cost has not been sufficiently reduced.

It is important to remember that the failure of a do command will 'cascade'. For example if we write

    (do A B)
    (do E F))

If any part of this fails the whole thing will fail, undoing anything which has been done.

15.1.5 choice

The rules for choice are simple:

  1. A choice will only present suitable alternatives to its selection rule, so the selection rule can only ever select a suitable alternative.
  2. If no suitable alternative is selected, the choice as a whole will fail and be unsuitable.

This means that a choice is suitable if and only if it contains a suitable alternative, and that suitable alternative is considered by the selection rule.

All selection rules except select.filter consider all of the suitable alternatives they are presented.

15.2 Changing suitability

Sometimes you may wish to change the suitability of an action. In the NHM, actions' suitability can only ever be reduced, as you can add a test either before or after doing the action which will cause it to fail under certain circumstances, but you can never change the action so that it succeeds in conditions where it would have failed. You have already seen how you can use flags for this, with the test-flags: argument that most measures will accept. You may require a more general rule, which lets you render a measure unsuitable for any reason.

In recent versions of the model you can do this using the fail-unless command within do

For example, say we had a measure like this:

(measure.standard-boiler fuel: mainsgas)

If we wished to amend this measure so that it was only suitable in urban houses we could write:

(do (fail-unless (house.morphology-is urban))
    (measure.standard-boiler fuel:mainsgas))

You could also write this using action.case as

(do (action.case (when (none (house.morphology-is urban)) (action.fail)))
    (measure.standard-boiler fuel:mainsgas))


(action.case (when (house.morphology-is urban)
                   (measure.standard-boiler fuel:mainsgas))
      default: (action.fail))

These are all equivalent statements; in each case the morphology of the hosue is tested before the measure is installed, and the installation fails if the morphology is not urban.

Sometimes you may wish to check suitability after the fact; for example, you might say that an insulation measure is not suitable if it insulates a very small or very large area, as small areas are not worthwhile and large areas require a more expensive material.

In this case, you can again use fail-unless or action.case

(do (measure.wall-insulation type:cavity resistance: 0.01 thickness:50)
    (fail-unless (> 10000 (size.m2) 10)))


(do (measure.wall-insulation type:cavity resistance: 0.01 thickness:50)
    (action.case (when (none (> 10000 (size.m2) 10)) (action.fail))))

As well as fail-unless, there is a special command called consume which fails unless it can reduce a variable by a certain amount without making it negative; this is useful for defining things like supply chains or maximum emissions rules.

For example, say you wished to restrict installations of a certain measure to 100,000 units per year:

(def remaining-units on:simulation default: 0)

;; every year, reset the remaining units
(on.dates (regularly)
          (set #remaining-units 100000))

(on.dates ...
    (apply to: ...
        (do (consume #remaining-units)

Here we have a global counter called #remaining-units, and in the do command around the-measure, we use consume to make sure that we have enough units left to install; consume will decrease #remaining-units by the house's weight, unless that would make #remaining-units negative in which case it will fail and the-measure will not be installed.

You could write this out as

(do (decrease #remaining-units (house.weight))
    (fail-unless (>= #remaining-units 0))

So consume is just a shorthand for this.

15.3 Exercises on suitability

For each of these measures, consider its application to the listed houses and determine whether it will succeed, and explain why

  1. The simplest option

    • applied to any house
  2. The second simplest option

    • applied to any house
  3. A slightly less simple option (see the 9 section if unclear)

    (action.do-nothing test-flags: qwerty)
    • applied to a house with no flags
    • applied to a house with the flag qwerty
    • applied to a house with the flag qwertyuiop
  4. And again

    (action.do-nothing test-flags: !qwerty)
    • applied to a house with no flags
    • applied to a house with the flag qwerty
    • applied to a house with the flag qwertyuiop
  5. Another simple one

    • applied to any house
  6. Doobie doo

    (do (do (do)) (do))
    • applied to any house
  7. Do choose

    (choice select: (select.weighted 1)
    • applied to any house
  8. Do choose some more

    (choice select:
            (select.filter test: (house.region-is london)
                           selector: (select.weighted 1))
    • to any house
  9. Do fail a choice

        (choice select: (select.weighted 1)
        (choice select: (select.weighted 1)
    • to any house
  10. Do a measure involving 11 and 10.

    (do (measure.standard-boiler fuel:mains-gas capex:1000)
        (consume #boiler-expenditure (* (house.weight)
    • To a house suitable for a gas boiler
    • To a house not on the gas grid
  11. Do a measure later

    (do (consume #boiler-expenditure (* (house.weight)
        (measure.standard-boiler fuel:mains-gas capex:1000))
    • To a house suitable for a gas boiler
    • To a house not on the gas grid
  12. Total failure

        (when (= 1 2) (action.fail))
        default: (action.fail))
    • to any house
  13. Failure?

        (when (> 1 2) (action.fail))
        default: (do update-flags: hello))
    • to any house
  14. Success?

        (when (< 1 2) (action.fail))
        default: (do update-flags: hello))

15.3.1 Answers

  1. action.do-nothing is suitable
  2. action.fail is never suitable
  3. We are testing for the flag qwerty, so it needs to be there, i.e.
    • unsuitable
    • suitable
    • unsuitable
  4. Now we are testing for the absence of the flag qwerty , i.e.
    • suitable
    • unsuitable
    • suitable
  5. A do on its own is suitable, and does nothing
  6. This is suitable, because:
    1. do is suitable if its children are suitable, and
    2. (do (do)) is suitable because do is suitable if its children are suitable and the do on its own is suitable, and
    3. the final do is also suitable as it has no children to make it fail.
  7. A choice is suitable so long as (a) some of its alternatives are suitable and (b) the selector does not rule them out; here we have three alternatives which are all suitable (each one is an empty do), and a selector which does not rule any out. So, this is always suitable.
  8. This choice has suitable alternatives in do and action.do-nothing, but that is not enough; its selector select.filter has a test: which only passes in London, so the choice is suitable only for houses in London in which it does nothing and unsuitable everywhere else.
  9. The do will always fail, because it has two actions in it:
    1. a choice with the only alternative being action.fail, which must fail, and
    2. a choice which will always succeed, but we never get there because of the preceding guaranteed failure.
  10. Firstly we are putting in a boiler, and then we are using consume, so there are two possible sources of failure: (a) boiler is unsuitable or (b) consume doesn't work because the second argument is more than the first.
    • for the house which can have the boiler, we are suitable if #boiler-expenditure is not less than 1000 times the house's weight; this is because consume is only suitable if it can reduce the variable #boiler-expenditure by house.weight times capital-cost without making it negative.
    • if the house is not on the gas grid, the consume doesn't matter, as the house can't have the boiler so it can't have anything
  11. So this time we have made a probable coding mistake - this is a bit of a trick question. As explained in 11, capital-cost is a special command which tells you the capital cost of the current measure or action. When used in a consume inside a do like this, it is the capital cost of the do so far. When the consume is checked, this will be zero, as we haven't got to the measure yet, and hence consume will succeed unless #boiler-expenditure is negative because subtracting zero cannot otherwise make it negative.

    After that, we will put in the boiler unless we can't; if the boiler is suitable the whole thing is suitable, and otherwise it isn't.

  12. Both alternative actions are action.fail, so we always fail.
  13. one is never greater than two, so we never look at action.fail; we always fall to the default: which is suitable, because update-flags: does not affect suitability and do is sutiable.
  14. one is always less than two, so we always try action.fail and then fail. We never try the other option, so this is always unsuitable.

16 Hypotheses

Before reading about this, make sure you have read the preceding sections on packages and suitability, as they are important prerequisites.

16.1 In-depth explanation of choices, packages and failure

The NHM's simulation proceeds by making use of hypotheses; every change that the model thinks about, whether to a single house or a population of houses, starts of as a hypothesis, in which the model changes things speculatively.

The changes which happen when considering a hypothesis are isolated from the model's view of the real world, and can change anything about the world until the hypothesis is either discarded without any change to how things "really" are, or it is accepted and becomes the new status quo.

This basic feature is used to make commands like do and choice work; in a do, the changes being applied happen within the hypothesis that the do will succeed. If one of the changes is not successful, the do can safely throw away the hypothesis in the knowledge that there will be no side-effects. Similarly, the choice command produces n hypotheses, one for each of the alternatives, and discards n-1 of them. To further complicate matters, the model can make a hypothesis within another hypothesis16; this is what happens when you place a choice within a do or vice-versa.

These commands (do and choice) have a common factor: whichever hypothesis ends up being accepted as the new status quo, we know that nothing which happened in any of the discarded alternative hypotheses can affect the simulation (it can affect reports, of which more later; reports are outside the simulation's view of the world).

There are some other commands which will allow you to work something out in a hypothesis, but to store the result outside that hypothesis. You could think of this as being analogous to using your imagination, at least if your imagination were very effective. Let us look at the first of these commands, under

16.2 Using under to work out values in hypothetical situations

Imagine a scenario in which you are modelling the offer of several packages of measures to various houses, and the houses are deciding between these packages based on the annual heating bill they will have after installing each package:

    select: (select.minimum (house.fuel-cost))


package-1, package-2 and package-3 are standing in for some actual packages that we are considering, for ease of reading. In this case, we will take each package, and then pick the one for which the fuel cost after installation is least.

Now say that we wish to model something slightly different: say that the householder making the decision has a belief that electricity prices will rise and gas prices will fall in the future, and so we wish to make the choice on the basis of these different prices.

However, these prices do not yet pertain in the model, and indeed may never as the householder could be wrong in their belief. We must make the choice minimising how fuel cost would be if we were to have a different suite of prices.

The tool for this is the under command, which has the following form:

    <<some actions>>
    evaluate: <<value of interest>>

When the under command is used by the model, it does the following:

  1. Produce a new hypothesis, which looks the same as where the under is being worked out, but is isolated from it because it is a hypothesis
  2. Uses the actions to change things in the hypothesis
  3. Works out the argument to evaluate: in the hypothesis, so that it "sees" the changes that were made
  4. Discards the hypothesis, and produces the value from 3 back where the under was being computed

You can read this as saying, "without changing how things are now, imagine what would happen if we did the actions and worked out the evaluate: argument afterwards".

So returning to our example, we can evaluate the fuel cost under a different set of tariffs, reflecting the householder's beliefs without changing the tariffs which are actually used in the simulation:

 select: (select.minimum
             (fuel type:mainsgas (charge (* 0.01 house.meter-reading)))
             (fuel type:peakelectricity (charge (* 0.3 house.meter-reading)))
             (fuel type:offpeakelectricity (charge (* 0.3 house.meter-reading)))
           evaluate: (house.fuel-cost)))

In this case the model will create six hypotheses, which we can draw out in a hierarchy:

  1. From the initial state, before the choice:
    1. a hypothesis in which package-1 has been applied, in which under makes:
      1. A sub-hypothesis in which the tariffs have been changed and package-1 has been applied
    2. a hypothesis in which package-2 has been applied, in which under makes:
      1. A sub-hypothesis in which the tariffs have been changed and package-2 has been applied
    3. a hypothesis in which package-3 has been applied, in which under makes:
      1. A sub-hypothesis in which the tariffs have been changed and package-3 has been applied

In each of the sub-hypotheses, under will work out the house.fuel-cost, giving the fuel costs for:

  1. package-1 with the hypothetical tariff
  2. package-2 with the hypothetical tariff
  3. package-3 with the hypothetical tariff

However, each of these is disposed of after being used to work out the cost, and the choice will select one of the "first-level" hypotheses as the actual outcome.

16.2.1 under and suitability

There is a subtlety here, which is that the under command has to produce a value, but the actions in it may be unsuitable. When it is used, the under command will stoically try to apply whatever actions you have given it and then work out the result even if the actions failed; for example if you said:

    (measure.standard-boiler fuel:mainsgas ...)
    evaluate: (house.energy-use))

To work out the energy use a house would have if you installed a new gas boiler, you will get one of two results:

  1. The energy use after installing a new gas boiler, if the boiler was suitable, or
  2. The current energy use, if the boiler was unsuitable

This ambiguity can be avoided using variables; for example you could amend the above to say:

    (set #energy-use -1)
        (measure.standard-boiler fuel:mainsgas ...)
        (set #energy-use (house.energy-use)))
    evaluate: #energy-use)

This would do the following:

  1. Make a hypothesis for the under, which will be thrown away later no matter what
  2. Within that hypothesis, change the variable #energy-use to be -1, which we will use as a special value to indicate unsuitability
  3. Now within that (outer) hypothesis, make another (inner) hypothesis for the do command, in which we will
    1. Try and install a new gas boiler; if this fails, we will stop the do command with no effect on the outer hypothesis
    2. If it succeeds, we will store the new (house.energy-use) in #energy-use, and the new state of affairs will be accepted for the outer hypothesis
  4. Now the do has completed, and the outer hypothesis will either be:
    • #energy-use is -1, and no boiler went in, or
    • #energy-use is (house.energy-use) after a boiler went in, and a boiler went in
  5. Finally under evaluates #energy-use within the outer hypothesis, producing either -1 or the energy use after the boiler
  6. The outer hypothesis is discarded and the simulation proceeds having either a -1 or the value of interest available

At this point if the value is emitted into a report the special value -1 can be filtered out in your other tools, or if it's being used in the simulation it can be tested using action.case or function.case to determine what to do in the situation where the measure is unsuitable.

16.3 Other ways to use hypotheses; set under:, rise-in, fall-in and original

Using under can be computationally expensive - each use of under can only compute a single value, so if you want to calculate several different values under some hypothetical changes you need to write it out several times. The model is not clever enough to know when the work of making the changes can be reused, so each use of under creates its own hypothesis, does all the work of making the changes and updating derived values like the energy use, and then throws the hypothesis away. As a result, if you ask for ten different values each under the same assumption by writing (under A evaluate:X), (under A evaluate:Y), and so on, it is ten times as expensive as asking for one.

As an alternative the model provides special argument to set to allow you to calculate several values in a hypothesis all at once:

    [#variable-1  #variable-2  #variable-3]
    [(function-1) (function-2) (function-3)]
    under: [(action-1)

This will do the following:

  1. Produce a new hypothesis, wherein:
    1. Apply action-1 to change things in the hypothesis
    2. Apply action-2 to change things further in hypothesis
    3. Compute function-1 in the hypothesis, and store the result in #variable-1
    4. Compute function-2 in the hypothesis, and store the result in #variable-2
    5. Compute function-3 in the hypothesis, and store the result in #variable-3
  2. Copy the values of #variable-1, #variable-2 and #variable-3 out of the hypothesis and into the "real" values for the variables
  3. Discard the hypothesis

This will have the same effect as writing

(do (set #variable-1 (under (action-1) (action-2) evaluate:(function-1)))
    (set #variable-2 (under (action-1) (action-2) evaluate:(function-2)))
    (set #variable-3 (under (action-1) (action-2) evaluate:(function-3))))

But it will be much faster, as (action-1) and (action-2) need only happen once, only one hypothesis is generated, and any calculation which can be shared between the three functions will be (for example, uses of the energy calculation).

A very common need is to compute the change in values, as a result of a hypothetical action. For example, when defining a choice you might want to include only those alternatives for which the reduction in fuel cost is greater than the net cost of installation. You can do this with variables, writing something like this:

    (set #initial-bill (house.fuel-cost))
        select: (select.filter
                    test: (> (- #initial-bill (house.fuel-cost))
                    selector: (select.minimum (net-cost)))

Newer versions of the model offer a simpler syntax:

 select: (select.filter
          test: (> (fall-in (house.fuel-cost))
          selector: (select.minimum (net-cost)))

The command fall-in computes its argument twice and produces the difference; the first time is computed outside the current hypothesis and the second inside. In this case the choice command is responsible for creating the current hypothesis, so the (fall-in (house.fuel-cost)) will be the difference between the fuel cost before applying any alternative and the fuel cost after. The rise-in command does the natural reverse of this (the difference between the value inside and outside), and the original command provides direct access to the value outside.

16.4 Some "hypothesis diagrams" for commands

It can help to draw out how hypotheses are constructed and selected for different commands. In each of these diagrams a circle represents a state of affairs, a solid arrow the application of an action to that state of affairs, and a dashed arrow the production of some hypotheses.

Circles with a red outline are those whose hypotheses are discarded, and those with a double-circle outline are those which are selected.

A red solid arrow indicates a failed measure, and will therefore always lead to a red circle.

16.4.1 do

Imagine the command

(do (X)

In the event that both X and Y succeed, this will produce the following sequence:


In which the initial state A is converted to a hypothesis B, on which we apply X to get C and then apply Y to get D, which is then selected as the new state of affairs.

Alternatively, one of X or Y might fail; if X fails we have


and if Y fails we have


Remember that the do as a whole has failed if its contents fail; this is why the final arrow to the double circle is shown as red in the last two diagrams. If the do were within another do, the outer one would also fail, and so on. Indeed you could "embed" the diagram for one of these examples into the placeholders for actions that are displayed like X, Y and so on:


This illustrates something like:

   (do (Q) (P) (R))

in which Q fails, causing the entire process to fail.

If the outer do were instead do all:true:

(do all:true
   (do (Q) (P) (R))

The diagram would look more like this:


i.e. the inner do which failed has simply been skipped over, and we end up with a successful application of X and then Y.

16.4.2 choice

Let's consider a single choice:

    select: (select.minimum (O))



Imagine that that option Z is unsuitable, and options X and Y are both suitable but Y has the minimum value for objective O.

In this picture we also have square nodes, which indicate the evaluation of the selection rule in each hypothesis.


You can see how measure Z is eliminated by unsuitability, and measure X by the selection rule.

16.4.3 under

The hypotheses produced by under never become real, but they do produce some information back in their originating state:

(under (X)
       evaluate: (Y))


Here we start in state A, produce hypothesis B in which we apply X to get hypothesis C, wherein we compute Y per rectangle D; the dotted line shows the result of Y being communicated back into the accepted state F, whereas E shows how the hypothesis for the under is thrown away.

16.5 Output from within hypotheses

Output from reporting commands is sent to files immediately, and is not part of the simulation's state17; it is therefore not reversed when reporting was done within a hypothesis that is later discarded. This can lead to confusing output, but may also be quite useful; for example, if you apply the following action:

    (probe name: p
           (measure.standard-boiler ...))

The probe p will probably contain records which show a boiler being successfully installed in some dwellings (so long as it has been applied to dwellings for which a boiler is suitable). However, these dwellings will never actually receive a boiler, because the action.fail will cause the surrounding do to be unwound; it can never have a real effect.

If you were to amend the above to read:

(probe name:q
     (probe name: p
            (measure.standard-boiler ...))

The two probes would be correlated; each house presented would produce a row in both p and q, but all the rows in q would show failure whereas some of the rows in p would show success.

The reports generated by def-report include a special column called selected, which indicates whether the hypothesis in which a row of the report was computed was eventually made part of the true state, or whether it was later discarded.

16.6 TODO Exercises about hypotheses

17 Defining things so you don't have to repeat them

You have already seen the commands for defining and referring to variables, and reports. Some similar syntax exists for actions, tests, and functions, to wit:

(def-test house-is-in-london (house.region-is London))
(def-test house-is-large (> (house.total-floor-area) 10000))

Would define two reusable tests, which you could refer to anywhere as #house-is-in-london or #house-is-large, for example as:

(apply to: (filter #house-is-in-london)
       (action.case (when #house-is-large

As far as the model is concerned, this is akin to writing

(apply to: (filter (house.region-is London))
       (action.case (when (> (house.total-floor-area) 10000)

This is useful when you want to use a common definition in a lot of places, because:

  1. The reader can remember the meaning of a complex expression under a single name, and
  2. The editor can change the definition in a single place and have a universal effect.

It is not useful when you have a definition which is not being reused, as it merely adds some extra mental overhead for the reader.

The analogs for actions and functions are:

(def-action my-package
            (do (measure-1)

(apply to:(filter (house.is-suitable-for #my-package))


(def-function standard-capex 500)
;... later on
(measure.standard-boiler capex:#standard-capex)

Note how def-function need not define a complicated function; it can be a constant if you would prefer.

18 Templates and modules

The def-function, def-action and def-test (in the previous section) commands provide a simple way to reuse common fragments of code. They are efficient for the simulator, and stand out when written because of their hash symbol. However, they are inflexible in that they define a single, fixed unit of code. Often you will instead have a repeating pattern in your code, which you want to avoid reiterating again and again. The model provides this using what is called a preprocessor (for better or worse; it may not be the best tool but it is the only tool here). If you have used SAS macros (or lisp macros, or the R substitute / eval primitives, or written what people call "dynamic SQL"), these are all analogous. If you have not, don't worry, we will cover it all directly in any case.

18.1 An example template

The simplest and most useful facility offered by the preprocessor allows you to define templates. A template is like a pro-forma bit of scenario syntax with some "blank spaces" in it that need to be filled in before it is complete. To employ a template, you first define the pro-forma under a name of your choice, with placeholders for the blank spaces, and then you use the template by invoking the name you gave it.

To make this concrete, consider the following situation; imagine you have written a scenario, but the Minister phones through and says "It is essential that at no point, as a result of the Mauve Deal policy, that emissions due to biomass should exceed 9 million tonnes per year; what would this do to the simulation outcomes? Any time in the next ten to fifteen minutes would be fine so don't rush.".

You consult the NHM manual, put on your thinking hat, and determine that this can be done by restricting the suitability of the measures the scenario is installing; this is a matter of checking a postcondition, namely that after installing a measure if you have made emissions go up you haven't taken them over 9 million tonnes. Postconditions are a good fit for do and fail-unless, so you cook up this:

(do ;; the measure goes here
    ;; now for the postcondition
        ;; so we give up, unless either:
            ;; the emissions for this house have not gone up
            (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
            ;; or if they have gone up, the total is still fine
            (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000)

This seems pretty reasonable, but looking at the clock we notice that we only have six minutes left, and we don't relish the prospect of correctly adding this around each measure we have written, even though we wrote each measure down using #def-action so they are all nicely defined in one place.

What we want here is a template; the code above is clearly a bit of pro-forma; it is the same in all cases, except for the something-or-other, which is a blank we need to fill in.

To define our template, we use the special template command, which looks like this:

(template <<template name>> [<<list of placeholders>>]
          <<pro-forma goes here>>)

In our case, let us say the template is to be called enforce-biomass-emissions-limit; the first step is simply to pop in our pro-forma (a.k.a. the template body):

(template enforce-biomass-emissions-limit []
   (do ;; the measure goes here
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000)

So far so good, but how do we tell the model that we want to put a particular thing in place of something-or-other? The answer is in two parts:

  1. We put something in the list of placeholders, to tell the template that it needs something to pop into place, and then 2
  2. We refer to that placeholder in the pro-forma in the right place.

To make them stand out, placeholders are always written with an atpersand (@) at the start of them; we can add one into the template we are imagining now:

(template enforce-biomass-emissions-limit [@measure]
   (do ;; the measure goes here
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000)

This is now our whole template - we have a placeholder, called @measure, which represents the blank to fill in. Now to write out emissions-limited actions in our scenario, we don't have to copy-paste the pro-forma into place every time; instead we simply invoke the template thus:

(apply to:(some-houses)
           (measure.standard-boiler fuel:biomasswood ...)

When the model sees that we have used enforce-biomass-emissions-limit as a command, it will remember that it saw a template by that name, and effectively replace the use of the template with its body; it is exactly as though you had just written down

(apply to:(some-houses)
   (do ;; the measure goes here
      (measure.standard-boiler fuel:biomasswood ...)@measure
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000)

You can think of a template as being a scenario editing rule that you might give to a particularly unfortunate intern, except it is applied transparently and automatically to your scenario when you press run.

Returning to our story, you are now able to zip through all the actions which you have sensibly defined with def-action, and amend them, so where once you had:

(def-action policy-biomass-boiler
            (measure.standard-boiler fuel:biomasswood ...))

You now have

(def-action policy-biomass-boiler
                (measure.standard-boiler fuel:biomasswood ...)))

Success! The astute reader will note that this replacement is analogous to what we have just done - the change to each def-action is itself a kind of pro-forma edit that could be done by our unfortunate intern. Indeed, if you found yourself doing this very frequently, you could say:

(template def-policy-action [@name @action]
    (def-action @name @action))

    name: policy-biomass-boiler
    action: (measure.standard-boiler fuel:biomasswood ...))

And then a single change to def-policy-action could change all your defined actions at a swoop!

This should give you some feel of when to use a template: if you have a lot of code which is often similar but with minor variations, consider replacing it with a template. Of course, adding more templates to your scenario can make it harder to read, especially when someone first approaches it, as they have to understand what each template does before they can understand the code which uses those templates, so you must weigh the introduction of templates carefully.

18.2 Placeholders and invocation

In the preceding example, we showed how to write a single placeholder, which had a name, and where the thing to be filled into the placeholder was identified by using that name.

There are a few different options available when writing a placeholder, which are explained in more detail in the language reference.

Each placeholder connects an argument supplied when invoking the template to some places in the body of the template.

18.2.1 Supplying placeholders when invoking a template

In the example we have seen, we had a single placeholder, called @measure. The code to fill into the placeholder when invoking the template was indicated by the keyword argument measure:, i.e.

(template example [@widgets @sprockets]

Has two placeholders, @widgets and @sprockets, and the content to go in these when invoking the template is identified by using these as named arguments (if you don't know what a named argument is, refer back to the very start of this document):

(example widgets: 9 sprockets: 13)

Any placeholder written like this (as an atpersand-prefixed word like @this or @that) will be filled in by the corresponding named arguments supplied to the invocation (like this: or that:).

You can also fill in placeholders with unnamed arguments by using the placeholders @1, @2, @3 and so on.

Changing our example, we could say

(template enforce-biomass-emissions-limit [@1]
   (do ;; the measure goes here
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000)

If we write this, then when we invoke the template the placeholder @1 will be substituted for the first unnamed (or positional) argument, e.g.

(enforce-biomass-emissions-limit (my-measure))

Finally, there is a third special placeholder you can use, called @rest. This placeholder will receive the content of any positional arguments that are left after @1, @2 and so on (for the ones of these you have used).

This is useful for passing many inputs on to another command which expects many inputs - for example, if you were defining a template to do a certain kind of choice, you could say

(template min-cost-choice [@rest]
            select: (select.minimum (capital-cost))

You could then invoke this with as many arguments as you like; they would all be filled in for @rest, for example:

(min-cost-choice (A) (B) (C))
; is the same as
  select: (select.minimum (capital-cost))
  (A) (B) (C))

18.2.2 Different names for placeholders within a template

Sometimes you might want have names for placeholders to make the template definition easier to read, without having to use those names when you invoke the template.

You can do this by writing a placeholder name with two parts separated by a colon, like @A:B. This will mean that you refer to the placeholder in the template body as @B, but that the model supplies it when the template is used according to the rules for @A.

In our example template, you might say that enforce-biomass-emissions-limit is only ever going to have one argument, which is the measure, so we want to use the @1 type of placeholder so template users are not forever writing measure: .... However, when reading the template, @1 is a bit opaque, so we could write instead:

(template enforce-biomass-emissions-limit [@1:constrained-measure]
   (do ;; the measure goes here
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000)

Here we are calling the placeholder @constrained-measure inside the template, as a hint to the reader, but when invoking the template we still need only write (enforce-biomass-emissions-limit X).

18.2.3 Providing defaults for placeholders

Sometimes you may want to have placeholders which only need filling in some of the time. This can happen if you have a lot of 'typical' values of which only some will change, or if you want to add flexibility to a template without breaking it in the places where it is already used.

In our example, we can imagine wanting to change the type of fuel or the limit value we are filling in; say we wanted to make the limit controllable, we might change the template to be:

(template enforce-biomass-emissions-limit [@1:constrained-measure @limit]
   (do ;; the measure goes here
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) @limit)

Now when we use the template we can write (enforce-biomass-emissions-limit X limit: 10000000) if the limit should be different in a certain case. The down-side of this change is that now we must supply a limit: argument, as otherwise there is nothing to substitute there in the body.

To get around this we can give a default value for @limit; the syntax for this is (perhaps confusingly) to write the placeholder and its default value together in square brackets, thus:

(template enforce-biomass-emissions-limit [@1:constrained-measure [@limit 9000000]]
   (do ;; the measure goes here
      ;; now for the postcondition
          ;; so we give up, unless either:
              ;; the emissions for this house have not gone up
              (>= 0 (fall-in (house.emissions by-fuel:biomasswood)))
              ;; or if they have gone up, the total is still fine
              (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) @limit)

In this amended version, we are allowed to provide a limit: argument, but if we do not it will be as though we had written limit: 9000000.

18.3 Other preprocessor commands (macros)

Sometimes you might want a template to do something a bit more complicated than just producing some pro-forma. There are a small number of other model commands which instruct the pre-processor to do edits to your scenario. All of these commands start with a tilde (~) character, and are documented in the manual section on standard macros. When you are reading a scenario, remember that any command named with a tilde is a command to manipulate the scenario code inside it, not a command pertinent to houses, dates and so on.

Here are some of the macros you are most likely to encounter:


The join macro concatenates bits of text. For example, if you wrote

(~join this - that - the - other)

when the join is expanded it will be replaced with this-that-the-other.

This is useful for writing templates that need to do things like setting a flag with a particular name, or defining a set of variables which should all have related names, because you can use a template placeholder inside the ~join. For example,

(def (~join installation-count-of- @measure-name s))

in a template would expand to produce something like (def installation-count-of-boilers) if @measure-name were to contain boiler.


The maybe macro is for passing along named arguments that are optional, without supplying a default in the template where you are writing them. For example, imagine you want to define a template which installs a new boiler, and you have some optional arguments that you want to pass on to measure.standard-boiler:

(template my-boiler [[@capex] [@opex]]
    (measure.standard-boiler winter-efficiency:89% summer-efficiency: 85%
       (~maybe capex: @capex)
       (~maybe opex: @opex)))

The macro accepts either one or two arguments; if the second argument is supplied, it just expands to both the arguments without modifying them. If the second argument is missing, the maybe macro expands to nothing at all.

The effect of this in the example above, is that if a user writes:


then they get:

(measure.standard-boiler winter-efficiency:89% summer-efficiency: 85%)

whereas if they write:

(my-boiler capex: 1000)

then they get:

(measure.standard-boiler winter-efficiency:89% summer-efficiency: 85%
    capex: 1000)

The ~match macro is a bit like an if statement. It will look one bit of scenario code that you give it, and compare it to several other options, each of which can have some associated "output". If there is an output that is the same code as the input, it will expand to the output.

The thing to compare is the first argument to ~match, and then all the other arguments should be lists. The first thing in each list checked against the thing to compare, and if it's the same the macro outputs the rest of the list.

For example

(~match large
   [small 100]
   [medium 200]
   [large 300])

Will expand to the single output 200, because large is the first item in the third list.

Note that something like this:

(~match (house.built-form)
   [Detached 1]
   [EndTerrace 2]

will not work as you might guess it would on reading. This is because ~match is a macro rather than a normal command, and so it operates on the scenario code itself, not on houses. Macro output is determined before the model runs, so it can never depend on the particulars of houses or things going on in the simulation.

In this case, the match command will compare the literal text (house.built-form) with the literal text Detached, EndTerrace, and so on. The text (house.built-form) does not equal any of these values (in fact, it only equals (house.built-form)!), and so the macro will output nothing as there is no match.

The normal model commands you can use for this kind of thing are things like function.case, action.case, lookup (which can be programmed with ~lookup-table, of which more later).

The ~match command is mainly useful if you are writing a template and you want template users to be able to supply categories of some kind as arguments, and to output different code for different categories, or if you want to produce a useful error message if a user provides an argument to a template which you don't know how to deal with:

(template boiler [@fuel-type]
   ;; check the @fuel-type is OK:
   (~match @fuel-type
        [MainsGas] [Electricity] [Biomass]
        default: (~error No assumptions available in the boiler template for @fuel-type.))
   (measure.standard-boiler fuel: @fuel-type
     capex: (capex-for-fuel-type @fuel-type) ...))

The combinations macro is a bit like a loop, if you have used other programming languages. It is a bit complicated to get to grips with initially, but there isn't that much to it in the end.

What it does, is take several lists of commands, and go through each possible combination made by taking exactly one command from each list, putting each combination inside another command that you give it.

To start with, here is an example where there is one list of commands to combine, and the command to put each combination inside is do.

  template: do

template: gives the outer command, and the list in brackets gives a list of two things to pick between.

This will output the following scenario code when it is expanded:

(do (measure.wall-insulation))
(do (measure.loft-insulation))

This is because there is one list to choose from, so the only combinations to go through are given by each member of that list.

Now here is a more complicated example:

   template: do


This will produce four combinations, which are what you get by choosing one thing from each list:

(do (measure.wall-insulation)
(do (measure.wall-insulation)
(do (measure.loft-insulation)
(do (measure.loft-insulation)

You can have as many lists as you would like, but remember that the number of combinations produced is the product of the length of each list, so 4 lists each having 6 things in gives you \(6^4 = 1296\) possible combinations, 10 lists of two things gives \(2^10 = 1024\) and so on. If a single combination is expensive to simulate or validate, all of the combinations will be exponentially moreso.

The template: argument can be the name of any model command, or any other template you have defined, so you can also use ~combinations to do things like defining a list of variables:

(do template:def [one two three four])


(def one)
(def two)
(def three)
(def four)

and so on.


Lookup table is a wrapper around the model's lookup command, so we should probably explain the lookup command first.

A lookup is a number-valued command (a function) that lets you conveniently write complicated conditional rules that depend on several other values. You can use a lookup wherever you would write a number, so as the capex: for a measure, or as the sample percentage for a sample and so on.

Here is a spurious example (it is intentionally spurious, to make clear that the values of entries can be anything you like - a lookup entry could contain any function):

  default: 1
  keys: [(house.built-form) (house.total-floor-area)]
  (entry [Detached     1000] (* 10 (house.total-floor-area)))
  (entry [SemiDetached 2000] (house.energy-use)))
  (entry [* *] 999))

When this lookup is computed for a house, the command does the following:

  1. It works out the values of the commands in the keys: argument
  2. It goes through each entry in the lookup in order, and compares the keys: values to the values in the list at the start of the entry, which we will call the matching rules.

    If an entry's matching rules all match their corresponding keys: value, then the lookup computes the second part of the entry, and produces that value.

  3. If no entry matched, then the lookup produces the value of the default: argument as a fall-back.

Each of the rules in an entry can be either a specific value, like a certain number or a categorical constant, or true or false for a boolean function in the key, or it can be a wildcard, written as a star * which will allow any value, or for numbers it can be a rule that defines a range of numbers like:

  • 13..15 which will match anything from 13 to 15
  • <10 which will match anything below 10
  • >10 which will match anything above 10

and equivalents using greater-than-equals and so on to include the end-point.

The ~lookup-table macro is a more convenient way to write these lookups. It lets you write one dimension of the lookup across the top of a table as column headings, which can make the rule easier to read.

For example a lookup which gives a time-series of values for each region of the country could be written as:

  row-keys: house.region
  column-key: sim.year

  [r         2016 2017 2018 2019 >2019]
  [London    1    2    3    4    5    ]
  [SouthWest 6    7    8    9    10   ]
  ;; and so on

This is just the same as a plain lookup as follows:

(lookup keys:[house.region sim.year]
   (entry [London 2016] 1)
   (entry [London 2017] 2)
   (entry [London 2018] 3)
   (entry [London 2019] 4)
   (entry [London >2019] 5)

   (entry [SouthWest 2016] 6)
   (entry [SouthWest 2017] 7)
   (entry [SouthWest 2018] 8)
   (entry [SouthWest 2019] 9)
   (entry [SouthWest >2019] 10))

18.4 When are templates and macros processed

An important point to understand about templates is that they are a pre-processing step applied to your scenario. When the model is asked to run a scenario, it does the following steps:

  1. Process all the include commands (of which more later), to produce one big scenario
  2. Transform all the templates within modules (of which more later), removing any modules
  3. Cut out all of the templates in the scenario and remember their definitions
  4. Find all invocations of templates or built-in macros in what's left, and expand them until there are no uses of templates or macros left
  5. Validate the resulting scenario
  6. If valid, actually run the resulting scenario:
    1. load the stock
    2. set up the event queue
    3. do events until there are none left or the end date is reached

This means that the behaviour of a template (or any other macro command) cannot depend on anything about a particular house; by the time the scenario comes to be executed, all such commands have been fully expanded into basic model commands.

18.5 Defining several templates with modules

Often you will be using templates, flags, and variables all together to add some behaviour to the model in a way that is (hopefully) readable and easy to use. When you so this, you will probably find that you write several templates which are all related to a particular bit of behaviour. To show that some templates are all related in some way, you could use a convention when naming them to show what bit of work they belong to. For example, if you were writing some templates to model supply chains for measures, you can imagine that you would have at least two templates:

  1. A template a bit like the emissions limit we saw above, which would prevent installation of a measure if the supply was insufficient
  2. Perhaps a template to 'refill' the supply chain
  3. Maybe a template to use when reporting on the contents of the supply chain

If you named these all things like supply-chain/limit-measure, supply-chain/refill, supply-chain/count-remaining and so on, and put them in a file called supply-chain.nhm, it would make it easy for a reader to keep track of. Remember that in NHM syntax a word can have most punctuation in it, so the slash here is no more special than the hyphen or any of the other letters; it's just part of the name.

Since this is a very common thing to want to do, there is a special preprocessor command called ~module, which edits the definitions of many templates so that they all have a common prefix on their name like this.

Imagine our example above; without knowing about ~module we might have written something like this:

(template supply-chain/limit-measure [...] ...)

(template supply-chain/refill [...] ...)

(template supply-chain/count-remaining [...] ...)

Instead of this, you can write:

(~module supply-chain
    (template limit-measure [...] ...)
    (template refill [...] ...)
    (template count-remaining [...] ...)

When the model reads this, it will transform all the templates' names to be prefixed with supply-chain/.

Often when you produce a reusable component like this, you will also have some definitions of variables or similar, which should be added to the scenario command's contents.

The convention for this is to include a template called init in your module which expects no arguments like so:

(~module supply-chain
    (template init[]
        (def remaining-boilers on:simulation default: 0)

    (template limit-measure[] ...)

If you follow this convention, you can use the special macro ~init-modules, which expands out to the contents of every template called init in each ~module that has been seen by the model.

Notice in the example we have written that we defined a variable remaining-boilers; ideally we would want to prefix this variable with the name of the module as well, so that it does not accidentally coincide with anyone else's variable definitions and so that it's clear where the variable came from (just as with the templates).

The ~module command helps with this as well, by modifying any word written in it which starts with a slash to have the module name as a prefix, so the above should instead be written

(~module supply-chain
    (template init[]
        (def /remaining-boilers on:simulation default: 0)

    (template limit-measure[] ...)

This would be read by the model as

(template supply-chain/init []
    (def supply-chain/remaining-boilers on:simulation default:0))

(template supply-chain/limit-measure [] ...)

This rule also applies to the common cases of words starting with an octothorpe and a slash, as these are often cross-references, and words starting with an exclamation mark and a slash, which are often used with names for flags.

For example you might write

(~module supply-chain
    (template init []
              (def /remaining on:simulation default:0))

    (template reset [@1:new-amount]
              (set #/remaining @new-amount))

    (template remaining-amount []

Which is analogous to writing

(template supply-chain/init []
          (def supply-chain/remaining on:simulation default:0))

(template supply-chain/reset [@1:new-amount]
          (set #supply-chain/remaining @new-amount))

(template remaining-amount []

18.6 include commands

Models of any complexity usually need splitting in to parts, because (a) they are hard to keep in mind as a single large file and (b) because some useful parts may be reusable in other work. The model provides the two commands include and include-modules to help with this.

The include command is older, and a little simpler, so we will explain that first. It is very simple - when the model is reading your scenario file, if it sees the include command, it uses the command's arguments to go and find another scenario file. This other file is the target of the include, and the file referred to is determined differently in the web-based NHM and the stand-alone application. The contents of that scenario file replace the use of the include command, so it is as though you had deleted the include command and copied and pasted its target into the place where it was. This process is recursive, so in the following situation:

    (include A))

and A referred to a scenario file containing

(on.dates (scenario-start)
          (include B)
          (include B))

and B referred to a scenario file containing

(apply (action.do-nothing))

The code read by the model would be

    (on.dates (scenario-start)
              (apply (action.do-nothing))
              (apply (action.do-nothing))))

Whilst include provides what you need to split your scenario up into parts, it is usually best if you are making a reusable part to create a module with the ~module command which we have already encountered. If you do this, you can use the include-modules command to include only the modules defined in a particular scenario. include-modules works exactly as include, except for two important differences:

  1. Only modules are considered; at the moment this means:
    • uses of the ~module command are kept, and
    • uses of the include-modules command are processed recursively
  2. Because only modules are included, cycles between modules are allowed. The code for module A can use include-modules to see the code for module B and vice-versa. This is not true when using include, in which case cycles of inclusion produce an error.

The section on best practice includes suggestions about how to organise code into modules, and how to use include-modules to include dependencies.

18.6.1 Includes in the web application

The NHM web application will be replaced by the desktop version for most purposes. However, if you are wanting to read or write scenarios in the web application you will need to know how include commands find their target.

All scenario files in the web application live in a database, and behind the scenes every save of a particular scenario is identified by a unique identifying number. This identifier is hard to remember and useless for reading purposes, so to include a save of a scenario you refer to it by the scenario's name and a version number. Each combination of name and number is guaranteed to refer to a specific save. When you want to include a save, you press the xi button by the name to grab a version number for it.

To include that save, you can then write (include name: <<the name>> version: <<the number>>). If you like, you may also leave out the name: and version: keywords, so

(include name: my-measures version: 5)

is equivalent to

(include my-measures 5)

18.6.2 Includes in the desktop application

In the desktop application, there is no list of all the scenarios in the world, and there are no version numbers. Instead there are only the files which the application knows about, which you can see in the project explorer view that is typically on the left.

To include one file from another, you simply give the include command the location for the file you are including. You can either provide the location of the included file relative to the including file, or you can provide an absolute location.

An absolute location always starts with a slash (/), and gives the hierarchy of folders containing the file separated by slashes. For example, if including a file called measures.nhm in a project called decc-files, you can refer to it as

(include /decc-files/measures.nhm)

A relative location is determined starting from the location of the including file. For example, if the aforementioned /decc-files/measures.nhm were to contain

(include other/capital-costs.nhm)

This would be referring to a file /decc-files/other/capital-costs.nhm; this is worked out by

  1. Taking off the last part of the location of the including file, to get /decc-files/
  2. Adding on the relative location written in the include command.

The useful thing about relative locations is that you can move a collection of files around or rename the folders containing them, and not have to update the includes.

18.7 TODO Exercises about templates

19 "Optimisation"

This section is written in scare quotes, as the NHM is not really an optimisation system. If you are familiar with optimising models which work by producing a linear or mixed-integer form for a problem and then solving it, it's important to realise that the kind of optimisation these tools do is not really applicable in the NHM. This is because it's possible to set up very non-linear behaviours in the NHM, for which no general optimisation method can work. In this sense the model is better suited to simulating what would happen if I did X, Y and Z than it is for answering questions like "what is the best way to achieve some goal G given limited resources and the following notion of what best means".

However, the model does provide a few limited facilities for optimising its behaviour in some simple circumstances which may meet some requirements of this kind.

19.1 Using repeat for goal-seeking

The repeat command re-runs part of a scenario repeatedly until a certain criterion is met. The effect of any changes made in that part of the scenario are all undone except for changes to selected variables, so each iteration of the repeat is like a separate trial of what would happen, given certain parameters for those variables. In this sense it's analogous to the sort of goal seeking function you may have used in spreadsheet programs before.

repeat is used within on.dates, and can contain commands like aggregate and apply, so it can repeat the application of a single moment in simulated time, but cannot repeat the execution of things which take place on several dates.

To use repeat you need to think of the command you are going to perform, and how it will be controlled by the variable that you are going to change. For example, imagine you wish to model the installation of measures with a subsidy, where:

  1. Only measures which break even will be installed
  2. The subsidy will be per kg of emissions avoided
  3. The amount of subsidy available is unlimited
  4. There is a target amount of emissions to avoid overall
  5. The objective is then to simulate the installation of measures at an "ideal" subsidy level. This ideal level is one where the minimum subsidy which achieves the target has been chosen. Note that this is not the same as giving each house the minimum subsidy that house requires to achieve the target, but instead describes a situation where only one subsidy level can be offered and it is the response of the houses which require the largest subsidy to break even which sets the subsidy level for everyone. This is why the repeat command is needed - we can only know whether a subsidy level does the job after we have made the offer, so we have to try the offer repeatedly with increasing amounts of subsidy until a suitable level is reached.

The repeat recipe for this has the following parts:

  1. The termination rule; a repeat command needs to know when to stop repeating and move on with the simulation
  2. The command to perform repeatedly
  3. The list of variables to preserve between repetitions - apart from changes to these variables, all the changes the repeat makes will be undone until the termination rule is met.
  4. The change we want to make to these variables - if we do not make some change then the repeat will not cause anything to change, and the model will run forever18.

In our example, these parts are:

  1. We want to stop when the total emissions reduction meets our goal:

    (> (summarize (aggregate.sum (fall-in house.emissions))) 10000)
  2. We want to install some measures with a subsidy per tonne of emissions avoided:

    (def subsidy-per-tonne on:simulation default:0)
    (on.dates regularly
         (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50
                                                            capex:(* 10 size.m2))
                           subsidy: (* #subsidy-per-tonne (fall-in house.emissions)))
             (fail-unless (<= net-cost 0)))))

    We have also made sure that the measure installation will fail unless the net cost after the subsidy is given is not positive.

  3. We can see that the code above will install a number of measures which depends on the value of #subsidy-per-tonne. A larger value will cause more installations, as the increasing subsidy will come to dominate the installation cost even in houses where the emissions reduction is not so large. Note however that this effect has an upper limit, as for some large subsidy level the capital cost will have been exceeded by the subsidy for all houses in which there is any reduction in emissions; at this point any further increase in subsidy level cannot cause more installations or reduce emissions any further as all the potential has been realized.

    Anyhow, this shows us the variable that we need to keep between steps; we are only interested in preserving the value of #subsidy-per-tonne, as this is what controls the rest of the behaviour.

    We can hence introduce our repeat into the on.dates command:

    (def subsidy-per-tonne on:simulation default:0)
    (on.dates regularly
       until: (> (summarize (aggregate.sum (fall-in house.emissions))) 10000)
       preserving: #subsidy-per-tonne
         (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50
                                                            capex:(* 10 size.m2))
                           subsidy: (* #subsidy-per-tonne (fall-in house.emissions)))
             (fail-unless (<= net-cost 0))))))
  4. Our code above is nearly finished but the astute reader may have noticed that there is no instruction given to change the value of the controlling variable, so the model will do this:

    • Arrive at the repeat; a hypothesis is produced in which to try out the change
    • Install the measure with subsidy level zero; this will fail in all cases, because the capex is always positive and the subsidy always comes out zero. Hence the net cost is always positive.
    • Undo the hypothesis, because the until condition is not met; no measures have been installed, so there is no reduction in emissons.
    • Try again at the first step

    To avoid this problem, we need to do something to increase the subsidy level:

    (def subsidy-per-tonne on:simulation default:0)
    (on.dates regularly
       until: (> (summarize (aggregate.sum (fall-in house.emissions))) 10000)
       preserving: #subsidy-per-tonne
         (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50
                                                            capex:(* 10 size.m2))
                           subsidy: (* #subsidy-per-tonne (fall-in house.emissions)))
             (fail-unless (<= net-cost 0))))
         (set #subsidy-per-tonne (+ 1 #subsidy-per-tonne))))

    Now the model will do something like this:

    • Arrive at the repeat, producing a hypothesis in which to try out the first subsidy level of zero pounds.
    • Perform the apply command with zero pounds resulting in no emissions reduction
    • Increase the #subsidy-per-tonne variable to take the value 1
    • Check the failure condition for the repeat; the test has failed and so the model will revert any changes that are made, except for those to so with the preserved variables
    • Now the model tries again, at the same starting point except that the subsidy level is higher. The same houses are offered the measure, except now some of them may break-even and keep the measure as a result.
    • The subsidy level is increased again to 2
    • Now the condition is checked again; let's say that 1 is not a sufficient subsidy level and so the changes are rolled back again.
    • The model tries again, offering the same houses the same measures except with a unit subsidy of 2. This time more houses will take up the subsidy. Let's say that 2 is a sufficient subsidy level. When the test is calculated this time, the repeat will exit and the model will continue.
    • The result is as though you had left out the repeat, and used some perfect foreknowledge of the outcome to select 2 as the correct subsidy level in the first place.

19.1.1 Avoiding infinite repetition

The model will detect situations where the controlling variables for a repeat statement are not changing. In these situations, because the model is deterministic an infinite loop is guaranteed, and the simulation will stop with an error.

It will also terminate with an error after a certain number of repetitions have been tried, as a safety measure.

However, if the commands you are repeating take a long time to run you may want to stop the loop early yourself. You can do this by defining your own variable which you use to count the number of goes around the loop and including it in the command to try, the list of preserved variables, and the termination rule:

(def subsidy-per-tonne on:simulation default:0)
(def iterations on:simulation default:0)
(on.dates regularly
  until: (or (> (summarize (aggregate.sum (fall-in house.emissions))) 10000)
             (> #iterations 10)) ; only ten tries
   preserving: [#subsidy-per-tonne  #iterations]
     (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50
                                                        capex:(* 10 size.m2))
                       subsidy: (* #subsidy-per-tonne (fall-in house.emissions)))
         (fail-unless (<= net-cost 0))))
   (set #iterations (+ 1 #iterations))     
   (set #subsidy-per-tonne (+ 1 #subsidy-per-tonne))))

19.1.2 Reporting from a repeat

As with other hypothesis-using commands, the model will undo any changes which are made to houses when a hypothesis is undone but it will not undo any reporting commands. Hence, if the model makes reporting output within a repeat command that output will appear even for the iterations of the repeat which were run to try out situations that were eventually discarded.

This is potentially useful, as it allows you to see the intermediate stages of the repeat (for example you could keep track of the number of goes round the loop as mentioned above, and output that into a report along with other quantities of interest to see how they respond to the change in the controlling factor).

It is also potentially confusing, as the report will contain descriptions of situations which did not end up actually happening in the model's canonical view of things. This can always be avoided just by moving any reporting statments so that they are outside the repeat command following it, in which case you will know that the repeat has finished and that you are seeing its final consequences.

19.2 Using in-order for delivery of measures according to a MAC

Another simple form of optimisation which the model can do is to offer measures to a population of houses in order of a user-defined function of how the house will be after taking a particular measure. This is a bit like the choice command which we have already seen, when used with a select.minimum or select.maximum rule; however, choice is an action which operates on a single house, so it will choose the best option for that house alone. When used with apply, the choice will still be presented with houses in a random order. As a result, in scenarios where there are a limited number of measures available, a choice may be configured to produce the best cost-benefit for each house in turn but this will not achieve the best cost-benefit overall because the random order from apply will choose a random selection of houses to have first dibs on the pool of measures or subsidy, and the houses in the random selection will probably not be those with the best cost-benefit in the population as a whole.

To achieve this limited form of global optimisation, the in-order command is provided. It can be used within an apply command, and works by using a hypothesis to work out the impact of installing each measure from a list of alternatives independently in each house from the targeted population.

The order implied by the hypothetical impact calculations is then used to sort the possible measure-house pairings, which are then performed non-hypothetically until they are exhausted.

To make this more concrete, here is an example:

We have a million pounds to spend on insulation; however, the cost to install insulation everywhere that could take it exceeds a million. It follows that we must allocate insulation in some way; we can model the limited supply by adding extra suitability rules to the insulation measure which make it fail if there is not enough money left.

(def budget on:simulation default: 1000000)
(def-action limited-insulation
  (do (measure.wall-insulation type:cavity thickness:50 u-value:0.5
                               capex: (+ 100 (* 10 size.m2)))
      (consume #budget (* net-cost house.weight))))

Using this example measure, running (apply #limited-insulation) is like allocating the insulation by lottery.

We might instead hope for a more technocratic solution implemented using in-order to select those houses which will achieve the greatest emissions reduction per pound spent.


      ;; the fall in emissions is the value we want to work out for each house
      ;; this is a value that can only be calculated by using a hypothesis as
      ;; it depends on what the measure does to a particular house
      ;; we divide by the cost, as we want cost effectiveness
      objective: (/ (fall-in house.emissions) net-cost)
      ;; we want to sort by decreasing order of fall in emissions, which is to say
      ;; we want to do the most cost effective possibility first, then the next and so on.
      ascending: false

In this example, we are offering a single measure to all of the houses; the model will proceed as follows:

  1. For each house, the model will create a hypothetical environment in which that house gets the measure (if the house is suitable for that measure) Because these hypotheses are independent they are each starting with the full budget left in the #budget variable, so no house is unsuitable because the budget ran out (unless it is a very large house, in which case we cannot afford it anyway).
  2. In that hypothesis, the model will work out the objective; this will be the fall in emissions due to the measure, divided by the cost of its installation.
  3. This produces a table (effectively) containing a row for each house and a single column for the insulation measure, each cell containing the cost-effectiveness of doing that measure for that house.
  4. Finally we sort the table in descending order, and then go through it doing the measures. Since this time when we do the measures they are not in independent hypotheses, doing the first measure will use up a bit of the budget, the next a bit more and so on until the budget starts to get tight. At this point some of the rows in the table will become infeasible, if they do not fit in the budget. The model will keep trying to do things in the specified order, but some things which were "marginally" suitable will have become unsuitable now that the money has run out.

If you give the in-order command more measures to consider, it will build a table which contains the objective value for every combination of houses and measures, and then run through that cell-by-cell in the desired order.

19.3 Optimisation exercises

  1. When using in-order, what is the difference between sorting by house.emissions in ascending order and sorting by (fall-in house.emissions) in descending order? What order will each of these two options tend to rank things in (i.e. what kind of houses will be at the top or bottom of the list)?
  2. When using repeat, what potential mistakes or hazards should you look out for?
  3. Write a scenario in which you use repeat to determine what resistance cavity wall insulation would be required to create a 5% reduction in carbon emissions if it were applied to all suitable houses at a thickness of 50mm
  4. Write a scenario in which you use in-order to find the maximum reduction in fuel bills manageable by installing 100,000 new boilers of a good efficiency.
  5. Think about how you could answer the above two questions without using the NHM's optimisation features; both of these problems can be solved simply with a probe report and a spreadsheet. Consider what this says about the situations in which you should use the facilities in the model, and when you should use tools outside the model.

19.3.1 Answers

  1. Ranking houses by raw emissions in ascending order will prioritise putting measures into houses which have low emissions after the measure, whether or not the measure is the cause. Houses which have low emissions after the measure probably also have low emissions before the measure, most likely just because they are small houses or are already quite efficient.

    Conversely, ranking houses by decreasing order of the reduction in emissions will prioritise putting measures into houses which most reduce their emissions as a result of the measure going in. A house with high emissions will be offered a measure before one with low emissions if the reduction due to putting in that measure is greater. In many cases this will effectively be the reverse order, because larger houses will normally have larger savings if a measure typically has a certain percentage effect.

  2. A repeat may trip you up in a number of ways:

    • Overshooting a target: if you are using repeat to search a range of values for one which produces some outcome, if the value starts too high then the repeat may finish successfully, but not optimally.
    • Hitting an upper limit: if the command being repeated simply cannot produce an outcome that allows the repeat to terminate then your scenario may run for a long time and eventually terminate with an error.
    • Failing to reset the controlling variables in a new year: if you are using a repeat to meet a goal in each year in a series of years then you need to remember to reset the repeat's controlling variables each time before it happens. Otherwise year 2 will start searching at the subsidy level for year 1, which you may not want.

    The way to overcome all of these potential problems is to investigate the response to the controlling variables manually before running your scenario in anger. You can do this either by writing your own probe and analysing it in a spreadsheet, or by producing a small scenario in which all that is happening is a repeat that scans through a sensible range of values and produces a report from within the repeat - this will let you plot the response of the outputs you are interested in to the variables that are being changed.

  3. This is an example of a question for which the model needs the repeat command, because we need to keep upping the resistance value for all the houses at once, so we have to roll back the changes we make to the whole stock until we find the desired outcome:

    (scenario start-date: 2016 end-date: 2016
              stock-id: a-stock
       (def resistance on:scenario default:0)
       (on.dates scenario-start
          (aggregate name:before
                     (aggregate.sum house.emissions))
          (repeat until: (>= 5%
                             (/ (summarize (aggregate.sum (fall-in house.emissions)))
                                (summarize (aggregate.sum (original house.emissions)))))
                  preserving: #resistance
                  (set #resistance (+ #resistance 0.01))
                  (apply (measure.wall-insulation thickness:50 resistance:#resistance
          (aggregate name:results
                     (aggregate.sum house.emissions)
                     (aggregate.mean #resistance))))
  4. This is a problem which is solvable either using in-order, as we want to work out how good a bunch of options for putting measures in houses would be, and then do the best until the money runs out.

    (scenario ...
      (def budget on:simulation default:0)
      (def-action limited-boiler
         (do (measure.standard-boiler
                capex:(+ 100 (* size.kw 13))
             (consume #budget (* house.weight net-cost))))
      (def-report effect-report
        (column name:cost value:capital-cost))
      (on.dates scenario-start
             report: effect-report
             objective: (/ net-cost (fall-in house.fuel-cost))
  5. Question 3 can be answered using a series of probe reports, each installing a measure with a different resistance. If each probe is inside a do and followed by an action.fail then the probe reports will be independent as no measures will really get installed. Such reports would be easy to analyse in a spreadsheet to determine the measure with the appropriate net effect. A similar but simpler solution is possible using def-report, as this can produce the total impact grouped by each variant of the measure.

    Question 4 can be answered by using a single probe report to output the cost effectiveness of doing the measure to the whole stock. Sorting this report by cost effectiveness will essentially reproduce what the model is doing internally.

    The important point here is that a lot of simple exploratory work like this is easier outside the model, and building up an understanding of your scenario by investigating in this way is often a good idea. The on-model features are intended for situations in which they are required, such as in a simulation where you wish to deliver an ideal order of measures each year for several years. In this case the houses affected in the first year need to be included in the baseline for the second, and so on, and so the choices of which houses end up with which measures need to be known on-model.

20 TODO Scenario assumptions, defaults, and SAP

An important question for your models will be what assumptions are being used, for which parts of the scenario.

Assumptions in the model cover several different areas, and most different kinds of assumption can be changed independently of one another. This flexibility can cause difficulties, making it hard to reason about what is happening in one bit of the scenario unless you have a good understanding of the whole thing.

To mitigate this risk, we recommend not using code that you don't understand, and trying hard to avoid having to use potentially conflicting or divergent assumptions within the same piece of work.

Rather than having a single block or some blocks of assumptions which are always present in the model, assumptions are instead present at various points, and at each point may change some or all of the state of the model to reflect a certain overall combination. The main moments when assumptions are imposed are:

  1. Built into the model are assumptions which dictate the details of things like the energy calculation.
  2. When a stock is created, the stock creation process itself represents a large assumption about how to interpret the housing survey, and it also applies some specific numerical assumptions about the physical structure of a house, like u- and k- values for walls and windows, etc.

    When the model loads a stock, the initial condition for all these aspects of the houses is determined by these assumptions.

  3. Once a house is taken from the stock, some other information is added to it by the model which describes those things about the house that are not stored in the stock. These are things like:

    • The weather conditions to use in the energy calculation
    • The heating behaviour for the house; its heating schedule and temperatures
    • The tariffs for the house; how its fuel is priced
    • The carbon emissions factors used in calculating emissions

    All of these quantities have default values which are described below; the defaults can be overridden by setting up appropriate rules in your scenario to change them

  4. As the model runs a house may be changed by things that the simulation has been instructed to do; for example, if you install an insulation measure in a house, the house's u-values for the relevant part will be changed in the way the measure says to change them.

    Thus a measure can take a house whose u-values are given by one set of assumptions from the stock, and change it in a way that no longer aligns with those assumptions. This change does however still reflect a particular assumption, which is whatever the scenario author wrote about the measure; for example they may have assumed that their insulation has a certain resistance. Adding a layer of such insulation may nevertheless produce a u-value which was not in the original stock at all.

    As a scenario author, it is your job to try and make sure that the parts you assemble in your scenario have compatible assumptions in this way.

  5. Finally, in a scenario there may be hypothetical changes assembled around a particular calculation which further amend the state of affairs but only in a way which affects that calculation. This is what the under command is for; for example, as just suggested, you may have a scenario in which a measure has introduced a u-value which is sensible from the point of view of the evidence about that measure, but then you may want to do some calculation using a different u-value which is approved for a certain purpose.

    The under command will let you do this by using an action which changes the house's u-values directly, without interfering in the operation of the rest of the scenario.

20.1 Efficiency

Changing the assumptions in the model using under is flexible, but it is not efficient. This is because creating the hypothesis, making the desired changes, and then reevaluating the energy use and any other down-stream effects takes the model a certain amount of time, and the model is not clever enough to avoid repeating the work when many values are being computed using similar or identical hypotheses.

Because of this it is a good idea to try and avoid setting up lots of different combinations of assumptions in your scenarios; if you have one main set of evidence, try and use this evidence consistently and make sure that each of the steps described above

20.2 SAP assumptions

20.3 Default values

21 TODO In-use factors

21.1 TODO Why in-use factors cannot be properly integrated with a simulation model

21.2 TODO How to implement them anyway if you really have to

22 TODO Assembling the parts

This section contains a few examples about how you can model behaviours which require a few different NHM features to be used together. Each example may end up using several features, but you can return to them after reading up on the features being introduced as you go.

22.1 TODO A calculation of technical potential

22.2 TODO A simulation model

22.2.1 Installing some measures [the basics]

22.2.2 Tracking measures installed and their effects [flags, reports]

22.2.3 Moving definitions to one place [definitions]

22.2.4 Adding in some off-model suitability rules [flags, sampling rules]

22.2.5 Limiting the supply of measures [variables, action.case]

22.2.6 Modelling household behaviour [choices]

22.2.7 Manipulating behaviour with a subsidy

22.2.8 Tidying up repeated code [templates and modules, includes]

22.2.9 Optimising measure delivery [in-order]

22.2.10 Investigating sensitivity to key variables [batch]

22.2.11 Cost recovery on bills [tariffs, costs]

23 TODO Quality and Best Practice

23.1 TODO General principles

  1. Reading code is harder than writing it, and harder still is changing it once it is written. This is a fairly universal law, which is always worth thinking about when writing for both computer and human audiences.
  2. Writing down some code is quite easy; however writing down correct code is quite hard. A corollary of this is that if you have some code which does not work properly, you should not be afraid of starting from scratch (perhaps with your old code as a reference).

23.2 Embedding QA checks into your scenario with assert

There are several good reasons to make tests for your scenario (or for any computer program). Tests can be expressed for humans to run (for example, a QA checklist of some kind), or they can be expressed for computers to run. The NHM has a command to help with the latter called assert.

  1. It helps you think through what you actually intend the program to do: What is a correct outcome? What is an incorrect one? How do you tell the difference?
  2. It helps you be confident that the program does what you intended: This is especially useful if you change the program, as programs are like wallpaper - if you push a bubble down in one place it often pops up somewhere else.
  3. It helps communicate to other people what the program is supposed to do: To some degree, reading a test may explain what the expected way to use some code is, and how it should behave.

An assertion is a claim of fact; in the NHM, the assert command provides a way to make a check that a claim of fact is true in the simulation's opinion. If the simulation finds that the claim is not true after all, it will stop early and produce an error in the errors.txt file.

The assert command is a sibling of apply; you can use it inside on.dates, and when the simulator gets to the date it will check the assertion is upheld.

For example, to check the simulation's belief in the identity of integers, you could say


    (assert (= 1 1)))

If this code were included in your scenario, when the scenario started the model would evaluate the logical statement "is 1 equal to 1", and if that were not the case it would produce an error and the simulation would stop.

This is a didactic example which you are not likely to want to use, but you might well want to do something like check that a certain number of houses have a particular flag on them:

    (assert (> (summarize (aggregate.where (house.flags-match interesting-flag)))

This will check that at least 1 million houses have the flag interesting-flag. Assertions like this will be checked once; if you want to make an assertion about each house in a population you can give assert a set of houses with the over: argument. Then the test will be checked for each house, and if any house fails the simulation will stop with an error.

For example if you wanted to check that all the flats in your stock had less than 10,000 square metres of floor area as a pre-condition on running your scenario, you could say

(on.dates (scenario-start)
   (assert over: (filter (any
                            (house.built-form-is ConvertedFlat)
                            (house.built-form-is PurposeBuiltHighRiseFlat)
                            (house.built-form-is PurposeBuiltLowRiseFlat)))
      (< (house.total-floor-area) 10000)))

The later section on writing good modules shows how you can and should include assertions and test scenarios as part of any reusable bit of code.

23.3 Writing good modules

If you have been re-factoring your code regularly into small testable parts, and writing QA tests for those parts, you will probably find some modules naturally "fall out". See the section on templates and modules above if you do not know what a module is, or feel like you do know what it is but are not really sure how they work.

We would suggest following these conventions for all modules which you write:

  1. Ideally, no more than one module in a file
  2. Ideally, the file should be named for the module in it
  3. A module's file should include-modules all the modules it depends on
  4. A module's file should include documentation comments and a test scenario
  5. Everything which you want to use from a module should be enclosed in a template
  6. A module should have an init template which sets it up, as far as possible

For example, imagine designing a module which is intended to model the hard-to-treat status of wall insulation in houses.

First we should choose a name, for example htt-walls as we are modelling hard to treat walls To start with let's have a module with nothing but an empty init template:

(~module htt-walls
         (template init []))

Now before we dive in, let's imagine the things which we might want to present to the code which is going to be using our module. You could call this the module's interface; it is a set of templates that we expect other people or code to be using. It is worth thinking carefully about the interface, as changing it later will be more difficult (since other code will have made use of it). As far as hard-to-treat is concerned, we will want to provide a template which people can use to find out whether or not a house's insulation is hard to treat.

(~module htt-walls
         (template init []
            ;; at the moment, nothing doing here.

         (template is-htt []
            ;; this template produces a logical test
            ;; if the test is true, then the house will be
            ;; hard-to-treat when installing insulation

Now users of our module will perhaps employ the command (htt-walls/is-htt) to target hard-to-treat houses, or to adjust the costs of their measures or similar. Note that we have added a comment which tells a reader what the template is all about.

One way to model hard-to-treat insulation would be to add a flag to each house that has HTT status; if we decide to use that approach, we can fill in the is-htt template:

(~module htt-walls
         (template init []
            ;; at the moment, nothing doing here.

         (template is-htt []
            ;; this template produces a logical test
            ;; if the test is true, then the house will be
            ;; hard-to-treat when installing insulation
            (house.flags-match /hard-to-treat))

So now we are saying that if the house has a flag htt-walls/hard-to-treat, then it will be considered hard to treat. However, users of our module do not and should not need to care about this. It is an implementation detail; we could instead change this to use house.static-property-is to look at an external dataset, or to use some other technique. This underlines the importance of choosing an appropriate interface.

Continuing with our flag approach, imagine we have some evidence about hard-to-treat insulation; say that there should be a certain count of hard-to-treat properties having a certain built form in each region of the UK. We want to set up the important flags as part of our init template, so that later when people use the is-htt template they get what they are expecting.

We might write something like this:

(~module htt-walls
  (template init []
     ;; sets up the wall insulation flags
     (apply to:(sample 1234 (filter (all (house.region-is southwest)
                                         (house.any-wall has-construction:any-cavity
                                                         has-cavity-insulation: false))))
        (house.flag /hard-to-treat))

     (apply to:(sample 4567 (filter (all (house.region-is london)
                                         (house.any-wall has-construction:any-cavity
                                                         has-cavity-insulation: false))))
        (house.flag /hard-to-treat))
     ;; ... and so on ...

  (template is-htt []
     ;; this template produces a logical test
     ;; if the test is true, then the house will be
     ;; hard-to-treat when installing insulation
     (house.flags-match /hard-to-treat))

So now a user of our module need only follow the convention that they (include-modules htt-walls.nhm), (~init-modules) in their scenario, and then they can use is-htt with wild abandon.

Looking at the above, you might think "there's a lot of repetition in the application of the flags", and you would be right. In this case we should probably add another template, to avoid copy-paste errors; however, this template does not want to be part of the module's interface, as it is to do with the details of how the module works, and other code outside the module should probably not depend on any of those details. The language has no built-in way to distinguish between interface and non-interface templates, but a simple convention is to use some punctuation at the start of the template's name, for example a hyphen or an underscore. Using an underscore as the convention for non-interface content, we might write:

(~module htt-walls
  (template init []
     ;; sets up the wall insulation flags
     (_setup_flags count: 1234
                   con: anycavity
     (_setup_flags count: 4567
                   con: anycavity
     ;; ... and so on ...

  (template _setup-flags [@count @region @con [@cav] [@ext] [@int]]
       ;; used internally to set the flags on houses
       ;; we will sample @count houses from @region for which
       ;; construction is @con (per house.any-wall has-construction:...)
       ;; and for which the given kinds of insulation are around
        to: (sample @count
                    (filter (house.region-is @region)
                            (house.any-wall has-construction: @con
                                            (~maybe has-internal-insulation: @int)
                                            (~maybe has-external-insulation: @ext)
                                            (~maybe has-cavity-insulation: @cav))))
         (action.flag /hard-to-treat)))

  (template is-htt []
     ;; this template produces a logical test
     ;; if the test is true, then the house will be
     ;; hard-to-treat when installing insulation
     (house.flags-match /hard-to-treat))

The technical details of this are not hugely important; the key thing being illustrated is that although we have rearranged the internals of our module, users of the interface do not need to care.

Having defined our module, before we go and integrate it into other potentially complex bits of work, we should test it independently of any other bits. The recommended way to do this is to add a scenario to the top of the module's file; this scenario will be stripped out when the file is included via include-modules, but can contain assertions and reports to help you test that just this module works properly.

In our case we might want to generate a report on the HTT status broken down by different regions, and maybe assert that the total count is what we would expect:

;; test scenario at the top:
;; probably a good place to write some documentation
;; explaining:
;;   - roughly how to use the module
;;     - ensure you have used ~init-modules
;;     - use the (htt-walls/is-htt) command to determine if a house is htt
;;   - the methodology behind it, links to sources of data, etc.
;;     - based on X Y and Z (limitations: A B and C)

    start-date: 2015
    end-date: 2015
    stock-id: test-stock

    (~init-modules) ;; this will init our module below

    (on.dates 2015
       ;; now report on the counts
       (aggregate name: htt-count
                  divide-by: [(house.region)

       ;; also use an assertion to check the result more directly
        (= 5801 (summarize (aggregate.where (htt-walls/is-htt)))))

;; module definition
(~module htt-walls
  (template init []
     ;; sets up the wall insulation flags
     (_setup_flags count: 1234
                   con: anycavity
     (_setup_flags count: 4567
                   con: anycavity
     ;; ... and so on ...

  (template _setup-flags [@count @region @con [@cav] [@ext] [@int]]
       ;; used internally to set the flags on houses
       ;; we will sample @count houses from @region for which
       ;; construction is @con (per house.any-wall has-construction:...)
       ;; and for which the given kinds of insulation are around
        to: (sample @count
                    (filter (house.region-is @region)
                            (house.any-wall has-construction: @con
                                            (~maybe has-internal-insulation: @int)
                                            (~maybe has-external-insulation: @ext)
                                            (~maybe has-cavity-insulation: @cav))))
         (action.flag /hard-to-treat)))

  (template is-htt []
     ;; this template produces a logical test
     ;; if the test is true, then the house will be
     ;; hard-to-treat when installing insulation
     (house.flags-match /hard-to-treat))

This is a reasonably complete example of a single module; it has the following properties:

  1. A reader, upon looking at it, can first see something about how to use it and how it works
  2. Along with some documentation, the reader is given a concrete example of some commands used in context, which they can run and fiddle with
  3. This concrete example shows some relevant outputs, and also will produce an error if certain invariants are not preserved
  4. The embedded QA logic gives the reader some idea of the degree of reliability they should expect
  5. The module itself is divided into interface and non-interface templates
  6. Each template has a little comment explaining something about it

When writing a module which depends in turn on other modules, you should include-modules those dependencies at the top level in the module's file; for example, if our project included a module defining some insulation measures, which used the hard to treat module to control the capital cost of some measures, that might look a bit like this:

;; wall-insulation-measures.nhm - defines some wall insulation measures

(include-modules htt-walls.nhm) ;; capex depends on whether a wall is htt
                                ;; so we use the htt module here

;; demo scenario - should have some tests really
(scenario stock-id:some-stock
   (on.dates scenario-start
      ;; check that when we put insulation in a house which is HTT
      ;; the capex is 1000
        over: (all (filter (htt-walls/is-htt))
                   (house.is-suitable-for (wall-insulation-measures/cavity)))
        (= 1000 (under (wall-insulation-measures/cavity) evaluate:(capital-cost))))

(~module wall-insulation-measures
         (template init []
                   (def-action /cavity-insulation-action
                         capex: (function.case
                                    (when (htt-walls/is-htt)
                                    default: 500)
                         resistance: 0.01

         (template cavity [] #/cavity-insulation-action))

Because it transitively includes htt-walls.nhm, another scenario can just include-modules wall-insulation-measures.nhm and expect it to work correctly, without having to know that wall-insulation-measures currently uses htt-walls.

23.4 TODO My scenario doesn't work

23.4.1 What to do if your scenario is doing something unexpected; warnings and errors

23.4.2 How to make scenarios that are likely to work

23.4.3 How to make scenarios that run quickly



Precisely, any subset from all the possible subsets, as the boiler failures are independent. The expectation would be for 100 dwellings to replace their boiler.


This example uses the default weighting behaviour of the model; there are other options described in the manual.


Modelling such a policy should raise a different question, about whether a model like the NHM can usefully represent changes to very small populations, given the intrinsic error which comes from using BREDEM and housing surveys.


This sampling method biases the mean down a bit when the quantum is large (because there are fewer samples which match exactly). In the NHM, the sampling method has a correction for this which removes the bias and reduces the variance in the mean, but it is harder to analyse.


The temperatures will eventually equalize at the mean of \(T_1\) and \(T_2\) weighted by the thermal masses involved, following an exponential decay as the \(H\) reduces. However, because the thermal mass of the whole outside world is much larger than that of the dwelling, we say that the outside world is not noticeably heated by the dwelling.


There are some other ways you can use the NHM: there is a command-line tool which can run and validate scenarios, which you can use with tools like emacs, and there is a web-based system for running it which may well have been deprecated by the time you read this. If you are in DECC you might call this the "Standalone" NHM.


If you are unfamiliar with menu bars and menus, you may wish to suspend learning the NHM application, and learn more about the basics of graphical interfaces to computers.


At this point you will have noticed that we are writing about terms that we have not defined. The actual terms in the language are explained later on.


Again, if you are not familiar with this term read ahead first.


Not in Microsoft Word though - Word is not a text editor. Do not do this; it will shred your work.


A line is a series of characters ending in the ASCII 'line feed' character (this is character 10 in the ASCII table), and a tab is character 9 in the ascii table. This is about the simplest way to store a table, and will be understood by almost any tool.


You could go to case level by saying divide-by: [house.survey-code], but this is not a good idea.


The model does not currently provide facilities for users to define or manipulate any data other than flags and numbers.


This design choice reflects the fact that different policies tend to have different rules for accounting, and that making the model intelligently determine precisely what things should be accounted for would be difficult and likely frustrating.


If you have seen the blockbuster film Inception, it is essentially like this but more exciting.


This is fairly reasonable, as the model cannot read its own reports back in so as to make decisions; writing to a report should not affect the sequence of events in the simulation. There is a minor exception here, which is that writing reports which use random numbers is allowed to change the random sequence, but reporting on random numbers seems like an odd thing to do.


Actually, the model will detect if nothing has changed at all between two steps of a repeat, and if this happens it will terminate and produce an error. However, you will still need to guard against situations where something changes but the termination condition is not met.

Author: CSE

Created: 2016-05-31 Tue 10:43