Learning the National Household Model
Table of Contents
- 1. Overview
- 2. Reading scenario language
- 3. The NHM desktop application
- 3.1. Downloading the application
- 3.2. Installing the application
- 3.3. Using the application
- 3.4. Editing scenarios efficiently
- 3.5. Navigating between files and definitions
- 3.6. Viewing files side-by-side
- 3.7. Viewing the top and bottom of a long file simultaneously
- 3.8. Finding other actions
- 3.9. Accessing the manual
- 3.10. Glossary
- 4. A simple scenario, by example
- 5. Making things happen
- 5.1. When do things happen; what dates, how ties are broken
- 5.2. Making reports; making changes to the housing stock
- 5.3. Targetting who things happen to; selecting houses with tests or at random
- 5.4. Things which happen by themselves; carbon factors, weather, opex and bills
- 5.5. Exercises about scheduling events
- 5.6. Exercises about sampling houses
- 6. Getting information out
- 6.1. An overview of the reporting commands
- 6.2. Using
aggregate
to produce summaries of the stock - 6.3. Using
probe
andprobe.aggregate
to see what has happened - 6.4. Using
def-report
and thereport:
argument - 6.5. Other reports; what measures have gone into which houses; who spent what money
- 6.6. Exercises about reports
- 7. Insulation measures
- 8. TODO Heating measures
- 9. Flags
- 9.1. Marking houses as different with flags
- 9.2. Using flags to select a set of houses
- 9.3. Using flags to affect suitability, and making actions also do flags
- 9.4. Patterns for flags, and the negation shorthand
- 9.5. Using flags to change a number, like a cost
- 9.6. Reporting on what flags a house has
- 9.7. Exercises about flags
- 10. Variables
- 11. Money
- 11.1. How does the model think about money
- 11.2. How to find out the costs of things
- 11.3. Costs for measures
- 11.4. Costs for fuel
- 11.5. Examining costs on-model
- 11.6. Paying for measures with subsidies or loans
- 11.7. Adding costs rather than subsidies
- 11.8. Setting up arbritrary payments in the future (obligations)
- 11.9. Distinguishing particular costs
- 11.10. Computing present values, prediction, ways of discounting things
- 11.11. TODO Exercises about money
- 12. Ephemeral values like
net-cost
,capital-cost
,size.m2
- 13. Tariffs
- 14. Choices and packages of measures
- 15. Measure suitability and failure
- 16. Hypotheses
- 16.1. In-depth explanation of choices, packages and failure
- 16.2. Using
under
to work out values in hypothetical situations - 16.3. Other ways to use hypotheses;
set
under:
,rise-in
,fall-in
andoriginal
- 16.4. Some "hypothesis diagrams" for commands
- 16.5. Output from within hypotheses
- 16.6. TODO Exercises about hypotheses
- 17. Defining things so you don't have to repeat them
- 18. Templates and modules
- 19. "Optimisation"
- 20. TODO Scenario assumptions, defaults, and SAP
- 21. TODO In-use factors
- 22. TODO Assembling the parts
- 22.1. TODO A calculation of technical potential
- 22.2. TODO A simulation model
- 22.2.1. Installing some measures [the basics]
- 22.2.2. Tracking measures installed and their effects [flags, reports]
- 22.2.3. Moving definitions to one place [definitions]
- 22.2.4. Adding in some off-model suitability rules [flags, sampling rules]
- 22.2.5. Limiting the supply of measures [variables, action.case]
- 22.2.6. Modelling household behaviour [choices]
- 22.2.7. Manipulating behaviour with a subsidy
- 22.2.8. Tidying up repeated code [templates and modules, includes]
- 22.2.9. Optimising measure delivery [in-order]
- 22.2.10. Investigating sensitivity to key variables [batch]
- 22.2.11. Cost recovery on bills [tariffs, costs]
- 23. TODO Quality and Best Practice
If you're reading this, maybe you are interested in learning to use the National Household Model (NHM). This guide should supply a fairly complete understanding of what the NHM is and how best to use it; however, it isn't exhaustive, and there are some references you will need to grasp if you want to fully follow how the model works.
This document starts with an overview that describes roughly how the model works, without getting into the specifics of the language, and then a section on reading the language and using the manual before you get to actually doing anything. Many people find it a bit difficult to learn something without diving in immediately and wildly pushing some buttons; if you think this describes you, you may want to skim-read the first two parts and go the first simple scenario. However, understanding how to read scenario language is ultimately essential to being able to use the model well, and skipping the required learning is likely to prove a false economy.
Many of the sections below introduce features of the model which are quite abstract when taken on their own, but are useful when combined to achieve complex effects. We have tried to include motivating examples as you go along, but in some cases a realistic example would involve the use of several different features in concert, and we have tried to avoid producing examples which depend on understanding material not yet discussed. If as a result you find that the examples in some sections are too abstract, have a look in the 22 section near the end. It contains several realistic examples which bring together the primitive features which are each explained in their own section.
1 Overview
The NHM is a programming language for a discrete event simulator that simulates energy-related behaviour for domestic dwellings using a SAP-like energy calculation. The dwelling data is mostly derived from the EHS (although other data may be used). Let's look at the italicised parts in a bit more detail:
1.1 A programming language
The NHM is programmable; it might be better to think of it as a tool for making models rather than a particular model, as most of the things you will want it to simulate you will have to explain to it in your program. It does not simulate very much apart at all from what your program instructs. It is designed to make certain kinds of simulation program easy to write, so they can have a short description, but you will still have to write something.
For those readers who have used programmable tools before, the NHM will have some degree of familiarity; you write a scenario, which is just some text obeying a special syntax, and then ask the model to run your scenario. It simulates what it's been instructed to simulate, and produces any outputs (typically big tables about what happened) that it's been instructed to emit. There is more about the syntax in the section on reading scenario language
You can then take the outputs and use them to inform policy decisions.
1.2 Discrete-event simulation
A discrete-event simulation is one where time passes in fits and starts between events, and nothing happens otherwise. An event is an instantaneous change in the state of affairs that happens at a particular time. Other types of models might use differential equations to represent values which change continuously in time, but in a discrete event model everything stays the same until an event comes along, at which moment there is a corresponding change to how things are. Then everything stays in its new arrangement until the next event, and so on.
The model uses the syntax in your scenario to set up the sequence of events which are going to happen in the simulation. This sequence (the event queue) is a list of 'things to do', sorted by the date on which they should be done. The events that a scenario defines usually describe a command for changing something, like "take a random 10% of semi-detached houses in the south-west, and offer each one some loft insulation", or a command to make some output, like "record how many semi-detached houses in the south-west have loft insulation".
Once the model has read your scenario and set up its future events, it simply goes through the list and does them in order until it's either reached the maximum date you want to simulate, or there are no events left to think about. An important thing to understand here is that at a particular point in simulated time, the model has an idea of how the world is. When an event happens, it can change this representation of the world, so that the next event is presented with a different state of affairs. This is the analog of the arrow of time which seems to exist in reality - things which happen in the past may affect the future, but not vice-versa.
Figure 1: The sequence of events. Each circle represents a state of affairs, and each arrow the execution of an event. S0 is how things are to start with; then we insulate some 10% of the houses in S0 (E1). S1 is how things are once some insulation has been handed out. Next, from S1 E2 takes place: we hand out new boilers for some other 10% of the houses, producing S2. Note that we would expect 1% of the population to get lucky and get both measures, if the two samples are independent. The size and cost of a boiler can depend on how well-insulated the house is, so the fact that E1 has affected some house may affect the outcome of E2 for that house, and if we simulated E2 followed by E1 the result may be different.
1.3 Dwellings and the stock
The state of affairs mentioned in the previous section is mostly defined by the condition of a collection of dwellings; a dwelling is a place where one or more households reside, typically a flat or a house (as distinct from a building, which may contain several dwellings, or a household, of which a dwelling may contain several).
The initial state (S0 in 1) is defined by a stock, which is a file containing descriptions of the dwellings that should exist to start with, which we will call cases. The stock which the model will use is specified in the scenario, so the same scenario can be run against different initial conditions by referring to a different stock.
Each case in the stock has a weight associated with it, which indicates how many dwellings in the real world correspond with that description. For example, there might be 5,000 dwellings which are to be represented by a description of a medium-sized flat in London, whereas there may only be 500 dwellings represented by a description of a detached house in the North of Scotland. The total weight of all the cases in the stock should equal the number of dwellings in regions which the stock covers.
1.3.1 Weighting
Although a particular population of dwellings may be the same from the point of view of the stock (and so be represented by a single case with the appropriate weight), these dwellings might have different experiences during the simulation. For example, if a case represents 1000 dwellings, and they all have a 10 year old boiler, we might expect each of those dwellings' boilers to have a 10% chance of needing replacement in the next year. At the end of the year, some subset of the 1000 dwellings1 will have had to get a new boiler, and will no longer be in the same condition as their original cohort.
In an ideal world, the model would handle this by simulating each dwelling independently: the original case of 1000 would be modelled as 1000 distinct dwellings, each able to take a different course. Unfortunately with current computers the NHM cannot efficiently simulate the ~25 million dwellings that would make up a national stock. Instead, the model takes a position somewhere between the two extremes of simulating cases (which are of varying weights, some quite large) and simulating individual dwellings (of which there would be too many). Each case is divided up into several simulated dwellings, using a number called the quantum; a simulated dwelling (afterwards just referred to as a dwelling, except clarification is required) will still represent multiple real dwellings, but (a) most simulated dwellings will have roughly the same weight, and (b) the weight can be smaller than the large case weights in the stock.
For a case of weight \(w\), using a quantum \(q\), \(\lfloor w/q \rfloor\) dwellings will be created with weight \(q\), and one extra dwelling will be produced with weight \(w - q * \lfloor w/q \rfloor\), where this is not zero (\(\lfloor x \rfloor\) is the greatest integer below \(x\)).
To give an example of how this works, imagine a stock summarised by the following table; each row is a case, with a unique identifier (and all the other information stored about a a case omitted, on which more later):
Case ID | Weight |
---|---|
\(A\) | 1000 |
\(B\) | 100 |
\(C\) | 500 |
\(D\) | 700 |
Now let us see what dwellings would be simulated with different quanta2:
With a quantum of 1000, the 4 dwellings produced are:
\(A_1: 1000, B_1:100, C_1:500, D_1:700\)
This is the extreme where cases and simulated dwellings have a 1:1 relationship.
With a quantum of 500, the 6 dwellings produced are:
\(A_{1,2}: 500, B_1:100, C_1:500, D_1:500, D_2:200\)
Wholly divisible cases \(A\) and \(C\) produce simulated dwellings with a weight of \(500\); \(B\) and \(D\) each produce a remainder with weight less than the quantum.
With a quantum of 100, the 23 dwellings produced are:
\(A_{1..10}:100, B_1:100, C_{1..5}:100, D_{1..7}:100\)
In this case all the simulated dwellings are of equal weight, as all cases are neatly divisible, and each simulated dwelling represents 100 real dwellings. Consequently, the smallest 'unit of decision' is 100 identical houses - changes can only happen to 100 houses at a time, all at once.
Note that decreasing the quantum increases the number of simulated dwellings, but decreases their weights so that the gross weight is maintained. As the number of simulated dwellings increases, the running time and memory requirements of the model increase. Since the main impact of quantisation is on random processes, for some kinds of simulation it is sensible to use a large quantum so that the model will run more quickly; for example, investigating 'technical potential' may be fine using 1 dwelling per case, whereas a policy which is expected to affect very small populations might require a correspondingly small quantum3.
1.3.2 What is in a stock; creating stocks
We have said that a stock contains descriptions of dwellings, but without any more detail. Each dwelling is considered as a hierarchical structure, composed of various different parts:
- Some basic attributes, like region, and built form
- Information about the technologies in the house
- This is a description of the heating, lighting and water heating systems
- Information about the structure of the house
- This describes the physical condition of the house, in terms of :
- a series of storeys, each of which has some walls, which have u-values, and so on.
- four elevations, each describing the doors and windows in that elevation
- This describes the physical condition of the house, in terms of :
- Information about the people in the house, and finally
- Some additional properties, which are miscellaneous extra data that the model doesn't really understand
Creating these hierarchical descriptions is itself quite an involved task; there are some parts of the NHM collectively called the Stock Importer which convert some different sorts of input data into the form required for a stock. These input data are either the various national surveys (the English Housing Survey and so on), or an intermediate set of tables called "DTO" tables. The format of these tables and a full description of the process are out of scope for this document; they are documented in the manual.
1.4 Exercise: thinking about weights
Imagine a stock which contains cases as described in table 1.
Case ID | Weight | Floor area |
---|---|---|
C1 | 100 | 1000 |
C2 | 400 | 1500 |
C3 | 300 | 800 |
C4 | 500 | 1200 |
C5 | 100 | 300 |
- What simulated dwellings and what gross weight would be produced using a quantum of:
- 50
- 100
- 150
- 500
Here is a procedure for sampling from the set of simulated dwellings:
To sample dwellings of weight \(w\), shuffle the dwellings into a random order, and then take as many from the beginning of the order up to but not exceeding total weight \(w\).
If we take a sample of weight 500, what is the expected floor area, and the variance in expected floor area, for a quantum of:
- 1
- 50
- 100
- 150 An analytical answer may be unfeasible here, and for 500, but you can simulate this with Excel quite easily
- 500
1.4.1 Solutions
- Writing \(n\) dwellings from case \(C\) with weight \(w\) as \(n \times C(w)\):
- \(100 \times C1(1), 400 \times C2(1), 300 \times C3(1), 500 \times C4(1), 100 \times C5(1)\), with a gross weight of 1400
- \(2 \times C1(50), 8 \times C2(50), 6 \times C3(50), 10 \times C4(50), 2 \times C5(50)\), with a gross weight of 1400
- \(1 \times C1(100), 4 \times C2(100), 3 \times C3(100), 5 \times C4(100), 1 \times C5(100)\), with a gross weight of 1400
- \(1 \times C1(100), 2 \times C2(150), 1 \times C2(100), 2 \times C3(150), 3 \times C4(150), 1 \times C4(50), 1 \times C5(100)\), with a gross weight of 1400
- \(1 \times C1(100), 1 \times C2(400), 1 \times C3(300), 1 \times C4(500), 1 \times C5(100)\), with a gross weight of 1400
For the expectation of the equally divisible situations, we can do this analytically
Consider the random variable produced for the mean floor area of drawing 700 dwellings when the quantum is 1; it is well-known that the sample mean (which is what we are computing) is an unbiased estimator of population mean, and that shuffling produces an unbiased sample, so this is just the expected floor area of the population (about 1121).
Now consider the case where the weight is wholly divisible for all the cases (say a weight of 50); rather than taking the sample mean we are taking a weighted sample mean, but by the linearity of expectations this is the same thing (since every sampled dwelling has weight 50).
For the unequally divisible situations, the analysis is too long to write out here. However, it is easy to approximate computationally. A quick simulation produces the following table for the data above:
Quantum Mean sample mean Deviation in sample mean 1 1121 9.0553851 50 1120 62.633857 100 1123 95.739229 150 1118 110.45361 200 1115 126.01587 250 1107 144.81367 300 1118 158.23716 350 1105 188.58685 400 1091 207.32583 450 1095 219.25328 500 1083 236.59248 The key points to note are: (a) the mean is reasonable4, and (b) the deviation in the mean grows
1.5 SAP-like energy calculation
The NHM is a domestic energy model. It uses the dwelling descriptions in the stock to predict dwellings' energy uses using the BREDEM model, which is the basis for SAP. Many of the odd results which can be produced by the NHM are attributable to features of the SAP method, in which the meaning or use of certain terms can be quite counterintuitive. We would recommend that you read the SAP document before using the NHM in anger, or at least look through the SAP worksheet5. We also provide a brief overview here, which also explains a little bit of building physics for those new to the entire area.
A SAP calculation breaks down into a few parts, which are (in rough order of signifiance):
- Space heating
- Water heating
- Lighting and appliance use
1.5.1 Heat demand; Newton's law, mean temperatures, u-values
Heat demand is mostly calculated using a version of Newton's law of cooling, which states that the rate at which heat flows from one body to another is proportional to the difference in temperature between them. Writing this as an equation
\[ H = k \cdot (T_1 - T_2) \]
So if the two bodies are at the same temperature, there will be no net flow of heat, at 1 degree there will be \(k\) Watts, and at \(x\) degrees difference there will be \(k\cdot x\) Watts. The constant of proportionality \(k\) is known as the heat loss coefficient or just heat loss; a higher heat loss means that more energy is needed to keep your house warm. In SAP, \(k\) is composed of two parts:
- The fabric heat loss, which is to do with heat flow through the building's surfaces
The ventilation heat loss, which is to do with heat lost due to air circulating between the inside and outside of the house
The fabric heat loss is defined to be
\[ \sum_s A_s \cdot u_s \]
where for each external surface of the building \(A_s\) is the area of that surface and \(u_s\) is the u-value. From this you can see that if you double the u-value of a surface or its area you double its contribution to the heat loss. The ventilation heat loss is defined by a more complex rule, but is less likely to be significant; if you are interested, see the SAP worksheet.
Once we know \(k\) for a dwelling, we are on our way to working out how much energy is needed to keep it warm; if the temperature inside is \(T_1\) and outside it is \(T_2\), then heat will be flowing out of the dwelling into the environment at \(H = k \cdot (T_1 - T_2)\). In an unheated dwelling this would reduce \(T_1\) and increase \(T_2\), and this process would continue until eventually their temperatures equalized6. In SAP, we presume that the dwelling's occupants operate their heating system to counteract this tendency, maintaining the internal temperature at \(T_2\). To do so, they must replace the loss \(H\) with an equal heat input to the building supplied by two sources:
- Converting primary energy into heat in the space heater (burning gas, or heating up a storage heater), and
- Incidental heat gains from other sources of heat like lights and appliances, and solar gains from the sun's radiation.
A little more is written about u-values and their relationship to insulation and r-values below, in How does the model think about insulation.
In this sketch we have not explained where \(T_1\), \(T_2\) or u-values come from; these are all important factors affecting energy use, so we will describe them a little:
- Internal and external temperatures
SAP is a monthly energy calculation, so external temperatures are typically regional monthly averages; these values can be changed away from the SAP defaults in your NHM scenario.
Internal temperature is more complicated, but because it is one of the significant factors for overall energy use it is worth understanding. The calculation is affected by several different parameters; looking only at the parameters might lead you to imagine a more sophisticated technique than is really used, but all of these parameters reduce to some adjustment of \(T_1\) in the equation above. The list of parameters is:
Living area temperature
This represents the intended temperature in the main part of the house when the heating is on.
Rest-of-dwelling temperature / temperature difference
The temperature in the rest of the dwelling can be defined as another temperature, or as a temperature difference from the living area temperature. In the second case, the difference is adjusted before being added.
Living area fraction
This is simply a number indicating what fraction of the floor area is the 'living area'; this value is computed in the stock import and stored by each house.
Heating schedule
The heating schedule is a pattern of on/off times for the heating system, by day of the week. Typically this is set to morning/evening heating on weekdays, and all-day heating at the weekend.
Responsiveness of main heating system
SAP defines heating systems to have a 'responsiveness'; as far as we know this is not a quantity with a measurable definition, but represents something about how the heating system responds to the required output, so a heating system with a thermostat is more responsive than one without.
Type of heating system
Some heating systems and combinations of heating controls will lead to an adjustment in the living area temperature, or the heating schedule. These kinds of adjustments are mostly detailed in SAP tables 4a,b,c etc.
Thermal mass parameter
The thermal mass parameter of the house is related to its construction and size, and used to determine the rate of cooling of the house; this affects the slope of the downward segments in figure 2.
Figure 2 illustrates how all of these factors essentially boil down to a single mean value; this is why although the energy calculator refers to heating patterns, living area fractions and so on, it is not actually able to produce useful in-day load profiles, nor does it understand the difference between night and day temperatures, and so on.
1.5.2 Internal and solar gains
Some heat is produced within the dwelling but not by the heating system. For example, lights, electrical appliances, cooking, people's bodies and the hot water system all produce some heat as a side-effect of their operation. As well as this, some of the sun's heat is captured within the house, mostly through the windows.
In SAP, these gains are quantified by a variety of different rules, and all contribute to meeting some of the need for heat; as a result, the heat output required from a heating system is usually a bit less than the raw heat output required to maintain a sensible internal temperature. The amount of demand met by gains is modulated by a "gains utilisation factor", which appears to have the effect that increases in gains have a diminishing usefulness for meeting the need for heat.
In rough summary, the determinants for gains are:
- Lights and appliances
- Both of these are electrical in SAP, and all of the electricity they consume is taken to produce heat within the dwelling. The requirement is determined by a polynomial in the floor area of the dwelling. For lights, the demand is also modulated by a model of the sunlight coming in the windows.
- Metabolic gains
- Each person in the house is presumed to contribute a certain fixed wattage in gains.
- Hot water gains
- The production, use, and storage of hot water can provide a lot of gains. The main components of this are:
- Heat lost at the tap
- Heat lost in distribution to the tap
- Heat lost in the primary circuit, which is affected by whether the pipes are insulated or not
- Heat lost from any heat storage; this can be a large term if the house has a large tank with poor insulation
- Cooking gains
- The SAP score does not include cooking, but BREDEM does; the NHM includes a cooking model, which can be excluded from energy use figures if required (the documentation for
house.energy-use
explains how). Cooking demands are all converted into gains, and are modelled by a simple regression.
1.5.3 Hot water
The hot water calculation is quite complex, and for a full description you should read BREDEM or SAP. The demand for heat for hot water is calculated in a few steps:
- The base demand for hot water is calculated, which is a function of the number of occupants in the house
- The heat content of this water is calculated using the specific heat of water and an assumed monthly temperature increase from the mains water to the hot water that's produced
- To this base demand various additional demands are added to account for losses; the kinds of losses depend on the type of hot water system, but include
- combi boiler losses, perhaps associated with the repeated heating of a combi boiler
- distribution losses in the pipework of the house
- primary circuit losses, in houses which have a hot water tank and hence a primary circuit
- tank losses, in houses which have a hot water tank
- Finally demand met by any solar thermal system is netted off, using a complex rule which computes heat available from the sun, and meets some amount of the demand less than that, modulated by a utilisation factor. As the demand for hot water shrinks, so the effectiveness of solar hot water to meet a fraction of that demand shrinks, in this model.
1.5.4 Meeting demand; mysterious adjustments
Once all of these factors have been considered, a calculation of the fuel requirement can be made; for many heating systems which involve a boiler, this is complicated by the determination of an effective efficiency, which is a function of the winter and summer efficiencies of the boiler and the total heat output for space and water heating. This adjustment reflects the fact that a boiler may be more efficient if it is used more, perhaps because when the boiler is only producing hot water in the summertime it is more likely to be starting from cold.
This overview should give you a feel for the important factors in a BREDEM or SAP type calculation, but it does not go into any of the many quirks and details which can confound any direct understanding of its ways. Most of the key values we have mentioned (efficiencies, demand temperatures, responsiveness, hot water losses and so on) are adjusted by any of a number of footnotes, tables, subheadings and so on in the SAP document.
For example, when determining the hot water efficiency of a heat pump, if the heat pump is supplying all of the hot water its efficiency is reduced. Similarly, if a dwelling has the right combination of heating controls, for some heating systems the efficiency of the system will be changed, for some the demand temperature will be changed, and for some the final consumption will be multiplied by a scaling factor.
If you wish to have a good feel for this undeniably important part of the model, we can only recommend that you read the SAP document in order to know the ins and outs of these various adjustments and correction factors.
2 Reading scenario language
Language is often divided into syntax and semantics. Roughly speaking, syntax is the study of what forms are structurally valid or correct, with reference only to the parts of speech involved, and semantics is about the meaning associated with those forms.
For example, in the English language you might say that the words
In in in flask hath bear qualitative. Of! Of!. fruitsome noisesome sponge quadrilateral ambition
are syntactically invalid; no matter what interpretation you put on the nouns and verbs, so long as they remain nouns and verbs the words cannot be taken to form a valid sentence. There are some things we can see here that typically never happen in English; two occurrences of the word in normally do not follow one another, an exclamation point is never followed by a full stop and so on.
The (famous) sentence
Colourless green ideas sleep furiously
has perfectly acceptable syntax, but its semantics are all over the place; it is nonsensical to apply the adjectives colourless and green to the same referent, and the noun in this case (idea) is not the kind of thing that has a colour anyway, nor is it an actor which can sleep, and finally one might wonder about whether it is possible to sleep furiously. Syntax is often expressed with some sort of diagram giving a hierarchy which relates the different syntactic elements together. In this case, we could say something like ([colourless green] ideas) (sleep furiously), to indicate that colourless green is an adjective phrase affecting the noun ideas to give a noun-phrase, and furiously is an adverb used with sleep to give a verb-phrase, which combines with the noun-phrase to make a sentence.
Anyhow, English syntax is not the subject of this document, but hopefully this analogy helps, and it should illustrate how natural language has quite a formal structure that is made easy by familiarity rather than anything innate.
In this document we will first cover the NHM syntax in detail; then we will go into the semantics. Dividing syntax from semantics is, for many people, a mentally gruelling and unnatural thing to do, and there is every possibility that in reading this bit of the manual you will be thinking some of the following:
- Why must I learn all these technicalities? Surely I could just get on with something?
- Why does the author keep going on about the distinction between meaning and syntax? Surely it's all academic anyway?
- Why do so many of these examples use nonsense words? What do they have to do with my work?
- What's the point of all these parentheses and brackets and so on?
- If octopuses could live on land, would we have them as pets like we do cats and dogs?
To answer at least some of these questions, it is helpful to try the mental exercise of 'playing computer': When a you read a human language, especially one you are unfamiliar with, you probably infer the syntax from what you already know about the meaning. You have enormous tacit knowledge about what kinds of words go where, and what a sentence is likely to be trying to say from its context; you can understand from the top down. Because the NHM language is for a computer to read rather than a human, none of this is helpful; instead it is a hindrance. The computer has no tacit knowledge, and no understanding of context; in fact is has no knowledge or understanding. Instead, it can only process the language by following a precisely defined set of rules, and a lot of those rules are the syntax; it can only form its model of the code from the bottom up. Where you can use your contextual understanding to resolve ambiguity, the computer must rely only on these rules. Learning the syntax is an exercise in trying to do this bottom-up reading yourself, so as to imagine how the computer is 'reading' the scenario. The better you are able to do this, the easier you will find it to be productive with the tool in the end. The syntax section of the manual tries to help with this by presenting questions of syntax without the context that you would normally reach for to form a top-down understanding.
2.1 Syntax
We now come onto the syntax for NHM scenarios; this defines what scenarios the model can understand, and how to read their structure.
The scenario language is a kind of special-purpose programming language. A lot of languages work by describing a series of steps to do1. Some parts of an NHM scenario form a list of steps, but as a rule it does not proceed by processing one line at a time in sequence.
Instead, the statements in a scenario should to be interpreted as instructions that have meaning given to them by the context in which they are written.
Every statement in a scenario one of four things:
- A literal, which is a word, number or other sequence of characters except spaces, or any sequence of characters between double quotation marks (including spaces)
hello
is a literalhello world
is two literals,hello
andworld
103.4
is a literal90%
is a literal01/01/2014
is a literal*my-long.thing!*
is a literal
- An expression (or s-expression or symbolic expression), which is exactly one literal followed by zero or more other statements enclosed in parentheses
(
and)
. Expressions are usually commands to the model, and the first literal gives the name of the command. The rest of the things are arguments to the command; they belong to the command and affect what it does in some way. Different commands accept different arguments.(hello)
is an expression, which is invoking a command calledhello
(+ 1 2 3)
is an expression, which is invoking the command+
with three literals1
,2
and3
as inputs. In this case+
does what you would think as do numbers, and this is an instruction to sum the values 1, 2 and 3.(measure.wall-insulation type: Cavity)
is an expression invoking the commandmeasure.wall-insulation
, and giving it the argumentstype:
andCavity
. Arguments which end with a colon (:
) tell the command something about the intended meaning of the following argument, so hereCavity
is the type ofmeasure.wall-insulation
(+ (* 1 2) 3)
is an expression invoking+
with two arguments; the first argument is another expression(* 1 2)
, and the second is the literal3
. Again, this means what you might expect: multiply 1 by 2, and then add 3 to the result.(1 + 2)
is an expression, but it does not mean "add 1 and 2", or "supply 1 and 2 to +". If it meant anything it would be supply the literals+
and2
to a command called1
; however the model has no command called1
.((1 + 3) * (4 + 5))
is not a legal expression, because the head is not a literal.()
is not a legal expression, because it has no head
- A list, which is a series of other statements enclosed in brackets
[
and]
[1 2 3 4]
is a list of the four literals1
,2
,3
, and4
.[(+ 1 2) 8]
is a list with two things in; the expression(+ 1 2)
and the literal8
[]
is a list with nothing in it
An infix expression, which is a normal mathematical expression enclosed in braces
{
and}
. Infix expressions must be space separated, because the names of NHM literals can contain characters that look like mathematical operators. Infix expressions are translated automatically into standard expressions when they are read. You don't have to use infix expressions anywhere if you don't want to, but in a few places they may make things easier to read.{1 + 2}
is an infix expression which is read as the expression(+ 1 2)
{1+2}
is an infix expression which is read as the expression(1+2)
, which is not+
applied to1
and2
.{9 * x + 33 / 7}
is an infix expression which is read as the expression(+ (* 9 x) (/ 33 7))
.
When you read an expression, it can be helpful to mentally translate it into a description of an instruction; here are some examples: (we will cover what the terms written actually mean, and how to use them later)
(measure.wall-insulation resistance:0.01 thickness:50 capex:(* 1.1 (size.m2)))
This can be read as
Use the command
measure.wall-insulation
, giving it aresistance
of 0.01, athickness
of 50, and acapex
determined by using the command*
with 1.1 and the result of the commandsize.m2
.(apply (measure.standard-boiler) to: (filter (house.region-is London)))
Reads as
Use the
apply
command, giving it the commandmeasure.standard-boiler
, and telling it thatto
is the result of using thefilter
command with thehouse.region-is
command givenLondon
.These translations are ignorant of the meaning of all the terms. We have tried (and sometimes succeeded) to make it so that the names of commands and of their named arguments have a guessable meaning. In the examples given, you can hopefully determine something about what the commands written will actually do. Every command has a page in the manual which should describe what arguments it takes, and what it does with them.
It can be helpful to think of s-expressions as hierarchies, a bit like an org-chart. For example, the expression
(a (b c d) (e f g (h (i j) k)))
could be illustrated as in figure 2. Just as in a real org-chart, entities which are higher up (like managers) are more important than those lower down (human resources); the managers direct the actions of their subordinates but because they are ideas people they do not have to understand the implied detail. They need only be concerned with the requirements they and the form of the outputs they will produce. In the examples above, you might say that
measure.wall-insulation
delegates the task of computing a number for the argumentcapex:
to the expression(* 1.1 (size.m2))
; in this case*
does not 'know' that its output is to be used for capex, andmeasure.wall-insulation
does not 'know' how to multiply two numbers together.Figure 2: A hierarchy describing the expression (a (b c d) (e f g (h (i j) k))). Arrows point to nodes which are more general, or important, or controlling. The node
a
controlsb
ande
, and may determine their environment and use both their outputs;e
can only see the effect ofb
insofar asa
relates them.
2.1.1 Summary: basic syntax
- Everything on a line following a
;
is ignored as a comment - The text is broken up into parts according to
- brackets and parentheses
- white space (spaces or line breaks)
- the colon character
:
- NHM syntax describes a hierarchy
- The hierarchy is spelled out by parentheses
(
and)
- The hierarchy is spelled out by parentheses
- The first word after a
(
names a command- In this a word is any sequence of characters except the ones mentioned above, e.g.:
- this-is-a-word
- this.is.a.word
- this/is-a-word
- THISISAWORD
- Everything after the first word up to the corresponding
)
is an argument to that command
- In this a word is any sequence of characters except the ones mentioned above, e.g.:
- Commands that are higher up in the hierarchy use or control lower down commands
- Commands are usually not "aware" of their siblings at equal level in the hierarchy
- So:
(this and that)
is the commandthis
applied to the argumentsand
, andthat
.(example [1 2 3] hello)
is the commandexample
applied to the arguments[1 2 3]
andhello
[1 2 3]
is a list containing the three words1
,2
and3
(first (second x y) z)
is the commandfirst
applied to two arguments:- the command
second
applied to the wordsx
andy
, and - the word
z
- the command
2.1.2 Examples by comparison with prefix notation
If you find the syntax hard to read, you may find it helps to look at some equivalent syntax written as infix expressions versus the prefix form used by the NHM. Excel (for example) typically uses an infix syntax for formulae.
To make it easier to follow, these examples use some syntax which has the same meaning in Excel as it does in the NHM (once written in prefix form). However, most syntax does not have a directly translatable meaning like this, and you will need to learn the meanings of the commands used from the manual in order to read or write it.
Infix (Excel) | Prefix (NHM) |
---|---|
1 |
1 |
1+2 |
(+ 1 2) |
3*4 |
(* 3 4) |
1+2+3*4 |
(+ 1 2 (* 3 4)) |
1/2 + 9 |
(+ (/ 1 2) 9) |
MAX(1,2,3) |
(max 1 2 3) |
MAX(2*2, 2+2) |
(max (* 2 2) (+ 2 2)) |
SIN(X)/X |
(/ (sin x) x) |
SIN(COS(X)) |
(sin (cos x)) |
3>2 |
(> 3 2) |
-sin(x) |
(- (sin x)) |
Things you might have noticed include:
- All functions must have parentheses, even the common
+
,-
,*
and/
- As a result, there is no precedence rule apart from the parentheses. There is no need to think about whether
a*b+c*d
is the product ofa
andb
summed with the product ofc
andd
or whether it is the product ofa
, the sum ofb
andc
, andd
. - The functions which don't have special treatment in infix world like
SIN
andCOS
are written in the same way, except that:- The function's name is after the opening
(
, and - The function's arguments are separated by white space not commas (this unambiguous because of 1. and 2. above)
- The function's name is after the opening
2.1.3 Named or keyword arguments
At the start of this section we saw some examples like (measure.wall-insulation type:cavity)
.
This is the command measure.wall-insulation
given two words, type:
and cavity
(remember that :
is a character that splits up the input, like white space is).
These two words supply a single argument called a keyword argument, or a named argument.
The first word type:
tells the command how it is to use the second argument, in this case cavity
.
This is in contrast to unnamed or positional arguments, in which the command determines how to use the argument by where it is.
For example, when dividing two numbers in prefix notation we write (/ a b)
.
This means to divide a
by b
; the /
command knows that the first argument a
is the numerator and the second argument b
the denominator.
If we swapped a
and b
around to get (/ b a)
the /
command would take b
as the numerator and a
as the denominator.
This syntax is easy to remember and read only because we were all taught about /
at school; it is a familiar function and we know it expects two parameters, and so on.
Other model commands (like measure.wall-insulation
) may also take many parameters, some of which may be optional; it would be difficult to remember which arguments were which if their meaning was given by their position.
Named arguments solve this problem by letting us forget about the order of the arguments, and giving us a hint about meaning when we read a command (although the ultimate meaning is still best found from the manual).
For example, returning to wall insulation we might see any of the following:
This
(measure.wall-insulation type: cavity thickness: 50 resistance: 0.01)
or
(measure.wall-insulation resistance: 0.01 thickness: 50 type: cavity)
or
(measure.wall-insulation type: cavity resistance: 0.01 thickness: 50)
or
(measure.wall-insulation type: cavity resistance: 0.01 thickness: 50 capex: 500)
In each case, we know that measure.wall-insulation
is being given:
- a
type:
ofcavity
- a
resistance:
of0.01
- a
thickness:
of50
In addition, in the final example there is another argument capex:
being given the value 500
.
In summary, if you see a word ending with a :
within a command, you know it is a keyword, and that the thing following it is being given to the command as a keyword argument.
That means that you can look up that keyword argument in the manual page for the command to know how the model will use it.
The command is responsible for using its arguments, and will use them in the order it needs to - the order they have been written is not important except when it has meaning for the command (like for /
).
For keyword arguments, the order never matters.
2.1.4 Mixing keyword arguments and positional arguments
Some commands may accept both arguments which need a keyword and arguments which do not need a keyword. These arguments can be mixed up; for example
(a-made-up-command hat: fedora 103 swindle: no 900 700 fire-station: kensington)
should be read as the following:
- a use of the command
a-made-up-command
, given the arguments- by position (these have no keyword, and swapping them around would change the meaning):
- 103
- 900
- 700
- by keyword (the order is not important):
- fire-station:
- kensington
- hat:
- fedora
- swindle:
- no
- by position (these have no keyword, and swapping them around would change the meaning):
Any of the following writings should be read the same!
(a-made-up-command hat: fedora swindle: no fire-station: kensington 103 900 700)
(a-made-up-command 103 900 700 hat: fedora swindle: no fire-station: kensington)
(a-made-up-command 103 900 700 hat: fedora swindle: no fire-station: kensington)
(a-made-up-command 103 900 hat: fedora swindle: no fire-station: kensington 700)
However, as a convention we would suggest using the form
(a-made-up-command hat: fedora swindle: no fire-station: kensington 103 900 700)
where the keyword arguments are written out first and the positional arguments follow. It is all the same to the model, but for the human reader this consistency makes it less likely that the eye skips over something by accident. For the same reason, it is often a good idea to avoid putting several keyword arguments on the same line, and so on. However, the question of how to layout your code is really a matter of judgement, and you should make a decision based on the particular circumstance rather than trying to adopt a universal principle.
2.1.5 List arguments
Sometimes a command may accept more than one value for one of its arguments.
For example, the model contains a command aggregate
which produces tables of summary statistics, and this command can accept a keyword argument over:
which tells it about which populations you want summary statistics for.
Since you might want summaries for several populations, you might want to be able to put in several values for over:
.
The way to express this to the model is by putting the multiple values in brackets []
, so for aggregate
(which we describe in detail later), you would write something like:
(aggregate ... over: [first second third] ... )
This should be read as:
- use the command
aggregate
, with the arguments- over:
- several values supplied, to wit:
first
second
third
This is quite different from
(aggregate ... over: first second third ... )
which (as explained above) would be read as
- use the command
aggregate
with the arguments- by position:
second
third
- by keyword:
- over:
first
- by position:
and different still from
(aggregate ... over: (first second third) )
which would be read as
- use the command
aggregate
with the arguments- over:
- having the single value:
- a use of the command
first
, with the positional argumentssecond
, andthird
- a use of the command
The order of values in a list can be significant, so rearranging a list may change the meaning of the command using it.
Lists may be used as values with some keyword arguments and some positional arguments.
If an argument is allowed to be a list, it is equivalent to write [the-value]
and just the-value
, i.e. a single item will be taken as a list containing only that one item.
2.1.6 Syntax errors; things you should never write
Here is a quick list of things which should always/never happen in any valid scenario. When you are getting started, you will probably write some syntactic mistakes as you get used to the general shape of things. Checking this list will help you see some common mistakes.
- every opening bracket
[
, parenthesis(
, quote ="= or brace{
should always have a matching closing symbol - a parenthesis should never follow another parenthesis immediately, like
((
- a parenthesis should never close immediately, like
()
- a parenthesis should always have a word immediately after it, like
(scenario...
- a parenthesis should never have a keyword immediately after it, like
(capex: ...
- a keyword should never have another keyword immediately after it, like
first: second:
- a keyword should always have an expression after it, like
first: something
orfirst: (something else)
orfirst:[]
- a keyword should never be before a closing symbol, like
first: )
orfirst: ]
- a keyword may appear after a closing symbol, like
b:
in(Q a: (W) b: (E))
- a keyword may appear after a closing symbol, like
- a keyword should never be written directly within a list argument, like
[first: x]
- they may appear within expressions that are in a list, like
[blah (something first:x) qwer]
- they may appear within expressions that are in a list, like
- an open bracket should never follow an open parenthesis, like
([ ...
As an exercise, try explaining why each of these is true.
2.2 Exercises: read and write some syntax
- For each of these bits of syntax, which sentence gives a description consistent with this section of the manual?
Note that the syntax written here may not have any semantics assigned to it in the NHM, but it is syntactically OK, so you could define some terms in a scenario that would give it a meaning.
Further note if any of them seem like they look like a mistake.
(house.energy-use)
- calculate house.energy minus use
- invoke the command
house.energy
with the unnamed argumentuse
- invoke the command
house.energy-use
(house.energy-use by-fuel:MainsGas)
- invoke the command
house.energy-use
with the unnamed argumentby-fuel:MainsGas
- invoke the command
house.energy-use
withMainsGas
for the named argumentby-fuel:
- invoke the command
(+ 1 house.energy-use by-fuel: mainsgas)
- invoke the command
+
with two unnamed arguments,1
andhouse.energy-use
, and the keyword argumentby-fuel:
beingmainsgas
- invoke the command
+
with the arguments1
, and- the command
house.energy-use
given a keyword argumentby-fuel:
beingmainsgas
- the command
- invoke the command
(aggregate.count name:how-many)
- invoke the command
aggregate
with the unnamed argumentcount
and and the keyword argumentname:
beinghow-many
- invoke the command
aggregate.count
with the keyword argumentname:
beinghow-many
- invoke the command
(+ 10 (* 4 (house.total-floor-area)) 3)
- invoke the command
+
with the arguments*
4
andhouse.total-floor-area
and3
- invoke the command
+
with the arguments10
, and:- the command
*
with the arguments4
and- the command
house.total-floor-area
, and 3
- the command
- the command
- invoke the command
+
with the arguments10
, and:- the command
*
with the arguments4
and- the command
house.total-floor-area
- the command
3
- the command
- invoke the command
(/ 8 (+ 3 4))
- invoke the command
+
with the arguments/ 8
and3
and4
- invoke the command
/
with two arguments,8
and- the command
+
itself having two arguments,3
and4
- the command
- invoke the command
(8 / 9 + 3)
- invoke the command
/
with8
and9
and then invoke the command+
with3
- invoke the command
8
with the arguments/
,9
,+
and3
- invoke the command
(1+2)
- invoke the command
1
with+
and2
as arguments - invoke the command
+
with1
and2
as arguments - invoke the command
1+2
with no arguments
- invoke the command
- For each of these bits of syntax, write your own outline description
(/ 1 2 3 4 5)
(+ + + + plus)
(+ (1 2) (/ 3) 4)
(+ (1 2) (/ 3 4))
(+ 1 2 (/ 3 4))
(+ 1 2 / 3 4)
(log base:9 9)
(log 9 base: 9)
(log (log 9) base: (log base:9 9))
(measure.standard-boiler fuel:mainsgas capex: (+ 500 (* 40 size.kw)))
(qwer [123 (qwer 303)] pip: [pop])
- For each of these outline descriptions, write equivalent syntax
- The word
fish
- A use of the command
fish
- The command
fish
applied to the single unnamed argument4
- The command
+
applied to two unnamed arguments:- the command
house.energy-use
, given the named argument:- by-fuel:
MainsGas
- the command
house.energy-use
, given the named argument:- by-fuel:
HouseCoal
- the command
- The word
- For each of these bits of syntax, indicate why it cannot be valid syntax
(+ 1 2 3
(+ ((* 3 4)))
[(house.energy-use by-fuel:) mainsgas]
([house.region] [house.built-form])
2.2.1 Answers
- Consistent descriptions are:
- 3. - there is one word in parentheses, which is the use of a command
- 2. - there are three words, the first being the command, the second ending with
:
and so a keyword - 1. - all the arguments are to
+
, because it has the parentheses - 2. - as for question 2
- 3. - the parentheses mean several levels of commands, and the
3
belongs to the+
not the*
- 2. - command always after the opening parenthesis
- 2. - command still always after the opening parenthesis
- 3. - command after the opening parenthesis; there are no spaces so it is one word,
1+2
.
- Our descriptions are:
/
applied to1
,2
,3
,4
, and5
+
applied to+
,+
,+
, andplus
+
applied to three arguments:1
applied to2
/
applied to3
- the word
4
+
applied to two arguments:1
applied to2
/
applied to3
and4
+
applied to three arguments1
2
/
applied to two arguments3
4
+
applied to 5 arguments:1
2
/
3
4
log
applied to two arguments:- the keyword argument
base:
being9
- a positional argument
9
- the keyword argument
- exactly the same as 7.
log
applied to two arguments- the keyword argument
base:
being:log
applied to two arguments- the keyword argument
base:
being9
- a positional argument
9
- the keyword argument
- a positional argument:
- the command
log
applied to the single argument9
- the command
- the keyword argument
- the command
measure.standard-boiler
, with two named arguments:- fuel:
- the word
mainsgas
- capex
- itself a command,
+
, applied to:500
- the command
*
applied to two arguments40
size.kw
- the command
qwer
applied to two arguments:- the keyword argument
pip:
, being:- a list with one value in, the word
pop
- a list with one value in, the word
- a positional argument, being alist of two values:
123
- the command
qwer
applied to the single argument303
- the keyword argument
- Each description can be written as:
fish
(fish)
(fish 4)
(+ (house.energy-use by-fuel:mainsgas) (house.energy-use by-fuel:housecoal))
- The errors are:
- There is no closing parenthesis for the opening one
- The use of the command
*
is itself being used as a command (double parentheses) - The keyword
by-fuel:
has no value; it is immediately followed by)
- The list
[house.region]
cannot be after a parenthesis, as a list cannot be the name of a command.
2.3 Semantics
By now we have covered the NHM syntax, but little to nothing about the semantics.
If you have a well-formed scenario the semantics say what simulation will if it runshappen.
However, not all syntactically correct scenarios can be made into a simulation, because not every command can be used in every place. For example, you cannot add a house's region to its floor area, because region is not something that can go within +
(ultimately because it's not a number). There is an exhaustive list of what commands can be used where in the language reference.
In truth most of the rest of this document is about the semantics, going into depth about the behaviour of different parts of the language, and what commands can be used where.
However, for familiarity here is a shallow overview of the semantics associated with common bits of syntax.
- Top level elements
There are two things you will see at the top level of a scenario
scenario
is the command declaring a single scenario. This will instruct the model to load some houses from a stock file, and then simulate changes to the houses and trigger the production of reportsbatch
is the command declaring a suite of related scenarios A batch contains ascenario:
argument, which is a scenario, and aninputs:
argument, which generates a table of parameters. The parameters are substituted into 'placeholders' in thescenario:
argument to produce many variants, which are run and then summarised together. This is mostly good for sensitivity analysis.
- Things which go inside a
scenario
Commands to make things happen;
on.dates
,on.change
Mostly you will use
on.dates
within the top level of a scenario. It sets up the simulation to do things when the model gets to particular dates. The things inside it are the things that will get done. The order in which things get done is determined by the date, and then by the line number.Commands to change things about how the model works
Along with
on.dates
, you may write a few things mostly starting withcontext.
which change 'global' settings in the model. For example,context.tariffs
sets up fuel prices,context.weather
changes the weather,context.carbon-factors
changes the carbon factors and so on.Definitions
Finally you may see definitions of variables and actions. The command
def
defines a variable, which will either be stored against each modelled house (like number of measures house has received) or as a single global counter (like number of measures given out in total) depending on theon:
argument.There are also commands
def-action
,def-test
,def-function
anddef-report
which merely serve to define things for later reuse; they have no direct effect on their own.
- Things which happen on particular dates
We mentioned
on.dates
which makes things happen; here are some of the things it can make happen:apply
is one of the most basic things you can make happen. When it happens, it finds a population of houses defined by itsto:
argument, and then goes through the actions it has to take applying each action to all the houses in turn.aggregate
is a command to produce a summary table about a population of houses. When it happens, rows are added to that table, so it lets you see the state of the stock at a moment in time.repeat
is a command that tries doing something to the stock again and again until a condition is met
- Measures and actions
apply
is what you use to change houses, using an action. A measure is a kind of action that represents a technology being put into a house. The NHM has lots of actions; the important ones are:action.case
, which lets you pick an action to take depending on how the house isdo
, which lets you combine a series of actions into onechoice
, which lets you pick an action to take based on how the house will be afterwardsaction.do-nothing
andaction.fail
, which are explained later
- Modules and templates
Along with everything described so far, you are likely to encounter, modules, templates and macros.
These are ways to make your scenario shorter to read and write by avoiding writing the same or similar things again and again. They are explained later, but basically
template
and any command starting with~
are special, and transform the scenario in some way before it gets run (and so before any stock is loaded or energy calculations are done or anything). - Includes
Finally if your scenario gets big enough or has parts that need reusing, it may be broken up into separate files. Any of the commands starting with
include
is akin to a mechanism for copy-pasting another file into your scenario when it is run.
3 The NHM desktop application
You are most likely to use the NHM by running the NHM desktop application7, which is a tool for editing scenarios with access to the help, and running them and looking at the results. If you are working with other people, you can use the git version control system to collaborate on scenarios, but this is by no means obligatory. This section of the manual will cover installing the application on your computer, using it to edit and run scenarios, and will direct you to other resources where you can learn more about it.
3.1 Downloading the application
The latest version of the NHM is always available from http://deccnhm.org.uk/standalone/. This presents four different builds for different operating systems. For the windows version, you will need to download cse.nhm.ide.application-win32.win32.x8664.zip. This requires you have:
- A 64-bit processor and associated Windows operating system
- Sufficient memory; this is probably 4GiB (gigabytes) at a minimum
It includes a version of the Java Runtime Environment (JRE) which it needs, so you do not need to install this. However, the applications offered for other platforms do not include this, so if you want to use the Mac OS or Linux versions you will need to install your own JRE.
3.2 Installing the application
The NHM requires very little installation - all that you need to do is unzip the zip file you downloaded in the previous step somewhere on your computer.
In windows you can usually do this by right-clicking on the zip file, and choosing Extract all...
If you are installing the application in a multi-user environment, where several users of the same machine may run it at the same time, you may need to do further setup. The Eclipse documentation explains more about this for system administrators.
3.3 Using the application
Now that you have the application installed, let's have a quick walk-through of the major features.
Start the application double-clicking on the National Household Model.exe
program.
If everything is working correctly, you should see a loading window like this:
Once the application has started up you will be presented with a blank slate for working on NHM scenarios. The window which appears will look something like this, except without all the arrows and boxes drawn on.
You can see in this window the main way the NHM interface works.
There is one big window, which has the usual menu bar at the, then a tool bar (the row of icons below the menu bar)8.
Below that the window is divided up into several non-overlapping rectangular panels.
Each panel may house some "views", which show information.
In the picture above, there are three panels, one on the left, one at the bottom right and one at the top right.
You can see that the left panel contains only the Project Explorer
view, and the panel at the bottom contains these four views:
Problems
Git Repositories
Model runs
Model run log
In the bottom panel, the Problems
view is currently displayed.
Clicking on the name of one of the other views would switch change to that view.
Each panel also has some little controls in the top right, a thin horizontal bar and a horizontal bar with a rectangle underneath it. The thin horizontal bar will shrink (minimize) the panel down to a narrow bar on the side of the window. The fat horizontal bar will expand (maximize) the panel to fill the whole window. The problems view and the project explorer view both have a cross (X) at the right of their name. Clicking this will close the view. They also both have a view menu, which is the little downward pointing arrow near the minimize and maximize buttons. This shows a menu with options relevant to the view. You can also move views around by dragging and dropping the tab with their name in on the screen.
Try clicking on any of these buttons to see what they do, to get a feel for the user interface.
If you have closed a view, or got confused about how things are, you can put things back to how they started by right clicking on the "NHM" button in the top right (by the house icon), and choosing Reset
:
You can also bring back individual views if you have closed them by looking in the Window
menu:
Finally in the middle of the window is the editor area. This is where your scenario text will be displayed. At the moment it is empty, because there is no scenario open. We will remedy this by creating a scenario to edit.
3.3.1 Creating a scenario
All work in the NHM application must be contained within a project, which will be shown in the Project Explorer
on the left.
To make a new project with a scenario in it, right click in the white space in the project explorer and choose New
-> Scenario Project
:
This will display a wizard, which is a series of steps to go through shown in a window on the screen. The first step of the new scenario project wizard looks like this:
Since we are making a new project, we need to have a name for the project.
This can be anything you like, but it is easiest if it does not contain spaces.
A project is really just a folder on your computer.
By default, it will go inside a folder which the NHM calls its workspace.
You can see this in the wizard as well.
Clicking Next >
to move on we get to the next page of the wizard:
This lets us pick a version of the model that we want to use.
The version you want to use is associated with the project, so every scenario in that project will be validated and run using the chosen version of the model.
The NHM desktop application can contain several versions of the model at once.
These can either be release versions, which should be reliable and will not change, or test versions which are still undergoing development.
When you (or your systems administrator) updates the NHM desktop application, it will get any newer versions of the model that are available.
If you have chosen one of the top two options here (latest release or latest version, including test versions) then if you update the NHM desktop application and a new release or new test version is available, that version will be used.
The NHM desktop application does not update itself unless you ask it to, so this need not be an intimidating prospect.
Once you have chosen a version of the model you can go on to the next page (or you can click Finish
to skip it, if you like):
This page is just for making a simple scenario to get started with.
It is not intelligent in any way, it just puts in some pro-forma text that is built into the application.
We will choose to have a quick-start scenario, and press Finish
.
Once the wizard is completed, an icon will appear in the Project Explorer
for your new project.
If you click on the +
next to this icon the project will "expand" to show you the scenario file that it contains.
Double click on that scenario file to open it.
You should see something like this:
The window looks like it did before, except that the blank panel now shows the scenario editor. There is also an error in the new scenario - there is no stock with the given name. This is shown in four places:
- In the scenario editor, in the left hand margin, there is a red X. If you point the mouse at this X, a "tool tip" will appear describing the error.
- In the scenario editor, the opening bracket of the scenario has a little red underline. This is because the error is to do with the
scenario
command in particular. The bracket which is underlined is for the command which is confusing the model. - In the right hand margin, there is a little red rectangle. This shows all the errors in the whole scenario, so if the scenario did not all fit on the screen, the right margin would show if there was an error off the bottom that you could not see on the left margin. Clicking on the error marks in the right margin will scroll the editor to the error.
- In the
Problems
view at the bottom, there is a list of errors. Each error is shown as a line in the list. Double-clicking on an error will find it in the editor.
Let's fix the error by importing a stock.
3.3.2 Importing a stock
To import a stock you must use another wizard.
Right click on the project icon in the Project Explorer
, and choose Import...
This will show a list of the import wizards that are known.
At the moment the stock import wizard is found under Other
:
Press Next >
and select a stock file to import by clicking Browse
Pressing Next >
again will show you a page where you need to select a version of the model to use.
The stock will be imported using the version you select.
Note that if a new version is released with a different stock importer, you will have to do the import again - the application will not change your stocks once they have been created.
Pressing Next >
will run the importer on the file, showing you a page of progress:
In the middle is a log of output that the stock importer produces.
This may help explain if there are problems in the stock.
In this example, there are no problems so we can press Finish
.
This will put the new stock file in our project, so we can use it in a scenario.
You should see it in the Project Explorer
view.
You can amend your scenario to use the new stock - one way to do this is to write out the name of the stock, but you can ask the application to type this for you.
Open the scenario, delete any existing text after the stock-id:
, and press Control-Space (i.e. depress the control key with one finger, then keeping it held down press and release the space bar, then release control).
You should see a suggestion box appear above the scenario editor, like this:
The suggestion box has two parts - on the left, a list of suggestions, and on the right help about each suggestion.
You can go up and down in the list with the up and down arrow keys, and you can choose a suggestion by pressing the return key.
Press return to select the stock you want to use, and the NHM will fill in the stock name for you.
Your scenario is now valid, but the application will not show this yet.
You have to save the scenario to have it re-validated.
You can tell that the scenario is not saved because there is a star (*
) in the title in the editor.
Press Control-S to save, and your screen should look like this:
Now the scenario has no errors, we could run it and inspect the results.
3.3.3 Running a scenario and viewing results
To run a scenario, you can either press the shortcut key Control-R, or click on the run button in the toolbar. For now, let's use the toolbar:
In this screenshot there is only one option for running, Run on this computer. You may have more options if your NHM knows about servers for running scenarios on. The user interface works the same way in either case. Once you have selected where to run your scenario, a new row will appear in the runs view:
This tells you how your scenario is getting along.
If you are using a remote server to run your scenarios, you can switch off the application and open it up again later to find out what happened.
If you are running the scenario locally, you must leave the application open to wait for the results.
When your scenario has finished, the State
column will show =Complete:
At this point, you can download the results of the run. To do this, either double-click on the run, or click on the button in the top left of the runs view which is like an arrow pointing into a tray. A dialog will open asking you where to save the results
The results will be placed in a folder contained in the folder you select, and will appear in the Project Explorer
:
The folder that is created has been named according to (a) the name of the scenario that was run and (b) the date when it was run.
You can see the output files that have been created - in this picture there is just counts.tab
, from the aggregate
report in the scenario.
You can open these tab
files within the nhm directly by double clicking, if you just want to inspect them:
They will open up in a new tab the editor panel.
Here you can see that the scenario is also still open.
However, you may want to open them up in another program, or send them in an email or something.
To open them using another program, you can right click on them, and select Open With
, Other
:
In the dialog that comes up, you can choose any editor built into the NHM or a program on your computer. For example, here is how to open the file using Excel:
If you want to open all tab
files in Excel when double clicking in the NHM, tick both Use this editor for all 'counts.tab' files and then Use it for all '*.tab' files as well.
Once this is done you can enjoy your tables in the familiar and comforting environment of Microsoft's premier table editor:
The result files and scenario files are not stored in any special way - they are just files in a folder on your computer.
This folder is called the workspace folder.
You can locate these files in Windows by right-clicking on any file in the Project Explorer
and choosing StartExplorer
, Show in File Manager
:
You can see that the Project Explorer
view is just showing you some files, which you can copy, email, or otherwise manipulate in whatever way you are used to:
To find out where the workspace is, you can either open a project in Windows explorer like this and navigate up a level, or you can choose Switch Workspace
, Other...
from the File
menu.
The window which comes up will contain the current workspace as the default:
You can have more than one workspace if you like, for different working contexts, and switch between them with this menu.
Now that we know how to run a scenario and look at the outputs, let's look at how to make slightly more complicated scenarios.
3.3.4 Creating more scenario files and including them
Usually your scenarios will be big enough to warrant being divided into several parts.
To make a new scenario file, either to include from another file or because you are writing a new scenario, you need to make a new file with the .nhm
file extension.
The easiest way to do this is to right click in the project explorer on the folder you want to put your new file in, and select New
, File
:
This will open a window where you can put in the file name.
Make sure it ends with .nhm
so that the application treats it as a scenario.
The newly created file will appear in the Project Explorer
, and you can double-click on it to open it.
It will open in another tab in the editor area:
Now we have our tabula rasa, we can begin typing some things.
In our example, let's say that we wish to define an NHM module.
We could type this all out, but let's save time and have the typing done for us.
Press Control-Space and the suggestions window will reappear.
In the list of suggestions, you should see a suggestion for ~module
.
Press return and it will type most of a module for you, selecting the name to fill in.
You can fill in a name, and then press tab to move to the next logical place to type something.
This suggestion has also put in an init
template for you, as this is NHM convention.
Click the cursor after the closing bracket for init
, but before the closing bracket for =module, and press Return to put in a new line.
Note that the editor has indented the line for you.
Now press Control-Space again, and choose the suggestion for entering a template definition:
Type the name of your template in the first placeholder (the name), and press tab to go to the next (the arguments).
Let's make a template with no arguments - just backspace to delete them.
Press tab again to place the cursor inside the template definition.
In our example we have called the template count-of-houses
:
We want the template to produce the count of the houses, so we are going to insert aggregate.count
9.
Type agg
to get started, and then press Control-Space again.
Some suggestions appear:
Press down until you get to aggregate.count
, and then press Return to insert it.
The cursor moves to the end of the command, before its closing parenthesis.
We want to give our aggregate a name, so press space to insert a space and then press Control-Space again:
The suggestions include name:
, which we want to supply.
Note that the help also tells us what name:
means in this context.
Press return and fill in any old name.
Now save the file.
You will notice that the model produces a validation error, saying that the file does not contain a scenario.
It is customary to put a scenario at the top of a file defining a module, which shows how the module works, and ideally helps you test its output independently of other modules.
We have typed one in which you can see below - try typing this yourself, pressing Control-Space whenever you have done a bit to see what the model suggests.
In this screenshot you can see that the auto-complete suggestions include user-defined templates which are available in your scenario.
Here we are filling in my-module/count-of-houses
into our test scenario, to show people how it is meant to be used.
After we have chosen it, note that hovering the mouse over it will show a tool-tip which tells you the definition of the template and where it came from:
Saving the new module file shows now validation errors, so we are ready to use it somewhere else. In an ideal world we would also have checked that our module did what we had intended it to do.
Switching back to my-project.nhm
, we can now make use of our module.
First we have to include it10; fortunately the auto-complete can help us with this.
At the top of my-project.nhm
(before the scenario), press Control-Space.
Note that one of the suggestions is include-modules
:
Choose this, press space to go to the first argument, and press Control-Space again to see suggestions for includes:
We can select the new module and press return to fill in its name
Now we are using some modules we should use ~init-modules
.
It doesn't actually matter in this example but it is good practice.
And now we can use our new template - in the existing aggregate
command, delete the old aggregate.count
command and press Control-Space.
In the list of suggestions, you can now see my-module/count-of-houses
.
This is the new template we just created!
Press return to insert it.
Note again that you can hover over the template to show its definition and where it came from:
Now we can run our main scenario again, to see if the results are different. Clicking run, notice that a window comes up asking for a bit of text to disambiguate the new run from existing runs. We type in 'with template', just so we know
In the runs window you can see a new run, which has our note after the Scenario
in brackets.
When it has finished you can download the results like before:
3.4 Editing scenarios efficiently
An NHM scenario is just a text file, so you could edit all your scenarios in Notepad11. However, the NHM application knows about the syntax for scenarios, and so can help you with your editing.
3.4.1 Auto-completion
We have already seen how pressing Control-Space will display a menu of suggestions that you can select from. These suggestions are relevant to the context in which you have pressed Control-Space; that is to say, the model will try only to suggest things which can legally be typed into the place where the insertion point is.
To make this a bit clearer here are some examples; in these examples, the vertical bar symbol |
denotes the insertion point where you pressed Control-Space.
agg|
The model will show anything that can be written starting withagg
at this point.(agg|
The model will show any commands that can be written that start withagg
at this point. Inserting a command will leave the cursor just before its closing parenthesis, so you can press space to start typing its arguments.(house.energy-use |
The model will show anything that can go withinhouse.energy-use
. This will be:- Other commands or values which can be used for the first unnamed argument to
house.energy-use
- The keyword part of keyword arguments which
house.energy-use
understands, likename:
orby-fuel:
- Other commands or values which can be used for the first unnamed argument to
(house.energy-use by-fuel: |
The model will show values that can be supplied for theby-fuel:
argument ofhouse.energy-use
The suggestions that the model displays are categorised. If you press Control-Space repeatedly, the suggestion window will cycle through the categories which it knows. You can see the displayed category in the bottom right of the suggestion window. For example, this list of suggestions is only showing keywords for keyword arguments; you can see this in the bottom right where it says "keywords":
3.4.2 Context help
The editor will show you help for anything that is written in a scenario.
If you hover the mouse over a word which you want help on, a tooltip will pop up describing that thing.
For example, here is what pops up when you hover over the aggregate
command:
If you hover the mouse over a keyword argument, the tooltip will tell you about what it means. If you hover over the left parenthesis of an unnamed argument, the tooltip will tell you about how that argument will be used.
3.4.3 Colouring in
The scenario editor colours in the text on the screen for you. This should make it easier to read. Different parts of the text are given different colours, which you can change if you like. Open the preferences:
And choose General
, Appearance
, Colors and Fonts
; there is a section in there called NHM Scenarios
, which lets you change the colours and shows you what they mean:
We find that the distinction between built-in commands and macros or templates is helpful; by default this is blue versus brown.
3.4.4 Automatic typing and useful keyboard shortcuts
Sometimes you cannot write everything you want by pressing Control-Space many times. In this case, the editor will try and help you to keep your parentheses correct, by typing or deleting some parentheses for you when you press certain keys. This can be confusing, and you can turn it off in the preferences if you want:
However, it is worth trying using the automatic typing for a little while as you may grow to like it. Here is a description of what the keys do, and how we would imagine you might use them:
- Adding and deleting parentheses
When you typing a parenthesis or delete a parenthesis, if the scenario is already balanced the editor will try and keep it balanced. If the scenario is not balanced, the editor will not try and do anything clever. In addition, if you use the selection to change more than one character at a time, say by selecting a region and deleting it or overwriting it, the editor will not do anything clever.
- When you type an opening parenthesis, the editor will insert the closing parenthesis for you, leaving the cursor in between the two, e.g:
given
blah blah |
, typing(
givesblah blah (|)
- When you type a closing parenthesis, the editor will move the cursor to the next closing parenthesis, rather than inserting one. This means that you can still type
()
to get()
. For example, given(blah | blah)
, typing)
will give(blah blah)|
- If there is only blank space between the insertion point and the next closing parenthesis, the blank space is deleted:
given
(blah| )
, typing)
will give(blah)|
. If this is causing particular confusion, consider the next part on moving parentheses around.
- If there is only blank space between the insertion point and the next closing parenthesis, the blank space is deleted:
given
- When you delete a parenthesis, its partner is deleted wherever it is in the text, e.g.
given,
(blah blah)|
, pressing backspace will giveblah blah|
. Because the insertion point is in the natural place you can still backspace over a lot of text. - If you select some text and type an opening parenthesis, a closing parenthesis will be added at the end of the selection, e.g.
given
something |blah blah| something
, typing(
givessomething (blah blah)| something
.
- When you type an opening parenthesis, the editor will insert the closing parenthesis for you, leaving the cursor in between the two, e.g:
given
- Moving parentheses around things you have already written; often if you are deleting or adding parentheses, what you are really wanting to do is to move something 'inside' an existing bracket or to put something out of a bracket. The editor has commands to do these two things for you
To move the next thing to be within the current bracket, you can press Control-Left Bracket (the key with
[
written on it, usually to the right of thep
key). For example: given(|) one (two three) four
, sequential presses of Control-Left Bracket will produce:(|one) (two three) four
(|one (two three)) four
(|one (two three) four)
This action 'absorbs' subsequent expressions into the current expression.
- The opposite action is to eject the last expression from the current expression. This is on the key Control-Right Bracket. For example:
given
(one (two three) four|)
, sequential presses of Control-Right Bracket will produce:(|one (two three)) four
(note the cursor is moved to the beginning of the current expression)(|one) (two three) four
(|) one (two three) four
- Moving the cursor around quickly
Often you want to move the cursor around the brackets quickly. Alt and the arrow keys are set up to do this.
- Alt-Up Arrow will move to the start of the current expression, or if already there to the start of containing expression, e.g.
given
(some (expression with things|))
, repeated presses will give:(some (|expression with things))
(|some (expression with things))
- Alt-Down Arrow will move to the start of the next expression within the current expression, e.g.:
given =(an | (expression (with) (sub-expressions))), repeated presses give:
(an (|expression (with) (sub-expressions)))
(an (expression (|with) (sub-expressions)))
.(an (expression (|with) (sub-expressions)))
- note because there is nothing within(with)
, nothing happens. We cannot go any further 'down'.
- Alt-Left Arrow will skip to the previous previous expression start or end at this level, or go up a level, e.g.
given
(a) (b) (c (d e)) (f)|
, repeated presses give:(a) (b) (c (d e)) |(f)
(a) (b) (c (d e))| (f)
(a) (b) |(c (d e)) (f)
(a) (b)| (c (d e)) (f)
(a) |(b) (c (d e)) (f)
(a)| (b) (c (d e)) (f)
|(a) (b) (c (d e)) (f)
- Alt-Right Arrow will skip the next expression start or end at this level, or go up a level. This is simply the reverse of Alt-Left Arrow.
- Alt-Up Arrow will move to the start of the current expression, or if already there to the start of containing expression, e.g.
given
You can get a summary of all these keys do by pressing F1
when in a scenario editor.
With a little learning, using combinations of these keys should let you do many kinds of edits much more quickly and without making errors.
For example, imagine you have some scenario code calculating a number, but you realize that you need to divide it by 1000.
Here is how things are to start with; the expression is as shown, and the cursor (|
) is before the expression.
(scenario ... (blah blah | (* 3.1415 (+ x y) in-use-factor)) )
Rather than worrying about precise cursor positioning, we can simply do the following:
Type a left parenthesis - we need more parentheses because we know we want another level in our expression. The editor types the right parenthesis for us:
(scenario ... (blah blah (|) (* 3.1415 (+ x y) in-use-factor)) )
Type the division symbol, as we are going to divide something:
(scenario ... (blah blah (/|) (* 3.1415 (+ x y) in-use-factor)) )
Absorb the next expression into the current expression, by typing Control-Left Bracket
(scenario ... (blah blah (/| (* 3.1415 (+ x y) in-use-factor))) )
Skip over the entire expression, so we can type the denominator, by typing Alt+Right Arrow
(scenario ... (blah blah (/ (* 3.1415 (+ x y) in-use-factor)|)) )
Press return, to make things nice and clear
(scenario ... (blah blah (/ (* 3.1415 (+ x y) in-use-factor) |)) )
- Type in the denominator
(scenario ... (blah blah (/ (* 3.1415 (+ x y) in-use-factor) 1000)) )
With practice, you may find that these commands allow you accurately edit scenario code with convenience and ease, and that many parenthesis-related woes are diminished. However, this may take some time to learn, and it is not for everyone.
3.4.5 Quick outline
In a long scenario you may find that scrolling up and down to find things becomes hard work.
Try pressing Control-o
(Control
and the o
key at the same time), to bring up the quick outline.
This shows the hierarchy of the scenario in a temporary window.
You can filter the quick outline by typing, select particular lines by pressing up and down, and you can press return to rapidly scroll to the selected line.
3.5 Navigating between files and definitions
If you have a lot of scenario files in your workspace, or a lot of template definitions, getting between them can be tiresome.
The application offers a key to jump quickly to a particular file, Control-Shift-R
(all three keys at once).
To find a specific template, you can press Control-Shift-T
.
3.6 Viewing files side-by-side
You have already seen how to open multiple editors, and switch between them by clicking on their tabs. You can also look at two (or more) editors at once if your screen is big enough, by dragging their tabs around on the screen. Try opening two editors, and then dragging the tab for one of them to the right hand side of the editor area. You should see a black rectangle showing where the editor will go if you let go, and when dragging to the right this should take up half of the editor area on the right. Let go, and you will have two editors open next to each other.
3.7 Viewing the top and bottom of a long file simultaneously
The above lets you open two files side-by-side or above each other.
Sometimes you want to open one file, but see two parts of it at once.
You can do this by typing Control-Shift-Minus
(Control-Underscore
), or Control-Left Brace
(Control-Shift-Left Bracket
), or looking in the Window
, Editor
menu.
3.8 Finding other actions
The application has all sorts of other useful functions, like search and replace across multiple files, local file history, team-working using git, comparisons between two files and so on. These functions can be hard to find, just like in any other program. Fortunately there is a quick way to search all the things the application knows how to do - the quick access box. This is the text box shown on the right hand side of the toolbar near the top of the window. Typing in here will pop up a list of all the commands, settings, help topics and so on that the application knows; for example, here is what happens if you type in "Run".
You can see a list of views that could be displayed on the screen, commands that can be taken in the current context, and preferences that you could change.
You can focus the quick access box by pressing Control-3
, so it is a quick way to do things for which you have forgotten the commands.
3.9 Accessing the manual
As well as context sensitive help within scenarios, the NHM application also provides the scenario language manual.
You can get to this by choosing Help Contents
in the Help
menu.
This will either open the help in a new window, or within the application, depending on your operating system.
In either case, the help will look something like this:
The "book" icons on the left are documentation for different parts of the application. You may have more such books than are shown in the screenshot, but you should have one entitled National Household Model. Clicking on this will show you the full manual for the latest version of the model that your application contains. This is where you can find the changelog, which will tell you about any recent changes to the model that you should be aware of.
The other documentation categories may tell you about
- Using EGit, the git client built into the NHM application
- The general principles for the application interface
3.10 Glossary
The NHM application is based on some software called Eclipse, which is what does the work of drawing windows on the screen, managing projects and so on. It has its own help section and terminology, and many of the menus and dialogs will refer to things using these words. Here is a quick list of a few of these words and what they mean:
- Workspace
- The workspace is a folder on your computer. The application looks in this folder for scenarios and projects. You can have more than one workspace, but you can only use one workspace at a time. It also holds special files containing any settings that you have changed in the application while using that workspace (for example, if you have changed the editor colours).
- Project
- The workspace contains projects. These are folders within the workspace folder which contain other files and folders. Every file in the application must belong to a project. They are the organising unit of work for the NHM application.
- Resource
- Sometimes the application will refer to "resources". This is its generic term for files, folders and projects.
- Workbench
- The main application window is called a "workbench window". You can open more than one workbench window by choosing
Window
,New Window
. Every workbench window has a menu bar at the top tool bar, and an editor area in the middle which may be surrounded by views. - Editor
- An editor is a part of the interface responsible for opening and maybe changing a file. The application contains different editors for different sorts of file, like the scenario editor for
.nhm
files. Editors appear in the editor area of a workbench window. - View
- A view is a part of the interface which displays some information but is not an editor. The
Project Explorer
is a view. Views are arranged into panels which surround the editor area. - Perspective
- A perspective is an arrangement of views and other icons in the interface. You will normally use the NHM perspective. This is indicated by the NHM button on the right hand end of the tool bar. You can use this to reset all the views to their default arrangement by right-clicking.
- Content Assist
- Content assist is the pop-up suggestions that appear when you press Control-Space.
- Platform
- The "platform" refers to the underlying machinery which makes the application work.
- Plugin / Feature
- A plugin is a bit of code which adds some behaviour to the application. A feature is a collection of plugins. For example, all the NHM related behaviour in the application is provided by a feature which is added to the bare minimum Eclipse needs to work. EGit is provided by another feature.
4 A simple scenario, by example
At this point you may be feeling like you have read a lot of background material, without getting on to anything concrete (or not, if you chose the option to skip the beginning so you could immediately do something). Without further ado, here is a complete, simple scenario which works and does something:
;; Text following a semicolon is a comment, which the model will ignore. ;; All scenarios start with the scenario command (scenario ;; these next are keyword arguments for the scenario ;; The scenario should start in 2015 start-date: 2015 ;; It should end in 2020 end-date: 2020 ;; This names a stock - the model will use this to find ;; the housing stock to use. Note that stocks are not built-in ;; to the model, nor is there any central repository of stocks ;; that it will know to refer to. In the standalone NHM, this just ;; refers to a stock file called my-houses.stock in the same folder ;; as the scenario file. stock-id: my-houses.stock ;; After the named arguments are the scenario's other arguments. ;; These usually either ;; a) Define things like variables or measures. ;; For example here we are defining a kind of wall insulation: (def-action ;; the first argument for def-action is a name my-insulation ;; the second argument is the definition of the measure, in this case ;; the measure we are using is measure.wall-insulation (measure.wall-insulation ;; capex: specifies the capital cost - here 100 + 50 * area capex: (+ 100 (* 50 (size.m2))) ;; the thickness of installed insulation thickness: 50 ;; the type of insulation to put in type: cavity ;; the thermal resistance resistance: 0.01)) ;; This definition didn't cause anything to happen - it's just telling the model ;; about the "my-insulation" measure we may want to use later. ;; b) set up things in the simulation which are "global", e.g. fuel costs ;; Here we set up a default tariff (one every house will be on at the start) (context.tariffs defaults:[ ;; this is the tariff (tariff ;; a tariff may cover several fuels, but this one ;; covers only one fuel (fuel type: MainsGas (charge (* 0.05 (house.meter-reading))))) ]) ;; c) cause things to happen at particular points in time. ;; on.dates is a command to do some things on particular dates (on.dates ;; its first argument defines the dates - regularly here stands in ;; for every year from scenario start date to end date. (regularly) ;; and then we tell it what we want to do on the dates: ;; here we are going to use the measure we defined on 5% ;; of detached houses (apply to: (sample 5% (filter (house.built-form-is Detached) (all-houses))) ;; the # refers to the my-insulation defined above #my-insulation) ;; and after that we will report on energy use (aggregate name:report (aggregate.mean name:energy (house.energy-use))) ) ;; d) instruct the model to turn on or off certain standard reports ;; here we turn on the transactions report (report.transactions))
We suggest you take this scenario, plug in a stock that you like, run it, and see what outputs you get!
4.1 Exercise: describe the scenario
Describe in detail what you think the example scenario above does when the model runs. Do not do this by guessing from what's written down; instead, look in the manual or use the context-sensitive help to read the documentation for each term that is used. Remember that the model is stupid, and only understands the scenario insofar as it is programmed to blindly perform certain calculations in response to it. Compare your answer to our description below.
4.2 Our description
This scenario simulates the following sequence of events:
- Loading the stock
- The stock is loaded from
my-houses.stock
, starting in 2015. This means that whatever houses whose build year is before or equal to 2015 will be considered to exist at the start of the run; houses built in subsequent years will be constructed at the start of their build year. - The scenario has no
quantum:
orweighting:
specified, which means it will take the defaults of400
andUniform
. As a result, the cases in the stock will be represented by simulated houses with weights of around 400, but most cases will probably produce houses with a slightly lower weight. Any cases whose weight is less than 400 will just be one house, with that weight. - There is a default tariff defined by
context.tariffs
; whenever a house is constructed it will get put on that tariff. Since nothing else speaks about tariffs, all the houses will be put on the tariff. - All other details are unspecified, and so fall to defaults; houses will be given a standard heating schedule, weather conditions, and so on. These defaults are described in an appendix to this document.
- The stock is loaded from
- Now we get to things which are going to be simulated. All of these things happen each year, from 2015 to 2020.
Firstly, we have the events caused by the
on.dates
rule; these will happen at the start of the year because:- The scenario dates are specified as years; these are shorthands for the first of January in each year
- The
on.dates
happensregularly
, which defaults to the anniversaries from the start date until the end date.
This rule says to do two things, an
apply
and anaggregate
. The apply will always happen before the aggregate in each year, as they are written in that order.The apply will first compute the set of houses defined by its
to:
argument. This is the job ofsample
, which will compute a subset of its second argument thus:sample
asksfilter
to produce a set of houses, which in turn asksall-houses
.all-houses
produces all the houses that currently exist, and gives them tofilter
;filter
looks at each one of those houses and asks(house.built-form-is Detached)
about them, keeping the ones where that is true. It gives these detached houses back tosample
.- now
sample
has all the detached houses, it picks a random subset having5%
of their weight, and gives this back toapply
Now
apply
has a set of houses to work on. It shuffles them into a random order (this doesn't matter for this scenario, but it could), and goes through them one at a time. For each house, it asks the measure named#my-insulation
to operate on the house if possible. The measure is defined up above theapply
, bydef-action
; you could just as well write the measure directly in theapply
The measure, when presented with a house, does the following:
- Checks if the house is suitable for the measure, according to the rules detailed in the manual. In this case, the measure is cavity wall insulation, so the house must have a wall without cavity insulation whose construction type indicates a cavity is present.
- If the house is not suitable, the measure fails, and the house is unchanged.
apply
does not care about this, and moves on to the next house. - If the house is suitable, the measure computes its size, and makes that available as
size.m2
; this is the sum of the area of all the suitable walls. - Then the measure computes the
capex:
rule; in this case fifty timessize.m2
, plus 100. Once thecapex:
rule has produced a result, the measure causes a transaction from the house to the market of that amount, so the house has paid for the measure.- Because we have
report.transactions
turns on at the bottom, this transaction will be recorded in the transaction log.
- Because we have
- Finally the measure changes the description of the house in some way; in this case, as it is an insulation measure, it goes to each suitable wall and:
- Marks the wall as having 50mm of cavity insulation
- Computes it a new u-value as \(u' = 1/(1/u + (0.01 \times 50))\); this is the resistance of the insulation (its unit resistance times thickness), plus the resistance of the existing wall (\(1/u\)), converted back to a u-value.
Once the measure has happened, the house's energy calculation result and so on may be different, as its thermal properties may have changed. Important things to note here are:
- The
to:
argument ofapply
did not know about the measure; it has blindly picked some houses, so whether or not they are suitable or have already had a measure before does not come into it unless you ask for it to. So, if you ask to apply a measure to a sample containing 10,000 houses, unless you have already made sure they are suitable, you may not install 10,000 measures. - The way the sample is written is important; 5% of detached houses is not the same as those houses which happen to be detached from a 5% sample of all the houses.
Now that
apply
is finished and probably changed some houses,on.dates
will tell theaggregate
to do its thing. When theon.dates
sets theaggregate
off, it will go through each of itsover:
sets and ask them to compute themselves. In this case, it will always beall-houses
(since we are using the default), which is just all the houses that exist at the point.Next, the aggregate will ask each of the aggregation rules it contains to compute a value for each of these sets. In this case, we have
aggregate.mean
;aggregate
will presentaggregate.mean
with the set of houses produced byall-houses
.aggregate.mean
will then go to each house in that set and present it tohouse.energy-use
, which will produce the energy calculator's estimate of energy use.aggregate.mean
takes all of these values and computes a weighted average of them, which is what it gives back to theaggregate
.Finally the aggregate will write down that value against the simulation's current date and the name of the set, so in this case a row will be produced in a report for group
all-houses
containing the current mean energy use.
Along with the explicit
on.dates
rule written in the scenario, there are two automatic rules which happen at the end of every year: houses pay their opex costs for particular technologies they have installed, and they pay their fuel bills according to their current tariffs. In our scenario there are no opex costs set up, so we can ignore that. However, we do have a tariff, so:For each house, at the end of the year, the tariffs are computed. All our houses are on a single tariff, which charges 5p per kWh of mains gas used. When the tariff is computed, all the charges it contains are calculated, and these produce transactions according to the relevant amounts.
In our scenario, these will be visible only in the transaction log report, which we have enabled.
- The simulation finishes on 01/01/2020; as a result we will see the final year's
on.dates
, but not the transactions for the tariffs which happen at the end of the year.
This is a detailed description of what happens, but it does not say much about what we would expect to see. Let us use our imaginations to guess about the effects we will observe in the outputs. For this we need to make up some statistics about our stock; let's say that there are 6 million detached houses, of which 3 million have uninsulated cavity walls. Let's also say that 80% of these houses are on gas and use gas as their heating fuel.
In the first year, we will sample 5% of the detached houses. This will give us 5% of 6 million, which is 300,000. These houses are as they were in the stock. Of these 300,000 we would expect roughly half to have an uninsulated cavity, by coincidence. However, we could get anything from 100% to 0% uninsulated. The actual distribution can be had with some elementary statistics, but the important point here is that the model didn't do anything clever to guess our intention. It just did the sampling as stated.
Anyhow, taking the expectation, we will go through these houses and about 150,000 of them will get cavity walls; they will spend some money, which we will see in the transaction log. The remaining houses will be unaffected.
Next the model will make our first year aggregate; this will just contain a single number for overall mean energy use. Again, we get what we asked for; the model does not guess that we wanted change in energy use and produce a baseline, so we cannot see the impact of our first year's efforts. Later chapters will explain more about how to report on the effects of measures in detail, and how to schedule things to happen at different times.
Now we come to the second year; again, we compute a sample of 5% of detached houses. This will be (with very high probability) a different sample to the previous year, but it may well intersect. We would expect about 5% of 5% to be in the intersection in this first year. As a result of this, the proportion which are uninsulated will have gone down a bit - to put it another way we are sampling 5% from a population which now contains (3 million - 150,000) uninsulated cavities. Note that our sample can still in-principle have any proportion of uninsulated houses, as there are more than 300,000 of each sort left to take.
Later chapters will explain how to avoid re-sampling by marking houses or targetting only suitable houses. Anyhow, in our second year we will in expectation insulate slightly fewer houses, and so we will see slightly fewer transactions.
The second year report will show some reduction in energy use; however, it will be quite small, as we are looking at the populatin total. Later chapters will explain how this report could summarise particular subpopulations so as to be more informative.
The end-of-year fuel cost transactions in the second year will also be slightly lower, for those houses which (a) have been insulated and (b) were gas heated.
In the third year, we will have insulated some more houses (although not quite as many in the second year), so our sample will be expected to contain yet fewer uninsulated houses; as a result the expected reduction in energy use and bills in the third year will be a bit lower than in the second.
This process will continue for subsequent years, with the energy use and expenditure asymptotically approaching the values expected for full insulation of cavities in detached properties. The expected shape of the curve is an exponential decay, as each time we shrink the population we are sampling from by a fixed proportion.
4.3 Exercise: amend the scenario
As you have hopefully discovered, this scenario does some things you might not want to do:
- It doesn't report on the direct effects of the measure being installed
- It doesn't avoid re-sampling houses that have been affected already
Try making some changes to get a better understanding of this (you may want to return to these questions after reading later chapters):
- Amend the scenario to report on more details
- Population level details about:
- How many houses are suitable for the measure (
house.is-suitable-for
) - How many houses are likely to be in the target set for the apply (
aggregate.sum
,aggregate over:
,house.is-suitable-for
,filter
,house.built-form-is
) - How fuel costs are affected for these houses as well as energy use (
house.fuel-cost
)
- How many houses are suitable for the measure (
- House level details about the houses targetted (
probe
) - Summaries of the specific target set (
probe.aggregate
)
- Population level details about:
- Amend the scenario to avoid re-sampling; there are two different ways to do this which mean different things:
- Using
house.flags-match
andupdate-flags:
to avoid affected houses - Using
house.is-suitable-for
to target only suitable houses in theapply
- Using
5 Making things happen
Now that you (hopefully) have a feeling for what the model is, this part will show you how to make it do simple tasks like producing reports and making basic changes to the state of the stock.
5.1 When do things happen; what dates, how ties are broken
As mentioned above, when the model runs your scenario it does the following:
- Sets the date to the
start-date:
of the scenario - Schedules things that are going to happen in the future
- Does the scheduled things in date order, until the
end-date:
of the scenario is reached
A small number of things are scheduled automatically:
- Loading the stock; this is always the first thing that happens
- Paying fuel bills and operational costs; this always happens annually, at the very end of the year. There is more information about how bills and operational costs are determined in the sections on Money and Tariffs.
Everything else that happens must be scheduled to happen by something written in your scenario. The simplest tool for scheduling is a command called on.dates
. If you look it up in the manual, you will see that it belongs at the top level in your scenario (i.e. as an argument to the scenario
command), and that it expects:
- An argument containing a list of dates, which say when something is to happen
- A series of arguments which define things that should happen
You can think of on.dates
as a device for attaching some free-floating things to do to some specific dates, so that the things will be done on those dates.
For example, you might write:
(scenario start-date: 01/01/2014 end-date: 01/01/2020 stock-id: my-stock (on.dates [01/01/2014 01/06/2014 01/01/2018] (insulate-some-houses) (make-a-report)))
This scenario will schedule the following:
- On 01/01/2014
- Load the stock
- Do the command
insulate-some-houses
(this is not a real command) - Do the command
make-a-report
(also not real)
- On 01/06/2014
- Do the command
insulate-some-houses
again - Do the command
make-a-report
again
- Do the command
- On 31/12/2014, 31/12/2015, 31/12/2016, 31/12/2017
- Pay fuel bills and operational costs; however, the default behaviour is that fuel and operational costs are zero, so effectively this does nothing
- On 01/01/2018
- Do the command
insulate-some-houses
for a third time - Do the command
make-a-report
for a third time
- Do the command
- On 31/12/2018, 31/12/2019
- Pay fuel bills and operational costs
- On 01/01/2020 the simulation finishes
If you want different things to happen on particular dates, you can put more than one on.dates
in a scenario. The scheduled dates are interleaved. Any ties are broken by the order in the scenario. For example,
(scenario start-date: 01/01/2014 end-date: 01/01/2020 stock-id: my-stock (on.dates [01/01/2015 01/01/2010] (do-thing-A)) (on.dates [01/01/2014 01/01/2010] (do-thing-B)))
Sets up the following schedule (ignoring fuel bills etc.):
- On 01/01/2014
- do
do-thing-B
(a made-up command)
- do
- On 01/01/2015
- do
do-thing-A
(also made-up)
- do
- On 01/01/2010
- do
do-thing-A
again - do
do-thing-B
again; this comes second because it is second in the scenario
- do
The reason questions of scheduling are important is the same as why scheduling is important in the real world: the things which have already happened determine how things are now, so doing A then B may not have the same result as doing B then A. If B is producing information in a report and A installs some measures, A then B will show you the outcome of doing A, whereas B then A will show you how things were before doing A, and then install the measures, without telling you anything about it (doing B then A then B again will show you before and after, of course).
5.1.1 Other sorts of dates
Alone with writing literal dates in on.dates
, you can also use the commands scenario-start
, scenario-end
, and regularly
to produce the scenario start date, end date, or a repeating series of dates. For example, if you write
(on.dates (regularly) (do-some-thing))
in your scenario, you will schedule do-some-thing
to happen annually from the start date of the scenario; you can provide arguments from:
, every:
and until:
to regularly
to change the dates it produces. For example, (regularly from:01/01/2020 every:"3 months" until:01/01/2030)
will produce quarterly dates from 01/01/2020 until 01/01/2030 (inclusive).
You can also write just the year, like 2015
as a shorthand for 01/01/2015
, and you can write two dates joined by two dots to represent an annual sequence starting on the first up to the second.
In this form, 2015..2020
is a shorthand for (regularly from:01/01/2015 until:01/01/2020 every:"1 year")
.
5.2 Making reports; making changes to the housing stock
Now we come to what sort of thing can you put in on.dates
to make something happen; there are a few options, but the most useful one is probably apply
. The manual for apply
will tell you that it expects two arguments:
- An action to apply as the first argument
- An optional named argument
to:
which is a command to select a set of houses
apply
can be thought of as a device for gluing a free-floating thing that you want to do to a population of houses. Putting this together with on.dates
you can create an instruction to do a given thing, to a given population, on some particular dates:
(scenario ; start date etc. omitted (on.dates (scenario-start) (apply (some-action) to: (some-houses))))
A lot more detail is given on what kind of thing you can use in place of (some-action)
or (some-houses)
later on. For now, let's omit to: (some-houses)
entirely (the manual explains that the default for to:
is (all-houses)
, which unsuprisingly represents all the houses in the model), and focus on (some-action)
.
An action is the broad term for something in the model which does something to a house; this includes
measures, which are actions that represent an installation of some technology in a house. Examples include:
measure.loft-insulation
andmeasure.wall-insulation
, which would insulate the loft or walls respectivelymeasure.standard-boiler
andmeasure.heat-pump
, which would install a new boiler or a heat pump
As you can see, most measure commands begin with
measure.
; this has no special meaning to the model, it's just to make the names distinctive.- Actions which put other actions together; for example you might want to install a package of several measures together, or choose between some alternatives
- Actions which output information about the condition of a house, which are covered in the next section.
5.3 Targetting who things happen to; selecting houses with tests or at random
As mentioned, by default apply
will operate on all the houses in the model. You can also use the commands filter
, union
, sample
, and bernoulli
to select subsets of the population.
filter
is a command which will produce a set of houses by using another logical test (true-or-false) command on each house, and only keeping those houses which pass the test. For example
(apply (some-action) to: (filter (house.has-loft) (all-houses)))
will apply some-action
only to those houses which are (a) in all-houses
(i.e. all the houses) and (b) for which house.has-loft
is true. house.has-loft
is an example of a boolean function or test. You can think of what happens here as:
- something triggers the
apply
(probablyon.dates
) apply
asks(filter ...)
to produce a set of houses to act on(filter ...)
asks(all-houses)
for a set of houses, getting back all the housesfilter
goes through each house in turn, and askshouse.has-loft
about that housefilter
responds toapply
with the houses which passed the test
apply
shuffles the houses that were chosen into a random orderapply
goes through each chosen house in turn, and asks(some-action)
to operate on that house
If you are looking for a particular number of houses, but you don't mind exactly which ones, you can use sample
; it will sample randomly from a set of houses to produce a result of a certain size. For example
(apply (some-action) to: (sample 15% (all-houses)))
will take the houses in all-houses
, pick a random subset of them whose size is 15% of the total, and then apply some-action
to them in a random order. sample
will interpret samples of size less than 1 as proportions (15% is equivalent to 0.15, as far as the model is concerned), and samples of size 1 or more as counts, so
(apply (some-action) to: (sample 1000000 (all-houses)))
will take the houses in all-houses
, pick a random subset of them whose size is 1 million, and then apply some-action
them in a random order.
You can combine sample
with filter
by using one as an argument to the other; for example
(sample 10% (filter (house.has-loft) (all-houses)))
produces a random 10% of those houses which have a loft. This is not the same as
(filter (house.has-loft) (sample 10% (all-houses)))
which instead takes a random 10% of all the houses, and then retains only those which happened to have a loft; the size of the second of these will probably be smaller than the first. When reading this, you can see that the command with the deepest number of brackets is worked out first; this is always true for the sampling commands (their precedence is as though for arithmetic).
If you want to sample houses at random using a per-house probability, you can use the bernoulli
command. It is like a probabilistic version of filter
. Where filter
expects a logical function about a house and uses it to deterministically rule the house in or out, bernoulli
takes a numerical function about a house, and uses it to compute a probability that the house will be included or excluded. For example
(apply (some-action) to: (bernoulli 10% (all-houses)))
does the following:
- again,
on.dates
will trigger theapply
on some dates on.dates
asksbernoulli
for some housesbernoulli
asksall-houses
for some houses, getting all the houses- for each of these houses in turn,
bernoulli
produces a uniform random number from 0 to 1; if the number is less than or equal to 0.1, the house is included, otherwise it is excluded bernoulli
returns all included houses to theapply
apply
takes each included house in turn, in a random order, and askssome-action
to act on the house
The difference between bernoulli
with 10% and sample
with 10% is that bernoulli
may pick more or less than 10% - it could pick 100% or 0%, if the random numbers came up right. In contrast, sample
will always pick 10%3.
5.4 Things which happen by themselves; carbon factors, weather, opex and bills
Along with things scheduled using on.dates
, there are a few things which are scheduled 'automatically' by the model. We have already touched on opex and bills, which are processed at the very end of each year. The scenario can also contain 'global' carbon factors and weather, which can be functions of time. If one of these is a function of time, the simulation will contain some automatically included events when the carbon factors or weather would change.
5.5 Exercises about scheduling events
Referring to the manual and using your knowledge of energy modelling, what are the significant differences likely to be between the following measure applications?
Naive:
(apply (measure.standard-boiler size: (size (house.peak-load)) capex: (* 100 (size.kw))) (measure.wall-insulation capex: 500 type: cavity resistance: 0.01 thickness: 50))
Less naive:
(apply (measure.wall-insulation capex: 500 type: cavity resistance: 0.01 thickness: 50) (measure.standard-boiler size: (size (house.peak-load)) capex: (* 100 (size.kw))))
Package:
(apply (do all:true (measure.wall-insulation capex: 500 type: cavity resistance: 0.01 thickness: 50) (measure.standard-boiler size: (size (house.peak-load)) capex: (* 100 (size.kw)))))
Hints:
- Higher up things tend to happen first
- Things which happen first can affect what comes after, but not vice-versa
- Packages affect suitability
Consider the following partial scenario, in which
A
,B
,C
,D
,E
,F
, andG
represent things that are to happen (e.g. application of measures, running reports etc.)(scenario start-date: 2005 end-date: 2011 (on.dates scenario-start A) (on.dates scenario-end B) (on.dates scenario-start C) (on.dates repeatedly E) (on.dates 1990..2015 F) (on.dates [2005 01/05/2010 02/01/2011] G))
In what order and on what dates are
A
toF
performed; note that some of them may happen more than once and some may happen on the same date but still have an order.
5.5.1 Answers:
There are two things going on here:
- Whether the insulation or boiler measure happens first (1 versus 2 and 3)
- Whether the measures are combined into a package or not (3 versus 1 and 2)
Installing the insulation first in this case will probably make the boilers cost less. This is because:
- The boiler's
capex
depends onsize.kw
(it's £100/kW) The boiler's size depends on
house.peak-load
. Reading the manual forhouse.peak-load
:This is determined using Newton's law of cooling, so peak load = temperature difference * specific heat loss.
Our knowledge of energy modelling tells us that if we install insulation we will reduce the specific heat loss, so if insulation goes in first the boiler will be smaller. On the other hand, the cost of insulation is only affected by the insulated area, so if the boiler goes in first, it is sized for the bigger heat loss, and then the insulation goes in exactly as before.
So, if the same houses were exposed to 1, 2, and 3 in separate scenarios, the results for 2 and 3 would have lower boiler costs.
The second difference is between 1 and 2, and 3; 1 and 2 are supplying two separate measures to the
apply
command, one after another. The documentation forapply
says about its unnamed arguments:the actions to apply; the first action will be applied to each house in turn, then the second will be applied to each house in turn, and so on.
So in the first two cases, if the houses are house A, B, … Z, we will be doing the following:
For case 1: go to house A and try to install a boiler, then go to house B and try to install a boiler, … then go to house Z and try to install a boiler, then go back to house A and try to install insulation, then go to house B and try to install insulation, and so on.
For case 2: go to house A and try to install insulation, then go to house B and try to install insulation, … then go to house Z and try to install insulation, then go back to house A and try to install a boiler, then go to house B and try to install a boiler, and so on.
Case 3 is quite different; we are using the
do
action, with the two measures inside it to create a package.
The commands happen as follows:
Date Command 01/01/2005 A,B,C,E,F,G 01/01/2006 E,F 01/01/2007 E,F 01/01/2008 E,F 01/01/2009 E,F 01/01/2010 E,F 01/05/2010 G 01/01/2011 E, F The commands happen left-to-right top-to-bottom in the table (so it goes A,B,C,E,F,G,E,F,E,F…)
Points to note:
- Nothing which lies outside the start and end dates happens
- Things which are on different dates happen in order
- Things on the same date happen according to their order in the scenario.
5.6 Exercises about sampling houses
- For each of these sampling commands, give a description of the sample that is produced which states the range of sizes of the sample, and anything else you may know about the houses in the sample.
(sample 50%)
(bernoulli 50%)
(sample 50% (filter (house.region-is London)))
(filter (house.region-is London) (sample 50%))
(filter (house.built-form-is Detached) (filter (house.region-is London)))
(union (filter (house.region-is London)) (filter (house.built-form-is Detached)))
- Write a sampling command to produce:
- The set of all the houses
- A random 5% of all the houses
- A random 100,000 houses
- All the houses in the South West
- 20% of the houses in the South West
- 20% of the houses in the South West and 10% of the houses in London
- 30% of the houses in the South West and in London
- A set in which each house has a 5% chance of membership
- A set in which a house's chance of membership is given by its floor area, clamped to 100.
- All the detached houses
- All the detached houses in the south west
- A set of detached houses, taken from a 10% sample of all the houses
- 10% of the detached hosues
5.6.1 Answers
- These commands produce:
- A sample containing 50% of all the houses that exist. Its size will (to within 1 quantum) be exactly 1/2 of the total weighted count.
- A sample containing somewhere between zero and all of the houses. Its size will, in expectation, be 1/2 of the total weighted count. Each case has a 50/50 chance of being in the set.
- A sample containing 50% of all the houses that are in London. Its size will (to within 1 quantum) be exactly 1/2 of the total weighted count in London. All the houses in it will be in London.
- A sample containing only the houses which turned out to be in London when a sample of 50% of all the houses was taken. All the houses in it will be in London. Its size will range from the size of London minus half the total population (or zero, if London is smaller than half the total population), up to whichever is less between the size of London and half the total population.
- A set containing all the houses which are both in London and are Detached.
- A set containing all the houses which are in London and all the houses which are Detached. It will contain houses which are not detached, and houses which are not in London, but every house in it will be either in London, detached, or both.
- You could produce these outputs with
(all-houses)
(sample 5% (all-houses))
, or equivalently just(sample 5%)
(sample 100000 (all-houses))
or(sample 100000)
(filter (house.region-is SouthWest) (all-houses))
or just(filter (house.region-is SouthWest))
(sample 20% (filter (house.region-is SouthWest)))
, reads as "sample 20% of [filter [houses in the south west] from all the houses]"(union (sample 20% (filter (house.region-is SouthWest))) (sample 10% (filter (house.region-is London))))
(sample 30% (union (filter (house.region-is SouthWest)) (filter (house.region-is London))))
(bernoulli 5%)
(bernoulli (min 100% (* 100% house.floor-area)))
(filter (house.built-form-is detached))
(filter (house.built-form-is detached) (filter (house.region-is southwest)))
, or(filter (all (house.built-form-is detached) (house.region-is southwest)))
(filter (house.built-form-is detached) (sample 10%))
(sample 10% (house.built-form-is detached))
6 Getting information out
We have now covered the idea that the simulation contains a house-centric model of the world, and that you can apply
different actions and measures to sets of houses to change that
We have not however said much about how to find out the consequences of the changes which we are simulating.
This is a significant omission, as all programs without some kind of output are equivalent; the whole reason for having a program is to get some output!
Outputs from the NHM are usually referred to as reports; normally they are tables, which the model stores in 'tab-separated values' format.
These are contained in files ending with the .tab
extension, and are simply plain text files in which each line represents a row of the table, and within each row the columns are delimited by 'tab' characters12.
These tab-separated files can be read using just about any tool, including Microsoft's Excel, R, SPSS, SAS and many others.
To persuade the NHM to produce a report, you need to include some commands in your scenario which instruct the simulator accordingly. It is not possible to extract any more information from a simulation than you requested after the fact without changing the scenario and re-running it, so if your scenario takes a long time to run take a moment to be sure you have the outputs you need before setting it in motion.
6.1 An overview of the reporting commands
There are four commands for producing reports which you are most likely to use:
aggregate
, a command which outputs summary information about populations of houses. This is useful for keeping track of high-level trends in your scenario.probe
, a command which outputs information about individual houses. This is useful for generating detailed data which you can process using other tools, or for digging into what is happening when the model produces confusing results.probe.aggregate
, a command which produces summary information about the impact of a specific action on a population of houses. It is the summarising analog forprobe
.def-report
and thereport:
keyword, which are relatively new commands that combine the features ofaggregate
andprobe
.
We will consider these in order
6.2 Using aggregate
to produce summaries of the stock
The aggregate
command is analogous to apply
in that it should be used within on.dates
.
Where apply
uses a sampling command to find a population of houses, and then does a measure to them whenever the surrounding on.dates
tells it to, aggregate
uses some sampling commands to find some populations and then uses some aggregation commands to work out summary statistics on those populations when the surrounding on.dates
tells it to.
The general form of aggregate
is:
(aggregate name: <<the name of the report file to produce>> over: [ <<a list of sampling commands>> ] divide-by: [ <<a list of cutting variables>> ] << a series of aggregations, like aggregate.count, aggregate.mean, etc. >> )
When an aggregate is triggered, it does the following:
- Compute the populations indicated by
over:
- For each population, for each dwelling in the population, work out the
divide-by:
values - For each sub-population induced by
divide-by:
values:- For each aggregation contained in the aggregate, work out that aggregation and record it in the result against the current date, the
over:
population and thedivide-by:
values.
- For each aggregation contained in the aggregate, work out that aggregation and record it in the result against the current date, the
For example, here is a very simple aggregate:
(aggregate name:example-1 over: [(all-houses)] divide-by: [] (aggregate.count name:count))
Let us say this is triggered at the start of the run on 01/01/2020; what happens is:
- The set
(all-houses)
is computed, which is all the houses - The
divide-by:
values are used to cut upall-houses
; there are none, so it is not divided - The resulting set, which is just all houses, is presented to
aggregate.count
, which produces the total weight This value is written down as a row in the report
example-1
, as follows:Date Group Count 01/01/2020 (all-houses) 24000000
If the on.dates
was set to trigger on several dates, each date would add a row to the table, producing something like:
Date | Group | Count |
---|---|---|
01/01/2020 | (all-houses) | 24000000 |
01/01/2021 | (all-houses) | 25000000 |
… |
6.2.1 Adding more groups
Let us add another group to the aggregate:
(aggregate name:example-2 over: [(all-houses) (filter (house.region-is London))] divide-by: [] (aggregate.count name:count))
The result is:
- The set
(all-houses)
is computed, which is all the houses- The
divide-by:
values are used to cut upall-houses
; there are none, so it is not divided - The resulting set, which is just all houses, is presented to
aggregate.count
, which produces the total weight - This value is written down as a row in the report
example-1
- The
The set
(filter (house.region-is London))
is computed, and the same process repeated; the final table now has two rows:Date Group Count 01/01/2020 (all-houses) 24000000 01/01/2020 (filter (house.region-is London)) 10000000
Note here that (all-houses)
and (filter (house.region-is London))
are intersecting, so the same houses are being represented in multiple rows.
The model will not guess if you mean to say "houses in london versus other houses"; you would have to define those two sets or use divide-by:
.
6.2.2 Using divide-by:
The divide-by:
argument effectively produces more groups, but in a convenient syntax.
It breaks each of the groups from the over:
argument down by a series of values about each house.
For example, we could amend the first example above to be divided up by built form:
(aggregate name:example-3 over: [(all-houses)] divide-by: [(house.built-form)] (aggregate.count name:count))
Now when triggered the aggregate does this:
- Compute
(all-houses)
, producing a set of all the houses - For each set produced (here just
all-houses
), work out(house.built-form)
, and break the set into subsets according to its distinct values. Built form is a categorical variable with the valuesMidTerrace
,EndTerrace
,SemiDetached
,Detached
,Bungalow
,ConvertedFlat
,PurposeBuiltLowRiseFlat
andPurposeBuiltHighRiseFlat
, so we will have a subset for each category that has some representatives. For each of these subsets, each aggregation is computed. In this case, that is just the count; we end up with the following table:
Date Group house.built-form count 01/01/2020 (all-houses) MidTerrace … 01/01/2020 (all-houses) EndTerrace … 01/01/2020 (all-houses) SemiDetached … … 01/01/2020 (all-houses) PurposeBuiltLowRiseFlat …
If we add more than one divide-by:
variable, we will have a further breakdown.
For example, we could add the region as well:
(aggregate name:example-4 over: [(all-houses)] divide-by: [(house.built-form) (house.region)] (aggregate.count name:count))
Now (all-houses)
will be divided up according to all the extant combinations of (house.built-form)
and (house.region)
, and a count produced for each distinct combination.
Unlike in the second example where we specified two groups in the over:
argument, for a given group the different divide-by:
rows are always mutually exclusive, because they vary in at least one of the dividing values.
6.2.3 Different kinds of aggregation
In the examples so far we have seen aggregate.count
.
This is a command which knows how to produce a number when another part of the model, like aggregate
, presents it with a set of houses; it produces the weighted count.
The sets of houses used are provided implicitly by the context in which aggregate.count
is used.
There are other commands which produce a number from a set of houses:
aggregate.mean
, which works out the weighted mean of another function For example:(aggregate.mean (house.total-floor-area))
works out the mean of the floor area when it is used on a set of houses
aggregate.sum
, which works out the weighted sum of another function For example:(aggregate.sum 1)
is the same asaggregate.count
. It goes to each house in the set, calculates the number1
, multiplies it by the weight, and sums the values.(aggregate.sum (house.energy-use))
adds up the weighted total energy use for all the houses in the set.
aggregate.n-tile
, which works out a weighted n-tile (like the median) of another function For example(aggregate.n-tile n: 0.5 (house.energy-use))
works out the median energy use for all the houses in the set
aggregate.where
, which works out another aggregate but only on houses which pass a given test For example(aggregate.where (house.region-is London) (aggregate.mean (house.energy-use)))
will work out the mean energy use for all the houses in the set which are in London. You may have noticed how this looks a bit like using one of the sampling commands on another one -aggregate.where
contains anaggregate.mean
, and modifies its input.
6.2.4 Using aggregate
effectively
Remember the rule for scheduling that we learned above! If an aggregate
command is triggered on the same date as an apply
command which changes some houses, in the rows for that date it will either (a) see the results, because it is written after the apply
, or (b) not see the results, because it is written before the apply
.
If you want your aggregate to always see the results of changes that are happening every year, you can trigger it before and after the changes in each year.
For example, in some code like this:
(on.dates [ 01/01/2015..01/01/2020 02/01/2015..02/01/2020 ] (aggregate name: report-which-sees-all ... )) (on.dates [2015 2016 2018] (apply ...))
The aggregate
will run on the first and second of January every year, and the apply
on the first of January 2015, 2016 and 2018.
Since the events on the first are tied, the tie will be broken by position and the aggregate will happen first.
As a result, the rows on the first will show how things started out in the year, and the rows on the second how they were affected.
However, if you really want to determine the effect of your actions, there are better ways - aggregate
is most useful for producing simple counts or sums that you can look at, for example to count up how many measures were installed in a given year.
6.3 Using probe
and probe.aggregate
to see what has happened
As we just saw, using aggregate
to determine exactly what is going on in a scenario can be quite difficult, and it is not intended for producing results that go down to house level13.
To help with this, the model offers the probe
command.
We have already seen the pattern of putting a command around another command of the same kind to change its behaviour with sample
, filter
and aggregate.where
; probe
follows this pattern, but for an action that you are applying to a house.
Putting a probe
around another action changes the action's behaviour so that it also makes a report which describes what happened whenever the action gets used.
For example, imagine we are installing a new boiler in some houses; the command for this is measure.standard-boiler
, so our scenario might have something like this in it:
... (on.dates ... (apply to: (sample 100000) (measure.standard-boiler fuel:MainsGas winter-efficiency: 85% update-flags: affected)) (aggregate name: heating-systems divide-by: [house.main-heating-system-type house.flags] (aggregate.count) (aggregate.sum (house.energy-use)))) ...
This example uses flags, which are described later on; here it suffices to say that when a house get a boiler it will also be "marked" as affected
, and that mark will appear in house.flags
which is being used to divide up the population in the aggregate.
We are hoping to install 100,000 boilers every year, and expecting that the overall energy use will go down by some amount. However, when we run the scenario, we have two problems:
- We don't seem to be installing 100,000 boilers every year, and
- The effect on energy use is not as much as we thought.
It seems that what we have told the model to do is not quite what we intended!
One way to diagnose this kind of problem is to use probe
; as described it goes around an existing measure or action.
In this scenario there is only one measure, which is the boiler, so we amend the example to include a probe; probes have the following form:
(probe name:<<name of report>> capture: [ << list of values to record>> ] <<another measure or action>> )
so adding the probe around our existing measure we get:
... (on.dates ... (apply to: (sample 100000) (probe name:boiler-investigation capture: [ (house.main-heating-system-type name:h) (house.flags name:f) (house.heating-efficiency measurement: winter of: PrimarySpaceHeating name: e) ] (measure.standard-boiler fuel:MainsGas winter-efficiency: 85% update-flags: affected))) (aggregate name: heating-systems divide-by: [house.main-heating-system-type house.flags] (aggregate.count) (aggregate.sum (house.energy-use)))) ...
You can see that the probe surrounds the measure, occupying the space it used to occupy; from the point of view of the other elements of the scenario, the probe
is kind-of "transparent", in that the meaning of the simulation itself is unchanged.
However, from our point of view the simulation is changed, in that we will have a new output file called dwelling/probe-boiler-investigation.tab
, which will help us in our diagnosis.
If you write a scenario like this now and run it, you will see that file contains some output of this form (some columns omitted for space reasons):
Date | dwelling-id | weight | h (before) | f (before) | e (before) | suitable | h (after) | f (after) | e (after) |
---|---|---|---|---|---|---|---|---|---|
2015 | 1234 | 500 | StandardBoiler | 0.8 | true | Condensing | affected | 0.85 | |
2015 | 5678 | 400 | StorageHeater | 1 | false | n/a | n/a | n/a | |
… | |||||||||
2016 | 1234 | 500 | Condensing | affected | 0.85 | true | Condensing | affected | 0.85 |
Each row in this table corresponds to a moment in the simulation when a particular simulated dwelling was sent to the probe
by the apply
as a result of the on.dates
triggering it.
You can see that every row contains the date at which the change is happening, a sequence number, the unique ID of the dwelling which is being considered, and the weight of that dwelling so that you can make weighted calculations if you need to.
As well as these values, each row also contains (1) the values given to capture:
about the house before the measure within the probe got applied, (2) an indication of whether the measure succeeded or not, and (3) when the measure did succeed columns providing the capture:
values after the measure's application.
In the example table above, you can see one dwelling was presented and got a measure (dwelling 1234), and another was presented and was not suitable (5678).
This report can therefore show us exactly which houses were considered, and how (if) they were affected.
In our hypothetical conundrum we might analyse the table and notice the following:
- On each chosen date, around 100,000 dwellings were presented to the probe; the sum of the weight column would be 100,000 when grouped by the date column
- However, under 100,000 dwellings had a successful measure installation. This explains why the aggregate contained a lesser effect than we expected.
- We might then filter the results to show only the cases where the measure did not install, and look for regularities. We might see that most of these houses have electric heaters of some sort; perhaps they are off the gas grid, and so not eligible for a gas boiler?
- Finally, we would notice that in later years we are operating on some houses which already have an 85% efficient mains gas boiler, and for which
house.flags
includesaffected
. This would remind us that thesample
command does not automatically understand that we want non-overlapping samples between years.
6.3.1 probe.aggregate
There is an analog to probe
which probes the aggregate effect of an action; it is like the aggregate
command combined with the probe
command.
For example, you can write
(apply (probe.aggregate name:aggregate-effect capture: [(aggregate.mean house.energy-use)] (measure.wall-insulation ...) ))
To see the before-and-after effect on mean energy use resulting from applying the wall insulation measure. The results will be broken down according to whether the measure succeeded.
As with the aggregate
command, you can also provide probe.aggregate
with a divide-by:
argument to cut the results according to different aspects of the house.
6.4 Using def-report
and the report:
argument
probe
and aggregate
were both added fairly early in the development of the model, and are a quick and efficient way to inspect bits of a scenario.
However, if you find that you want to probe a lot of actions, and also want some summary statistics produced for the same outputs, they can become quite tiresome to write down.
The def-report
, which was added more recently, gives an alternative way to report on houses.
You use it by defining a report in one place (in the top level of the scenario), and then by sending houses to that report from other places where you want to know what is going on.
Defining a report is a matter of using the def-report
command, which has the general form:
(def-report <<name of report>> (column ...) (column ...) (cut ...) (cut ...))
So for example, if we wanted a report which told us about the built form and energy use of houses we might write
(def-report example-report (column name: built-form value: house.built-form) (column name: energy-use value: house.energy-use))
Each column
here has a name:
, which is the text used to name the column, and then a value
which is a command used to compute the value for that column when a house is sent to the report.
The easiest way to send a house to the report is to use the send-to-report
action, for example:
(on.dates (scenario-start) (apply to: (all-houses) (send-to-report from:this-place #example-report)))
This is all similar to what you have seen before, except for the octothorpe (#
) symbol written before example-report
.
It is used to indicate that the word example-report
is a cross-reference to the report defined by the def-report
we wrote above.
In the NHM application's scenario editor, these cross-references will be coloured in differently to make them stand out.
Since send-to-report
is an action, we can put it inside apply
like any other action, and it gets used on all the houses in the to:
set; in this example, all the houses that exist.
Each time a house is sent to a report, two things happen:
- It is treated as though it were observed by a
probe
command; in this case, a row will be entered into a file calleddwelling/probe-example-report.tab
which contains the built form and energy use for the house. In addition it will contain thefrom:
argument forsend-to-report
, so that you can send houses to a report from several places and determine which rows came from where. - The values observed are summarised, and when the simulation date advances they may be contributed an aggregate summary of all the houses which were sent to the report since the last time the date changed. In our example, the summary will contain only the date, the
from:
column and a weighted count. However, you can add more summary statistics and aggregations by using thesummary:
argument of column and by adding thecut
command into thedef-report
6.4.1 Column summaries
Each column can have multiple summary values computed for it using the summary:
argument.
Let us amend our example report to use this facility
(def-report example-report (column name: built-form value: house.built-form summary: [(in detached semidetached)]) (column name: energy-use value: house.energy-use summary: [min max mean]))
This report will now generate the following summary statistics:
- the count of houses where built form is either
detached
orsemidetached
- the minimum, maximum and mean values for energy use
Available summaries are sum, mean, variance, min, max and n-tile (for numbers), count (for logical values), and in (for categorical values). The language reference describes these in more detail.
6.4.2 Cuts
Cuts are a way of replicating the divide-by:
argument in the aggregate
command.
Each cut in a def-report
produces another aggregate output, in which the results will be broken down by the columns named in the cut. These columns must be in the report. For our example, let us add a break down by built form:
(def-report example-report (cut name:my-first-cut built-form) (column name: built-form value: house.built-form summary: [(in detached semidetached)]) (column name: energy-use value: house.energy-use summary: [min max mean]))
The word built-form
in the cut refers to the column name:built-form
below it.
An aggregate report called my-first-cut-of-example-report
will be produced, in which the summary:
values are shown for all houses sent to the report, broken down by date of sending and the built form column.
6.4.3 Sending from other actions
One of the really useful things about def-report
is that you can send houses to it from many places.
This does not have to involve send-to-report
; every action in the model has the named argument report:
, which accepts a list of reports to send the house to.
The house will be sent to the report, before and after the action is taken, just as though it had been placed inside a probe
command.
When the report:
argument is used, the name:
of the sending command is used where the from:
argument is used in send-to-report
.
For example, if using the action (measure.standard-boiler name:my-boiler-measure report:#example-report)
, the example report would get rows with my-boiler-measure
in the from column.
6.4.4 A whole scenario
We could re-write our example probe scenario above using def-report
like this:
(def-report heating-systems (column name:h value: house.main-heating-system-type) (column name:f value: house.flags) (column name:e value: (house.heating-efficiency measurement:winter of:primaryspaceheating) summary: [mean min max]) (column name:energy value:house.energy-use summary: sum) (cut name:cut-heating-and-flags h f outcome)) (on.dates ... (apply to: (sample 100000) (measure.standard-boiler fuel:MainsGas winter-efficiency: 85% report: #heating-systems update-flags: affected)))
As written this is not much shorter, but you can see how if we were installing several measures in different places, sending them to the report would be easier.
6.4.5 Special columns
In the example above, we have shown one of the special columns that can be included in a cut
; these are:
outcome
; this column istrue
if the measure succeeds andfalse
otherwise. A measure will fail if it is unsuitable for the house. This is explained in more detail later.sent-from
; this column is thefrom:
attribute of the originatingsend-to-report
, or thename:
attribute of a measure used withreport:
selected
; this column istrue
if the condition being reported on ends up become part of the model's view of how things really are. This is relevant for hypotheses and choices, in which measures may be installed in a house hypothetically but where the hypothesis may never be realised. This is explained more in the later section on hypotheses and choices.
6.5 Other reports; what measures have gone into which houses; who spent what money
As well as the user-defined reports you can produce with probe
, aggregate
, probe.aggregate
, and def-report
with report:
, there are several built-in reports which produce other kinds of outputs that can help you analyse the results of a simulation.
6.5.1 report.aggregate-measure-costs
Produces a summary table detailing the number, cost, and size of measures which have been installed
6.5.2 report.fuel-costs
Reports annually on the total fuel cost paid by each house.
6.5.3 report.global-accounts
Reports on the balance of the model's global accounts; whenever a house receives or spends money, this is transferred from one of a number of global accounts. This report shows changes in the global accounts' balances as they change.
6.5.4 report.measure-installations
Produces a dwelling-level log of measure installations. When a measure is installed in a dwelling, a row is added to this file containing the type of measure, the cost, the size, the date, and the dwelling ID for which the measure was installed.
6.5.5 report.sequence
Produces a report detailing the sequence of events that was simulated. This can be useful to determine what order things are ending up happening in. It also contains information about the random number generator state, which allows you to determine at what point two simulations containing stochastic behaviour have diverged.
6.5.6 report.transactions
Produces a log of every transaction that has happened in the model. This is described more fully in the section on 11, but in summary each row describes the transfer of some money between a house and a global account, or in some cases between two global accounts. Such a transfer happens whenever a house receives a measure, or a subsidy, or pays its fuel bill, or similar.
6.6 Exercises about reports
- Amend an existing scenario to include a report which allows you to answer these questions:
- How many houses are there in each region?
- How many houses are there of each built form?
How many houses are there whose floor areas are
- 0-10m
- 10-100m
- 100-1000m
- 1000m-10000m and so on.
Hint: you could use the
log
andround
commands for this.- For each of these categories, what is the mean energy use and emissions
- Amend a scenario to emit a report from which you could scatter plot the relationship between floor area and modelled energy use by some different types of fuel.
- Amend a scenario in which you install a measure to emit a report which tells you exactly which houses were affected by the measure, and what the measure's impact on their carbon emissions (
house.emissions
) was. - Amend a scenario in which you install a measure to emit a report which tells you how many houses were affected by the measure, and what the measure's summed impact on their carbon emissions was, broken down by built form.
6.6.1 Answers
These are all things which you can do with the
aggregate
command, orcut
withindef-report
. We give an example of each:(aggregate name:exercise divide-by: (house.region) (aggregate.count name:count) (aggregate.mean name: mean-energy-use (house.energy-use)) (aggregate.mean name: mean-emissions (house.emissions)))
In this, the divide-by part could be replaced with
(house.built-form)
to answer question 2., or with(round (log (house.total-floor-area)))
for question 3. Alternatively all dividing factors could be included in the same report within square brackets, in which case the report would detail each possible combination of factors. Usingdef-report
instead:(def-report exercise (column name: energy-use value: (house.energy-use) summary: mean) (column name: emissions value: (house.emissions) summary: mean) (column name: house-region value: (house.region)) (column name: house-form value: (house.built-form)) (column name: house-area value: (round (log (house.total-floor-area)))) (cut name: by-region house-region) (cut name: by-form house-form) (cut name: by-area house-area)) (on.dates (scenario-start) (apply (send-to-report #exercise)))
This is a candidate for a
probe
report, ordef-report
, as for a scatter plot we want to see information about each case. Using a probe:(on.dates (scenario-start) (apply (probe name: p capture: [ (house.total-floor-area name: tfa) (house.energy-use name: total) (house.energy-use by-fuel: mainsgas name: gas) (house.energy-use by-fuel: electricity name: elec) ])))
The
def-report
equivalent would be(def-report example (column name: tfa value: (house.total-floor-area)) (column name: total value: (house.energy-use)) (column name: gas value: (house.energy-use by-fuel: mainsgas)) (column name: elec value: (house.energy-use by-fuel: electricity))) (on.dates (scenario-start) (send-to-report #example))
Again, this is a
probe
ordef-report
; if our existing scenario has something like this in it for the measure:(on.dates ... (apply ... (measure.wall-insulation ...) ) )
when using a
probe
we would wrap the measure in the probe so:(on.dates ... (apply ... (probe name:effects capture: [ (house.emissions) ] (measure.wall-insulation ...)) ))
alternatively with
def-report
(def-report effects (column name:emissions value: (house.emissions))) (on.dates ... (apply ... (measure.wall-insulation report: #effects ...) ))
This would be a job for
probe.aggregate
ordef-report
; again using the fragment above, we would have either(on.dates ... (apply ... (probe.aggregate name:effects capture: [(aggregate.sum (house.emissions)) (aggregate.count)] divide-by: (house.built-form) (measure.wall-insulation ...) )))
or
(def-report effects (column name: emissions value: (house.emissions) summary: sum) (column name: form value: (house.built-form)) (cut name: by-form form outcome)) (on.dates ... (apply ... (measure.wall-insulation report: #effects...) ))
7 Insulation measures
There are a few measures which perform insulation in the model; measure.wall-insulation
, measure.loft-insulation
, measure.floor-insulation
, measure.glazing
and so on. All of these work in fairly similar ways.
7.1 How does the model think about insulation
Before we come to the details of each measure, let's revisit how the energy calculator thinks about insulation. Insulation is important to the energy calculator because it affects the specific heat loss of the building; roughly speaking this is worked out as
\[ H = V + \sum_s A_s \cdot U_s \]
Where the sum is over external surfaces \(s\), and \(A_s\) is the surface's exposed area and \(U_s\) the u-value of the surface. \(V\) is an additional term covering heat losses due to ventilation, which is not so interesting here.
This value \(H\) is important because it is (roughly speaking) proportional to the heat required in the house, and so the fuel that must be burned to meet that requirement; halving \(H\) will roughly halve the fuel inputs for space heating.
In each modelled house, there are various external surfaces which contribute to the sum above:
- Walls
- Windows
- Doors
- Floors
- Roofs
For each of these, the model has a stored u-value and area5, and some information about what kind of insulation is present. When an insulation measure is applied to a house, it will change the u-value and the information about what kind of insulation is present.
You may be used to thinking of insulation in terms of a final u-value; in the interests of verisimilitude, the model prefers to use r-values (resistances). An r-value is simply the reciprocal of a u-value, but the important thing is that r-values can be added together; imagine you are applying the same insulation to two different walls. The first wall has very poor thermal performance (let us say a u-value of 10), whereas the second has good thermal performance (\(u = 0.5\)). Let us also say that the insulation material on its own has a u-value of 1 (neither excellent nor terrible).
The u-value of the wall after insulation should clearly not be the same in both cases; instead it is determined by the combined performance of the wall with the new material. The best-case value for this is the sum of the two layers' thermal resistances, so for wall 1:
\[ u' = \frac{1}{1/10 + 1/1} = 0.91 \]
whereas for wall 2:
\[ u' = \frac{1}{1/0.5 + 1/1} = 0.33 \]
Those measures in the model which add insulation to an existing surface are usually controlled by a (per-unit-thickness) resistance:
and thickness:
arguments, which are multiplied together to get a total resistance, which is then used to modify the existing u-value of the surface. In these cases you can specify a final u-value function if you prefer, in which case the impact of the insulation is independent of the initial condition of the wall.
Those measures which replace an existing insulated element (like glazing) are controlled by a simpler u-value, as the old material is being entirely replaced and so should not affect the end result.
Insulation measures also have some measure-specific rules around their suitability and the parts of a house which they affect:
7.1.1 Wall insulation
In the NHM, a building is composed of a series of storeys, and each storey has a floor plan which is a polygon in which each line is a wall. Each of these walls has a construction type, which indicates what the wall is made of, and a record of how much of three different kinds of insulation (internal, external and filled cavity) are found on the wall. Walls with different construction types can accept different sorts of insulation, summarised in this table:
Construction | Internal | External | Filled Cavity |
---|---|---|---|
Granite or Whinstone | Yes | Yes | No |
Sandstone | Yes | Yes | No |
Solid Brick | Yes | Yes | No |
Cob | Yes | Yes | No |
Cavity | Yes | Yes | Yes |
Timber Frame | Yes | Yes | Yes |
System Build | Yes | Yes | Yes |
Metal Frame | Yes | Yes | Yes |
(Party walls) | No | No | No |
Normally, a wall cannot have a form of insulation if it already has some insulation of that form, although this can be overridden if desired.
A house then is suitable for a given form of insulation if, for at least one of its walls:
- The wall's construction type will accept that type of insulation, per the table above
- The wall does not have any insulation of the given form so far.
Upon installation, the wall's u-value will be changed, and the wall will be marked as having some of the given form of insulation.
7.1.2 Loft insulation
Each storey in an NHM house may have some exposed surface on its top. All of these exposed surfaces are considered to form the insulatable roof area. The existing insulation thickness and u-value are stored, as well as information about the presence of a loft.
Unlike wall insulation, loft insulation can be installed when there is already some insulation present, when the measure allows it; this is controlled by the top-up:
argument.
Loft insulation is suitable when:
- There is some roof area
- The roof's construction type is not pitched slate or tiles
- The measure is a top-up measure and the existing thickness is less than the target thickness, or the measure is not a top-up and there is no insulation present
- The house has a loft
When top-up insulation is added, the resistance which is added is determined using the thickness topped up, so a top-up from 200mm to 300mm with a resistance of 0.01 will add a resistance of 1.
7.1.3 Glazing
Each of the four elevations (front, back, left and right) of a house in the NHM have a 'glazed proportion' (this is how the data is recorded in the EHS), and that fraction of each wall belonging to a given elevation is taken to be glazed. The glazed area may be broken down into different kinds of glazing, as some houses will have mixed types of glazing.
NHM glazing measures are simpler than the other insulation measures; installing glazing simply replaces all the glazing already in the house with the new sort, and this is always considered to be a suitable action to take.
Note that this means that you can replace double glazing with single glazing, if you so wish; the suitability rules are intended to be as basic as possible, preventing things which are infeasible rather than things which are undesirable.
7.1.4 Actions to set insulation (as opposed to measures)
As well as offering insulation measures, the model has some corresponding actions which will change the insulation state of parts of the house without respect to suitability and without charging any money.
These are action.set-loft-insulation
, action.set-wall-insulation
, action.set-glazing
and so on. They are not intended for modelling the installation of measures, but instead are there as a blunt instrument for changing a house into a given condition.
Even blunter instruments include action.reset-walls
and others, which will change the thermal properties of all the walls in a house in one go.
7.2 Capital cost of insulation, and size.m2
When an insulation measure is installed, it temporarily records the area which was insulated and makes it available through the size.m2
command. The value of size.m2
is particular to the measure which contains it in the scenario; it is not a record of the total area insulated so far or anything, but is an ephemeral value which should be used during the installation of a measure or package.
To make this more explicit, if you write
(measure.wall-insulation ... capex: (* 100 size.m2))
When the capex:
rule is worked out, size.m2
will produce the area insulated by this application of this measure to the current house.
Similarly, if you write:
(apply (measure.wall-insulation ... capex: (* 100 size.m2)) (measure.loft-insulation ... capex: (* 200 size.m2)))
The capex:
rule for wall-insulation will be 100 pounds per square meter of wall insulation installed by the wall insulation measure, and the capex:
rule for loft insulation will be 200 pounds per square meter of roof area that was insulated.
This hopefully illustrates how the value of size.m2
depends on where it is written. A side-effect of this is that if you write something like this:
(aggregate name: total-area (aggregate.sum size.m2))
To report on "total insulated area", this will not work; as far as the model is concerned, this means something like "report on the total area insulated by doing this report", which unsuprisingly is zero.
If you wish to report on ephemeral values like size.m2
, you must do one of two things:
Report on them using a
probe
in the right context:(probe capture: [size.m2] (measure.wall-insulation ... ))
or
(probe.aggregate capture: [(sum size.m2)] (measure.wall-insulation ...))
Store the ephemeral values into a variable from the right context, and then refer to them later:
(do (measure.wall-insulation ...) (increase #insulated-area size.m2))
The
do
andincrease
commands and variables have not been met yet, so you will have to skip ahead for more on these.
The cost produced by the capex:
rule is available using the command capital-cost
, which has the same behaviour; capital-cost
is the capital cost of the command containing it, so reporting on capital-cost
is asking for the capital cost of the report.
There is more about this in the section on money.
7.3 How to put in some insulation
Offering insulation measures is no different to using any other action: you use the apply
command to try and put the measure into a population of houses. Each measure has slightly different parameters; here are some examples:
Wall insulation:
(apply (measure.wall-insulation type: cavity thickness: 50 resistance: 0.002 capex: (* 10 size.m2)))
In this example we are offering wall insulation any construction that has a cavity. If any house has a wall with a cavity which is not already insulated, the wall will be marked as having 50mm of insulation, and its u-value changed by adding a resistance of \(50 \times 0.002\). The house will incur a cost of ten pounds per square metre of wall that was changed.
You can directly specify a u-value instead, if you do not want the initial condition of the wall to matter:
(apply (measure.wall-insulation type: cavity thickness: 50 u-value: 0.2 capex: (* 10 size.m2)))
The u-value, resistance and thickness arguments can all be given formulae instead of constants, which will be worked out each time the measure is used. Here is an example using a table of u-values which depend on the build year of the house:
(measure.wall-insulation ... u-value: (~lookup-table row-keys: house.sap-age-band [age u-value] [A 2] [B 1] [C 0.5] [D 0.25] [E 0.1] [* 0.05] ))
Loft insulation:
(apply (measure.loft-insulation thickness: 300 resistance: 0.02 capex: 500 top-up: false ))
In this example, we will install 300mm of loft insulation on any suitable roof which currently has no insulation at all (the
top-up:
argument determines this). The u-value will be updated by adding a resistance of 300 * 0.02, and the house will pay a flat fee of 500 pounds.When using
top-up: true
, the u-value will be updated by adding the resistance (300 - existing thickness) * 0.02.Glazing:
(apply (measure.install-glazing frame-factor:0.8 capex: (+ 100 (* 50 size.m2)) frame-type: metal gains-transmittance: 0.9 light-transmittance: 0.9 glazing-type: triple insulation-type: lowehardcoat u-value : 0.2 ))
As you can see, glazing has more parameters than other forms of insulation; this is because of its effect on internal gains and lighting usage in the BREDEM calculation. You can find sensible values for these parameters in SAP; the defaults used by the model if you do not specify values do not appear to be sensible as of this writing, so leaving them out is not a good idea. In this example, we will install some triple glazing with a u-value of 0.2, at a cost of 100 pounds plus 50 pounds per square metre of windows installed.
The measure will replace all the windows in the dwelling with windows of this standard.
To give a quick summary:
- frame factor
- the amount of the window area not taken up by the frame (and so passing light and heat)
- frame type
- the material the frame is made of; this has no direct effect, but needs to be recorded as you may want it later if you wish to impose different u-values on a house and these depend on the frame type
- gains transmittance
- a factor indicating what proportion of heat from the sun is included into the calculation of solar gains
- light transmittance
- a factor indicating how much light from the sun passes into the house, which has a small effect on the lighting demand
- glazing type
- the kind of glazing; as with frame type, this has no direct effect
- insulation type
- akin to glazing type and frame type
- u-value
- the thermal property of the window, which is significant
7.4 How to find out about insulation on houses
There are various commands which you can use to find out about insulation state, either for reporting purposes or to use to change scenario behaviour. These are all documented in more detail in the manual, but here is a summary list:
house.double-glazed-proportion
- produces the proportion of a house's glazing which is double glazed or better.
house.floor-insulation-thickness
- produces the thickness of any floor insulation
house.loft-insulation-thickness
- produces the thickness of any loft insulation
house.predominant-wall-type
- produces the predominant wall type; this is the wall type which has the greatest external area
house.floor-construction-type
- produces the type of floor construction for the ground floor
house.roof-construction-type
- produces the type of roof construction
house.has-loft
- tests whether a house has a loft or not
house.any-wall
a complex test which lets you determine whether any of the walls of a house satisfy conditions you are interested in. For example,
(house.any-wall has-cavity-insulation: true)
Will tell you whether the house has any walls for which there is some filled cavity insulation installed. Similarly
(house.any-wall has-cavity-insulation: false)
Will tell you whether the house has any walls which do not have cavity insulation (including those walls which cannot have cavity insulation).
(house.any-wall has-construction: anycavity)
Will tell you whether any of the walls have a cavity; these can be combined, so
(house.any-wall has-construction:anycavity has-cavity-insulation:false)
Will tell you whether a house has at least one wall which currently has no cavity insulation, and has one of the cavity construction types.
house.all-walls
- an analog of
house.any-wall
which only passes if every external wall a house has satisfies the conditions that you are interested in.
7.5 Exercises about insulation
Here is a drawing of a normal terraced house; the front elevation is on the left of the drawing.
- Work out its heat loss coefficient, as it is now (in this exercise we are disregarding some details; the extra height on storeys for the floor joists, ventilation losses etc; these factors are included in the model).
- Based on this, work out the amount of power required from a heat source to maintain a 7 degree temperature difference with the outside
- For each of the following measures, note which parts of the house change, and recalculate the heat loss coefficient and power to maintain a 7 degree temperature difference
(measure.wall-insulation type: internal thickness: 50 resistance: 0.01)
(measure.wall-insulation type: cavity thickness: 50 resistance: 0.02)
- Both of the above measures
(measure.glazing type:double u-value: 1)
(measure.glazing type:triple u-value: 0.2)
(measure.loft-insulation thickness: 200 resistance: 0.02)
- All of the above measures
7.5.1 Answers
The heat loss coefficient is the sum of these elements
Floor Element Area u heat loss G front wall 4 0.5 2. G front glazing 4 1 4 G left wall 4 2 8 G rear wall 2 2 4 G rear glazing 2 1.5 3. G rear wall (inset) 4 2 8 G right wall 4 2 8 G floor 20 0.8 16. G roof 8 2 16 1 front wall 4 2 8 1 front glazing 4 1 4 1 rear wall 4 0.5 2. 1 rear glazing 4 1.5 6. 1 roof 12 2 24 total 113. Notes:
- The areas of the front and back walls are reduced by the glazing in them
- The party walls contribute no heat loss
- The roof area of the ground floor is its area less the floor area of the first floor
- The power requirement is the heat loss multipled with the temperature difference, so \(113\times 7=791\text{watts}\)
This will change all the walls except the rear wall of the first floor, which already has internal insulation. The amended values will be
Floor Element Area u r u' delta heat loss G front wall 4 0.5 0.5 0.4 -0.4 G front glazing 4 1 0 1 0 G left wall 4 2 0.5 1. -4. G rear wall 2 2 0.5 1. -2. G rear glazing 2 1.5 0 1.5 0. G rear wall (inset) 4 2 0.5 1. -4. G right wall 4 2 0.5 1. -4. G floor 20 0.8 0 0.8 0. G roof 8 2 0 2. 0. 1 front wall 4 2 0.5 1. -4. 1 front glazing 4 1 0 1 0 1 rear wall 4 0.5 0 0.5 0. 1 rear glazing 4 1.5 0 1.5 0. 1 roof 12 2 0 2. 0. total -18.4 The final heat loss will then be 113 - 18.4 = 94.6, with a power requirement of 662 watts
This will change all the walls except the front wall of the ground floor, which already has a filled cavity. The amended values will be:
Floor Element Area u r u' delta heat loss G front wall 4 0.5 0 0.5 0. G front glazing 4 1 0 1 0 G left wall 4 2 1 0.66666667 -5.3333333 G rear wall 2 2 1 0.66666667 -2.6666667 G rear glazing 2 1.5 0 1.5 0. G rear wall (inset) 4 2 1 0.66666667 -5.3333333 G right wall 4 2 1 0.66666667 -5.3333333 G floor 20 0.8 0 0.8 0. G roof 8 2 0 2. 0. 1 front wall 4 2 1 0.66666667 -5.3333333 1 front glazing 4 1 0 1 0 1 rear wall 4 0.5 1 0.33333333 -0.66666668 1 rear glazing 4 1.5 0 1.5 0. 1 roof 12 2 0 2. 0. total -24.666667 The reduction in power requirement is then 171 watts.
Applying both the measures involves summing the r-values which we have applied in each case:
Floor Element Area u r u' delta heat loss G front wall 4 0.5 0.5 0.4 -0.4 G front glazing 4 1 0 1 0 G left wall 4 2 1.5 0.5 -6. G rear wall 2 2 1.5 0.5 -3. G rear glazing 2 1.5 0 1.5 0. G rear wall (inset) 4 2 1.5 0.5 -6. G right wall 4 2 1.5 0.5 -6. G floor 20 0.8 0 0.8 0. G roof 8 2 0 2. 0. 1 front wall 4 2 1.5 0.5 -6. 1 front glazing 4 1 0 1 0 1 rear wall 4 0.5 1 0.33333333 -0.66666668 1 rear glazing 4 1.5 0 1.5 0. 1 roof 12 2 0 2. 0. total -28.066667 This nicely illustrates how insulation has diminishing returns - the effect of applying both forms of insulation is much less than the sum of their effects when taken independently.
The glazing measure is not computed as a resistance but simply as a replacement of the basic u-value:
Floor Element Area u heat loss G front wall 4 0.5 2. G front glazing 4 1 4 G left wall 4 2 8 G rear wall 2 2 4 G rear glazing 2 1 2 G rear wall (inset) 4 2 8 G right wall 4 2 8 G floor 20 0.8 16. G roof 8 2 16 1 front wall 4 2 8 1 front glazing 4 1 4 1 rear wall 4 0.5 2. 1 rear glazing 4 1 4 1 roof 12 2 24 total 110. The only difference between the two kinds of glazing for our purposes is the u-value:
Floor Element Area u heat loss G front wall 4 0.5 2. G front glazing 4 0.2 0.8 G left wall 4 2 8 G rear wall 2 2 4 G rear glazing 2 0.2 0.4 G rear wall (inset) 4 2 8 G right wall 4 2 8 G floor 20 0.8 16. G roof 8 2 16 1 front wall 4 2 8 1 front glazing 4 0.2 0.8 1 rear wall 4 0.5 2. 1 rear glazing 4 0.2 0.8 1 roof 12 2 24 total 98.8 Applying loft insulation will affect both the roof parts only.
Floor Element Area u r u' delta heat loss G front wall 4 0.5 0 0.5 0. G front glazing 4 1 0 1 0 G left wall 4 2 0 2. 0. G rear wall 2 2 0 2. 0. G rear glazing 2 1.5 0 1.5 0. G rear wall (inset) 4 2 0 2. 0. G right wall 4 2 0 2. 0. G floor 20 0.8 0 0.8 0. G roof 8 2 4 0.22222222 -14.222222 1 front wall 4 2 0 2. 0. 1 front glazing 4 1 0 1 0 1 rear wall 4 0.5 0 0.5 0. 1 rear glazing 4 1.5 0 1.5 0. 1 roof 12 2 4 0.22222222 -21.333333 total -35.555555 Because the roof was a large surface with a bad u-value, the change is pretty significant.
Let us combine all these effects:
Floor Element Area u r u' delta heat loss G front wall 4 0.5 0.5 0.4 -0.4 G front glazing 4 1 4 0.2 -3.2 G left wall 4 2 1.5 0.5 -6. G rear wall 2 2 1.5 0.5 -3. G rear glazing 2 1.5 4.333 0.20001333 -2.5999733 G rear wall (inset) 4 2 1.5 0.5 -6. G right wall 4 2 1.5 0.5 -6. G floor 20 0.8 0 0.8 0. G roof 8 2 4 0.22222222 -14.222222 1 front wall 4 2 1.5 0.5 -6. 1 front glazing 4 1 4 0.2 -3.2 1 rear wall 4 0.5 1 0.33333333 -0.66666668 1 rear glazing 4 1.5 4.333 0.20001333 -5.1999467 1 roof 12 2 4 0.22222222 -21.333333 total -77.822142 Here we have derived the effective r-value for the replacement of the windows, as though we were insulating them rather than replacing them. The combined effect is a power reduction of about 540 watts.
8 TODO Heating measures
8.1 How does the model think about heating measures
Heating measures are harder to describe than insulation measures. You might think of a heating system as a thing which meets a need for some amount of heat by converting some fuel at some efficiency. In this view, installing a new heating system might change a house's heating fuel and efficiency, and so convert the same demand for heat into a different demand for heating fuel.
Whilst this is superficially correct, in a BREDEM or SAP type calculation it is complicated by two considerations:
- The choice of heating system may affect the demand for heat, for various reasons:
- Moving from point-of-use hot water to central hot water will introduce distribution losses
- Moving from an instantaneous combi boiler providing hot water to a standard boiler with a tank will introduce tank losses and primary circuit losses
- Some combinations of heating system and heating controls will change the demand temperature in the house; SAP table 4e lists these kinds of adjustments, and SAP table 9 also makes such an adjustment based on the control type column.
- The heating system responsiveness will change the demand temperature in the house; this is a dimensionless number between 0 and 1 which reflects the time a heating system takes to respond to demand for heat. Systems with a lower responsiveness are assumed to result in a higher average internal temperature. SAP table 4d gives the responsiveness for wet heating systems, and table 4a the responsiveness for other types of heating system.
- The efficiency of the heating system may be adjusted according to one of a number of rules; SAP table 4c lists the adjustments that are used. In addition the efficiency used for boilers is usually a function of their winter and summer efficiencies, which are combined in the harmonic ratio of the space heating and water heating heat demands.
In the model, when you install a heating measure, some other changes will be made to ensure that the result makes sense:
- If installing something which requires a central heating system, and the house does not have a central heating system, a central heating system will also be installed. This reflects the cost of fitting pipework, not installing a boiler. Measures which may have this effect have an additional named argument
system-capex:
, which is used to determine the capital cost of installation. - If installing something which requires a hot water tank, a new tank will be installed (even if there is an existing tank). The tank's insulation thickness and capacity are given by
cylinder-insulation-thickness:
andcylinder-volume:
respectively.
Most heating systems have a notion of size; you can control this using the size:
argument of the measure to enter a sizing rule. A sizing rule will typically look something like this:
(size min: 5 max:50 (house.peak-load))
The size used is calculated from the first unnamed argument (in the example, this is house.peak-load
, which is a reasonable choice. The peak load is the heat output required to keep the house warm on a cold day; by default to produce a 19 degree internal temperature against a -5 degree external temperature). The min:
and max:
arguments are optional, and restrict the measure's suitability.
The sizing rule is used in two ways:
- It is calculated before calculating the capital cost or operational cost, and the result is made available via the
size.kw
command. This allows you to compute a size for a heating system, and then use it in the pricing rule to affect the price - As mentioned, the
min:
andmax:
arguments restrict the suitability. If the sizing function produces a value below themin:
or above themax:
, the measure will be unsuitable. This is intended to reflect the fact that some technologies do not work well or are not available in some sizes.
You can use any function for the sizing function, so you could combine house.peak-load
with round
to represent a technology typically available in 10kW increments:
(size max:50 (round (house.peak-load) to: upper 10))
This indicates that we should take the peak load, but if it is 1kW we should round it up to 10, if it is 11kW we should round it to 20 and so on.
8.2 How to put in new heating systems
The commands for putting in new heating systems are:
measure.standard-boiler
This installs a standard condensing boiler, with a hot water tank and central heating system. The boiler will supply space and water heat.measure.combi-boiler
This measure installs a condensing combi boiler, with a central heating system. You can specify a storage combi boiler using thestorage-volume:
argument. The boiler will supply space and water heat.measure.heat-pump
This measure installs a heat pump, with a central heating system. The type of heat pump is given by thetype:
argument, and the efficiency by thecop:
argument. The heat pump will supply space and water heat.measure.storage-heater
This measure installs storage heaters for space heating.measure.room-heater
This measure installs a secondary room heater.measure.warm-air-system
This measure installs a warm air system; it can only ever replace an existing warm air system, so you can never increase the number of warm air systems in the stock.measure.district-heating
This measure installs a district heating system; as far as the model is concerned, this is a bit like a boiler whose fuel input isCommunityHeat
. The SAP rules about primary fuel inputs to community heat or CHP are not modelled, as there is no information about the kind of heat network a house would be connected to.measure.solar-dhw
This measure installs a solar water heating system. The SAP / BREDEM model for solar water heating is complicated, and the output from the heater depends on the demand for hot water in a complex, nonlinear way. Roughly speaking, the useful output of the solar water heater is greater in houses with a geater demand for hot water. This presumably represents the fact that the heat output is of a relatively low grade, and so will be of most benefit if the tank is cold.
8.3 TODO How to find out about heating systems and fuels
There are several commands which will tell you about the kind of heating system in a house.
8.4 TODO Exercises about heating
9 Flags
We have already briefly seen flags. Flags are one of the ways to store information on houses in the simulation. A useful metaphor for flags is that they are sticky labels with a single word written on them. You can stick as many labels to each house as you like, and you can test whether a house has the right combination of labels when filtering the population or applying an action.
9.1 Marking houses as different with flags
At the start of the run, there are never any flags on any of the houses.
Naturally our first question is: how do we put flags on some houses?
The simplest way to do this is with the action house.flag
:
(on.dates (scenario-start) (apply to:(sample 20%) (house.flag flag-A flag-B)))
This will:
- Produce a random sample of 20% of all the houses (if this is unfamiliar, read the preceding sections)
- Go through that 20% sample and use the
house.flag
action on all of them - This will mark each house with two flags,
flag-A
andflag-B
. You can use any word (without spaces) for a flag, soflag-A
andflag-B
are indicative here. It could behouse-of-interest
, orboiler-has-failed
or whatever you like.
Once a house has been given a flag, it will keep it until you remove it again. The model does not automatically do anything to the flags on houses, and it has no understanding of their names, so they are entirely your responsibility to manage. In this example, we have marked 1/5 of the houses, and that particular set will remain so marked until we change it. Because of this, if you flag some houses because they are in a certain state (for example, having a high energy use), the presence of the flag later in the run tells you that the houses used to have high energy use. This is useful, when you want to see how a population has changed, but potentially confusing if you don't bear in mind this is what's happening.
Now that you know how to put some flags on a house, we can look at how to find houses which have particular flags.
9.2 Using flags to select a set of houses
A house either has a flag or it doesn't, so the thing we are looking for is a logical test which will distinguish these cases.
The test for this is house.flags-match
, which will tell you whether a house has a certain combination of flags.
For example, to report on the set of houses having flag-A
, we might write
(aggregate name:houses-with-A over: [ (filter (house.flags-match flag-A)) ] (aggregate.count))
Here we are aggregating over:
the set (filter (house.flags-match flag-A))
, in which (house.flags-match flag-A)
will be true for any house that has flag-A
on it.
9.3 Using flags to affect suitability, and making actions also do flags
If you look in the manual pages for most actions and measures you will see that they accept the two named arguments test-flags:
and update-flags:
.
These let you do two things:
test-flags:
tells the action that, before it is applied, it should check something about the flags on the house, and fail if they are not how you need them to beupdate-flags:
tells the action that, after it has succeeded (if it succeeds), it should modify the flags on the house so that they are a certain way.
For example, you could restrict a measure to work only houses with the flag flag-A
like this:
(apply to: (all-houses) (measure.wall-insulation test-flags: flag-A ... ))
Note that this does not make the measure suitable for all houses with flag-A
; it makes the measure unsuitable for all houses without flag-A
, but if a house has flag-A
but is already unsuitable it does not become suitable.
This argument accepts a list, so you can also write
(apply to: (all-houses) (measure.wall-insulation test-flags: [flag-A flag-B] ... ))
to restrict the measure to houses with both flag-A
and flag-B
.
The dual to this is update-flags:
; say you wanted to mark a house when it had received some insulation from a particular measure, so that you could keep track of it later or report on it.
(apply to: (all-houses) (measure.wall-insulation update-flags: house-got-some-wall-insulation ... ))
In this you could refer to the set (house.flags-match house-got-some-wall-insulation)
, and it would contain any house to which the above measure was applied successfully. These houses would be guaranteed to have had some wall insulation.
If you wanted to split the population into houses which did and did not, out of the target population, you could write
(apply to:(sample 50%) (house.flag house-considered) (measure.wall-insulation ... update-flags: house-affected))
Now all the houses which were offered the measure (here a random 50%) will be flagged as house-considered
, but only those which got insulation will be flagged as house-affected
.
9.4 Patterns for flags, and the negation shorthand
All of the commands which work with flags actually work with patterns for flags.
A pattern is like a very short rule which lets you match a whole set of flags all at once.
The simplest patterns are plain words, like we have already written; the pattern flag-A
just matches the flag flag-A
.
More complicated patterns can include some special symbols:
the star symbol
*
will match any text. For example, the patternflag-*
matches any flag which starts withflag-
, so if you say(filter (house.flags-match flag-*))
you would get houses flagged withflag-A
andflag-B
andflag-some-other-flag
and so on.(filter (house.flags-match *))
gives the set of all houses which have got any flag on them at all.- The question mark
?
will match any single letter. For example,flag-?
matchesflag-A
andflag-B
, but does not matchflag-XY
becauseXY
is two letters. - You can write several alternatives like
<X,Y,Z>
. This pattern matchesX
orY
orZ
.
These parts can be combined, so for example <X-,Y->*
will match the flags X-
, Y-
, X-abcdef
, Y-qwert
, and so on and so forth.
You cannot use patterns when adding a flag to a house; for example if you were to say (house.flag *)
to try and add all possible flags to a house, the model will not validate your scenario.
There is one other special bit of flag syntax; prepending an exclamation point !
to any pattern negates it.
When testing a flag this means that no flags should match the pattern, rather than that at least one should.
When changing a flag, this means that all flags matching the pattern should be removed.
Here are some examples:
(house.flag !flag-A)
- this will remove just the flagflag-A
from any house it's applied to.(house.flag !flag-*)
- this will remove all flags which start withflag-
from any house it's applied to.(house.flag !*)
- this will remove all the flags from any house it's applied to this is because*
matches all the flags that are there, and!
means to remove them, because we are changing flags, rather than testing them.(house.flags-match !flag-A)
- this will be true for any house which does not have the flagflag-A
on it.(house.flags-match !flag-A flag-B)
- this will match any house which does not haveflag-A
AND does haveflag-B
.(house.flags-match !flag-A !flag-B)
- this will match any house which does not haveflag-A
AND does NOT haveflag-B
.(house.flags-match !flag-*)
- this will be true for any house on which no flag starts withflag-
(measure.standard-boiler ... test-flags:!got-boiler update-flags:got-boiler)
- this will install a boiler in any house which can have a boiler and does not have the flaggot-boiler
. If it succeeds, and installs a boiler, then it will add the flaggot-boiler
to the house. Notice this means that this measure (in the absence of other commands to remove thegot-boiler
flag) can only be applied successfully once to each house.
9.5 Using flags to change a number, like a cost
Because you can check a flag with a logical test, you can use it in other places to do things like controlling the capital cost of a measure.
For example, imagine you were modelling the distinction between easy to treat and hard to treat cavities in the stock; the model does not know about this to start with. Firstly, you would write some code to decide about which houses are hard to treat:
(on.dates (scenario-start) ; set up HTT flag; (apply to: (sample 10% (...)) ; say a random 10% of some kinds of houses are HTT (house.flag HTT))) ;; then later: (apply to: (all-houses) (measure.wall-insulation ... capex: (function.case (when (house.flags-match HTT) 1000) default: 500)))
In this example we have defined the capex with the command function.case
, which uses logical tests to switch between different values. Here it will be 1000 when the house has the HTT
flag on it, but only 500 otherwise.
Note that for this specific case you might instead want to do something different in a real scenario, like having two variants of the measure defined:
(def-action htt-cwi (measure.wall-insulation ... capex: 1000 test-flags: htt update-flags: got-htt-cwi)) (def-action normal-cwi (measure.wall-insulation ... capex: 500 test-flags: !htt update-flags: got-normal-cwi))
This uses the def-action
command which is explained later; this is just an example of something which you can model more than one way, and the 'right' way depends on what you are trying to do. You could either say that insulation is a measure, and in HTT
houses it costs more, or you could say that HTT insulation is a different measure to normal insulation, which costs more and for which only HTT
houses are eligible.
9.6 Reporting on what flags a house has
You can use (house.flags-match ...)
as a value in any report, for example in the capture:
argument of a probe
, or as a test in aggregate.where
, or as divide-by:
in aggregate
.
There is another useful command, house.flags
which produces all the flags on a house, separated by commas. This also accepts a pattern, to print all the flags which match the pattern.
For example, if you wanted to see all flags starting with got-
on a house you could output (house.flags got-*)
and you would see a value like got-boiler,got-insulation
if the house had those two flags on it. Similarly you could write (house.flags !got-*)
to see all flags which do not start with got-
This is a useful thing to use in divide-by:
or capture:
if you have set up your measures to put a consistent pattern of flags on houses when applied.
9.7 Exercises about flags
Here are some patterns together with sets of flags. In each case, write down the following
- Whether
house.flags-match
would be true, given the list of flags - What the value of
house.flags
would be in each case, if given the pattern - What the new set of flags would be if the patterns were used with
update-flags:
(if legal) - Whether the measure
(action.do-nothing test-flags: [...])
would be suitable, if the patterns were filled in the space.
Patterns | Flags |
---|---|
* |
|
* |
A B C |
!* |
A B C |
got-* |
got-boiler got-insulation gottfried |
!htt |
htt-house large-house |
!suitable |
unsuitable |
suitable eligible |
suitable large-house |
? |
A house B |
<?,??> |
aa a aaa b bbb |
9.7.1 Answers
*
- false (must have a flag)
- empty (no text output)
- Not a legal value for
update-flags:
- not suitable (must have a flag)
*
- true (has at least one matching flag)
- The text
A,B,C
- Not a legal value for
update-flags:
- suitable
!*
- false (must not have any flags)
- empty (no flags match !*)
- no flags left afterwards (removes them all)
- not suitable
got-*
- true (both
got-boiler
andgot-insulation
are matching= got-boiler,got-insulation
gottfried
- suitable
- true (both
!htt
- false (although there is
htt-house
, it is not exactlyhtt
) htt-house,large-house
, as they both match the pattern of not beinghtt
htt-house
andlarge-house
(unchanged, it does not match any to remove)- unsuitable
- false (although there is
!suitable
- true (the house does not have the flag
suitable
, so it's OK). Theunsuitable
flag is irrelevant. This shows how the model does not understand the meaning of what you write. unsuitable
, as this matches the rule of not being the wordsuitable
unsuitable
, as the flagsuitable
is not there to be removed.- the measure would be suitable
- true (the house does not have the flag
suitable
eligible
- false, as the flag
eligible
is not present and would be required suitable
, as this matches the pattern forsuitable
suitable
large-house
andeligible
- unsuitable
- false, as the flag
?
- true, as
A
andB
both match?
A,B
- Not a legal pattern for
update-flags:
- suitable
- true, as
<?,??>
- true, as the flags
aa
,a
,b
all amtch a,aa,b
- not a legal pattern for
update-flags:
- suitable
- true, as the flags
10 Variables
Flags are one way to store information on a house. They can store boolean values, as they are either there (true) or not there (false). Whilst you can do anything you like with boolean values and a supply of patience, it is often useful to be able to store other kinds of information. In the NHM, you can store numbers as well as flags14. These numbers can be stored against individual houses so that each house may have its own value, or globally for all the houses in the simulation.
10.1 Defining and accessing variables; variable scope
Whilst flags are defined just by being used, variables must be defined before you can use them.
This is done using the def
command, which always belongs directly within a scenario
.
def
has the following form:
(def <<variable name here>> default: <<default value here>> on: <<where the variable lives>>)
Let's address the arguments in turn:
- The variable name is the first positional argument, and it is required. It can be any word (including punctuation), and is what you will use to refer to the variable when changing or looking at its value
- The
default:
argument defines a starting value for the variable. If you do not give adefault:
, accessing the variable before storing a value in it will cause the simulation to halt with an error (see 23.4). - The
on:
argument defines where the variable "lives"; there are three options:house
. A variable defined to beon: house
may take a different value for each house; in this sense it is like a flag, except it stores a number rather than just being present or missing. This could be used to store something like the number of times a house has been considered by a policy, or the year when it was last given a subsidy, or similar.simulation
. A variable defined ason: simulation
is the same for all houses; it is more like a counter to keep track of something shared by all the houses. For example, you might use a simulation variable to record how many of a certain measure are available. This is shared by all the houses, as there should be one supply chain for everyone rather than one for each house.event
. A variable defined to beon: event
is like one definedon:house
except it is ephemeral; each time anapply
oraggregate
or similar command completes, the value is forgotten and goes back to its default (if any).
Now that we know how to define a variable, we come on to how to use them. There are two ways to use a variable:
- By accessing its value in a calculation or a report
- By changing its value with an action
To access a variable, you merely refer to its name; if you like, you can prefix this with an octothorpe (#
symbol) to make it clear that this is what you are doing.
For example, in this scenario:
(scenario ... (def my-variable on:house default: 10) (on.dates (scenario-start) (aggregate name: a-report (aggregate.sum (+ 3 #my-variable)) (aggregate.mean #my-variable))))
Will produce two columns in the report:
- The sum, which will be 13 times the number of houses modelled; this is because:
- The default value of the variable is 10, and we do not change it anywhere
aggregate.sum
is summing the value of the variable plus 3 over all the houses- ten plus three is thirteen.
- The mean value of the variable, which is just 10, because again we have not yet changed it.
The key point here is that anywhere where we would write a number, or a function that computes a number, we can refer to a variable instead.
If the variable is defined as on: house
, the value will be that for the house or houses being looked at, just as for other house-specific values like house.energy-use
.
Dual to accessing a variable is changing it. There are three commands for changing variables, which are all actions
set
For example, applying the action(set #x 3)
will store 3 in the variablex
increase
Applying(increase #x 10)
is equivalent to(set #x (+ 10 #x))
; it changes the stored value to be ten more than it is already.decrease
Decrease is the opposite of increase;(decrease #x 10)
will reduce the stored value of the variable by ten.
Plugging some of these into our previous example we can make some changes to the variable before we report on it:
(scenario ... (def my-variable on:house default: 10) (on.dates (scenario-start) (apply to: (sample 5%) (increase #my-variable 10)) (apply to: (sample 10%) (set #my-variable (* my-variable 1.1))) (apply to: (filter (house.region-is london)) (decrease #my-variable 1)) (aggregate name: a-report (aggregate.sum (+ 3 #my-variable)) (aggregate.mean #my-variable))))
Now there will be eight different states for the value of #my-variable
' across the population, depending on membership of the two samples or the filter to London:
- Unchanged, if not in London or either random sample, i.e. 10
- 9, if in London but neither random sample
- 11, if in the middle sample only
- 10, if in the middle sample and in London
- 20, if in the first sample only
- 19, if in the first sample and in London
- 22, if in the first and second samples
- 21, if in the first asn second samples and in London
Estimating the values produced in the report is left as an exercise for the reader.
The values stored in variables can be taken from other calculations, and the set
, increase
and decrease
commands can be used in any place where other actions might be used.
For example, to store the initial energy use for every house in a scenario, before any changes are made, you could write
(scenario (def original-energy-use on:house) (on.dates scenario-start (apply (set #original-energy-use (house.energy-use)))) ;; later on (on.dates ... (aggregate name: impacts (aggregate.mean (- #original-energy-use (house.energy-use))))))
This illustrates how variables hold the values put in them at particular moments in simulated time. In this case, after being set, the #original-energy-use
variable will not change. The house.energy-use
command on the other hand will always reflect the current energy use, so by reporting on the difference we can see the change in energy use that has resulted for each house by whatever changes the scenario has so far wrought.
In principle you now know everything there is to know about variables in the NHM; in practise you may wish to refer to a few examples of how variables can be used. Before we come to that you should read the subsequent sections, in particular those about choices and packages and suitability and failure.
Examples of how to use variables to model particular effects are given in 22
10.2 Exercises about variables
For each of the following apply
commands, describe the value or values of the variable #x
after the apply
is finished:
First
(def x on:house) ... (apply (set #x (house.energy-use)))
Changing to
simulation
(def x on:simulation) ... (apply (set #x (house.energy-use)))
Using
increase
(def x on:house) ... (apply (increase #x (house.energy-use)))
Adding a
default:
(def x on:simulation default:0) ... (apply (increase #x (house.energy-use)))
With a function
(def x on:simulation default:0) ... (apply (increase #x (* (house.weight) (house.energy-use))))
Referring to own value
(def x on:house default:10) ... (apply (increase #x (* #x #x)))
10.2.1 Answers
The answers are:
#x
will take several values stored against each house, containing the energy use of that house at the time the apply happened.#x
will be a single value for the whole simulation, which will contain the energy use of one house at random (whichever one comes through theapply
last).- The scenario will fail with errors, as
#x
has no default value, but we are trying toincrease
it, i.e. add a new value to its existing value, without ever providing an existing value. - With the default, and
on: simulation
, there will be one value which will have each house's energy use added to it. The sum that is produced will be unweighted. - Like 4, but weighted, as the global x will be increased by the weight-energy use product for each house.
- Each house will have
#x
equalling 110, as it has a default of 10, and(* #x #x)
is 100.
11 Money
The NHM includes a simple model of money, in which each house has a ledger of transactions it has participated in. This ledger can be manipulated and reported on to find out about how much money has been spent, and on what things.
11.1 How does the model think about money
Every house has a ledger of transactions. A transaction is a record of the transfer of an amount of money between a house and a named pool of money (representing something like the market, or a policy's spending) on a certain date. The ledger never contains transactions which have yet to happen; however, these can be predicted within the model if you wish to consider the future costs of an immediate action.
Transactions are caused by a few different things in the model:
- Installing measures
- Measures typically have a
capex:
argument, which accepts any number-valued command. When a measure is installed, if it succeeds, a transaction is made from the account of the house which received the measure to a ledger representing the market for all measures. - Operational costs
- Some measures have an
opex:
argument, which takes a number. When the measure is installed, the this value will be calculated and stored in the house. Until the thing put in by the measure is removed by another measure (say if a boiler is replaced), the house will make a transaction for this operational cost once a year at the very end of the year. The house is said to have an obligation to pay its operational costs. - Fuel costs
- Every year, at the end of the year (when the operational costs are also paid), the house has an obligation to settle for its energy use. This is done by evaluating the house's tariffs, which are discussed in more detail in the next section. A tariff has the opportunity to create a few different transactions for different parts of the bill.
- Financial actions
- The above are all of the built-in financial considerations in the model; however, if you wish to model other movements of money, there are various commands to do this:
finance.with-subsidy
, which encloses another measure and meets some or all of its cost with an immediate subsidyfinance.with-loan
, which encloses another measure and meets some or all of its cost with an immediate subsidy to be repaid over time as a loanfinance.with-obligation
, which encloses another measure and sets up an arbitrary obligation to pay some amounts in the futuremeasure.with-additional-cost
, which encloses another measure and adds an arbitrary extra transaction to itpay
, which allows for the transfer of money between non-house entities.
11.1.1 Inflation, income, investment and so on
The NHM does not model inflation, income, return from investments or anything like that. All transactions are assumed to be denominated in units of equivalent value. If you wish to account for inflation, we suggest modelling the prices of things in the NHM using real values, and adding the inflation-adjustment off-model.
11.2 How to find out the costs of things
The best way to make this subject concrete is to take an existing scenario (hopefully you have one by this point in the guide!) and add report.transactions
inside the scenario
command.
This will produce a report containing the model's entire transaction log.
Each row of this report shows everything the model knows about a transaction that has happened; the columns are:
- payer
- this is typically the ID of a dwelling, although money can be moved between policies
- weight
- this is the weight of the payer, which is usually the dwelling weight
- date
- the date when the transaction occurred
- payee
- this is the name of the entity which received the payment
- amount
- this is the weighted amount of money transferred
- address
- this is a slash-separated path in the scenario of the command which caused the transaction
- tags
- each transaction may have several tags on it which indicate something about the type of transaction; this is a comma-separated list of these tags. For example, any capital expenditure will have
:CAPEX
as a tag, any loan related transaction will have:loan
, and so on.
For most questions about money whose results are not needed on-model, it should be possible to produce the answer by processing the transactions log using another tool.
Now we will look into the different commands which produce transactions, and the ways to ask about the costs of particular operations within the model, for specific reporting or for making decisions.
11.3 Costs for measures
As mentioned, measures generate transactions to pay for their installation.
All measures have a capex:
argument; when a measure is installed, if it is suitable, it will compute its capex:
after the installation is finished and immediately produce a transaction which causes the payment of that amount from the house to a payee called :market
.
The transaction will have the tag :CAPEX
.
Measures with an opex:
argument compute its value after installation, and cause the future payment of that amount each year at the end of the year, tagged :OPEX
.
If the measure is later removed or replaced, the associated opex payments will stop.
11.4 Costs for fuel
As mentioned, fuel costs are incurred by every house at the end of the year.
The pricing structure is determined by the tariffs for the house; these are described later in the 13 section.
Tariff-related transactions are always tagged with their fuel and with the :fuel
tag.
You can add other tags on the charge
command if you want to, for disambiguation of the transaction log.
11.5 Examining costs on-model
For on-model questions, you can use several different commands to look at the costs associated with particular types of transactions on houses or accounts.
11.5.1 Immediate costs
For inspecting the immediate cost of commands which are currently happening, for example within a probe
report, or to compute the subsidy required for a newly installed measure, the model has the commands net-cost
and capital-cost
.
These commands add up the costs due to transactions which have happened to the current house as a result of the action which is computing the value; normally this is the action which contains the net-cost
or capital-cost
command (this is explained in more detail later, in the section on ephemeral values). capital-cost
sums the cost of all transactions which are tagged :CAPEX
, whereas net-cost
sums the cost of all transactions.
You can also parameterise net-cost
to sum a particular subset of transactions; this is documented in more detail in the manual, but you can for example write
(net-cost tagged:"!:CAPEX")
To determine the net cost of the current action excluding any :CAPEX
transactions.
The matching rule supplied to tagged:
follows the same form as the rules for matching flags a house, except it matches tags on the house's transactions instead.
11.5.2 Historical costs
Where net-cost
and capital-cost
allow you to examine the costs associated with what the model is currently doing, the commands house.sum-transactions
, house.balance
and account.balance
allow you to ask the model about the total expenditure that has happened so far.
The first of these, house.sum-transactions
, is like a version of net-cost
which includes all the transactions that have happened to a house; with no arguments it is a synonym for house.balance
, and will produce the net effect of all the money transferred in the simulation so far on a house's bank balance.
Remember that the model does not simulate receipt of income, so this will usually be negative and become moreso over time as the house pays for fuel and maintenance, and for any measures it receives (less any subsidies).
To find out about the total impact of a subset of transactions you can write, for example:
(house.sum-transactions tags: ":CAPEX")
This is equivalent to capital-cost
, but gives the capital expenditure of the current house over the whole simulation so far.
Finally, account.balance
is the reverse of house.balance
, telling you how much money was transferred from or to a particular "global account".
For example, writing
(account.balance ":market")
Will tell you the total amount spent on measures in the simulation.
11.6 Paying for measures with subsidies or loans
So far we have discussed only things which cost a house money, like installing measures or paying bills. Sometimes you may want to install measures but to pay for some part of their cost on behalf of the house.
There are two forms of subsidy available:
- Immediate subsidies which are not recouped, using
finance.with-subsidy
andfinance.fully
, or - Immediate subsidies which are recouped later by a loan, using
finance.with-loan
All of these three commands work in the same way: you write them around another action, and they do the action but also provide the subsidy and set up any loan.
For example:
(finance.fully (measure.standard-boiler fuel:mainsgas capex: 1000))
will perform the inner action (installing a new boiler, and hence costing 1000 pounds), and then (if the inner action is suitable), it will provide a subsidy whose value equals the net cost of performing the inner action (in this case, 1000 pounds).
As a result, if you were to write:
(probe capture: [(net-cost)] (finance.fully X))
the net-cost
captured would always be zero, no matter what X
was, because finance.fully
always provides a subsidy which cancels out the net cost of the thing it's financing.
In a less-generous policy, we might use finance.with-subsidy
, in which we have the option to calculate the value of the subsidy using a rule:
(finance.with-subsidy subsidy: (min 500 (net-cost)) (measure.standard-boiler fuel:mainsgas capex:1000))
In this example we use the rule that the subsidy is whatever is least of the net cost of the measure and 500, effectively clamping the upper end of the subsidy to 500 pounds. In this case if we were to add a probe
and capture the net-cost
for the entire action, it would be 500, as the house would incur the capital cost of 1000 and then receive the subsidy of 500.
Finally, if we were approaching the nadir of policy generosity we might be looking to saddle the house with a loan, which naturally the householders would only accept if it were economically rational. To model this you can use finance.with-loan
:
(finance.with-loan principal: (net-cost) rate: 5% term: 10 (measure.standard-boiler fuel:mainsgas capex:1000))
In this example, a house will install a boiler, incurring a capital cost of 1000 pounds, but then it will be given a 1000 pound immediate subsidy (the principal:
, which in this case is the net cost of 1000 pounds).
So far the house is doing well - it has a free boiler - but it also incurs the obligation to repay the loan, with interest, which happens on the anniversaries of the date of installation.
The loan payments are of equal size, and the interest on the remaining principal is compounded annually.
These finance commands can be combined; for example, you might want to pay for some of a measure using a subsidy, and then settle the remainder with a loan; here is a more complex example:
(finance.with-loan principal: (net-cost) rate: 6% term: 10 (finance.with-subsidy subsidy: (min 500 (* 2 (fall-in (house.emissions)))) (do (measure.standard-boiler fuel:mainsgas capex:1000) (measure.wall-insulation type:cavity capex: (+ 400 (* size.m2 10)))) ))
Although the loan is written at the top, it is the outermost thing and so it is considered last; you should read this as something like:
- Finance doing the following with a loan, where the principal of the loan should equal the net cost of the thing we are financing
- Finance doing the following with a subsidy, where the subsidy amount is whichever is least of 500 pounds, or double the fall in emissions
- Install as a package, the following:
- a boiler, at a cost of 1000 pounds
- wall insulation, at a cost of 400 pounds and 10 pounds per square meter insulated
- Install as a package, the following:
- Finance doing the following with a subsidy, where the subsidy amount is whichever is least of 500 pounds, or double the fall in emissions
Filling in some example figures we might have:
- Before we can do the loan, we must do the subsidy:
- Before we can do the subsidy, we must install the measures
- Installing the measures, we have
- boiler, at a cost of 1000 pounds
- insulation, at a cost of (say) 1500 pounds
- the net cost of all this is now 3500
- Now we can consider the subsidy amount; say we reduce emissions by 300 kgCO2/year, we will offer a subsidy of 500 as this is less than 600.
finance.with-subsidy
gives the house 500 pounds, so the net cost of the subsidised package as a whole is now 3000
- Before we can do the subsidy, we must install the measures
- Now we can work out the principal for the loan, which is the
net-cost
of the subsidised package; this is 3000 - The house is given 3000 pounds now, and made to repay this over 10 years at 6% compounded annually.
This example also illustrates how commands like net-cost
refer to the net cost of the action which contains them, rather than the net cost of everything that has happened in the scenario.
11.7 Adding costs rather than subsidies
The finance.with-subsidy
cannot produce costs for a household; if the subsidy:
argument is negative (which would represent a cost), no subsidy is offered.
The command measure.with-cost
is an equivalent which allows the addition of further costs to another action.
For example, if you wished to model a service where an advisor visits a house and charges them fifty pounds for the suggestion to decrease their thermostat setting, you could write
(measure.with-cost cost: 50 (action.set-heating-temperatures living-area-temperature: 19))
This would reduce the house's heating temperature, and charge the house 50 pounds for doing so.
11.8 Setting up arbritrary payments in the future (obligations)
The previous section shows how we can set up future payments by offering a loan.
There is one more finance command, finance.with-obligation
, which is more complicated than the other sorts.
This command allows you to commit a house to an arbitrary sequence of future payments.
It has two important arguments:
- amount:
the
amount:
argument gives a command for working out the amount that the house must pay (or if negative, receive). This command is computed each time the house has to make a payment, so it can change over time.As a consequence, you may want to store the amount a house should pay or receive in a variable, if the value of the command for computing it would change over time, or if it contains an ephemeral value like
net-cost
.- schedule:
- the
schedule:
argument determines when the obligation is due, and so when theamount:
argument will be used to compute a value for a transaction.
For example, to set up a house to receive 100 pounds every year for a decade, you could write
(finance.with-obligation amount: -100 ; negative as it is a cost schedule: (periodic-payment interval: "1 year" lifetime: "10 years") ;; we need to finance something! (action.do-nothing))
The amount is computed each year, so we might write:
(finance.with-obligation amount: (sim.year) schedule: (periodic-payment interval: "1 year" lifetime: "10 years") ;; we need to finance something! (action.do-nothing))
This would make the house pay the value of the current year (so 2016 pounds in 2016, 2017 in 2017 and so on) each year for ten years.
This is because the sim.year
command when used produces the simulation's current year, and the future payments are computed in the year when they are due.
11.9 Distinguishing particular costs
As mentioned, all transactions have a set of tags on them; most finance-related commands can either filter the transactions they include by including or excluding transactions which have certain tags, or can put specific tags on the transactions they generate.
Actions which generate transactions will typically have the arguments:
- tags:
- a list of tags to write on every transaction that the action is responsible for, and
- counterparty:
- the name of the global account which is on the other side of the transaction from a house.
11.10 Computing present values, prediction, ways of discounting things
As mentioned, the model makes no attempt to represent things like cost of capital, inflation, or any other financial matters. All the model captures is the movement of particular amounts of money from place to place; you can set the prices for these things, and it is up to you whether you wish to use real (inflation-adjusted) prices in the future parts of the simulation or not. Similarly, if you wish to compute the discounted value of running a particular policy for the wider world, it is up to you to work this out off-model, bearing in mind the decisions you have made about what prices to use and what they ought to mean15.
However, there are circumstances in which a scenario needs to compute a discounted value on model. In the NHM this is usually a matter of prediction, rather than accounting; whilst the model does not represent discounted or inflated figures as the ground truth, it does allow you to model the prediction of future values or costs.
For example, you might want to represent a situation in which a house makes a decision between some alternatives which have an immediate cost in exchange for a future benefit based on their expected (possibly discounted) value over some horizon.
There are a couple of simple commands which produce predictions for the current year:
house.fuel-cost
predicts the costs generated by the obligation to pay the fuel bill at the end of the current year, assuming that the house continues to use energy at the current rate and remains on the same tariffhouse.annual-cost
predicts all of the costs the house is expecting to incur in the next year, or a subset which have certain tags
Together with these is the command future-value
, which computes the value of another command each year for some number of years into the future, insofar as this can be predicted, and adds up those values.
For example, you can write
(future-value horizon: 3 (house.annual-cost))
to predict the annual cost over the next three years. This will compute the annual cost thrice, each time in a hypothesis, one which looks like the current date, then 1 year hence, and finally two years hence. The sum of these values will be produced.
Because the future-value
command computes its argument multiple times in a new hypothesis for each year up to the horizon, it can also produce discounted sums by multiplying the value produced in each future year by an appropriate discount factor.
You can write whatever discount function you like, but the model includes exponential-discount
and hyperbolic-discount
which probably cover most needs.
For example, if you wished to calculate the future costs of a house's obligations including an exponential discount, you could write
(future-value horizon: 10 (exponential-discount rate:5% (house.annual-cost)))
The exponential-discount
command determines the discount factor in each year and multiplies its first argument by it, in this case the house's annual cost. Putting this all together, you can calculate a "net present value" for doing something by writing
(+ (capital-cost) ;; the immediate capex (future-value horizon:10 (exponential-discount rate:5% (house.annual-cost))))
Note here how the capital-cost
is not inside the future-value
but is added to it; this is because the future-value
of capital-cost
is zero, since in the future hypotheses being produced no further measures are being installed.
The astute reader may be wondering how exponential-discount
works; after all, to work out a discount factor for each future year it needs to know how far into the future the prediction has got when it is worked out.
The answer is that the hypotheses generated by future-value
do provide access to the simulation's true current year as well as to the hypothetical year using the notion of different levels of foresight.
When you use future-value
, you can specify the argument predict:
to specify a the foresight levels which should be granted to the house.
These determine which things should change as the prediction moves into the future, and which things should be opaque.
For example, if you wished to do the present value calculation above, but allowing the house perfect foresight about future fuel prices but not future carbon factors or weather, you could write:
(+ (capital-cost) ;; the immediate capex (future-value predict: [tariffs] horizon:10 (exponential-discount rate:5% (house.annual-cost))))
In this, the model would compute the discounted annual cost using future fuel prices for the house's current tariffs, but without future carbon factors, weather, or anything else time-sensitive.
If the house's tariff included a charge which depended on a house's emissions, whose unit price changed over time, the future-value
above would reflect the changes in the rate, but would not reflect any changes in emissions factors included in the scenario.
If you wish your own scenario functions to have different degrees of predictability, you can use the foresight:
argument supplied on time-sensitive functions; for example, if you had a function which changed based on the year:
(function.case (when (= (sim.year) 2020) 10) default: 9)
You can control the degree of predictability of this function by providing a foresight:
argument to sim.year
. Say any future-value
which is capable of predicting fuel prices ought to be able to predict the value of this function, you could write
(function.case (when (= (sim.year foresight: tariffs) 2020) 10) default: 9)
If you were then to predict the future value of this function for 2 years from 2019 with future-value
, you could get one of two outcomes:
(future-value horizon: 2 predict: [] (function.case (when (= (sim.year foresight: tariffs) 2020) 10) default: 9))
In this case, where we have said that nothing can be predicted, the function would be computed twice:
- Once in hypothetical 2019, where the function would produce 9
- Once in hypothetical 2020, but where
sim.year
would be blinded to the hypothetical year, and so would still appear to be 2019, so the function would produce 9
The value would be 18.
If we change to say predict: [tariffs]
, the result will instead be 19, as we have
- An evaluation in hypothetical 2019, producing 9
- Another in hypothetical 2020, including for the
sim.year
, producing 10.
This also answers the question above of how exponential-discount
can work; you can always compute the number of years into the future a function is being predicted by the following expression:
(- (sim.year foresight:always) (sim.year foresight:never))
The first term in the difference is always the year being predicted, and the second is always the true year; these levels of foresight are special and can never be prevented or overridden, so this always evaluates to zero in the first hypothetical year, one in the second and so on.
An exponential discount factor of (say) 5% is then just
(/ 1 (pow 105% (- (sim.year foresight:always) (sim.year foresight:never))))
Indeed, this is exactly the value computed by exponential-discount
given a rate of five percent.
11.11 TODO Exercises about money
12 Ephemeral values like net-cost
, capital-cost
, size.m2
Most model commands' meanings depend on the context where they are used.
The commands net-cost
, capital-cost
, size.m2
and size.kw
all refer to the closest action containing them, which you might call the "current command", so their meaning is "the net cost of the current command so far", and so on.
To make this clearer, here are some examples:
(finance.with-subsidy subsidy: (net-cost) (do (measure.standard-boiler capex: 500 ...) (measure.wall-insulation capex: 200 ...)))
In this example, net-cost
refers to the net cost of the finance.with-subsidy
so far; since this is before the subsidy has been given (because we're working out the amount), this means that it is the net cost of the inner do
command, which is 500 + 200 = 700.
If we were to add a further do
command around the finance.with-subsidy
like so:
(do (finance.with-subsidy subsidy: (net-cost) (do (measure.standard-boiler capex: 500 ...) (measure.wall-insulation capex: 200 ...))) (set #variable (net-cost)))
The second use of net-cost
's nearest containing action is the outermost do
, and the net cost of that at the time when the set is worked out is zero.
This is because by the time we get to the set
the actions which are part of the do
have done the following:
- Spent 500 pounds on a boiler (net cost of the outer do is now 500)
- Spent 200 pounds on insulation (net cost of the outer do is now 700)
- Offer a subsidy equal to the net cost of the inner do, i.e. 700; at this point the net cost of the whole action is 700-700 = 0
As a result the variable #variable
will be set to zero.
The capital-cost
command is similar, except it only includes capital costs, so if we were to say
(do (finance.with-subsidy subsidy: (net-cost) (do (measure.standard-boiler capex: 500 ...) (measure.wall-insulation capex: 200 ...))) (set #variable-1 (net-cost)) (set #variable-2 (capital-cost)))
The outcome would be that #variable-1
is zero but #variable-2
is 700 (as it is just the capital expenditure, not the subsidy).
The sizing commands size.m2
and size.kw
are normally used in the capex:
arguments for insulation or heating measures, in which context their closest containing action is the measure itself, so they refer to its size:
(measure.standard-boiler size: (size min:0 max:1000 (log (house.total-floor-area))) capex: (+ 100 (* 500 (size.kw))))
In this example, the size for the boiler is determined to be the log of the house's floor area, and the capital cost is 100 plus 500 times that size (as explained in the section on heating measures, the size has no significance apart from its effect on size.kw
and things defined in terms of it).
The same rules apply to the propagation of size.kw
as to capital-cost
and net-cost
; for example, if you write:
(do (measure.loft-insulation ...) (measure.wall-insulation ...) (set #variable-1 (size.m2)))
Then the variable #variable-1
will contain the area in square metres insulated by the closest containing action, which is the do
command; this is the area insulated by the loft insulation measure plus the area insulated by the wall insulation measure. If you wanted to record each one separately you could write something like:
(do (do name:A (measure.loft-insulation ...) (set #variable-1 (size.m2))) (do name:B (measure.wall-insulation) (set #variable-2 (size.m2))))
In this case, the first size.m2
's closest containing action is the do
which we have given the name A
, and the second's is the do named B
, so #variable-1
is the area of loft insulation and #variable-2
the area of wall insulation.
The closest containing action can equally well be something like a probe report:
(probe name:p capture: [(capital-cost)] (do .... ))
In this example, the capital-cost
command will be computed twice, and both times it will give the capital cost of the closest containing action, which is the probe
.
The first time, the capital cost will be zero, because the application of probe has incurred no capital costs. The second, it will be the capital cost incurred by applying the probe, which is the capital cost of doing the action inside the probe.
12.1 Places where ephemeral values will not work, or will be confusing
The context-sensitive nature of ephemeral values creates some potential for confusion:
- Ephemeral values will not work in
aggregate
reports; this is because there is no containing action when the value is worked out in the report. Asking an aggregate report to computecapital-cost
is like asking for the capital cost of the report; the model has no way of knowing that you mean to say the capital cost of the measures you put in in the last year, or over the simulation so far, or similar. - Ephemeral values will not work as expected in 16 produced by the
under
command orset under:
; this is the value will be the value for the actions applied in the hypothesis. The model has no way of knowing if you mean to ask for the value outside the hypothesis, and always produces the value inside, even if it happens to be zero (as it will be for many commands).
13 Tariffs
The previous section mentions the fuel bill and tariffs, without going into detail about how they work. This part of the guide explains how to define and use tariffs. Before getting into the language, let's go through how tariffs are represented in the NHM.
An NHM tariff is a package of rules which define how to calculate the bill a house should pay for a subset of fuels. Each house can be on several different tariffs at once, but they can only have one tariff for each fuel, and if they take a tariff they must use it for all the fuels it covers. For example, imagine you have the following tariffs available:
Dual fuel
Fuel Unit Price Electricity 10p Gas 5p Cheap electricity
Fuel Unit Price Electricity 8p Expensive gas
Fuel Unit Price Gas 10p
A given house could either take the Dual fuel tariff, or the Cheap electricity and Expensive gas tariffs. However, they could not have Cheap electricity and Dual fuel, to get the best of both worlds and pay 8p/5p.
The rules in a tariff may be almost arbitrarily complex; each fuel covered by the tariff has a rule, and the rule may contain multiple charges, where each charge has a function to compute the payment a house should make; this function may depend on the amount of energy the house has used, or it may be a constant value, or any other function that you can write.
A house's fuel bill is worked out and paid at the end of the year, every year; by default the model puts all the houses on an 'empty' tariff for each fuel, which makes no charges, so unless you write down some tariffs nobody will pay any bills.
13.1 Tariff syntax
Tariffs are normally defined in the context.tariffs
element in the scenario.
This has two named arguments, defaults:
and others:
, each of which can accept a list of tariff definitions.
The tariffs in defaults:
will be added to all the houses at the start of the run. Because of this, only one of these tariffs may cover each fuel. The tariffs in others:
can be any collection of tariffs, which houses may be switched onto later.
For example:
(scenario ... (context.tariffs defaults: [A B C] others: [X Y Z]))
The tariffs A
, B
and C
will be added to each house when it is created; X
, Y
and Z
will have no effect unless they are explicitly used by action.set-tariffs
(of which more later).
A tariff definition is either tariff
or tariff.simple
. Let's look at an example of tariff
first:
(tariff name: dual-fuel (fuel type: peakelectricity (charge (* 0.1 house.meter-reading))) (fuel type: offpeakelectricity (charge (* 0.1 house.meter-reading))) (fuel type: mainsgas (charge (* 0.05 house.meter-reading))))
This is a tariff implementing the Dual fuel example above; the house will be charge 10p/unit for peak and off peak electricity, and 5p for mains gas.
The house.meter-reading
command has some special behaviour:
- At any point in time, it evaluates to the total number of units of fuel that have not yet been paid for.
- Within a
fuel
command in atariff
, it refers to the unpaid for units of that specific fuel.
This also illustrates another point of interest: the Electricity
fuel type has no consumption in tariffs, in order to prevent accidental double-counting. It is always broken down into PeakElectricity
and OffPeakElectricity
, whether or not the house is on a tariff that has different charges for these two categories.
To complete the example above, we could write something like this:
(context.tariffs defaults: [ (tariff name: dual-fuel (fuel type: peakelectricity (charge (* 0.1 house.meter-reading))) (fuel type: offpeakelectricity (charge (* 0.1 house.meter-reading))) (fuel type: mainsgas (charge (* 0.05 house.meter-reading)))) ] others: [ (tariff name: cheap-electricity (fuel type: peakelectricity (charge (* 0.08 house.meter-reading))) (fuel type: offpeakelectricity (charge (* 0.08 house.meter-reading)))) (tariff name: expensive-gas (fuel type: mainsgas (charge (* 0.1 house.meter-reading)))) ] )
We could either write the dual-fuel
tariff in defaults, or cheap-electricity
and/or expensive-gas
, but we could not write dual-fuel
with either of the others, as dual-fuel
is incompatible with both of the others.
Now we can look at a few different kinds of tariff that we could write:
13.2 Examples of different tariffs you can write
13.2.1 Simple unit rates, and standing charges
Unit rates can be written using tariff
and charge
, as seen above; additionally the command tariff.simple
gives a slightly simpler syntax for unit price + standing charge type tariffs:
(tariff.simple name: dual-fuel (function.simple-pricing fuel: mainsgas unit-price: 0.05) (function.simple-pricing fuel: peakelectricity unit-price: 0.1) (function.simple-pricing fuel: offpeakelectricity unit-price: 0.1))
This has a slightly flatter structure and may be easier to read, but is otherwise interchangeable.
Along with the price, function.simple-pricing
accepts a standing-charge:
argument, but there is a subtlety to using it: the standing charge is applied to anyone who is on the tariff, whether or not they actually use any of the given type of fuel.
This means that you can put someone on a tariff like this:
(tariff.simple name:biomass (function.simple-pricing fuel: biomasswood unit-price: 0.06 standing-charge:100))
and they will be charged 100 pounds per year even if they use no biomasswood
fuel.
To write a standing charge which does not get applied if you use no units, you can write it out using the more complex tariff structure:
(tariff name: biomass (fuel type: biomasswood (charge (* house.meter-reading 0.06)) ;6p per unit (charge (function.case (when (> house.meter-reading 0) 100) default: 0)) ;; 100 pounds if we have used more than 0 units ))
Here we have used function.case
to define the standing charge part - the value of the charge is 100 only when the meter reading exceeds 0.
13.2.2 Time-series in tariffs
The examples we have shown so far have simple functions for prices.
However, each tariff is computed again each year, so you can use more complex price functions, including ones which change over time.
There are several ways to write a time-series in the NHM; the most readable is probably using the ~lookup-table
macro command, which translates into a use of the lookup
command. Here is an example:
(tariff name: gas-changing-over-time (charge (* house.meter-reading (~lookup-table row-keys: sim.year [YEAR PRICE] [<2000 0.1] [2000..2020 0.15] [2021 0.14] [>2021 0.13]))))
In this example, the charge will be calculated as the meter reading times the value drawn from the table; the table is keyed on the current year, so the cost will change when the tariff is computed in different years.
13.2.3 Rising-block tariffs
Another kind of tariff you could model is a rising-block tariff (indeed, a tariff with a standing charge is a simple form of rising-block tariff). For example, say we wanted to charge 2p/unit for the first 500kWh of gas used, and 8p/unit after that; we could write:
(tariff name:rising-block (fuel type:mainsgas (charge (* 0.02 (min 500 house.meter-reading))) ; first 500 units (charge (* 0.08 (max 0 (- house.meter-reading 500)))) ; after 500 units ))
13.2.4 Tariffs with a straight tax
Within a tariff you can use the net-cost
command to find the net cost of computing the cost for the current fuel so far.
For example, we could amend our rising block tariff to include a 20% tax:
(tariff name:rising-block (fuel type:mainsgas (charge (* 0.02 (min 500 house.meter-reading))) ; first 500 units (charge (* 0.08 (max 0 (- house.meter-reading 500)))) ; after 500 units (charge (* 20% net-cost)) ;; 20% of whatever the previous charges have cost for this fuel ))
Note that the net-cost
is not for the tariff
as a whole but just for the fuel
, so this would not affect other fuel
commands in the same tariff
.
To do this, you would need to introduce the same charge
into each fuel
.
13.2.5 Tariffs with an emissions tax
Tariffs do not have to be based on house.meter-reading
; you can introduce other components if you like.
For example, here we charge 1 pound per unit of emissions associated with each fuel:
(tariff name: dual-fuel (fuel type: peakelectricity (charge (* 0.1 house.meter-reading)) (charge (* 1 (house.emissions by-fuel: peakelectricity)))) (fuel type: offpeakelectricity (charge (* 0.1 house.meter-reading)) (charge (* 1 (house.emissions by-fuel: offpeakelectricity)))) (fuel type: mainsgas (charge (* 0.05 house.meter-reading)) (charge (* 1 (house.emissions by-fuel: mainsgas)))))
The charge
has been added into each fuel
; we have had to specify the fuel
twice, because house.emissions
is not like house.meter-reading
, and does not have special behaviour based on being inside a fuel
; only house.meter-reading
works like this.
13.3 Switching people's tariffs
Now we have seen how to define tariffs, and how to set the default tariffs for a scenario, we should look at how to change people's tariffs.
When using these commands, remember that the model will not switch people back off particular tariffs unless you tell it to.
For example, if you put someone on an economy 7 type tariff because they have a storage heater, and then you replace their storage heater with a boiler, the model will not switch them on to another tariff for you. They will continue to be on their original tariff until such time as the action.set-tariffs
command is used to change them onto another tariff.
13.4 Policies that modify your bill whatever tariff you are on (like the Warm Home Discount)
Along with tariffs, a house's bill may include so-called extra charges.
An extra charge is what it sounds like - an additional charge (or subsidy) added on to your bill after the tariffs have been computed.
Unlike tariffs, extra charges are not exclusively responsible for particular fuels.
They can be added to a house with action.extra-fuel-charge
, and removed with action.remove-fuel-charge
.
For example, if you wanted to offer all houses a subsidy of 100 pounds toward their fuel bill you could write
(apply (action.extra-fuel-charge (extra-charge -100)))
The value is negative because it is a subsidy.
In this example, even houses with a bill of less than 100 pounds would receive a 100 pound subsidy.
If you wanted the subsidy to be up to 100 pounds you can refer to the cost of the other tariffs using net-cost
again:
(apply (action.extra-fuel-charge (extra-charge (- (min 100 net-cost)))))
Here we have offered a subsidy of 100 pounds or the net-cost
of the other tariffs, whichever is least.
If you wanted the net cost of a specific fuel, you can associate a fuel type with the extra charge, which makes net-cost
aware only of the costs for that fuel.
(apply (action.extra-fuel-charge (extra-charge fuel: mainsgas (- (min 100 net-cost)))))
Here we have offered a subsidy of up to 100 pounds off the gas bill. Unfortunately the division of electricity into peak and off-peak applies here as well, making writing an equivalent rule for two fuels a little more difficult:
(apply (action.extra-fuel-charge (extra-charge (- (min 100 (net-cost tagged: ":<peak,offpeak>electricity"))))))
Rather than associating the charge with a fuel, we have used the tagged:
argument of net-cost
to refer only to transactions tagged with either :peakelectricity
or :offpeakelectricity
.
To remove a charge, you will need to refer to it by its name:
. For example, if you had offered all houses a mains gas subsidy as above, except with a name
(apply (action.extra-fuel-charge (extra-charge name: mains-gas-subsidy fuel: mainsgas (- (min 100 net-cost)))))
But then you wanted to find some houses and remove the subsidy from them, you would elsewhere write something like:
(apply to:(sample 10%) (action.remove-fuel-charge #mains-gas-subsidy))
In this, #mains-gas-subsidy
refers to the extra-charge
above whose name:
argument is mains-gas-subsidy
.
You can also remove all extra charges from a house just by applying (action.remove-fuel-charge)
. This is useful if you are computing a hypothetical fuel cost, of which more later.
13.5 TODO Exercises about tariffs
14 Choices and packages of measures
So far we have mostly looked at how to apply individual measures.
You can put several measures directly inside an apply
command, but they are not applied to the houses one at a time; instead the first measure is applied (potentially failing) to all the houses, then the second, …
To apply a series of measures to a specific house, the model offers some other commands which compose different actions together. These produce new, compound actions which do things like:
- Install both A and B, or neither
- Install whichever is cheapest of A and B
- Install A in rural houses and B in urban houses
- Install whichever is cheaper of A and B in rural houses, or whichever is cheaper of C and D in urban houses
Compositions like this can be repeated ad infinitum, as each combination is itself an action which can be further combined. In any case, all the commands to combine actions have a common "shape"; they each take some n actions and put them together into one single action.
14.1 Picking between alternatives
First let us consider two ways to turn n candidate actions into one by selecting one as the one to do, and disregarding the rest. The first way makes the selection based on information which is independent of the effects of any of the candidates. The action to take is chosen before we do it, and so even before we know whether it will work. The second way is to make the selection after the fact, with knowledge of the effects that each candidate will have.
14.1.1 Before the fact (if statements)
To make a choice before the fact, the model provides the action.case
command.
It ties each candidate together with a logical test; when the action is applied to a house, the first candidate whose associated test passes for that house is used, and the action.case
behaves as that candidate.
For example, if you wanted to install measure A in rural houses and measure B in urban houses, you might write:
(action.case (when (house.morphology-is rural) A) (when (house.morphology-is urban) B))
Each of the when
commands in the action.case
has two arguments:
- The test, and
- The candidate action
Upon being applied action.case
goes through the when
conditions in the order they are written down, working out each test in turn.
Once it encounters a test that passes (produces a true
value), it behaves as the associated action.
It does not consider any subsequent when
commands after the action has been performed, so it will only ever use one of the candidates.
If your conditions are not exhaustive, you can use the default:
argument to specify a 'fallback' action that will be applied if and only if none of the when
commands' tests pass.
This is one way to write an "if" type action:
(action.case (when (> 1000 (house.total-floor-area)) (measure.floor-insulation ...)) default: (action.do-nothing))
In this example, we will apply measure.floor-insulation
if (> 1000 (house.total-floor-area))
, and we will (successfuly) do nothing otherwise.
In truth, action.do-nothing
is redundant here, as it is used as the default:
even if you do not write it down; hence writing (action.case)
is equivalent to writing (action.do-nothing)
.
The suitability and success of action.case
(of which more later) is equal to the suitability of the action which is chosen, so the when
that is selected need not be a suitable option.
If you wish to restrict yourself to suitable alternatives, you can include (house.is-suitable-for ...)
in the when
's test.
14.1.2 After the fact (least-cost, etc.)
action.case
is all very well when we can decide on our candidate without thinking about what the candidate will do.
However, we often want to make a choice between alternatives based on something about their outcome; for example, we might want to choose the cheapest option, or the most cost-effective.
The command for this is called choice
, and has the following form:
(choice select: <<rule for picking the winner>> <<option A>> <<option B>> <<option C>> ... )
Rather than using a simple logical test, as in action.case
, choice
uses a so-called selector for picking the winning candidate.
These selection commands are provided so that you can decide between the multiple alternatives based on information about all of their outcomes.
For example, there is no logical test in the NHM which can tell you which option has the least cost, as the tests are only defined to know about a single house.
Instead, you can use the selection command select.minimum
, which will choose the candidate for which a certain value is minimum, after the fact.
Let us look at minimising emissions by installing a measure:
(choice select: (select.minimum house.emissions) <<option A>> <<option B>> <<option C>>)
This choice command will do the following;
- From the current state of affairs, it will produce three hypotheses (of which more later)
- A hypothesis in which only option A has taken place
- A second in which only option B has taken place
- A third in which only option C has taken place
- It will offer these hypotheses to the selection rule, which will pick one of them to come true.
If any of option A, B or C prove to be unsuitable, their hypotheses will not be offered to the selection rule.
In the case of
select.minimum
, this means:- Calculate
house.emissions
according to hypothesis A; this is the emissions that will result from doing option A - Calculate the equivalent for hypotheses B and C
- Compare these three values, and select whichever hypothesis has the minimum value.
- Calculate
So this new combined action does whichever of the three options is suitable and has minimum emissions after the fact.
There are several other selection commands:
select.maximum
; this is hopefully self-explanatory by contrast with =select.minimum.select.weighted
; this performs a weighted random selection; for example, if we write(choice select: (select.weighted 1) A B C)
The choice will produce three hypotheses for A, B and C, and then ask
select.weighted
to pick one.select.weighted
will in turn compute the value of its argument (here, just 1) in each hypothesis (similarly to howselect.minimum
does), but rather than taking the minimum it will:- Compute the total weight of all suitable options; in this case the argument is always 1, so the total weight is 3
- Randomly select one of the alternatives using its weight normalised by the total as a probability. In this case, that means that each option has a 1/3 chance of being selected. If you were to apply such a choice to 1000 houses, you would expect about 333 to get option A, 333 to get option B, and 333 to get option C (assuming they are all suitable for all the alternatives). Note how the weight is a bit different to a probability, as a result of the normalisation.
select.filter
; this reduces the range of candidates using a logical test, and then uses another selection command to pick from the reduced list of candidates. For example, if you wished to select the option with minimum emissions whose net cost of application was not greater than 1000 pounds, you could write:(choice select: (select.filter test: (< net-cost 1000) selector: (select.minimum house.emissions)) A B C)
This will:
- Produce the three hypotheses for A, B and C (again, allowing for suitability)
- Ask
select.filter
to pick one; it will compute thetest:
argument in each case, and collect up those for which the test passes. For example this might just be B and C, if A were too expensive. select.filter
will then askselect.minimum
to pick an alternative from this restricted subset; the result will then be one of B or C.
select.fallback
; this is a slightly complicated rule. It allows you to try one way of selecting between alternatives, but "fall back" to another way if the first way does not admit any of the alternatives; this is only really relevant in combination withselect.filter
. For example, you might want to say "if any of the options reduces energy use to less than 10000 units, do the cheapest; otherwise, do that which has the maximum reduction".(select.fallback (select.filter test:(< house.energy-use 10000) selector: (select.minimum net-cost)) (select.minimum house.energy-use))
In this example the first selector will be applied, but if it admits none of the alternatives (because they all fail the
test:
), it will use the second selector instead.
14.1.3 Choices to do with before-to-after differences
All of the choices shown above are a pure function of the state of affairs after the alternative has been considered. It does not matter what the house was like beforehand; in any situation where the outcomes are the same, the same choice will be made. However, some models must involve a function of the change resulting from an alternative; if you are considering cost-effectiveness, this is to do with the ratio of effect to cost, and effect is typically a change. In recent versions of the model there are two ways to refer to these kinds of changes. One involves storing the quantity of interest before the change in a variable, and is explained in the section on variables, and one involves referring to the original value of the quantity of interest, which is explained in the section on hypotheses. To summarise quickly here, one may either write:
(do (set #initial-value (house.emissions)) (choice select: (select.maximum (/ (- #initial-value house.emissions) net-cost)) A B C ))
or (but only in recent model versions)
(choice select: (select.maximum (/ (fall-in house.emissions) net-cost)) A B C)
For more about these see the respective sections below.
14.1.4 action.do-nothing
in choices
We have mentioned several times that choices offer only the suitable alternatives to their selection rule.
Suitability rules are discussed in more detail below; however, we mention here that choice
will only ever select a suitable alternative, and if there are no suitable alternatives it will itself be unsuitable and will fail.
If you wish to ensure that a choice is always suitable, you can include action.do-nothing
as one of the alternatives; it is always suitable, and so (so long as it is not ruled out by a use of select.filter
) it is always available for selection.
14.1.5 Ties in choices
In all cases, ties are resolved by order of writing in the scenario; for example if using (select.minimum x)
, if two candidates both produce minimal x
the first one written will be selected.
14.2 How to combine several options together; making packages
The other way to assemble n actions into one is to do them in a defined order; the model provides the do
command for this.
Its syntax is quite simple:
(do A B C D ...)
When applied to a house, this command will first apply measure A, and then apply measure B to the house (in the condition left by applying A), then apply measure C to the house (in the condition left by B), and so on.
The do
command has special behaviour in two aspects: first, it uses 16 to ensure that only suitable packages can have any effect, and second it allows you to see and record "ephemeral" values like net-cost
and size.m2
whose values depend where they are being evaluated in the scenario.
Remember that any action applied to a house can go one of two ways:
- The action succeeds, because it is suitable for the house, and it may have some effect on the house like installing a boiler or changing some flags (this last part is not obligatory; there are two basic commands which are suitable and succeed but have no effect, namely
action.do-nothing
and an empty(do)
statement). - The action fails, because it is unsuitable for the house, in which case it is forbidden for the action to have any effect on the house, and if it does it's a bug in the model.
Bearing this in mind there is a natural question for do
: what happens if one of the measures that we are doing is unsuitable, and fails?
The answer depends on the all:
argument to do
:
14.2.1 All-or-nothing packages
When do
is used with all: true
, or with no value given for all:
as the default is true
, the do
action is only successful when every action inside it completes successfully.
If all but the last action in a do
command succeed, the whole do
fails, and it follows from the rules about suitability mentioned above that the do
as a whole must have no effect; that is to say, all of the effects of the successful actions in the do are reversed if action fails.
For example, the action:
(do (measure.wall-insulation type:cavity ...) (measure.standard-boiler fuel:mainsgas ...))
Is a package giving the all-or-nothing proposition of cavity wall insulation and a new boiler together, or neither one. Only a house on the gas grid with an uninsulated cavity-type wall could be be affected by this action, as for any other house one of the measures would fail precluding the other from having an effect.
14.2.2 Try-everything packages
If the all:
argument is false
, the do
command will resolve the problem differently; it will ignore the success or failure of the actions within it and simply try to do them one after another, and it will always be considered to succeed.
The effect of the do
will be the compounded effect of whichever actions where successful, since none of the unsuccessful actions will have had any effects (as this is forbidden).
Changing our previous example:
(do all:false (measure.wall-insulation type:cavity ...) (measure.standard-boiler fuel:mainsgas ...))
We have an action that can have four outcomes:
- A house on gas, with an uninsulated wall
- Wall insulation is installed and then a boiler is, successfully
- A house off gas with an uninsulated wall
- Wall insulation is installed, successfully
- A house on gas, without an uninsulated wall
- A boiler is installed, successfully
- A house off gas without an uninsulated wall
- Nothing happens at all, successfully
This form of the do
command just throws some measures at a house without caring what sticks.
14.3 Access to ephemeral values
As mentioned, the do
command has special handling of ephemeral values; when using the following commands within a do
set
increase
decrease
consume
fail-unless
The values of net-cost
, capital-cost
, size.m2
, size.kw
, original
, rise-in
and fall-in
are all computed with respect to the effect of the do
command so far.
For example, if we amend the example above:
(do (set #x-1 (net-cost)) (measure.wall-insulation capex: 100 type:cavity ...) (set #x-2 (net-cost)) (measure.standard-boiler capex: 200 fuel:mainsgas ...) (set #x-3 (net-cost)))
The variables #x-1
, #x-2
and #x-3
will take the values 0, 100, and 300 respectively; this is because when the first set
happens the net cost of the do
is zero (as it has done nothing to spend money), when the second happens the net cost is 100 as the do
has bought some insulation for 100 pounds, and when the third happens it is 300 as the do
has bought both 100 pounds of insulation and 200 pounds of boiler.
The same rules about suitability apply here, so the outcome would either be these values of the variables and the two measures installed, or the variables unchanged and neither measure installed.
The choice
command has a similar rule for its select:
argument; when values in the select:
are computed they are computed with respect to a particular alternative, and so net-cost
is the net cost of doing the alternative, capital-cost
the capital cost, (rise-in x)
the increase in x
caused by doing that alternative and so on.
14.4 Exercises about choices and packages
- For each of these situations, which outcome or outcomes would you expect to see (you may want to read the section above on flags first):
Applying the action
(action.case (when (house.region-is London) (house.flag london)) (when (house.built-form-is Detached) (house.flag detached)) (when (> house.total-floor-area 500) (house.flag large)))
to each of these houses, all having no flags
- A detached house in london with a floor area of 400
- A terraced house in london with a floor area of 600
- A terraced house in wales with a floor area of 500
- A terraced house in wales with a floor area of 600
- A detached house in wales with a floor area of 600
Applying the action
(do (house.flag A) (house.flag B) (house.flag C))
to a house with no flags
Applying the action
(choice select: (select.minimum house.total-floor-area) (house.flag A) (house.flag B) (house.flag C))
to a house with no flags
Applying the action
(choice select: (select.weighted 5) (house.flag A) (house.flag B) (house.flag C))
to a house with no flags
Applying the action
(do (house.flag A) (action.do-nothing test-flags: !A) (house.flag B))
to a house with no flags
Applying the action
(do all:false (house.flag A) (action.do-nothing test-flags: !A update-flags: C) (house.flag B))
to a house with no flags
Applying the action
(do (action.do-nothing test-flags: !A update-flags: C) (house.flag A) (house.flag B))
to a house with no flags
Applying the action
(choice select: (select.minimum 0) (do (house.flag A) (action.fail)) (house.flag B))
To a house with no flags
Applying the action
(choice select: (select.minimum #x) (do (house.flag A) (set #x 1)) (do (house.flag B) (set #x 2)) (do (house.flag C) (set #x 3)))
To a house with no flags or variables set
Applying the action
(choice select: (select.maximum #x) (do (house.flag A) (set #x 1)) (do (house.flag B) (set #x 2)) (do (house.flag C) (set #x 3)))
To a house with no flags or variables set
Applying the action
(choice select: (select.weighted #x) (do (house.flag A) (set #x 1)) (do (house.flag B) (set #x 2)) (do (house.flag C) (set #x 3)))
To a house with no flags or variables set
Applying the action
(choice select: (select.filter test: (> #x 1) selector: (select.minimum #x)) (do (house.flag A) (set #x 1)) (do (house.flag B) (set #x 2)) (do (house.flag C) (set #x 3)))
To a house with no flags or variables set
- Write some code to do each of the following actions
- Install three measures
There are some more exercises related to choices and packages in the section on suitability and failure.
14.4.1 Answers
- The effects will be
By type of house:
- The flag
london
will be added - The flag
london
will be added - No change, but the action will be a success; this is because of the
default:
beingaction.do-nothing
unless otherwise specified - The flag
large
will be added - The flag
detached
will be added
If you have added more than one flag in any case, remember that
action.case
will only ever do one of its alternatives- The flag
- The flags
A
,B
andC
will all be added - The flag
A
will be added; this is because:choice
only ever adds one flaghouse.total-floor-area
is the same for all the alternatives, so there is a tie- ties are resolved by order of writing and
A
is at the top
- One third of the time, the flag
A
will be applied, one third of the timeB
will be applied, and one third of the timeC
will be applied. - The action will always fail, and hence have no effect. This is because:
- The first action in the do will always succed, and leave the house having flag
A
- The second action will fail whenever the house has flag
A
, i.e. always - The third action will never get a look-in, because the second action will always fail
- The effect of the first action will always be reversed because a failed action may never have effects on the house, and the
do
has failed
- The first action in the do will always succed, and leave the house having flag
- The house will always have just the flags
A
andB
added. This is because:all:
isfalse
, meaning thatdo
will plod along trying each of its actions and ignoring whether they fail or not(house.flag A)
will succeed, addingA
(action.do-nothing test-flags:!A update-flags:C)
will fail, because of the presence ofA
; however,do
will keep going becauseall:false
(house.flag B)
will succeed, addingB
- The house will always end up with the flags
A
,B
andC
. This is because although theaction.do-nothing test-flags: !A
would fail if the flagA
were present, thedo
is performing its actions in sequence and we have not got to(house.flag A)
by that point yet. These last three examples illustrate how the effects of actions feed-through to each other. - The house will always end up with the flag
B
. This is becausechoice
can never select an unsuitable/failed alternative, and the first alternative will always fail. The house will always end up with the flag
A
and the variablex
being 1. The three alternatives will produce the following three hypothetical situations:- flag
A
is present andx
is 1 - flag
B
is present andx
is 2 - flag
C
is present andx
is 3
Of these, the first minimises
x
, so this is the outcome that will be selected.- flag
- The converse of the previous example; flag
C
andx
equals 3 will happen. - The three hypotheses will be selected with differing probabilities:
- flag
A
is present andx
is 1, with probability 1/6 - flag
B
is present andx
is 2, with probability 2/6, or 1/3 - flag
C
is present andx
is 3, with probability 3/6, or 1/2
- flag
The house will always end up with flag
B
andx
being 2, because: The three hypotheses will be produced as before:- flag
A
is present andx
is 1 - flag
B
is present andx
is 2 - flag
C
is present andx
is 3
First
select.filter
is used, and knocks out the first alternative because x is not greater than 1:flagA
is present andx
is 1- flag
B
is present andx
is 2 - flag
C
is present andx
is 3
Then
select.minimum
is used, and chooses the first of the remaining alternatives, becausex
is least in that case.- flag
15 Measure suitability and failure
In the NHM an action is something that you use to change a house. Actions may have one of two outcomes:
- The action may succeed, and can then change the state of affairs in some way, for example
- installing insulation in a house, and spending some money to do so
- changing a user-defined variable on a house
- putting in a boiler AND putting in some insulation together
- writing in a report and installing double glazing
- doing nothing, but with a merry sense of success
- The action can fail, in which case it definitely does not change how things are
Whether an action succeeds or fails is determined by its suitability. An action which fails is unsuitable (and any unsuitable measure will fail), whereas one which succeeds is suitable (and any suitable measure will succeed).
Measures have certain built-in suitability criteria, which are defined within the model code which enacts them.
For example:
- a gas boiler measure will not be suitable for a house which is not on the gas grid.
- cavity wall insulation will not be suitable for a house which has only solid walls
These suitability criteria are documented in the language reference for each measure.
15.1 Suitability of compound actions
All other actions also have suitability rules, which are also documented in the language reference.
These rules are more complex for the compound actions like do
, choice
and action.case
.
It is important to understand these rules if you use the compound actions, because they can (a) cause your actions to fail when you might expect them to succeed, and also (b) you can use them to model complicated behaviour like supply chains, post-hoc constraints, random outcomes and all sorts of other effects.
The sections on these commands describe their suitability rules a little, but this section covers them in more detail, explaining how they can be put together.
15.1.1 Actions using test-flags:
In the section on flags we saw that most actions in the model have the test-flags:
argument, which makes the action check the presence or absence of certain flags as a way of changing its suitability.
Here are some examples:
(measure.wall-insulation type:cavity test-flags: A ...)
; this measure will only be suitable if (a) it would be suitable anyway (i.e. the wall insulation rules are satisfied), AND (b) the house has the flagA
on it.(measure.wall-insulation type:cavity test-flags: [A B] ...)
; this measure will only be suitable if it would be suitable anyway AND the house has BOTH the flagsA
andB
on it.(measure.wall-insulation type:cavity test-flags: [A !B] ...)
; this measure will only be suitable if it would be suitable anyway AND the house has the flagA
on it AND the house DOES NOT have the flagB
on it.(measure.wall-insulation type:cavity test-flags: * ...)
; this measure will only be suitable if it would be suitable anyway AND the house has at least one flag on it(measure.wall-insulation type:cavity test-flags: !* ...)
; this measure will only be suitable if it would be suitable anyway AND the house has NO flags on it(measure.wall-insulation type:cavity test-flags: !B* ...)
; this measure will only be suitable if it would be suitable anyway AND the house has NO flags on it that start withB
The test-flags:
argument can only make an action less suitable for houses; it will not remove any existing suitability rules.
It is checked before the other suitability rules are checked, as a precondition; if it does not pass, the measure immediately stops and produces a failure.
15.1.2 action.do-nothing
and action.fail
The special action action.do-nothing
always succeeds (UNLESS you use test-flags:
on it, in which case it can fail).
The special action action.fail
always fails, no matter what.
These two are useful in two ways:
action.do-nothing
is useful in achoice
, where you want to be sure that there is always a feasible option.action.fail
is useful withinaction.case
, when you want to trigger a failure under certain circumstances.
15.1.3 action.case
action.case
has the suitability/success behaviour of whichever of the actions in it end up happening.
Recall that action.case
may contain several different alternatives, thus:
(action.case (when A X) (when B Y) default: Z)
In this case, if X
is suitable, but the test A
does not pass for a house, and B
does pass but Y
is not suitable, action.case
will pick Y
and also be unsuitable.
The choice of alternative is taken before any consideration of suitability, so having a suitable option does not mean it will be chosen.
If you wanted this, you could augment the tests to check suitability as well:
(action.case (when (all A (house.is-suitable-for X)) X) (when (all B (house.is-suitable-for Y)) Y) default:Z)
In this example, each when
also includes the test for suitability, so the action.case
will never attempt an unsuitable alternative.
15.1.4 do
The do
command is quite complicated; its suitability behaviour depends on the value of the all:
argument it is supplied:
- If
all:
istrue
(or not supplied),do
is only suitable if each action inside it is suitable, at the point when they are applied. Thus if any action inside thedo
fails, the entiredo
will fail. - If
all:
isfalse
,do
is always suitable (unless it also hastest-flags:
, which are checked as a precondition). Even if all the actions inside thedo
fail, it will succeed, and the effects of any successful changes will be kept (unless thedo
is inside a furtherdo
which turns out to fail).
When we say that the do
is suitable if the actions inside are suitable at the point when they are applied, we mean this; consider the following do
statement:
(do (action.do-nothing test-flags: !A update-flags: A) (action.do-nothing test-flags: !A))
Imagine a house which does not have the flag A
; if we consider each of the actions inside the do
in turn, they both appear to be suitable for the house. After all, it does not have the flag A
.
However, a successful application of the first action will add the flag A
to the house, and as a result the second action will be unsuitable once the first has happened.
Hence this do
statement is always unsuitable and will always fail.
When a do
command fails, its effects are 'rolled back' - in the example above, not only is the do
statement perpetually unsuitable, it also has no effect.
This is because although the first action will put the A
flag on the house, the failure of the second causes the failure of the do
, and a failed action may not have any effects.
More realistic examples have the same all or nothing behaviour:
(do (measure.wall-insulation ...) (measure.standard-boiler fuel:mainsgas))
A house exposed to this action will receive either the wall insulation and the boiler or neither; it must be suitable for both, and it must be suitable for the boiler after its walls have been insulated.
One useful effect of this is to allow action.case
to 'undo' things after the fact.
For example, imagine we want to install a boiler, but only if the fuel bill is sufficiently reduced:
(do (measure.standard-boiler ...) (action.case (when (< (fall-in house.fuel-cost) 500) action.fail)))
We can only know the fall in fuel bill by installing the boiler (as we have to do an energy calculation to figure it out). In this example we install the boiler, and then use action.case
to trigger action.fail
only if after the boiler is installed the fuel cost has not been sufficiently reduced.
It is important to remember that the failure of a do
command will 'cascade'. For example if we write
(do (do A B) ... (do E F))
If any part of this fails the whole thing will fail, undoing anything which has been done.
15.1.5 choice
The rules for choice
are simple:
- A choice will only present suitable alternatives to its selection rule, so the selection rule can only ever select a suitable alternative.
- If no suitable alternative is selected, the choice as a whole will fail and be unsuitable.
This means that a choice is suitable if and only if it contains a suitable alternative, and that suitable alternative is considered by the selection rule.
All selection rules except select.filter
consider all of the suitable alternatives they are presented.
15.2 Changing suitability
Sometimes you may wish to change the suitability of an action.
In the NHM, actions' suitability can only ever be reduced, as you can add a test either before or after doing the action which will cause it to fail under certain circumstances, but you can never change the action so that it succeeds in conditions where it would have failed.
You have already seen how you can use flags for this, with the test-flags:
argument that most measures will accept.
You may require a more general rule, which lets you render a measure unsuitable for any reason.
In recent versions of the model you can do this using the fail-unless
command within do
For example, say we had a measure like this:
(measure.standard-boiler fuel: mainsgas)
If we wished to amend this measure so that it was only suitable in urban houses we could write:
(do (fail-unless (house.morphology-is urban)) (measure.standard-boiler fuel:mainsgas))
You could also write this using action.case
as
(do (action.case (when (none (house.morphology-is urban)) (action.fail))) (measure.standard-boiler fuel:mainsgas))
or
(action.case (when (house.morphology-is urban) (measure.standard-boiler fuel:mainsgas)) default: (action.fail))
These are all equivalent statements; in each case the morphology of the hosue is tested before the measure is installed, and the installation fails if the morphology is not urban
.
Sometimes you may wish to check suitability after the fact; for example, you might say that an insulation measure is not suitable if it insulates a very small or very large area, as small areas are not worthwhile and large areas require a more expensive material.
In this case, you can again use fail-unless
or action.case
(do (measure.wall-insulation type:cavity resistance: 0.01 thickness:50) (fail-unless (> 10000 (size.m2) 10)))
or
(do (measure.wall-insulation type:cavity resistance: 0.01 thickness:50) (action.case (when (none (> 10000 (size.m2) 10)) (action.fail))))
As well as fail-unless
, there is a special command called consume
which fails unless it can reduce a variable by a certain amount without making it negative; this is useful for defining things like supply chains or maximum emissions rules.
For example, say you wished to restrict installations of a certain measure to 100,000 units per year:
(def remaining-units on:simulation default: 0) ;; every year, reset the remaining units (on.dates (regularly) (set #remaining-units 100000)) (on.dates ... (apply to: ... (do (consume #remaining-units) (the-measure))))
Here we have a global counter called #remaining-units
, and in the do
command around the-measure
, we use consume
to make sure that we have enough units left to install; consume
will decrease #remaining-units
by the house's weight, unless that would make #remaining-units
negative in which case it will fail and the-measure
will not be installed.
You could write this out as
(do (decrease #remaining-units (house.weight)) (fail-unless (>= #remaining-units 0)) (the-measure))
So consume
is just a shorthand for this.
15.3 Exercises on suitability
For each of these measures, consider its application to the listed houses and determine whether it will succeed, and explain why
The simplest option
(action.do-nothing)
- applied to any house
The second simplest option
(action.fail)
- applied to any house
A slightly less simple option (see the 9 section if unclear)
(action.do-nothing test-flags: qwerty)
- applied to a house with no flags
- applied to a house with the flag
qwerty
- applied to a house with the flag
qwertyuiop
And again
(action.do-nothing test-flags: !qwerty)
- applied to a house with no flags
- applied to a house with the flag
qwerty
- applied to a house with the flag
qwertyuiop
Another simple one
(do)
- applied to any house
Doobie doo
(do (do (do)) (do))
- applied to any house
Do choose
(choice select: (select.weighted 1) (do) (do) (do))
- applied to any house
Do choose some more
(choice select: (select.filter test: (house.region-is london) selector: (select.weighted 1)) (action.do-nothing) (do) (fail))
- to any house
Do fail a choice
(do (choice select: (select.weighted 1) (action.fail)) (choice select: (select.weighted 1) (action.do-nothing)))
- to any house
Do a measure involving 11 and 10.
(do (measure.standard-boiler fuel:mains-gas capex:1000) (consume #boiler-expenditure (* (house.weight) (capital-cost))))
- To a house suitable for a gas boiler
- To a house not on the gas grid
Do a measure later
(do (consume #boiler-expenditure (* (house.weight) (capital-cost))) (measure.standard-boiler fuel:mains-gas capex:1000))
- To a house suitable for a gas boiler
- To a house not on the gas grid
Total failure
(action.case (when (= 1 2) (action.fail)) default: (action.fail))
- to any house
Failure?
(action.case (when (> 1 2) (action.fail)) default: (do update-flags: hello))
- to any house
Success?
(action.case (when (< 1 2) (action.fail)) default: (do update-flags: hello))
15.3.1 Answers
action.do-nothing
is suitableaction.fail
is never suitable- We are testing for the flag
qwerty
, so it needs to be there, i.e.- unsuitable
- suitable
- unsuitable
- Now we are testing for the absence of the flag
qwerty
, i.e.- suitable
- unsuitable
- suitable
- A
do
on its own is suitable, and does nothing - This is suitable, because:
do
is suitable if its children are suitable, and(do (do))
is suitable becausedo
is suitable if its children are suitable and thedo
on its own is suitable, and- the final
do
is also suitable as it has no children to make it fail.
- A
choice
is suitable so long as (a) some of its alternatives are suitable and (b) the selector does not rule them out; here we have three alternatives which are all suitable (each one is an emptydo
), and a selector which does not rule any out. So, this is always suitable. - This
choice
has suitable alternatives indo
andaction.do-nothing
, but that is not enough; its selectorselect.filter
has atest:
which only passes in London, so the choice is suitable only for houses in London in which it does nothing and unsuitable everywhere else. - The
do
will always fail, because it has two actions in it:- a
choice
with the only alternative beingaction.fail
, which must fail, and - a
choice
which will always succeed, but we never get there because of the preceding guaranteed failure.
- a
- Firstly we are putting in a boiler, and then we are using
consume
, so there are two possible sources of failure: (a) boiler is unsuitable or (b) consume doesn't work because the second argument is more than the first.- for the house which can have the boiler, we are suitable if
#boiler-expenditure
is not less than 1000 times the house's weight; this is because consume is only suitable if it can reduce the variable#boiler-expenditure
byhouse.weight
timescapital-cost
without making it negative. - if the house is not on the gas grid, the
consume
doesn't matter, as the house can't have the boiler so it can't have anything
- for the house which can have the boiler, we are suitable if
So this time we have made a probable coding mistake - this is a bit of a trick question. As explained in 11,
capital-cost
is a special command which tells you the capital cost of the current measure or action. When used in aconsume
inside ado
like this, it is the capital cost of thedo
so far. When theconsume
is checked, this will be zero, as we haven't got to the measure yet, and henceconsume
will succeed unless#boiler-expenditure
is negative because subtracting zero cannot otherwise make it negative.After that, we will put in the boiler unless we can't; if the boiler is suitable the whole thing is suitable, and otherwise it isn't.
- Both alternative actions are
action.fail
, so we always fail. - one is never greater than two, so we never look at
action.fail
; we always fall to thedefault:
which is suitable, becauseupdate-flags:
does not affect suitability anddo
is sutiable. - one is always less than two, so we always try
action.fail
and then fail. We never try the other option, so this is always unsuitable.
16 Hypotheses
Before reading about this, make sure you have read the preceding sections on packages and suitability, as they are important prerequisites.
16.1 In-depth explanation of choices, packages and failure
The NHM's simulation proceeds by making use of hypotheses; every change that the model thinks about, whether to a single house or a population of houses, starts of as a hypothesis, in which the model changes things speculatively.
The changes which happen when considering a hypothesis are isolated from the model's view of the real world, and can change anything about the world until the hypothesis is either discarded without any change to how things "really" are, or it is accepted and becomes the new status quo.
This basic feature is used to make commands like do
and choice
work; in a do
, the changes being applied happen within the hypothesis that the do
will succeed. If one of the changes is not successful, the do
can safely throw away the hypothesis in the knowledge that there will be no side-effects.
Similarly, the choice
command produces n hypotheses, one for each of the alternatives, and discards n-1 of them.
To further complicate matters, the model can make a hypothesis within another hypothesis16; this is what happens when you place a choice
within a do
or vice-versa.
These commands (do
and choice
) have a common factor: whichever hypothesis ends up being accepted as the new status quo, we know that nothing which happened in any of the discarded alternative hypotheses can affect the simulation (it can affect reports, of which more later; reports are outside the simulation's view of the world).
There are some other commands which will allow you to work something out in a hypothesis, but to store the result outside that hypothesis.
You could think of this as being analogous to using your imagination, at least if your imagination were very effective.
Let us look at the first of these commands, under
16.2 Using under
to work out values in hypothetical situations
Imagine a scenario in which you are modelling the offer of several packages of measures to various houses, and the houses are deciding between these packages based on the annual heating bill they will have after installing each package:
(choice select: (select.minimum (house.fuel-cost)) (package-1) (package-2) (package-3))
package-1
, package-2
and package-3
are standing in for some actual packages that we are considering, for ease of reading. In this case, we will take each package, and then pick the one for which the fuel cost after installation is least.
Now say that we wish to model something slightly different: say that the householder making the decision has a belief that electricity prices will rise and gas prices will fall in the future, and so we wish to make the choice on the basis of these different prices.
However, these prices do not yet pertain in the model, and indeed may never as the householder could be wrong in their belief. We must make the choice minimising how fuel cost would be if we were to have a different suite of prices.
The tool for this is the under
command, which has the following form:
(under <<some actions>> evaluate: <<value of interest>> )
When the under
command is used by the model, it does the following:
- Produce a new hypothesis, which looks the same as where the
under
is being worked out, but is isolated from it because it is a hypothesis - Uses the actions to change things in the hypothesis
- Works out the argument to
evaluate:
in the hypothesis, so that it "sees" the changes that were made - Discards the hypothesis, and produces the value from 3 back where the
under
was being computed
You can read this as saying, "without changing how things are now, imagine what would happen if we did the actions and worked out the evaluate:
argument afterwards".
So returning to our example, we can evaluate the fuel cost under a different set of tariffs, reflecting the householder's beliefs without changing the tariffs which are actually used in the simulation:
(choice select: (select.minimum (under (action.set-tariffs (tariff (fuel type:mainsgas (charge (* 0.01 house.meter-reading))) (fuel type:peakelectricity (charge (* 0.3 house.meter-reading))) (fuel type:offpeakelectricity (charge (* 0.3 house.meter-reading))) )) evaluate: (house.fuel-cost))) (package-1) (package-2) (package-3))
In this case the model will create six hypotheses, which we can draw out in a hierarchy:
- From the initial state, before the choice:
- a hypothesis in which
package-1
has been applied, in whichunder
makes:- A sub-hypothesis in which the tariffs have been changed and
package-1
has been applied
- A sub-hypothesis in which the tariffs have been changed and
- a hypothesis in which
package-2
has been applied, in whichunder
makes:- A sub-hypothesis in which the tariffs have been changed and
package-2
has been applied
- A sub-hypothesis in which the tariffs have been changed and
- a hypothesis in which
package-3
has been applied, in whichunder
makes:- A sub-hypothesis in which the tariffs have been changed and
package-3
has been applied
- A sub-hypothesis in which the tariffs have been changed and
- a hypothesis in which
In each of the sub-hypotheses, under
will work out the house.fuel-cost
, giving the fuel costs for:
package-1
with the hypothetical tariffpackage-2
with the hypothetical tariffpackage-3
with the hypothetical tariff
However, each of these is disposed of after being used to work out the cost, and the choice will select one of the "first-level" hypotheses as the actual outcome.
16.2.1 under
and suitability
There is a subtlety here, which is that the under
command has to produce a value, but the actions in it may be unsuitable.
When it is used, the under
command will stoically try to apply whatever actions you have given it and then work out the result even if the actions failed; for example if you said:
(under (measure.standard-boiler fuel:mainsgas ...) evaluate: (house.energy-use))
To work out the energy use a house would have if you installed a new gas boiler, you will get one of two results:
- The energy use after installing a new gas boiler, if the boiler was suitable, or
- The current energy use, if the boiler was unsuitable
This ambiguity can be avoided using variables; for example you could amend the above to say:
(under (set #energy-use -1) (do (measure.standard-boiler fuel:mainsgas ...) (set #energy-use (house.energy-use))) evaluate: #energy-use)
This would do the following:
- Make a hypothesis for the
under
, which will be thrown away later no matter what - Within that hypothesis, change the variable
#energy-use
to be -1, which we will use as a special value to indicate unsuitability - Now within that (outer) hypothesis, make another (inner) hypothesis for the
do
command, in which we will- Try and install a new gas boiler; if this fails, we will stop the
do
command with no effect on the outer hypothesis - If it succeeds, we will store the new
(house.energy-use)
in#energy-use
, and the new state of affairs will be accepted for the outer hypothesis
- Try and install a new gas boiler; if this fails, we will stop the
- Now the
do
has completed, and the outer hypothesis will either be:#energy-use
is -1, and no boiler went in, or#energy-use
is(house.energy-use)
after a boiler went in, and a boiler went in
- Finally
under
evaluates#energy-use
within the outer hypothesis, producing either -1 or the energy use after the boiler - The outer hypothesis is discarded and the simulation proceeds having either a -1 or the value of interest available
At this point if the value is emitted into a report the special value -1 can be filtered out in your other tools, or if it's being used in the simulation it can be tested using action.case
or function.case
to determine what to do in the situation where the measure is unsuitable.
16.3 Other ways to use hypotheses; set
under:
, rise-in
, fall-in
and original
Using under
can be computationally expensive - each use of under
can only compute a single value, so if you want to calculate several different values under some hypothetical changes you need to write it out several times.
The model is not clever enough to know when the work of making the changes can be reused, so each use of under
creates its own hypothesis, does all the work of making the changes and updating derived values like the energy use, and then throws the hypothesis away.
As a result, if you ask for ten different values each under
the same assumption by writing (under A evaluate:X)
, (under A evaluate:Y)
, and so on, it is ten times as expensive as asking for one.
As an alternative the model provides special argument to set
to allow you to calculate several values in a hypothesis all at once:
(set [#variable-1 #variable-2 #variable-3] [(function-1) (function-2) (function-3)] under: [(action-1) (action-2)])
This will do the following:
- Produce a new hypothesis, wherein:
- Apply
action-1
to change things in the hypothesis - Apply
action-2
to change things further in hypothesis - Compute
function-1
in the hypothesis, and store the result in#variable-1
- Compute
function-2
in the hypothesis, and store the result in#variable-2
- Compute
function-3
in the hypothesis, and store the result in#variable-3
- Apply
- Copy the values of
#variable-1
,#variable-2
and#variable-3
out of the hypothesis and into the "real" values for the variables - Discard the hypothesis
This will have the same effect as writing
(do (set #variable-1 (under (action-1) (action-2) evaluate:(function-1))) (set #variable-2 (under (action-1) (action-2) evaluate:(function-2))) (set #variable-3 (under (action-1) (action-2) evaluate:(function-3))))
But it will be much faster, as (action-1)
and (action-2)
need only happen once, only one hypothesis is generated, and any calculation which can be shared between the three functions will be (for example, uses of the energy calculation).
A very common need is to compute the change in values, as a result of a hypothetical action.
For example, when defining a choice
you might want to include only those alternatives for which the reduction in fuel cost is greater than the net cost of installation.
You can do this with variables, writing something like this:
(do (set #initial-bill (house.fuel-cost)) (choice select: (select.filter test: (> (- #initial-bill (house.fuel-cost)) (net-cost)) selector: (select.minimum (net-cost))) (action-1) (action-2) (action-3) ))
Newer versions of the model offer a simpler syntax:
(choice select: (select.filter test: (> (fall-in (house.fuel-cost)) (net-cost)) selector: (select.minimum (net-cost))) (action-1) (action-2) (action-3))
The command fall-in
computes its argument twice and produces the difference; the first time is computed outside the current hypothesis and the second inside.
In this case the choice
command is responsible for creating the current hypothesis, so the (fall-in (house.fuel-cost))
will be the difference between the fuel cost before applying any alternative and the fuel cost after.
The rise-in
command does the natural reverse of this (the difference between the value inside and outside), and the original
command provides direct access to the value outside.
16.4 Some "hypothesis diagrams" for commands
It can help to draw out how hypotheses are constructed and selected for different commands. In each of these diagrams a circle represents a state of affairs, a solid arrow the application of an action to that state of affairs, and a dashed arrow the production of some hypotheses.
Circles with a red outline are those whose hypotheses are discarded, and those with a double-circle outline are those which are selected.
A red solid arrow indicates a failed measure, and will therefore always lead to a red circle.
16.4.1 do
Imagine the command
(do (X) (Y))
In the event that both X and Y succeed, this will produce the following sequence:
In which the initial state A is converted to a hypothesis B, on which we apply X to get C and then apply Y to get D, which is then selected as the new state of affairs.
Alternatively, one of X or Y might fail; if X fails we have
and if Y fails we have
Remember that the do
as a whole has failed if its contents fail; this is why the final arrow to the double circle is shown as red in the last two diagrams.
If the do
were within another do
, the outer one would also fail, and so on.
Indeed you could "embed" the diagram for one of these examples into the placeholders for actions that are displayed like X
, Y
and so on:
This illustrates something like:
(do (X) (do (Q) (P) (R)) (Y))
in which Q
fails, causing the entire process to fail.
If the outer do were instead do all:true
:
(do all:true (X) (do (Q) (P) (R)) (Y))
The diagram would look more like this:
i.e. the inner do
which failed has simply been skipped over, and we end up with a successful application of X
and then Y
.
16.4.2 choice
Let's consider a single choice:
(choice select: (select.minimum (O)) (X) (Y) (Z))
Imagine that that option Z
is unsuitable, and options X
and Y
are both suitable but Y
has the minimum value for objective O
.
In this picture we also have square nodes, which indicate the evaluation of the selection rule in each hypothesis.
You can see how measure Z
is eliminated by unsuitability, and measure X
by the selection rule.
16.4.3 under
The hypotheses produced by under
never become real, but they do produce some information back in their originating state:
(under (X) evaluate: (Y))
Here we start in state A, produce hypothesis B in which we apply X to get hypothesis C, wherein we compute Y per rectangle D; the dotted line shows the result of Y being communicated back into the accepted state F, whereas E shows how the hypothesis for the under
is thrown away.
16.5 Output from within hypotheses
Output from reporting commands is sent to files immediately, and is not part of the simulation's state17; it is therefore not reversed when reporting was done within a hypothesis that is later discarded. This can lead to confusing output, but may also be quite useful; for example, if you apply the following action:
(do (probe name: p (measure.standard-boiler ...)) (action.fail))
The probe p
will probably contain records which show a boiler being successfully installed in some dwellings (so long as it has been applied to dwellings for which a boiler is suitable).
However, these dwellings will never actually receive a boiler, because the action.fail
will cause the surrounding do
to be unwound; it can never have a real effect.
If you were to amend the above to read:
(probe name:q (do (probe name: p (measure.standard-boiler ...)) (action.fail)))
The two probes would be correlated; each house presented would produce a row in both p
and q
, but all the rows in q
would show failure whereas some of the rows in p
would show success.
The reports generated by def-report
include a special column called selected
, which indicates whether the hypothesis in which a row of the report was computed was eventually made part of the true state, or whether it was later discarded.
16.6 TODO Exercises about hypotheses
17 Defining things so you don't have to repeat them
You have already seen the commands for defining and referring to variables, and reports. Some similar syntax exists for actions, tests, and functions, to wit:
(def-test house-is-in-london (house.region-is London)) (def-test house-is-large (> (house.total-floor-area) 10000))
Would define two reusable tests, which you could refer to anywhere as #house-is-in-london
or #house-is-large
, for example as:
(apply to: (filter #house-is-in-london) (action.case (when #house-is-large (xyz))))
As far as the model is concerned, this is akin to writing
(apply to: (filter (house.region-is London)) (action.case (when (> (house.total-floor-area) 10000) (xyz))))
This is useful when you want to use a common definition in a lot of places, because:
- The reader can remember the meaning of a complex expression under a single name, and
- The editor can change the definition in a single place and have a universal effect.
It is not useful when you have a definition which is not being reused, as it merely adds some extra mental overhead for the reader.
The analogs for actions and functions are:
(def-action my-package (do (measure-1) (measure-2))) ... (apply to:(filter (house.is-suitable-for #my-package)) #my-package)
or
(def-function standard-capex 500) ;... later on (measure.standard-boiler capex:#standard-capex)
Note how def-function
need not define a complicated function; it can be a constant if you would prefer.
18 Templates and modules
The def-function
, def-action
and def-test
(in the previous section) commands provide a simple way to reuse common fragments of code.
They are efficient for the simulator, and stand out when written because of their hash symbol.
However, they are inflexible in that they define a single, fixed unit of code.
Often you will instead have a repeating pattern in your code, which you want to avoid reiterating again and again.
The model provides this using what is called a preprocessor (for better or worse; it may not be the best tool but it is the only tool here).
If you have used SAS macros (or lisp macros, or the R substitute
/ eval
primitives, or written what people call "dynamic SQL"), these are all analogous.
If you have not, don't worry, we will cover it all directly in any case.
18.1 An example template
The simplest and most useful facility offered by the preprocessor allows you to define templates. A template is like a pro-forma bit of scenario syntax with some "blank spaces" in it that need to be filled in before it is complete. To employ a template, you first define the pro-forma under a name of your choice, with placeholders for the blank spaces, and then you use the template by invoking the name you gave it.
To make this concrete, consider the following situation; imagine you have written a scenario, but the Minister phones through and says "It is essential that at no point, as a result of the Mauve Deal policy, that emissions due to biomass should exceed 9 million tonnes per year; what would this do to the simulation outcomes? Any time in the next ten to fifteen minutes would be fine so don't rush.".
You consult the NHM manual, put on your thinking hat, and determine that this can be done by restricting the suitability of the measures the scenario is installing; this is a matter of checking a postcondition, namely that after installing a measure if you have made emissions go up you haven't taken them over 9 million tonnes.
Postconditions are a good fit for do
and fail-unless
, so you cook up this:
(do ;; the measure goes here (something-or-other) ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000) )))
This seems pretty reasonable, but looking at the clock we notice that we only have six minutes left, and we don't relish the prospect of correctly adding this around each measure we have written, even though we wrote each measure down using #def-action
so they are all nicely defined in one place.
What we want here is a template; the code above is clearly a bit of pro-forma; it is the same in all cases, except for the something-or-other
, which is a blank we need to fill in.
To define our template, we use the special template
command, which looks like this:
(template <<template name>> [<<list of placeholders>>] <<pro-forma goes here>>)
In our case, let us say the template is to be called enforce-biomass-emissions-limit
; the first step is simply to pop in our pro-forma (a.k.a. the template body):
(template enforce-biomass-emissions-limit [] (do ;; the measure goes here (something-or-other) ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000) ))))
So far so good, but how do we tell the model that we want to put a particular thing in place of something-or-other
?
The answer is in two parts:
- We put something in the list of placeholders, to tell the template that it needs something to pop into place, and then 2
- We refer to that placeholder in the pro-forma in the right place.
To make them stand out, placeholders are always written with an atpersand (@
) at the start of them; we can add one into the template we are imagining now:
(template enforce-biomass-emissions-limit [@measure] (do ;; the measure goes here @measure ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000) ))))
This is now our whole template - we have a placeholder, called @measure
, which represents the blank to fill in.
Now to write out emissions-limited actions in our scenario, we don't have to copy-paste the pro-forma into place every time; instead we simply invoke the template thus:
(apply to:(some-houses) (enforce-biomass-emissions-limit measure: (measure.standard-boiler fuel:biomasswood ...) ))
When the model sees that we have used enforce-biomass-emissions-limit
as a command, it will remember that it saw a template by that name, and effectively replace the use of the template with its body; it is exactly as though you had just written down
(apply to:(some-houses) (do ;; the measure goes here (measure.standard-boiler fuel:biomasswood ...)@measure ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000) ))))
You can think of a template as being a scenario editing rule that you might give to a particularly unfortunate intern, except it is applied transparently and automatically to your scenario when you press run.
Returning to our story, you are now able to zip through all the actions which you have sensibly defined with def-action
, and amend them, so where once you had:
(def-action policy-biomass-boiler (measure.standard-boiler fuel:biomasswood ...))
You now have
(def-action policy-biomass-boiler (enforce-biomass-emissions-limit measure: (measure.standard-boiler fuel:biomasswood ...)))
Success! The astute reader will note that this replacement is analogous to what we have just done - the change to each def-action
is itself a kind of pro-forma edit that could be done by our unfortunate intern.
Indeed, if you found yourself doing this very frequently, you could say:
(template def-policy-action [@name @action] (def-action @name @action)) (def-policy-action name: policy-biomass-boiler action: (measure.standard-boiler fuel:biomasswood ...))
And then a single change to def-policy-action
could change all your defined actions at a swoop!
This should give you some feel of when to use a template: if you have a lot of code which is often similar but with minor variations, consider replacing it with a template. Of course, adding more templates to your scenario can make it harder to read, especially when someone first approaches it, as they have to understand what each template does before they can understand the code which uses those templates, so you must weigh the introduction of templates carefully.
18.2 Placeholders and invocation
In the preceding example, we showed how to write a single placeholder, which had a name, and where the thing to be filled into the placeholder was identified by using that name.
There are a few different options available when writing a placeholder, which are explained in more detail in the language reference.
Each placeholder connects an argument supplied when invoking the template to some places in the body of the template.
18.2.1 Supplying placeholders when invoking a template
In the example we have seen, we had a single placeholder, called @measure
.
The code to fill into the placeholder when invoking the template was indicated by the keyword argument measure:
, i.e.
(template example [@widgets @sprockets] ...)
Has two placeholders, @widgets
and @sprockets
, and the content to go in these when invoking the template is identified by using these as named arguments (if you don't know what a named argument is, refer back to the very start of this document):
(example widgets: 9 sprockets: 13)
Any placeholder written like this (as an atpersand-prefixed word like @this
or @that
) will be filled in by the corresponding named arguments supplied to the invocation (like this:
or that:
).
You can also fill in placeholders with unnamed arguments by using the placeholders @1
, @2
, @3
and so on.
Changing our example, we could say
(template enforce-biomass-emissions-limit [@1] (do ;; the measure goes here @1 ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000) ))))
If we write this, then when we invoke the template the placeholder @1
will be substituted for the first unnamed (or positional) argument, e.g.
(enforce-biomass-emissions-limit (my-measure))
Finally, there is a third special placeholder you can use, called @rest
.
This placeholder will receive the content of any positional arguments that are left after @1
, @2
and so on (for the ones of these you have used).
This is useful for passing many inputs on to another command which expects many inputs - for example, if you were defining a template to do a certain kind of choice, you could say
(template min-cost-choice [@rest] (choice select: (select.minimum (capital-cost)) @rest ))
You could then invoke this with as many arguments as you like; they would all be filled in for @rest
, for example:
(min-cost-choice (A) (B) (C)) ; is the same as (choice select: (select.minimum (capital-cost)) (A) (B) (C))
18.2.2 Different names for placeholders within a template
Sometimes you might want have names for placeholders to make the template definition easier to read, without having to use those names when you invoke the template.
You can do this by writing a placeholder name with two parts separated by a colon, like @A:B
.
This will mean that you refer to the placeholder in the template body as @B
, but that the model supplies it when the template is used according to the rules for @A
.
In our example template, you might say that enforce-biomass-emissions-limit
is only ever going to have one argument, which is the measure, so we want to use the @1
type of placeholder so template users are not forever writing measure: ...
.
However, when reading the template, @1
is a bit opaque, so we could write instead:
(template enforce-biomass-emissions-limit [@1:constrained-measure] (do ;; the measure goes here @constrained-measure ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) 9000000) ))))
Here we are calling the placeholder @constrained-measure
inside the template, as a hint to the reader, but when invoking the template we still need only write (enforce-biomass-emissions-limit X)
.
18.2.3 Providing defaults for placeholders
Sometimes you may want to have placeholders which only need filling in some of the time. This can happen if you have a lot of 'typical' values of which only some will change, or if you want to add flexibility to a template without breaking it in the places where it is already used.
In our example, we can imagine wanting to change the type of fuel or the limit value we are filling in; say we wanted to make the limit controllable, we might change the template to be:
(template enforce-biomass-emissions-limit [@1:constrained-measure @limit] (do ;; the measure goes here @constrained-measure ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) @limit) ))))
Now when we use the template we can write (enforce-biomass-emissions-limit X limit: 10000000)
if the limit should be different in a certain case.
The down-side of this change is that now we must supply a limit:
argument, as otherwise there is nothing to substitute there in the body.
To get around this we can give a default value for @limit
; the syntax for this is (perhaps confusingly) to write the placeholder and its default value together in square brackets, thus:
(template enforce-biomass-emissions-limit [@1:constrained-measure [@limit 9000000]] (do ;; the measure goes here @constrained-measure ;; now for the postcondition (fail-unless ;; so we give up, unless either: (any ;; the emissions for this house have not gone up (>= 0 (fall-in (house.emissions by-fuel:biomasswood))) ;; or if they have gone up, the total is still fine (< (summarize (aggregate.sum (house.emissions by-fuel: biomasswood))) @limit) ))))
In this amended version, we are allowed to provide a limit:
argument, but if we do not it will be as though we had written limit: 9000000
.
18.3 Other preprocessor commands (macros)
Sometimes you might want a template to do something a bit more complicated than just producing some pro-forma.
There are a small number of other model commands which instruct the pre-processor to do edits to your scenario.
All of these commands start with a tilde (~
) character, and are documented in the manual section on standard macros.
When you are reading a scenario, remember that any command named with a tilde is a command to manipulate the scenario code inside it, not a command pertinent to houses, dates and so on.
Here are some of the macros you are most likely to encounter:
~join
The join macro concatenates bits of text. For example, if you wrote
(~join this - that - the - other)
when the join is expanded it will be replaced with
this-that-the-other
.This is useful for writing templates that need to do things like setting a flag with a particular name, or defining a set of variables which should all have related names, because you can use a template placeholder inside the
~join
. For example,(def (~join installation-count-of- @measure-name s))
in a template would expand to produce something like
(def installation-count-of-boilers)
if@measure-name
were to containboiler
.~maybe
The maybe macro is for passing along named arguments that are optional, without supplying a default in the template where you are writing them. For example, imagine you want to define a template which installs a new boiler, and you have some optional arguments that you want to pass on to
measure.standard-boiler
:(template my-boiler [[@capex] [@opex]] (measure.standard-boiler winter-efficiency:89% summer-efficiency: 85% (~maybe capex: @capex) (~maybe opex: @opex)))
The macro accepts either one or two arguments; if the second argument is supplied, it just expands to both the arguments without modifying them. If the second argument is missing, the maybe macro expands to nothing at all.
The effect of this in the example above, is that if a user writes:
(my-boiler)
then they get:
(measure.standard-boiler winter-efficiency:89% summer-efficiency: 85%)
whereas if they write:
(my-boiler capex: 1000)
then they get:
(measure.standard-boiler winter-efficiency:89% summer-efficiency: 85% capex: 1000)
~match
The
~match
macro is a bit like an if statement. It will look one bit of scenario code that you give it, and compare it to several other options, each of which can have some associated "output". If there is an output that is the same code as the input, it will expand to the output.The thing to compare is the first argument to
~match
, and then all the other arguments should be lists. The first thing in each list checked against the thing to compare, and if it's the same the macro outputs the rest of the list.For example
(~match large [small 100] [medium 200] [large 300])
Will expand to the single output
200
, becauselarge
is the first item in the third list.Note that something like this:
(~match (house.built-form) [Detached 1] [EndTerrace 2] ...)
will not work as you might guess it would on reading. This is because
~match
is a macro rather than a normal command, and so it operates on the scenario code itself, not on houses. Macro output is determined before the model runs, so it can never depend on the particulars of houses or things going on in the simulation.In this case, the match command will compare the literal text
(house.built-form)
with the literal textDetached
,EndTerrace
, and so on. The text(house.built-form)
does not equal any of these values (in fact, it only equals(house.built-form)
!), and so the macro will output nothing as there is no match.The normal model commands you can use for this kind of thing are things like
function.case
,action.case
,lookup
(which can be programmed with~lookup-table
, of which more later).The
~match
command is mainly useful if you are writing a template and you want template users to be able to supply categories of some kind as arguments, and to output different code for different categories, or if you want to produce a useful error message if a user provides an argument to a template which you don't know how to deal with:(template boiler [@fuel-type] ;; check the @fuel-type is OK: (~match @fuel-type [MainsGas] [Electricity] [Biomass] default: (~error No assumptions available in the boiler template for @fuel-type.)) (measure.standard-boiler fuel: @fuel-type capex: (capex-for-fuel-type @fuel-type) ...))
~combinations
The combinations macro is a bit like a loop, if you have used other programming languages. It is a bit complicated to get to grips with initially, but there isn't that much to it in the end.
What it does, is take several lists of commands, and go through each possible combination made by taking exactly one command from each list, putting each combination inside another command that you give it.
To start with, here is an example where there is one list of commands to combine, and the command to put each combination inside is
do
.(~combinations template: do [(measure.wall-insulation) (measure.loft-insulation)])
template:
gives the outer command, and the list in brackets gives a list of two things to pick between.This will output the following scenario code when it is expanded:
(do (measure.wall-insulation)) (do (measure.loft-insulation))
This is because there is one list to choose from, so the only combinations to go through are given by each member of that list.
Now here is a more complicated example:
(~combinations template: do [(measure.wall-insulation) (measure.loft-insulation)] [(measure.standard-boiler) (measure.heat-pump)])
This will produce four combinations, which are what you get by choosing one thing from each list:
(do (measure.wall-insulation) (measure.standard-boiler)) (do (measure.wall-insulation) (measure.heat-pump)) (do (measure.loft-insulation) (measure.standard-boiler)) (do (measure.loft-insulation) (measure.heat-pump))
You can have as many lists as you would like, but remember that the number of combinations produced is the product of the length of each list, so 4 lists each having 6 things in gives you \(6^4 = 1296\) possible combinations, 10 lists of two things gives \(2^10 = 1024\) and so on. If a single combination is expensive to simulate or validate, all of the combinations will be exponentially moreso.
The
template:
argument can be the name of any model command, or any other template you have defined, so you can also use~combinations
to do things like defining a list of variables:(do template:def [one two three four])
is
(def one) (def two) (def three) (def four)
and so on.
~lookup-table
Lookup table is a wrapper around the model's
lookup
command, so we should probably explain thelookup
command first.A
lookup
is a number-valued command (a function) that lets you conveniently write complicated conditional rules that depend on several other values. You can use alookup
wherever you would write a number, so as thecapex:
for a measure, or as the sample percentage for asample
and so on.Here is a spurious example (it is intentionally spurious, to make clear that the values of entries can be anything you like - a lookup entry could contain any function):
(lookup default: 1 keys: [(house.built-form) (house.total-floor-area)] (entry [Detached 1000] (* 10 (house.total-floor-area))) (entry [SemiDetached 2000] (house.energy-use))) (entry [* *] 999))
When this lookup is computed for a house, the command does the following:
- It works out the values of the commands in the
keys:
argument It goes through each
entry
in the lookup in order, and compares thekeys:
values to the values in the list at the start of the entry, which we will call the matching rules.If an entry's matching rules all match their corresponding
keys:
value, then the lookup computes the second part of the entry, and produces that value.- If no entry matched, then the lookup produces the value of the
default:
argument as a fall-back.
Each of the rules in an entry can be either a specific value, like a certain number or a categorical constant, or true or false for a boolean function in the key, or it can be a wildcard, written as a star
*
which will allow any value, or for numbers it can be a rule that defines a range of numbers like:13..15
which will match anything from 13 to 15<10
which will match anything below 10>10
which will match anything above 10
and equivalents using greater-than-equals and so on to include the end-point.
The
~lookup-table
macro is a more convenient way to write these lookups. It lets you write one dimension of the lookup across the top of a table as column headings, which can make the rule easier to read.For example a lookup which gives a time-series of values for each region of the country could be written as:
(~lookup-table row-keys: house.region column-key: sim.year [r 2016 2017 2018 2019 >2019] [London 1 2 3 4 5 ] [SouthWest 6 7 8 9 10 ] ;; and so on )
This is just the same as a plain
lookup
as follows:(lookup keys:[house.region sim.year] (entry [London 2016] 1) (entry [London 2017] 2) (entry [London 2018] 3) (entry [London 2019] 4) (entry [London >2019] 5) (entry [SouthWest 2016] 6) (entry [SouthWest 2017] 7) (entry [SouthWest 2018] 8) (entry [SouthWest 2019] 9) (entry [SouthWest >2019] 10))
- It works out the values of the commands in the
18.4 When are templates and macros processed
An important point to understand about templates is that they are a pre-processing step applied to your scenario. When the model is asked to run a scenario, it does the following steps:
- Process all the
include
commands (of which more later), to produce one big scenario - Transform all the templates within modules (of which more later), removing any modules
- Cut out all of the templates in the scenario and remember their definitions
- Find all invocations of templates or built-in macros in what's left, and expand them until there are no uses of templates or macros left
- Validate the resulting scenario
- If valid, actually run the resulting scenario:
- load the stock
- set up the event queue
- do events until there are none left or the end date is reached
This means that the behaviour of a template (or any other macro command) cannot depend on anything about a particular house; by the time the scenario comes to be executed, all such commands have been fully expanded into basic model commands.
18.5 Defining several templates with modules
Often you will be using templates, flags, and variables all together to add some behaviour to the model in a way that is (hopefully) readable and easy to use. When you so this, you will probably find that you write several templates which are all related to a particular bit of behaviour. To show that some templates are all related in some way, you could use a convention when naming them to show what bit of work they belong to. For example, if you were writing some templates to model supply chains for measures, you can imagine that you would have at least two templates:
- A template a bit like the emissions limit we saw above, which would prevent installation of a measure if the supply was insufficient
- Perhaps a template to 'refill' the supply chain
- Maybe a template to use when reporting on the contents of the supply chain
If you named these all things like supply-chain/limit-measure
, supply-chain/refill
, supply-chain/count-remaining
and so on, and put them in a file called supply-chain.nhm
, it would make it easy for a reader to keep track of.
Remember that in NHM syntax a word can have most punctuation in it, so the slash here is no more special than the hyphen or any of the other letters; it's just part of the name.
Since this is a very common thing to want to do, there is a special preprocessor command called ~module
, which edits the definitions of many templates so that they all have a common prefix on their name like this.
Imagine our example above; without knowing about ~module
we might have written something like this:
(template supply-chain/limit-measure [...] ...) (template supply-chain/refill [...] ...) (template supply-chain/count-remaining [...] ...)
Instead of this, you can write:
(~module supply-chain (template limit-measure [...] ...) (template refill [...] ...) (template count-remaining [...] ...) )
When the model reads this, it will transform all the templates' names to be prefixed with supply-chain/
.
Often when you produce a reusable component like this, you will also have some definitions of variables or similar, which should be added to the scenario
command's contents.
The convention for this is to include a template called init
in your module which expects no arguments like so:
(~module supply-chain (template init[] (def remaining-boilers on:simulation default: 0) ) (template limit-measure[] ...) ... )
If you follow this convention, you can use the special macro ~init-modules
, which expands out to the contents of every template called init
in each ~module
that has been seen by the model.
Notice in the example we have written that we defined a variable remaining-boilers
; ideally we would want to prefix this variable with the name of the module as well, so that it does not accidentally coincide with anyone else's variable definitions and so that it's clear where the variable came from (just as with the templates).
The ~module
command helps with this as well, by modifying any word written in it which starts with a slash to have the module name as a prefix, so the above should instead be written
(~module supply-chain (template init[] (def /remaining-boilers on:simulation default: 0) ) (template limit-measure[] ...) ... )
This would be read by the model as
(template supply-chain/init [] (def supply-chain/remaining-boilers on:simulation default:0)) (template supply-chain/limit-measure [] ...) ...
This rule also applies to the common cases of words starting with an octothorpe and a slash, as these are often cross-references, and words starting with an exclamation mark and a slash, which are often used with names for flags.
For example you might write
(~module supply-chain (template init [] (def /remaining on:simulation default:0)) (template reset [@1:new-amount] (set #/remaining @new-amount)) (template remaining-amount [] #/remaining))
Which is analogous to writing
(template supply-chain/init [] (def supply-chain/remaining on:simulation default:0)) (template supply-chain/reset [@1:new-amount] (set #supply-chain/remaining @new-amount)) (template remaining-amount [] #supply-chain/remaining)
18.6 include
commands
Models of any complexity usually need splitting in to parts, because (a) they are hard to keep in mind as a single large file and (b) because some useful parts may be reusable in other work.
The model provides the two commands include
and include-modules
to help with this.
The include
command is older, and a little simpler, so we will explain that first.
It is very simple - when the model is reading your scenario file, if it sees the include
command, it uses the command's arguments to go and find another scenario file.
This other file is the target of the include, and the file referred to is determined differently in the web-based NHM and the stand-alone application.
The contents of that scenario file replace the use of the include
command, so it is as though you had deleted the include
command and copied and pasted its target into the place where it was.
This process is recursive, so in the following situation:
(scenario ... (include A))
and A
referred to a scenario file containing
(on.dates (scenario-start) (include B) (include B))
and B
referred to a scenario file containing
(apply (action.do-nothing))
The code read by the model would be
(scenario (on.dates (scenario-start) (apply (action.do-nothing)) (apply (action.do-nothing))))
Whilst include
provides what you need to split your scenario up into parts, it is usually best if you are making a reusable part to create a module with the ~module
command which we have already encountered.
If you do this, you can use the include-modules
command to include only the modules defined in a particular scenario.
include-modules
works exactly as include
, except for two important differences:
- Only modules are considered; at the moment this means:
- uses of the
~module
command are kept, and - uses of the
include-modules
command are processed recursively
- uses of the
- Because only modules are included, cycles between modules are allowed.
The code for module
A
can useinclude-modules
to see the code for moduleB
and vice-versa. This is not true when usinginclude
, in which case cycles of inclusion produce an error.
The section on best practice includes suggestions about how to organise code into modules, and how to use include-modules
to include dependencies.
18.6.1 Includes in the web application
The NHM web application will be replaced by the desktop version for most purposes.
However, if you are wanting to read or write scenarios in the web application you will need to know how include
commands find their target.
All scenario files in the web application live in a database, and behind the scenes every save of a particular scenario is identified by a unique identifying number.
This identifier is hard to remember and useless for reading purposes, so to include a save of a scenario you refer to it by the scenario's name and a version number.
Each combination of name and number is guaranteed to refer to a specific save.
When you want to include a save, you press the xi
button by the name to grab a version number for it.
To include that save, you can then write (include name: <<the name>> version: <<the number>>)
.
If you like, you may also leave out the name:
and version:
keywords, so
(include name: my-measures version: 5)
is equivalent to
(include my-measures 5)
18.6.2 Includes in the desktop application
In the desktop application, there is no list of all the scenarios in the world, and there are no version numbers. Instead there are only the files which the application knows about, which you can see in the project explorer view that is typically on the left.
To include one file from another, you simply give the include
command the location for the file you are including.
You can either provide the location of the included file relative to the including file, or you can provide an absolute location.
An absolute location always starts with a slash (/
), and gives the hierarchy of folders containing the file separated by slashes.
For example, if including a file called measures.nhm
in a project called decc-files
, you can refer to it as
(include /decc-files/measures.nhm)
A relative location is determined starting from the location of the including file.
For example, if the aforementioned /decc-files/measures.nhm
were to contain
(include other/capital-costs.nhm)
This would be referring to a file /decc-files/other/capital-costs.nhm
; this is worked out by
- Taking off the last part of the location of the including file, to get
/decc-files/
- Adding on the relative location written in the
include
command.
The useful thing about relative locations is that you can move a collection of files around or rename the folders containing them, and not have to update the includes.
18.7 TODO Exercises about templates
19 "Optimisation"
This section is written in scare quotes, as the NHM is not really an optimisation system. If you are familiar with optimising models which work by producing a linear or mixed-integer form for a problem and then solving it, it's important to realise that the kind of optimisation these tools do is not really applicable in the NHM. This is because it's possible to set up very non-linear behaviours in the NHM, for which no general optimisation method can work. In this sense the model is better suited to simulating what would happen if I did X, Y and Z than it is for answering questions like "what is the best way to achieve some goal G given limited resources and the following notion of what best means".
However, the model does provide a few limited facilities for optimising its behaviour in some simple circumstances which may meet some requirements of this kind.
19.1 Using repeat
for goal-seeking
The repeat
command re-runs part of a scenario repeatedly until a certain criterion is met.
The effect of any changes made in that part of the scenario are all undone except for changes to selected variables, so each iteration of the repeat is like a separate trial of what would happen, given certain parameters for those variables. In this sense it's analogous to the sort of goal seeking function you may have used in spreadsheet programs before.
repeat
is used within on.dates
, and can contain commands like aggregate
and apply
, so it can repeat the application of a single moment in simulated time, but cannot repeat the execution of things which take place on several dates.
To use repeat you need to think of the command you are going to perform, and how it will be controlled by the variable that you are going to change. For example, imagine you wish to model the installation of measures with a subsidy, where:
- Only measures which break even will be installed
- The subsidy will be per kg of emissions avoided
- The amount of subsidy available is unlimited
- There is a target amount of emissions to avoid overall
- The objective is then to simulate the installation of measures at an "ideal" subsidy level. This ideal level is one where the minimum subsidy which achieves the target has been chosen. Note that this is not the same as giving each house the minimum subsidy that house requires to achieve the target, but instead describes a situation where only one subsidy level can be offered and it is the response of the houses which require the largest subsidy to break even which sets the subsidy level for everyone. This is why the repeat command is needed - we can only know whether a subsidy level does the job after we have made the offer, so we have to try the offer repeatedly with increasing amounts of subsidy until a suitable level is reached.
The repeat recipe for this has the following parts:
- The termination rule; a repeat command needs to know when to stop repeating and move on with the simulation
- The command to perform repeatedly
- The list of variables to preserve between repetitions - apart from changes to these variables, all the changes the repeat makes will be undone until the termination rule is met.
- The change we want to make to these variables - if we do not make some change then the repeat will not cause anything to change, and the model will run forever18.
In our example, these parts are:
We want to stop when the total emissions reduction meets our goal:
(> (summarize (aggregate.sum (fall-in house.emissions))) 10000)
We want to install some measures with a subsidy per tonne of emissions avoided:
(def subsidy-per-tonne on:simulation default:0) (on.dates regularly (apply (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50 capex:(* 10 size.m2)) subsidy: (* #subsidy-per-tonne (fall-in house.emissions))) (fail-unless (<= net-cost 0)))))
We have also made sure that the measure installation will fail unless the net cost after the subsidy is given is not positive.
We can see that the code above will install a number of measures which depends on the value of
#subsidy-per-tonne
. A larger value will cause more installations, as the increasing subsidy will come to dominate the installation cost even in houses where the emissions reduction is not so large. Note however that this effect has an upper limit, as for some large subsidy level the capital cost will have been exceeded by the subsidy for all houses in which there is any reduction in emissions; at this point any further increase in subsidy level cannot cause more installations or reduce emissions any further as all the potential has been realized.Anyhow, this shows us the variable that we need to keep between steps; we are only interested in preserving the value of
#subsidy-per-tonne
, as this is what controls the rest of the behaviour.We can hence introduce our
repeat
into theon.dates
command:(def subsidy-per-tonne on:simulation default:0) (on.dates regularly (repeat until: (> (summarize (aggregate.sum (fall-in house.emissions))) 10000) preserving: #subsidy-per-tonne (apply (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50 capex:(* 10 size.m2)) subsidy: (* #subsidy-per-tonne (fall-in house.emissions))) (fail-unless (<= net-cost 0))))))
Our code above is nearly finished but the astute reader may have noticed that there is no instruction given to change the value of the controlling variable, so the model will do this:
- Arrive at the repeat; a hypothesis is produced in which to try out the change
- Install the measure with subsidy level zero; this will fail in all cases, because the capex is always positive and the subsidy always comes out zero. Hence the net cost is always positive.
- Undo the hypothesis, because the
until
condition is not met; no measures have been installed, so there is no reduction in emissons. - Try again at the first step
To avoid this problem, we need to do something to increase the subsidy level:
(def subsidy-per-tonne on:simulation default:0) (on.dates regularly (repeat until: (> (summarize (aggregate.sum (fall-in house.emissions))) 10000) preserving: #subsidy-per-tonne (apply (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50 capex:(* 10 size.m2)) subsidy: (* #subsidy-per-tonne (fall-in house.emissions))) (fail-unless (<= net-cost 0)))) (set #subsidy-per-tonne (+ 1 #subsidy-per-tonne))))
Now the model will do something like this:
- Arrive at the repeat, producing a hypothesis in which to try out the first subsidy level of zero pounds.
- Perform the
apply
command with zero pounds resulting in no emissions reduction - Increase the
#subsidy-per-tonne
variable to take the value 1 - Check the failure condition for the repeat; the test has failed and so the model will revert any changes that are made, except for those to so with the preserved variables
- Now the model tries again, at the same starting point except that the subsidy level is higher. The same houses are offered the measure, except now some of them may break-even and keep the measure as a result.
- The subsidy level is increased again to 2
- Now the condition is checked again; let's say that 1 is not a sufficient subsidy level and so the changes are rolled back again.
- The model tries again, offering the same houses the same measures except with a unit subsidy of 2. This time more houses will take up the subsidy. Let's say that 2 is a sufficient subsidy level. When the test is calculated this time, the repeat will exit and the model will continue.
- The result is as though you had left out the repeat, and used some perfect foreknowledge of the outcome to select 2 as the correct subsidy level in the first place.
19.1.1 Avoiding infinite repetition
The model will detect situations where the controlling variables for a repeat statement are not changing. In these situations, because the model is deterministic an infinite loop is guaranteed, and the simulation will stop with an error.
It will also terminate with an error after a certain number of repetitions have been tried, as a safety measure.
However, if the commands you are repeating take a long time to run you may want to stop the loop early yourself. You can do this by defining your own variable which you use to count the number of goes around the loop and including it in the command to try, the list of preserved variables, and the termination rule:
(def subsidy-per-tonne on:simulation default:0) (def iterations on:simulation default:0) (on.dates regularly (repeat until: (or (> (summarize (aggregate.sum (fall-in house.emissions))) 10000) (> #iterations 10)) ; only ten tries preserving: [#subsidy-per-tonne #iterations] (apply (do (finance.with-subsidy (measure.wall-insulation u-value:0.5 type:cavity thickness: 50 capex:(* 10 size.m2)) subsidy: (* #subsidy-per-tonne (fall-in house.emissions))) (fail-unless (<= net-cost 0)))) (set #iterations (+ 1 #iterations)) (set #subsidy-per-tonne (+ 1 #subsidy-per-tonne))))
19.1.2 Reporting from a repeat
As with other hypothesis-using commands, the model will undo any changes which are made to houses when a hypothesis is undone but it will not undo any reporting commands. Hence, if the model makes reporting output within a repeat command that output will appear even for the iterations of the repeat which were run to try out situations that were eventually discarded.
This is potentially useful, as it allows you to see the intermediate stages of the repeat (for example you could keep track of the number of goes round the loop as mentioned above, and output that into a report along with other quantities of interest to see how they respond to the change in the controlling factor).
It is also potentially confusing, as the report will contain descriptions of situations which did not end up actually happening in the model's canonical view of things. This can always be avoided just by moving any reporting statments so that they are outside the repeat command following it, in which case you will know that the repeat has finished and that you are seeing its final consequences.
19.2 Using in-order
for delivery of measures according to a MAC
Another simple form of optimisation which the model can do is to offer measures to a population of houses in order of a user-defined function of how the house will be after taking a particular measure.
This is a bit like the choice
command which we have already seen, when used with a select.minimum
or select.maximum
rule; however, choice is an action which operates on a single house, so it will choose the best option for that house alone. When used with apply
, the choice will still be presented with houses in a random order.
As a result, in scenarios where there are a limited number of measures available, a choice may be configured to produce the best cost-benefit for each house in turn but this will not achieve the best cost-benefit overall because the random order from apply will choose a random selection of houses to have first dibs on the pool of measures or subsidy, and the houses in the random selection will probably not be those with the best cost-benefit in the population as a whole.
To achieve this limited form of global optimisation, the in-order
command is provided.
It can be used within an apply
command, and works by using a hypothesis to work out the impact of installing each measure from a list of alternatives independently in each house from the targeted population.
The order implied by the hypothetical impact calculations is then used to sort the possible measure-house pairings, which are then performed non-hypothetically until they are exhausted.
To make this more concrete, here is an example:
We have a million pounds to spend on insulation; however, the cost to install insulation everywhere that could take it exceeds a million. It follows that we must allocate insulation in some way; we can model the limited supply by adding extra suitability rules to the insulation measure which make it fail if there is not enough money left.
(def budget on:simulation default: 1000000) (def-action limited-insulation (do (measure.wall-insulation type:cavity thickness:50 u-value:0.5 capex: (+ 100 (* 10 size.m2))) (consume #budget (* net-cost house.weight))))
Using this example measure, running (apply #limited-insulation)
is like allocating the insulation by lottery.
We might instead hope for a more technocratic solution implemented using in-order
to select those houses which will achieve the greatest emissions reduction per pound spent.
(on.dates scenario-start (apply (in-order ;; the fall in emissions is the value we want to work out for each house ;; this is a value that can only be calculated by using a hypothesis as ;; it depends on what the measure does to a particular house ;; we divide by the cost, as we want cost effectiveness objective: (/ (fall-in house.emissions) net-cost) ;; we want to sort by decreasing order of fall in emissions, which is to say ;; we want to do the most cost effective possibility first, then the next and so on. ascending: false #limited-insulation)))
In this example, we are offering a single measure to all of the houses; the model will proceed as follows:
- For each house, the model will create a hypothetical environment in which that house gets the measure (if the house is suitable for that measure)
Because these hypotheses are independent they are each starting with the full budget left in the
#budget
variable, so no house is unsuitable because the budget ran out (unless it is a very large house, in which case we cannot afford it anyway). - In that hypothesis, the model will work out the objective; this will be the fall in emissions due to the measure, divided by the cost of its installation.
- This produces a table (effectively) containing a row for each house and a single column for the insulation measure, each cell containing the cost-effectiveness of doing that measure for that house.
- Finally we sort the table in descending order, and then go through it doing the measures. Since this time when we do the measures they are not in independent hypotheses, doing the first measure will use up a bit of the budget, the next a bit more and so on until the budget starts to get tight. At this point some of the rows in the table will become infeasible, if they do not fit in the budget. The model will keep trying to do things in the specified order, but some things which were "marginally" suitable will have become unsuitable now that the money has run out.
If you give the in-order
command more measures to consider, it will build a table which contains the objective value for every combination of houses and measures, and then run through that cell-by-cell in the desired order.
19.3 Optimisation exercises
- When using
in-order
, what is the difference between sorting byhouse.emissions
in ascending order and sorting by(fall-in house.emissions)
in descending order? What order will each of these two options tend to rank things in (i.e. what kind of houses will be at the top or bottom of the list)? - When using
repeat
, what potential mistakes or hazards should you look out for? - Write a scenario in which you use
repeat
to determine what resistance cavity wall insulation would be required to create a 5% reduction in carbon emissions if it were applied to all suitable houses at a thickness of 50mm - Write a scenario in which you use
in-order
to find the maximum reduction in fuel bills manageable by installing 100,000 new boilers of a good efficiency. - Think about how you could answer the above two questions without using the NHM's optimisation features; both of these problems can be solved simply with a probe report and a spreadsheet. Consider what this says about the situations in which you should use the facilities in the model, and when you should use tools outside the model.
19.3.1 Answers
Ranking houses by raw emissions in ascending order will prioritise putting measures into houses which have low emissions after the measure, whether or not the measure is the cause. Houses which have low emissions after the measure probably also have low emissions before the measure, most likely just because they are small houses or are already quite efficient.
Conversely, ranking houses by decreasing order of the reduction in emissions will prioritise putting measures into houses which most reduce their emissions as a result of the measure going in. A house with high emissions will be offered a measure before one with low emissions if the reduction due to putting in that measure is greater. In many cases this will effectively be the reverse order, because larger houses will normally have larger savings if a measure typically has a certain percentage effect.
A repeat may trip you up in a number of ways:
- Overshooting a target: if you are using repeat to search a range of values for one which produces some outcome, if the value starts too high then the repeat may finish successfully, but not optimally.
- Hitting an upper limit: if the command being repeated simply cannot produce an outcome that allows the repeat to terminate then your scenario may run for a long time and eventually terminate with an error.
- Failing to reset the controlling variables in a new year: if you are using a repeat to meet a goal in each year in a series of years then you need to remember to reset the repeat's controlling variables each time before it happens. Otherwise year 2 will start searching at the subsidy level for year 1, which you may not want.
The way to overcome all of these potential problems is to investigate the response to the controlling variables manually before running your scenario in anger. You can do this either by writing your own probe and analysing it in a spreadsheet, or by producing a small scenario in which all that is happening is a repeat that scans through a sensible range of values and produces a report from within the repeat - this will let you plot the response of the outputs you are interested in to the variables that are being changed.
This is an example of a question for which the model needs the repeat command, because we need to keep upping the resistance value for all the houses at once, so we have to roll back the changes we make to the whole stock until we find the desired outcome:
(scenario start-date: 2016 end-date: 2016 stock-id: a-stock (def resistance on:scenario default:0) (on.dates scenario-start (aggregate name:before (aggregate.sum house.emissions)) (repeat until: (>= 5% (/ (summarize (aggregate.sum (fall-in house.emissions))) (summarize (aggregate.sum (original house.emissions))))) preserving: #resistance (set #resistance (+ #resistance 0.01)) (apply (measure.wall-insulation thickness:50 resistance:#resistance type:cavity))) (aggregate name:results (aggregate.sum house.emissions) (aggregate.mean #resistance))))
This is a problem which is solvable either using in-order, as we want to work out how good a bunch of options for putting measures in houses would be, and then do the best until the money runs out.
(scenario ... (def budget on:simulation default:0) (def-action limited-boiler (do (measure.standard-boiler capex:(+ 100 (* size.kw 13)) size:house.peak-load) (consume #budget (* house.weight net-cost)))) (def-report effect-report (column name:cost value:capital-cost)) (on.dates scenario-start (apply (in-order report: effect-report objective: (/ net-cost (fall-in house.fuel-cost)) #limited-boiler))))
Question 3 can be answered using a series of probe reports, each installing a measure with a different resistance. If each probe is inside a
do
and followed by anaction.fail
then the probe reports will be independent as no measures will really get installed. Such reports would be easy to analyse in a spreadsheet to determine the measure with the appropriate net effect. A similar but simpler solution is possible usingdef-report
, as this can produce the total impact grouped by each variant of the measure.Question 4 can be answered by using a single probe report to output the cost effectiveness of doing the measure to the whole stock. Sorting this report by cost effectiveness will essentially reproduce what the model is doing internally.
The important point here is that a lot of simple exploratory work like this is easier outside the model, and building up an understanding of your scenario by investigating in this way is often a good idea. The on-model features are intended for situations in which they are required, such as in a simulation where you wish to deliver an ideal order of measures each year for several years. In this case the houses affected in the first year need to be included in the baseline for the second, and so on, and so the choices of which houses end up with which measures need to be known on-model.
20 TODO Scenario assumptions, defaults, and SAP
An important question for your models will be what assumptions are being used, for which parts of the scenario.
Assumptions in the model cover several different areas, and most different kinds of assumption can be changed independently of one another. This flexibility can cause difficulties, making it hard to reason about what is happening in one bit of the scenario unless you have a good understanding of the whole thing.
To mitigate this risk, we recommend not using code that you don't understand, and trying hard to avoid having to use potentially conflicting or divergent assumptions within the same piece of work.
Rather than having a single block or some blocks of assumptions which are always present in the model, assumptions are instead present at various points, and at each point may change some or all of the state of the model to reflect a certain overall combination. The main moments when assumptions are imposed are:
- Built into the model are assumptions which dictate the details of things like the energy calculation.
When a stock is created, the stock creation process itself represents a large assumption about how to interpret the housing survey, and it also applies some specific numerical assumptions about the physical structure of a house, like u- and k- values for walls and windows, etc.
When the model loads a stock, the initial condition for all these aspects of the houses is determined by these assumptions.
Once a house is taken from the stock, some other information is added to it by the model which describes those things about the house that are not stored in the stock. These are things like:
- The weather conditions to use in the energy calculation
- The heating behaviour for the house; its heating schedule and temperatures
- The tariffs for the house; how its fuel is priced
- The carbon emissions factors used in calculating emissions
All of these quantities have default values which are described below; the defaults can be overridden by setting up appropriate rules in your scenario to change them
As the model runs a house may be changed by things that the simulation has been instructed to do; for example, if you install an insulation measure in a house, the house's u-values for the relevant part will be changed in the way the measure says to change them.
Thus a measure can take a house whose u-values are given by one set of assumptions from the stock, and change it in a way that no longer aligns with those assumptions. This change does however still reflect a particular assumption, which is whatever the scenario author wrote about the measure; for example they may have assumed that their insulation has a certain resistance. Adding a layer of such insulation may nevertheless produce a u-value which was not in the original stock at all.
As a scenario author, it is your job to try and make sure that the parts you assemble in your scenario have compatible assumptions in this way.
Finally, in a scenario there may be hypothetical changes assembled around a particular calculation which further amend the state of affairs but only in a way which affects that calculation. This is what the
under
command is for; for example, as just suggested, you may have a scenario in which a measure has introduced a u-value which is sensible from the point of view of the evidence about that measure, but then you may want to do some calculation using a different u-value which is approved for a certain purpose.The under command will let you do this by using an action which changes the house's u-values directly, without interfering in the operation of the rest of the scenario.
20.1 Efficiency
Changing the assumptions in the model using under
is flexible, but it is not efficient. This is because creating the hypothesis, making the desired changes, and then reevaluating the energy use and any other down-stream effects takes the model a certain amount of time, and the model is not clever enough to avoid repeating the work when many values are being computed using similar or identical hypotheses.
Because of this it is a good idea to try and avoid setting up lots of different combinations of assumptions in your scenarios; if you have one main set of evidence, try and use this evidence consistently and make sure that each of the steps described above
20.2 SAP assumptions
20.3 Default values
21 TODO In-use factors
21.1 TODO Why in-use factors cannot be properly integrated with a simulation model
21.2 TODO How to implement them anyway if you really have to
22 TODO Assembling the parts
This section contains a few examples about how you can model behaviours which require a few different NHM features to be used together. Each example may end up using several features, but you can return to them after reading up on the features being introduced as you go.
22.1 TODO A calculation of technical potential
22.2 TODO A simulation model
22.2.1 Installing some measures [the basics]
22.2.2 Tracking measures installed and their effects [flags, reports]
22.2.3 Moving definitions to one place [definitions]
22.2.4 Adding in some off-model suitability rules [flags, sampling rules]
22.2.5 Limiting the supply of measures [variables, action.case]
22.2.6 Modelling household behaviour [choices]
22.2.7 Manipulating behaviour with a subsidy
22.2.8 Tidying up repeated code [templates and modules, includes]
22.2.9 Optimising measure delivery [in-order]
22.2.10 Investigating sensitivity to key variables [batch]
22.2.11 Cost recovery on bills [tariffs, costs]
23 TODO Quality and Best Practice
23.1 TODO General principles
- Reading code is harder than writing it, and harder still is changing it once it is written. This is a fairly universal law, which is always worth thinking about when writing for both computer and human audiences.
- Writing down some code is quite easy; however writing down correct code is quite hard. A corollary of this is that if you have some code which does not work properly, you should not be afraid of starting from scratch (perhaps with your old code as a reference).
23.2 Embedding QA checks into your scenario with assert
There are several good reasons to make tests for your scenario (or for any computer program).
Tests can be expressed for humans to run (for example, a QA checklist of some kind), or they can be expressed for computers to run.
The NHM has a command to help with the latter called assert
.
- It helps you think through what you actually intend the program to do: What is a correct outcome? What is an incorrect one? How do you tell the difference?
- It helps you be confident that the program does what you intended: This is especially useful if you change the program, as programs are like wallpaper - if you push a bubble down in one place it often pops up somewhere else.
- It helps communicate to other people what the program is supposed to do: To some degree, reading a test may explain what the expected way to use some code is, and how it should behave.
An assertion is a claim of fact; in the NHM, the assert
command provides a way to make a check that a claim of fact is true in the simulation's opinion.
If the simulation finds that the claim is not true after all, it will stop early and produce an error in the errors.txt
file.
The assert
command is a sibling of apply
; you can use it inside on.dates
, and when the simulator gets to the date it will check the assertion is upheld.
For example, to check the simulation's belief in the identity of integers, you could say
(on.dates (scenario-start) (assert (= 1 1)))
If this code were included in your scenario, when the scenario started the model would evaluate the logical statement "is 1 equal to 1", and if that were not the case it would produce an error and the simulation would stop.
This is a didactic example which you are not likely to want to use, but you might well want to do something like check that a certain number of houses have a particular flag on them:
(on.dates (scenario-start) (assert (> (summarize (aggregate.where (house.flags-match interesting-flag))) 1000000)))
This will check that at least 1 million houses have the flag interesting-flag
.
Assertions like this will be checked once; if you want to make an assertion about each house in a population you can give assert
a set of houses with the over:
argument.
Then the test will be checked for each house, and if any house fails the simulation will stop with an error.
For example if you wanted to check that all the flats in your stock had less than 10,000 square metres of floor area as a pre-condition on running your scenario, you could say
(on.dates (scenario-start) (assert over: (filter (any (house.built-form-is ConvertedFlat) (house.built-form-is PurposeBuiltHighRiseFlat) (house.built-form-is PurposeBuiltLowRiseFlat))) (< (house.total-floor-area) 10000)))
The later section on writing good modules shows how you can and should include assertions and test scenarios as part of any reusable bit of code.
23.3 Writing good modules
If you have been re-factoring your code regularly into small testable parts, and writing QA tests for those parts, you will probably find some modules naturally "fall out". See the section on templates and modules above if you do not know what a module is, or feel like you do know what it is but are not really sure how they work.
We would suggest following these conventions for all modules which you write:
- Ideally, no more than one module in a file
- Ideally, the file should be named for the module in it
- A module's file should
include-modules
all the modules it depends on - A module's file should include documentation comments and a test scenario
- Everything which you want to use from a module should be enclosed in a template
- A module should have an
init
template which sets it up, as far as possible
For example, imagine designing a module which is intended to model the hard-to-treat status of wall insulation in houses.
First we should choose a name, for example htt-walls
as we are modelling hard to treat walls
To start with let's have a module with nothing but an empty init
template:
(~module htt-walls (template init []))
Now before we dive in, let's imagine the things which we might want to present to the code which is going to be using our module. You could call this the module's interface; it is a set of templates that we expect other people or code to be using. It is worth thinking carefully about the interface, as changing it later will be more difficult (since other code will have made use of it). As far as hard-to-treat is concerned, we will want to provide a template which people can use to find out whether or not a house's insulation is hard to treat.
(~module htt-walls (template init [] ;; at the moment, nothing doing here. ) (template is-htt [] ;; this template produces a logical test ;; if the test is true, then the house will be ;; hard-to-treat when installing insulation ) )
Now users of our module will perhaps employ the command (htt-walls/is-htt)
to target hard-to-treat houses, or to adjust the costs of their measures or similar.
Note that we have added a comment which tells a reader what the template is all about.
One way to model hard-to-treat insulation would be to add a flag to each house that has HTT status; if we decide to use that approach, we can fill in the is-htt
template:
(~module htt-walls (template init [] ;; at the moment, nothing doing here. ) (template is-htt [] ;; this template produces a logical test ;; if the test is true, then the house will be ;; hard-to-treat when installing insulation (house.flags-match /hard-to-treat)) )
So now we are saying that if the house has a flag htt-walls/hard-to-treat
, then it will be considered hard to treat.
However, users of our module do not and should not need to care about this.
It is an implementation detail; we could instead change this to use house.static-property-is
to look at an external dataset, or to use some other technique.
This underlines the importance of choosing an appropriate interface.
Continuing with our flag approach, imagine we have some evidence about hard-to-treat insulation; say that there should be a certain count of hard-to-treat properties having a certain built form in each region of the UK.
We want to set up the important flags as part of our init
template, so that later when people use the is-htt
template they get what they are expecting.
We might write something like this:
(~module htt-walls (template init [] ;; sets up the wall insulation flags (apply to:(sample 1234 (filter (all (house.region-is southwest) (house.any-wall has-construction:any-cavity has-cavity-insulation: false)))) (house.flag /hard-to-treat)) (apply to:(sample 4567 (filter (all (house.region-is london) (house.any-wall has-construction:any-cavity has-cavity-insulation: false)))) (house.flag /hard-to-treat)) ;; ... and so on ... ) (template is-htt [] ;; this template produces a logical test ;; if the test is true, then the house will be ;; hard-to-treat when installing insulation (house.flags-match /hard-to-treat)) )
So now a user of our module need only follow the convention that they (include-modules htt-walls.nhm)
, (~init-modules)
in their scenario
, and then they can use is-htt
with wild abandon.
Looking at the above, you might think "there's a lot of repetition in the application of the flags", and you would be right. In this case we should probably add another template, to avoid copy-paste errors; however, this template does not want to be part of the module's interface, as it is to do with the details of how the module works, and other code outside the module should probably not depend on any of those details. The language has no built-in way to distinguish between interface and non-interface templates, but a simple convention is to use some punctuation at the start of the template's name, for example a hyphen or an underscore. Using an underscore as the convention for non-interface content, we might write:
(~module htt-walls (template init [] ;; sets up the wall insulation flags (_setup_flags count: 1234 region:southwest con: anycavity cav:false) (_setup_flags count: 4567 region:london con: anycavity cav:false) ;; ... and so on ... ) (template _setup-flags [@count @region @con [@cav] [@ext] [@int]] ;; used internally to set the flags on houses ;; we will sample @count houses from @region for which ;; construction is @con (per house.any-wall has-construction:...) ;; and for which the given kinds of insulation are around (apply to: (sample @count (filter (house.region-is @region) (house.any-wall has-construction: @con (~maybe has-internal-insulation: @int) (~maybe has-external-insulation: @ext) (~maybe has-cavity-insulation: @cav)))) (action.flag /hard-to-treat))) (template is-htt [] ;; this template produces a logical test ;; if the test is true, then the house will be ;; hard-to-treat when installing insulation (house.flags-match /hard-to-treat)) )
The technical details of this are not hugely important; the key thing being illustrated is that although we have rearranged the internals of our module, users of the interface do not need to care.
Having defined our module, before we go and integrate it into other potentially complex bits of work, we should test it independently of any other bits.
The recommended way to do this is to add a scenario
to the top of the module's file; this scenario
will be stripped out when the file is included via include-modules
, but can contain assertions and reports to help you test that just this module works properly.
In our case we might want to generate a report on the HTT status broken down by different regions, and maybe assert that the total count is what we would expect:
;; test scenario at the top: ;; probably a good place to write some documentation ;; explaining: ;; - roughly how to use the module ;; - ensure you have used ~init-modules ;; - use the (htt-walls/is-htt) command to determine if a house is htt ;; - the methodology behind it, links to sources of data, etc. ;; - based on X Y and Z (limitations: A B and C) (scenario start-date: 2015 end-date: 2015 stock-id: test-stock (~init-modules) ;; this will init our module below (on.dates 2015 ;; now report on the counts (aggregate name: htt-count divide-by: [(house.region) (htt-walls/is-htt)] (aggregate.count)) ;; also use an assertion to check the result more directly (assert (= 5801 (summarize (aggregate.where (htt-walls/is-htt))))) )) ;; module definition (~module htt-walls (template init [] ;; sets up the wall insulation flags (_setup_flags count: 1234 region:southwest con: anycavity cav:false) (_setup_flags count: 4567 region:london con: anycavity cav:false) ;; ... and so on ... ) (template _setup-flags [@count @region @con [@cav] [@ext] [@int]] ;; used internally to set the flags on houses ;; we will sample @count houses from @region for which ;; construction is @con (per house.any-wall has-construction:...) ;; and for which the given kinds of insulation are around (apply to: (sample @count (filter (house.region-is @region) (house.any-wall has-construction: @con (~maybe has-internal-insulation: @int) (~maybe has-external-insulation: @ext) (~maybe has-cavity-insulation: @cav)))) (action.flag /hard-to-treat))) (template is-htt [] ;; this template produces a logical test ;; if the test is true, then the house will be ;; hard-to-treat when installing insulation (house.flags-match /hard-to-treat)) )
This is a reasonably complete example of a single module; it has the following properties:
- A reader, upon looking at it, can first see something about how to use it and how it works
- Along with some documentation, the reader is given a concrete example of some commands used in context, which they can run and fiddle with
- This concrete example shows some relevant outputs, and also will produce an error if certain invariants are not preserved
- The embedded QA logic gives the reader some idea of the degree of reliability they should expect
- The module itself is divided into interface and non-interface templates
- Each template has a little comment explaining something about it
When writing a module which depends in turn on other modules, you should include-modules
those dependencies at the top level in the module's file; for example, if our project included a module defining some insulation measures, which used the hard to treat module to control the capital cost of some measures, that might look a bit like this:
;; wall-insulation-measures.nhm - defines some wall insulation measures (include-modules htt-walls.nhm) ;; capex depends on whether a wall is htt ;; so we use the htt module here ;; demo scenario - should have some tests really (scenario stock-id:some-stock (~init-modules) (on.dates scenario-start ;; check that when we put insulation in a house which is HTT ;; the capex is 1000 (assert over: (all (filter (htt-walls/is-htt)) (house.is-suitable-for (wall-insulation-measures/cavity))) (= 1000 (under (wall-insulation-measures/cavity) evaluate:(capital-cost)))) )) (~module wall-insulation-measures (template init [] (def-action /cavity-insulation-action (measure.wall-insulation capex: (function.case (when (htt-walls/is-htt) 1000) default: 500) type:cavity thickness:50 resistance: 0.01 ))) (template cavity [] #/cavity-insulation-action))
Because it transitively includes htt-walls.nhm
, another scenario can just include-modules
wall-insulation-measures.nhm
and expect it to work correctly, without having to know that wall-insulation-measures
currently uses htt-walls
.
23.4 TODO My scenario doesn't work
23.4.1 What to do if your scenario is doing something unexpected; warnings and errors
23.4.2 How to make scenarios that are likely to work
23.4.3 How to make scenarios that run quickly
Footnotes:
Precisely, any subset from all the possible subsets, as the boiler failures are independent. The expectation would be for 100 dwellings to replace their boiler.
This example uses the default weighting behaviour of the model; there are other options described in the manual.
Modelling such a policy should raise a different question, about whether a model like the NHM can usefully represent changes to very small populations, given the intrinsic error which comes from using BREDEM and housing surveys.
This sampling method biases the mean down a bit when the quantum is large (because there are fewer samples which match exactly). In the NHM, the sampling method has a correction for this which removes the bias and reduces the variance in the mean, but it is harder to analyse.
The temperatures will eventually equalize at the mean of \(T_1\) and \(T_2\) weighted by the thermal masses involved, following an exponential decay as the \(H\) reduces. However, because the thermal mass of the whole outside world is much larger than that of the dwelling, we say that the outside world is not noticeably heated by the dwelling.
There are some other ways you can use the NHM: there is a command-line tool which can run and validate scenarios, which you can use with tools like emacs, and there is a web-based system for running it which may well have been deprecated by the time you read this. If you are in DECC you might call this the "Standalone" NHM.
If you are unfamiliar with menu bars and menus, you may wish to suspend learning the NHM application, and learn more about the basics of graphical interfaces to computers.
At this point you will have noticed that we are writing about terms that we have not defined. The actual terms in the language are explained later on.
Again, if you are not familiar with this term read ahead first.
Not in Microsoft Word though - Word is not a text editor. Do not do this; it will shred your work.
A line is a series of characters ending in the ASCII 'line feed' character (this is character 10 in the ASCII table), and a tab is character 9 in the ascii table. This is about the simplest way to store a table, and will be understood by almost any tool.
You could go to case level by saying divide-by: [house.survey-code]
, but this is not a good idea.
The model does not currently provide facilities for users to define or manipulate any data other than flags and numbers.
This design choice reflects the fact that different policies tend to have different rules for accounting, and that making the model intelligently determine precisely what things should be accounted for would be difficult and likely frustrating.
If you have seen the blockbuster film Inception, it is essentially like this but more exciting.
This is fairly reasonable, as the model cannot read its own reports back in so as to make decisions; writing to a report should not affect the sequence of events in the simulation. There is a minor exception here, which is that writing reports which use random numbers is allowed to change the random sequence, but reporting on random numbers seems like an odd thing to do.
Actually, the model will detect if nothing has changed at all between two steps of a repeat, and if this happens it will terminate and produce an error. However, you will still need to guard against situations where something changes but the termination condition is not met.