forked from Ensembles/ert
-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
301 lines (216 loc) · 11.7 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
_________________________________
/ \
| ______ ______ _______ |
| | ____| | __ \ |__ __| |
| | |__ | |__) | | | |
| | __| | _ / | | |
| | |____ | | \ \ | | |
| |______| |_| \_\ |_| |
| |
| Ensemble based Reservoir Tool |
\_________________________________/
------------------------------------------------------------------------
1. ERT
2. ECLIPSE utilities.
3. Building ERT
3.1 CMake settings you might want to adjust
4. The code:
4.1 The different libraries
4.2 The general structure
4.3 Python wrappers
5. Tests
------------------------------------------------------------------------
1. ERT
ERT - Ensemble based Reservoir Tool is a tool for managing en ensemble
of reservoir models. The initial motivation for creating ERT was a as
tool to do assisted history matching with Ensemble Kalman Filter
(EnKF). Very briefly use of EnKF can be summarized as:
1. Sample initial reservoir parameters from a (Gaussian) prior
distribution.
2. Simulate the ensemble of of reservoir forward in time through
part of the historical period.
3. Load the results, compare with observed data, update the
parameters and the state of reservoir and restart the
simulations.
This recipe is quite complex technically, and in particular involves
the ability to read and write input and output files from the
reservoir simulator (i.e. ECLIPSE in the case of ERT), run simulations
with arbitrary external programs, plotting data and so on. This
implies that a quite significant technical machinery must be in place
before the EnKF algorithm as such can be utilizied. This in particular
applies to real industry reservoir models, where typically
imperfections of all kinds flourish.
The initial motivation for creating ERT was to be able to use the EnKF
algorithm for history matching. Currently ERT is more used with the
Ensemble Smoother and also purely as a worklfow manager; herding a
large collection of reservoir models through the required simulations
steps.
2. ECLIPSE Utilities
ERT has a quite large amount of code devoted to reading and writing
the ECLIPSE output files (grid/rft/restart/init/summary). In addition
there is also reasonable support for reading and writing the grdecl
input files, but there is no general .DATA file parser. The ability to
read and write ECLIPSE output files is valuable in many reservoir
applications, and it is possible to only build and use the libecl
(with support libraries) library for working with ECLIPSE files. In
fact the default build setup is to only build the ECLIPSE related
library and utilities. This part of the ERT distribution can also be
built on Windows with Visual Studio (albeit with maaaany warnings) and
with MinGW.
3. Building ERT
CMake is the build system for ERT. The top level CMakeLists.txt file
is located in the devel/ directory, and this CMakeLists.txt file
includes individual CMakeLists.txt files for the different libraries.
Building with CMake is performed like this:
1. Create a build directory, this can in principle be anywhere in
the filesystem. At the same level as the devel/ directory is a
practical choice.
2. Go to the build directory and invoke the command:
ccmake <path/to/directory/containing/CMakeLists.txt>
Go through several 'configure' steps with CMake and generate
native build files.
3. Exit ccmake and invoke the native build system, i.e. ordinaray
make on Linux.
4. Subsequent builds can be performed using just the native make
command, as in step 3.
3.1 CMake settings you might want to adjust
The main setting you should adjust is BUILD_ERT which is default to
OFF, i.e. by default only the ECLIPSE related utilities will be
built. The build system has numerous configurations checks; the
ECLIPSE utilities should build on Windows, but to build all of ERT you
will need a Linux (Posix) system.
4. The code
The code is mainly written as a collection of libraries written in C.
4.1 The different libraries
The different libraries are:
libert_util: This library is a collection of utilities of various
sorts; if C++ had been chosen as implementation language most of
these utilities would probably have been in libstdc++.
libgeometry: This is a very small geometry library; the main code
is a small implementantion of an alorithm to determine whether a
point is inside a polyhedron. The ECLIPSE library has some
geometry related code which should be moved here.
libwell: This library will load well information from an ECLIPSE
restart file. This is mainly for the purpose of visualization
of the existing wells, and can not be used to update or model
the well configuration.
libecl: This library will read and (partly) write the various
binary ECLIPSE files, including GRID/EGRID, summary, INIT,
restart and RFT files. There is also support for reading an
writing grdecl formatted files, but there is no support for
general parsing of the ECLIPSE input format.
----------------------------------------------------------------------------
librms: This is a library for reading and writing RMS Roff
files. It turns out that ECLIPSE file formats is the most common
external file format for RMS and the ROFF support is not
essential.
libconfig: This library implements a parser for the ERT config file
format, this format is used in the main ERT configuration file,
and also in several small special topic configuration files used
by ERT. The config format parsed by this library was inspired by
the ECLIPSE format, in retrospect that was a mistake - it should
have been based on a standard format like xml.
To confuse things evan further the libconfig library implements
/two/ config formats, the 'second format' is implemented in the
file conf.c, and only used as format for the observations in
ERT.
libplot: A *very* simple librarry for creating plots, just exactly
satisfies the needs of ERT.
libanalysis: The EnKF algorithm is implemented in this library.
libjob_queue: This library implements a system to manage and run
simulations in the form of external programs. The library has a
queue manager, and a system with drivers which communicate with
the underlying system. Currently the library has a LSF driver to
work with LSF, a LOCAL driver which starts simulations on the
current workstation and a RSH driver which submits to a
'cluster' of workstation with ssh.
libenkf: This is the main functionality which is ERT specific; this
library is to large.
4.2 General structure
The code is written in C, but conventions give a 'scent of object
oriented'. Common to a very large part of the code is the following:
- Every file 'xxx' implements a datatype 'xxx_type' - the naming
convention is quite strong.
- All the struct defintions are in the source files, i.e. external
scope must access the fields of a structure through accessor
functions.
- All functions which operate on a type 'xxx_type' take a pointer
to xxx_type as the first argument, the structure closely resemble
the 'self' argument used when implementing Python classes.
- Memory management is manual; however there are some conventions:
* Functions allocating storage have _alloc_ as part of the name.
* For all functions xxx_alloc() allocating storage there should
be a matching xxx_free() to discard the objects.
* Containers can optionally destroy their content is the content
is installed with a destructor.
- In libert_util/src/type_macros.h there is a macro based
'type-system' which is used to runtime check casts of (void *).
4.3 Python wrappers
Some of the code, in particular the ECLIPSE related functionality, has
been wrapped in Python. Using these wrappers it is quite easy work
with ECLIPSE files. The python wrappers are quite well documented both
in the devel/python/docs directory and in the Python classes
themselves.
5. Test
The ert codebase has a small but increasing test coverage. The tests
are typically located in a per-library subdirectory tests/. The test
framework is based on ctest which basically works like this:
1. In the CMakeLists.txt file testing is enabled with
ENABLE_TESTING().
2. Tests are added with:
add_test( test_name test_executable arg1 arg2 arg3 ... )
If the executable exits with status 0 the test has passed,
otherwise it has failed. The executable run by the test can
typically be an executable built as part of the solution, but can
in principle be an arbitrary executable like a dedicated test
runner or e.g. the Python interpreter.
5.1 Test of C code
The main part of the testing infrastructure is small C applications
which are added like this:
add_executable( test_exe test_source.c )
target_link_libraries( test_exe lib )
add_test( test_name ${EXECUTABLE_OUTPUT_PATH}/test_exe commandline_arg1 commandline_arg2 )
Where the two first lines create the test executable in the normal
way, and the last line adds it as a test.
5.2 Test of Python Code
In devel/python/test there are several files with Python tests, these
files are executable files and they are invoked directly from the
commandline. A limited number of the tests have been integrated in the
ctest system.
5.3 Test names
The tests in the cmake build system follow the naming convention of
the library which functionality they are testing - i.e. all tests in
the libecl library have a name starting with 'ecl' and all tests in
the config library start with 'config'. The options -R and -E to ctest
can be used to include and exclude tests based on their name
ctest -R ecl # Run all tests mathing the regexp 'ecl'
ctest -E ecl # Run all tests NOT matching the regexp 'ecl'
5.4 Test labels
Using the cmake set_property() function it is possible to assigne
labels to the test, and the -L and -LE options to ctest can be used to
limit which tests to run. A test can only have one label; in the
current ctest setup different labels are combined into one composite
label with a ":" separator, i.e.
set_property( TEST test_xxx PROPERTY LABELS StatoilData:Python)
will set the 'StatoilData' and 'Python' properties on test_xxx. The
labels currently maintained in the ert test setup are:
StatoilData: This implies that the test makes use of Statoil
internal data. If you are in Statoil/Bergen you can see more
details in file devel/test-data/README for how to make this data
available.
If you are not in Statoil/Bergen you must pass the option: "-EL
StatoilData" to ctest to skip all the tests which require
Statoil data.
StatoilBuild: There is one python test which makes use of Statoil
internal configuration data, this test is labeled with
StatoilBuild. If you want to run this test you must set the
cmake option ECL_LOCAL_TARGET to point to a file which contains
some local configuration settings, e.g. where the ECLIPSE binary
is installed.
Python: This label is used to indicate that we are talking about a
Python test.
LSF: This labels indicates that the test needs a working LSF
environment to run.
ctest -L Statoil # Run all tests labeled with Statoil - both
# StatoilData and StatoilBuild
ctest -EL "Statoil|LSF" # Exclude all tests labeled with Statoil or LSF.