Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem with updating the database #512

Open
KarolPlewka opened this issue Dec 15, 2023 · 0 comments
Open

Problem with updating the database #512

KarolPlewka opened this issue Dec 15, 2023 · 0 comments
Labels
bug

Comments

@KarolPlewka
Copy link

Describe the bug
When I try to update the database as it's described in readme

docker exec -it nominatim sudo -u nominatim nominatim replicati on --project-dir /nominatim

I've got following error:

`2023-12-15 11:03:18: Using project directory: /nominatim
2023-12-15 11:03:22: Starting indexing rank (0 to 4) using 1 threads
2023-12-15 11:03:22: Starting rank 1 (using batch size 1)
2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 1

2023-12-15 11:03:22: Starting rank 2 (using batch size 1)
2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 2

2023-12-15 11:03:22: Starting rank 3 (using batch size 1)
2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 3

2023-12-15 11:03:22: Starting rank 4 (using batch size 1)
2023-12-15 11:03:22: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 4

2023-12-15 11:03:22: Starting indexing boundaries using 1 threads
2023-12-15 11:03:22: Starting boundaries rank 4 (using batch size 1)
2023-12-15 11:07:32: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 4

2023-12-15 11:07:32: Starting boundaries rank 5 (using batch size 1)
2023-12-15 11:07:34: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 5

2023-12-15 11:07:34: Starting boundaries rank 6 (using batch size 1)
2023-12-15 11:07:34: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 6

2023-12-15 11:07:34: Starting boundaries rank 7 (using batch size 1)
2023-12-15 11:07:34: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 7

2023-12-15 11:07:34: Starting boundaries rank 8 (using batch size 1)
2023-12-15 11:07:54: Done 1/1 in 19 @ 0.052 per second - FINISHED boundaries ran k 8

2023-12-15 11:07:54: Starting boundaries rank 9 (using batch size 1)
2023-12-15 11:07:54: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 9

2023-12-15 11:07:54: Starting boundaries rank 10 (using batch size 1)
2023-12-15 11:07:54: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 10

2023-12-15 11:07:54: Starting boundaries rank 11 (using batch size 1)
2023-12-15 11:07:54: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 11

2023-12-15 11:07:54: Starting boundaries rank 12 (using batch size 1)
2023-12-15 11:07:56: Done 10/10 in 1 @ 5.226 per second - FINISHED boundaries ra nk 12

2023-12-15 11:07:56: Starting boundaries rank 13 (using batch size 1)
2023-12-15 11:07:56: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 13

2023-12-15 11:07:56: Starting boundaries rank 14 (using batch size 1)
2023-12-15 11:07:56: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 14

2023-12-15 11:07:56: Starting boundaries rank 15 (using batch size 1)
2023-12-15 11:07:56: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 15

2023-12-15 11:07:56: Starting boundaries rank 16 (using batch size 1)
2023-12-15 11:07:58: Done 30 in 2 @ 13.975 per second - boundaries rank 16 ETA ( seconds): 3.79
2023-12-15 11:07:59: Done 43 in 2 @ 15.975 per second - boundaries rank 16 ETA ( seconds): 2.50
2023-12-15 11:07:59: Done 58 in 3 @ 18.211 per second - boundaries rank 16 ETA ( seconds): 1.37
2023-12-15 11:08:00: Done 76 in 3 @ 19.508 per second - boundaries rank 16 ETA ( seconds): 0.36
2023-12-15 11:08:00: Done 83/83 in 4 @ 20.342 per second - FINISHED boundaries r ank 16

2023-12-15 11:08:00: Starting boundaries rank 17 (using batch size 1)
2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 17

2023-12-15 11:08:00: Starting boundaries rank 18 (using batch size 1)
2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 18

2023-12-15 11:08:00: Starting boundaries rank 19 (using batch size 1)
2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 19

2023-12-15 11:08:00: Starting boundaries rank 20 (using batch size 1)
2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 20

2023-12-15 11:08:00: Starting boundaries rank 21 (using batch size 1)
2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 21

2023-12-15 11:08:00: Starting boundaries rank 22 (using batch size 1)
2023-12-15 11:08:00: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 22

2023-12-15 11:08:00: Starting boundaries rank 23 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 23

2023-12-15 11:08:01: Starting boundaries rank 24 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 24

2023-12-15 11:08:01: Starting boundaries rank 25 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED boundaries rank 25

2023-12-15 11:08:01: Starting indexing rank (5 to 25) using 1 threads
2023-12-15 11:08:01: Starting rank 5 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 5

2023-12-15 11:08:01: Starting rank 6 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 6

2023-12-15 11:08:01: Starting rank 7 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 7

2023-12-15 11:08:01: Starting rank 8 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 8

2023-12-15 11:08:01: Starting rank 9 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 9

2023-12-15 11:08:01: Starting rank 10 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 10

2023-12-15 11:08:01: Starting rank 11 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 11

2023-12-15 11:08:01: Starting rank 12 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 12

2023-12-15 11:08:01: Starting rank 13 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 13

2023-12-15 11:08:01: Starting rank 14 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 14

2023-12-15 11:08:01: Starting rank 15 (using batch size 1)
2023-12-15 11:08:01: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 15

2023-12-15 11:08:01: Starting rank 16 (using batch size 1)
2023-12-15 11:08:04: Done 30 in 3 @ 9.773 per second - rank 16 ETA (seconds): 1. 43
2023-12-15 11:08:07: Done 39 in 6 @ 6.188 per second - rank 16 ETA (seconds): 0. 81
2023-12-15 11:08:08: Done 44/44 in 6 @ 6.471 per second - FINISHED rank 16

2023-12-15 11:08:08: Starting rank 17 (using batch size 1)
2023-12-15 11:08:08: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 17

2023-12-15 11:08:08: Starting rank 18 (using batch size 1)
2023-12-15 11:08:09: Done 42/42 in 1 @ 28.632 per second - FINISHED rank 18

2023-12-15 11:08:09: Starting rank 19 (using batch size 1)
2023-12-15 11:08:09: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 19

2023-12-15 11:08:09: Starting rank 20 (using batch size 1)
2023-12-15 11:08:12: Done 30 in 2 @ 12.033 per second - rank 20 ETA (seconds): 4 .57
2023-12-15 11:08:12: Done 42 in 2 @ 15.544 per second - rank 20 ETA (seconds): 2 .77
2023-12-15 11:08:13: Done 57 in 3 @ 16.251 per second - rank 20 ETA (seconds): 1 .72
2023-12-15 11:08:13: Done 73 in 4 @ 18.217 per second - rank 20 ETA (seconds): 0 .66
2023-12-15 11:08:14: Done 85/85 in 4 @ 18.828 per second - FINISHED rank 20

2023-12-15 11:08:14: Starting rank 21 (using batch size 1)
2023-12-15 11:08:14: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 21

2023-12-15 11:08:14: Starting rank 22 (using batch size 1)
2023-12-15 11:08:16: Done 36/36 in 2 @ 16.190 per second - FINISHED rank 22

2023-12-15 11:08:16: Starting rank 23 (using batch size 1)
2023-12-15 11:08:16: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 23

2023-12-15 11:08:16: Starting rank 24 (using batch size 1)
2023-12-15 11:08:16: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 24

2023-12-15 11:08:16: Starting rank 25 (using batch size 1)
2023-12-15 11:08:17: Done 6/6 in 0 @ 11.812 per second - FINISHED rank 25

2023-12-15 11:08:17: Starting indexing rank (26 to 30) using 1 threads
2023-12-15 11:08:17: Starting rank 26 (using batch size 1)
2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 26

2023-12-15 11:08:17: Starting rank 27 (using batch size 1)
2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 27

2023-12-15 11:08:17: Starting rank 28 (using batch size 1)
2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 28

2023-12-15 11:08:17: Starting rank 29 (using batch size 1)
2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 29

2023-12-15 11:08:17: Starting rank 30 (using batch size 20)
2023-12-15 11:08:17: Done 0/0 in 0 @ 0.000 per second - FINISHED rank 30

2023-12-15 11:08:17: Starting rank 0 (using batch size 1)
2023-12-15 11:08:19: Done 190 in 2 @ 91.970 per second - rank 0 ETA (seconds): 4 0.00
2023-12-15 11:08:19: Done 281 in 2 @ 101.934 per second - rank 0 ETA (seconds): 35.20
2023-12-15 11:08:20: Done 382 in 3 @ 113.848 per second - rank 0 ETA (seconds): 30.63
2023-12-15 11:08:21: Done 495 in 3 @ 126.950 per second - rank 0 ETA (seconds): 26.58
2023-12-15 11:08:21: Done 621 in 4 @ 141.106 per second - rank 0 ETA (seconds): 23.02
2023-12-15 11:08:22: Done 762 in 4 @ 157.434 per second - rank 0 ETA (seconds): 19.74
2023-12-15 11:08:22: Done 919 in 5 @ 178.687 per second - rank 0 ETA (seconds): 16.51
2023-12-15 11:08:22: Done 1097 in 5 @ 197.430 per second - rank 0 ETA (seconds): 14.04
2023-12-15 11:08:23: Done 1294 in 5 @ 217.052 per second - rank 0 ETA (seconds): 11.86
2023-12-15 11:08:23: Done 1511 in 6 @ 230.583 per second - rank 0 ETA (seconds): 10.23
2023-12-15 11:08:24: Done 1741 in 7 @ 236.004 per second - rank 0 ETA (seconds): 9.02
2023-12-15 11:08:25: Done 1977 in 8 @ 243.116 per second - rank 0 ETA (seconds): 7.78
2023-12-15 11:08:25: Done 2220 in 8 @ 252.347 per second - rank 0 ETA (seconds): 6.53
2023-12-15 11:08:27: Done 2472 in 10 @ 243.733 per second - rank 0 ETA (seconds) : 5.73
2023-12-15 11:08:28: Done 2715 in 10 @ 250.063 per second - rank 0 ETA (seconds) : 4.61
2023-12-15 11:08:28: Done 2965 in 11 @ 260.127 per second - rank 0 ETA (seconds) : 3.48
2023-12-15 11:08:29: Done 3225 in 12 @ 259.552 per second - rank 0 ETA (seconds) : 2.48
2023-12-15 11:08:31: Done 3484 in 14 @ 242.811 per second - rank 0 ETA (seconds) : 1.59
2023-12-15 11:08:34: Done 3726 in 16 @ 221.532 per second - rank 0 ETA (seconds) : 0.65
2023-12-15 11:08:35: Done 3869/3869 in 18 @ 214.391 per second - FINISHED rank 0

2023-12-15 11:08:35: Starting interpolation lines (location_property_osmline) (u sing batch size 20)
Traceback (most recent call last):
File "/usr/local/bin/nominatim", line 14, in
exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module',
File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 264, in nomi natim
return parser.run(**kwargs)
File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 126, in run
return args.command.run(args)
File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/replication.py", li ne 196, in run
self._update(args)
File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/replication.py", li ne 161, in _update
indexer.index_full(analyse=False)
File "/usr/local/lib/nominatim/lib-python/nominatim/indexer/indexer.py", line 140, in index_full
if self.index_by_rank(26, 30) > 1000:
File "/usr/local/lib/nominatim/lib-python/nominatim/indexer/indexer.py", line 178, in index_by_rank
total += self._index(runners.InterpolationRunner(analyzer), 20)
File "/usr/local/lib/nominatim/lib-python/nominatim/indexer/indexer.py", line 234, in _index
runner.index_places(pool.next_free_worker(), part)
File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", l ine 201, in next_free_worker
return next(self.free_workers)
File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", l ine 209, in yield_free_worker
if thread.is_done():
File "/usr/local/lib/nominatim/lib-python/nominatim/db/async_connection.py", l ine 159, in is_done
if self.conn.poll() == psycopg2.extensions.POLL_OK:
psycopg2.errors.InternalError: Splitting of Point geometries is unsupported
CONTEXT: PL/pgSQL function osmline_update() line 77 at assignment

`
Container was run few months ago with following parameters:

docker run -it --shm-size=1g -e PBF_URL=https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/pbf/planet-latest.osm.pbf -e REPLICATION_URL=https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/replication/day/ -e IMPORT_STYLE=extratags -e UPDATE_MODE=continuous -v nominatim-data:/datadrive -p 8080:8080 --name nominatim mediagis/nominatim:4.2
To Reproduce
docker exec -it nominatim sudo -u nominatim nominatim replicati on --project-dir /nominatim

Expected behavior
Update should be completed without any error

Desktop / Server (please complete the following information):

  • OS & Version: Ubuntu 20.04
  • Docker Version: 20.10.21
  • Nominatim Version: 4.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@KarolPlewka