How could I limit the number of CPU threads used by MOTIS?

[kalon33]

On my local server, I don't want that MOTIS uses all my CPU resources. How can I limit it to use a given amount of CPU threads?

[felixguendling]

The --num_threads=4 flag or num_threads=4 INI property of the config.ini can be set to configure the number of threads.

[kalon33]

Thanks for your answer Felix, but it doesn't seem to work for graph building at least. I tried ./motis --num_threads=24 and got 40 using latest release. Le 2022-07-18 15:56, Felix Gündling a écrit :

The --num_threads=4 flag or num_threads=4 INI property of the config.ini can be set to configure the number of threads. -- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you authored the thread.Message ID: ***@***.***>

Links: ------ [1] #256 (comment) [2] https://github.com/notifications/unsubscribe-auth/AADQPO3MODQ6GZYE7FKHAK3VUVPANANCNFSM534KGHXA

[felixguendling]

That's true. Currently, MOTIS uses several external dependencies (some written by us like PPR or tiles, some by others like OSRM). Some of them use their own thread-pool implementation. The MOTIS code (in this repo) currently does not have control over the number of threads created in these third-party thread-pools. The number of threads you can specify via `--num_threads=X" controls the thread pool MOTIS uses when the server is actually running.

I created an issue for this: #258
However, currently I don't have the time resources to fix this.

[kalon33]

Thanks Felix for your explanations and for planning to fix this, even if it's not right now. Is there a resource friendly way to deal with refreshing GTFS data everyday (if possible without downtime?) and keeping other data untouched? In my situation with French railroad (SNCF) and Paris region data that I'm willing to test, stops are not changing or very marginally day after day (like routes, and roads from OSM), but a new dataset is generated everyday, that changes trip_ids, hence the mapping with GTFS-RT feeds. Thanks for your help. Le 2022-07-18 18:30, Felix Gündling a écrit :

That's true. Currently, MOTIS uses several external dependencies (some written by us like PPR or tiles, some by others like OSRM). Some of them use their own thread-pool implementation. The MOTIS code (in this repo) currently does not have control over the number of threads created in these third-party thread-pools. The number of threads you can specify via `--num_threads=X" controls the thread pool MOTIS uses when the server is actually running. I created an issue for this: #258 [1] However, currently I don't have the time resources to fix this. -- Reply to this email directly, view it on GitHub [2], or unsubscribe [3]. You are receiving this because you authored the thread.Message ID: ***@***.***>

Links: ------ [1] #258 [2] #256 (reply in thread) [3] https://github.com/notifications/unsubscribe-auth/AADQPO55KBLNHJRY5MJJN3TVUWBBVANCNFSM534KGHXA

[kalon33]

Thank you for your help! It now works (livemap notably), will try routing. How may I know that realtime is working properly/is used? How can I change interface language?

[felixguendling]

The language can be changed using the "?lang=fr" parameter. Example: https://europe.motis-project.de/?lang=en

The default language can be changed here:

motis/ui/web/js/main.js

Line 49 in d7ec276

let language = langParam || "de";

How may I know that realtime is working properly/is used?

There are different indicators:

• The log shows how many updates were processed (including errors if a trip could not be found, etc.).
• Also you'll find delay information in the routing results ("+10" for 10 min delay).
• You can also change the coloring of the vehicles to be based on delay (red=high delay, green=on-time) in the bottom right of the web UI

[kalon33]

Thanks, I'm currently trying to optimise RAM use. Is there a way to specify to MOTIS a proxy server for realtime data retrieval?

[kalon33]

And I've this kind of errors with GTFS-RT data :
[erro][2022-07-21T10:17:10Z+0000][timer.cc:50] error in timer RIS GTFS-RT Update: Illegal byte sequence
Why?

[kalon33]

OK, I removed one of the feeds and I don't have this anymore, but I've that:
[erro][2022-07-21T10:19:43Z+0000][timer.cc:50] error in timer RIS GTFS-RT Update: sslv3 alert handshake failure (SSL routines, CONNECT_CR_SRVR_HELLO)

[felixguendling]

Illegal byte sequence

I would guess this comes from the Protocol Buffers parser and means that the downloaded file was not a valid protobuf file. Not much we can do here. I could not reproduce this message with the GTFS-RT URLs in your config.ini, so I could not look into it in detail.

sslv3 alert handshake failure

I also got this error with your config. Seems like this was missing; https://stackoverflow.com/a/59225060
I added it to our networking library. MOTIS v0.8.2 will be out asap with the fix.

Is there a way to specify to MOTIS a proxy server for realtime data retrieval?

Not yet. I commited simple proxy support. However, I don't think it's usable for HTTPS-based GTFS-RT streams. This requires establishing a connection to the GTFS-RT server via the proxy using the CONNECT method which is currently not implemented. I'll try to implement it. In the meanwhile I'll release the fix for the SSL SNI error.

[felixguendling]

Thank you for your feedback!

I tested it with your data feed and added a small output similar to OTP:

[info][2022-07-21T12:22:18Z+0000][gtsfrt_parser.cc:104] sncf_tgv: parsed 517 GTFS-RT updates: 145/517 (28.0464%) successful
[info][2022-07-21T12:22:18Z+0000][gtsfrt_parser.cc:104] caee: parsed 12 GTFS-RT updates: 12/12 (100%) successful
[info][2022-07-21T12:22:18Z+0000][gtsfrt_parser.cc:104] bordeauxaero: parsed 2 GTFS-RT updates: 2/2 (100%) successful
[info][2022-07-21T12:22:18Z+0000][logging.cc:23] [flush] starting

msg types:
                   delay:       951

evs:
                   total:     16105

trips:
                   total:       951

updates
                   total:     16105
                   found:     16105

So especially the high failure rate for the sncf_tgv caught my eye.

I checked some of these messages by hand and I think they are sending updates for trips that don't exist - at least not in the GTFS-static files they provide.

First case is trip OCESN9894F876490:2022-07-20T00:33:34Z from message unable to find GTFS trip {GTFS-TRIP trip_id=sncf_tgv_OCESN9894F876490:2022-07-20T00:33:34Z, start_date=2022-07-21 00:00}:

➜  schedule grep -r "OCESN9894F876490:2022-07-20T00:33:34Z" sncf_tgv 
sncf_tgv/stop_times.txt:OCESN9894F876490:2022-07-20T00:33:34Z,15:12:00,15:12:00,StopPoint:OCETGV-87212027,0,,0,1,
sncf_tgv/stop_times.txt:OCESN9894F876490:2022-07-20T00:33:34Z,15:47:00,15:50:00,StopPoint:OCETGV-87142109,1,,0,0,
sncf_tgv/stop_times.txt:OCESN9894F876490:2022-07-20T00:33:34Z,16:27:00,16:30:00,StopPoint:OCETGV-87171926,2,,0,0,
sncf_tgv/stop_times.txt:OCESN9894F876490:2022-07-20T00:33:34Z,17:01:00,17:07:00,StopPoint:OCETGV-87271494,3,,0,0,
sncf_tgv/stop_times.txt:OCESN9894F876490:2022-07-20T00:33:34Z,17:57:00,18:08:00,StopPoint:OCETGV-87223263,4,,0,0,
sncf_tgv/stop_times.txt:OCESN9894F876490:2022-07-20T00:33:34Z,18:43:00,18:43:00,StopPoint:OCETGV-88140010,5,,1,0,
sncf_tgv/trips.txt:FR:Line::53F2E1D2-2532-4CBF-9213-90D7962D5212:,000040,OCESN9894F876490:2022-07-20T00:33:34Z,9894,0,3093,
➜  schedule grep -r "OCESN9894F876490:2022-07-20T00:33:34Z" sncf_tgv
➜  schedule grep 000040 sncf_tgv/calendar.txt sncf_tgv/calendar_dates.txt 
sncf_tgv/calendar.txt:000040,0,0,0,0,0,0,0,20220721,20220820
sncf_tgv/calendar_dates.txt:000040,20220815,1

So it exists with service id 000040 which says it never operates (calendar.txt), except on 20220815. So the error message in MOTIS is correct, that it doesn't operate today.

---

Second example is trip OCESN6077F1199309:2022-07-21T00:35:46Z from message unable to find GTFS trip {GTFS-TRIP trip_id=sncf_tgv_OCESN6077F1199309:2022-07-21T00:35:46Z, start_date=2022-07-21 00:00} with at date 2022-07-21 00:00

➜  schedule grep -r OCESN6077F1199309:2022-07-21T00:35:46Z sncf_tgv   
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,15:07:00,15:07:00,StopPoint:OCETGV INOUI-87686006,0,,0,1,
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,18:11:00,18:27:00,StopPoint:OCETGV INOUI-87751008,1,,0,0,
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,19:11:00,19:15:00,StopPoint:OCETGV INOUI-87755009,2,,0,0,
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,20:09:00,20:12:00,StopPoint:OCETGV INOUI-87757526,3,,0,0,
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,20:35:00,20:39:00,StopPoint:OCETGV INOUI-87757625,4,,0,0,
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,20:47:00,20:50:00,StopPoint:OCETGV INOUI-87757674,5,,0,0,
sncf_tgv/stop_times.txt:OCESN6077F1199309:2022-07-21T00:35:46Z,21:07:00,21:07:00,StopPoint:OCETGV INOUI-87756056,6,,1,0,
sncf_tgv/trips.txt:FR:Line::3C11D08B-6F1C-416C-A388-63B4074018CB:,000215,OCESN6077F1199309:2022-07-21T00:35:46Z,6077,1,723,
➜  schedule grep 000215 sncf_tgv/calendar.txt sncf_tgv/calendar_dates.txt
sncf_tgv/calendar.txt:000215,0,0,0,0,0,0,0,20220721,20220820
sncf_tgv/calendar_dates.txt:000215,20220722,1
sncf_tgv/calendar_dates.txt:000215,20220724,1
sncf_tgv/calendar_dates.txt:000215,20220725,1
sncf_tgv/calendar_dates.txt:000215,20220729,1
sncf_tgv/calendar_dates.txt:000215,20220731,1
sncf_tgv/calendar_dates.txt:000215,20220801,1
sncf_tgv/calendar_dates.txt:000215,20220802,1
sncf_tgv/calendar_dates.txt:000215,20220803,1
sncf_tgv/calendar_dates.txt:000215,20220804,1
sncf_tgv/calendar_dates.txt:000215,20220805,1
sncf_tgv/calendar_dates.txt:000215,20220807,1
sncf_tgv/calendar_dates.txt:000215,20220808,1
sncf_tgv/calendar_dates.txt:000215,20220809,1
sncf_tgv/calendar_dates.txt:000215,20220810,1
sncf_tgv/calendar_dates.txt:000215,20220811,1
sncf_tgv/calendar_dates.txt:000215,20220812,1
sncf_tgv/calendar_dates.txt:000215,20220814,1
sncf_tgv/calendar_dates.txt:000215,20220815,1
sncf_tgv/calendar_dates.txt:000215,20220816,1
sncf_tgv/calendar_dates.txt:000215,20220817,1
sncf_tgv/calendar_dates.txt:000215,20220818,1
sncf_tgv/calendar_dates.txt:000215,20220819,1

This one has more days where it operates, however not the 21st of July (today). So from looking only very quickly into it, it looks like there are messages for non-existent trips.

However, the map shows that there are delays:

I'll try to dig deeper what's happening with these trips and look at the protocol buffers message directly. Maybe there's an error in MOTIS in handling/finding the trips. Any help is much appreciated. Maybe I'm misinterpreting the specifications? Or am I missing something else?

[felixguendling]

Update: looked into the message. This is the update message for the first example of my previous post:

[erro][2022-07-21T12:38:52Z+0000][gtsfrt_parser.cc:99] Exception on entity OCESN9894F876490:2022-07-20T00:33:34Z for message with timestamp 1658407104: unable to find GTFS trip {GTFS-TRIP trip_id=sncf_tgv_OCESN9894F876490:2022-07-20T00:33:34Z, start_date=2022-07-21 00:00}, message=id: "OCESN9894F876490:2022-07-20T00:33:34Z"
trip_update {
  trip {
    trip_id: "OCESN9894F876490:2022-07-20T00:33:34Z"
    start_time: "15:12:00"
    start_date: "20220721"
  }
  stop_time_update {
    departure {
      delay: 0
      time: 1658409120
    }
    stop_id: "StopPoint:OCETGV-87212027"
  }
  stop_time_update {
    arrival {
      delay: 0
      time: 1658411220
    }
    departure {
      delay: 0
      time: 1658411400
    }
    stop_id: "StopPoint:OCETGV-87142109"
  }
  stop_time_update {
    arrival {
      delay: 0
      time: 1658413620
    }
    departure {
      delay: 0
      time: 1658413800
    }
    stop_id: "StopPoint:OCETGV-87171926"
  }
  stop_time_update {
    arrival {
      delay: 0
      time: 1658415660
    }
    departure {
      delay: 0
      time: 1658416020
    }
    stop_id: "StopPoint:OCETGV-87271494"
  }
  stop_time_update {
    arrival {
      delay: 0
      time: 1658419020
    }
    departure {
      delay: 0
      time: 1658419680
    }
    stop_id: "StopPoint:OCETGV-87223263"
  }
  stop_time_update {
    arrival {
      delay: 0
      time: 1658421780
    }
    stop_id: "StopPoint:OCETGV-88140010"
  }
  timestamp: 1658405515
}

From my understanding, this trip is not contained in the static timetable data:

➜  schedule grep -r "OCESN9894F876490:2022-07-20T00:33:34Z" sncf_tgv
➜  schedule grep 000040 sncf_tgv/calendar.txt sncf_tgv/calendar_dates.txt 
sncf_tgv/calendar.txt:000040,0,0,0,0,0,0,0,20220721,20220820 # doesn't operate at all
sncf_tgv/calendar_dates.txt:000040,20220815,1 # exception for 15th of August where it operates

So I would assume that at least in this example, there is a data error on the data provider side?

[kalon33]

It seems so, indeed. Static data is refreshed once a day at night, and we use the latest available. I now have delays displayed too. That's nice!
Would it be possible to have some logo or message in the vehicle or trip details indicating that we have RT information for it, but it is on time (and maybe a different color on the livemap when no delay information is available, like grey or black) ? Currently I couldn't find any difference when we have RT info and vehicle/trip is on time, and when we don't have one. Thanks again!

[kalon33]

Here are the informations I have under 0.8.2:
[erro][2022-07-21T14:58:33Z+0000][ris.cc:320] input source "url: https://proxy.transport.data.gouv.fr/resource/sncf-ter-gtfs-rt-trip-updates, auth: none": Illegal byte sequence

[felixguendling]

Thank you for your input!

v0.8.3 is building currently: https://github.com/motis-project/motis/releases/tag/v0.8.3

It colors the trains without delay infos in grey.

Illegal byte sequence

Should also be fixed. Can you try again please?
Thank you! :-)

[kalon33]

Nice! That part is working well! Thanks a lot for your reactivity and efficiency! RT feeds are parsed and every problem in them is very well explained now. About the livemap, it seems that grey doesn't bring a lot of contrast with the tracks/roads color. Maybe a deep blue would be better if it's not used elsewhere (Sorry for pointing towards grey in the first place)? Would it also be possible to have some way to discard an itinerary once we calculated it? Currently I can't get rid of it, even by reloading the interface. Then, could you add some possibility to move the map with the mouse instead of using the keyboard? Would it also be possible for the android app to configure an alternative server to use for custom deployments? And last but not least, since I got MOTIS working, even without RT data, I have some memory use problems.I don't know if it's linked but when I open the web interface on Firefox on MacOS or on Android (maybe Chrome too?), Memory use rises after some moves on the map (zoom, etc) to use more that the double it uses before, from 8Gb to >16Gb (maybe even more, but my server memory is then full) in my configuration.I got these kind of messages in the server console, before it got killed either by me or the OOM killer: ``` [warn][2022-07-21T20:10:03Z+0000][dispatcher.cc:121] target not found: /gbfs/info [net::web_server] write: Broken pipe [net::web_server] write: Broken pipe [net::web_server] write: Broken pipe [net::web_server] write: Broken pipe [net::web_server] write: Bad file descriptor ``` It doesn't seem to happen when I'm browsing the interface with Firefox under Linux (Ubuntu 22.04LTS), but maybe I didn't try enough to get it triggered. Do you know what could be the problem? Is there some more information I could collect to help fixing this? Again, thanks a lot for your help.

[kalon33]

I'm not exactly sure what you mean? The search fields (from, to, time, forward/backward) are seeded from local storage. I would consider this useful.

I mean that once I made an itinerary research, I can't get rid of it, it's still displayed on the map, even if I reload the web interface. There's no "reset" button to clear it which would also make it disappear from the map.

I tried to reproduce but it didn't happen. Do you mean the memory usage of the MOTIS process on the server or of the web browser process on the client?

I mean the process on the server. I don't know why but it seems that in some situations, memory use rises a lot.

This just means that a client disconnected without properly closing the connection before. Maybe it's a performance optimization on modern web browsers. At least nothing to be worried about. Maybe we should just not log it.
OK, so same thing for [warn][2022-07-21T20:10:03Z+0000][dispatcher.cc:121] target not found: /gbfs/info and [net::web_server] write: Bad file descriptor?

This morning I hit another problem, it seems that with the current version, the web interface doesn't display properly with Firefox 102.0.1 or Chromium 103.0.5060.53 (Ubuntu 22.04LTS). See
this screencast where I first load the URL, then hit Ctrl+R to reload without cache, and then Ctrl+R again with developer console opened.

Thanks for your help, and sorry for asking so much things...

[felixguendling]

I don't know why but it seems that in some situations, memory use rises a lot.

Can you create a screencast with what you do in the UI vs htop with MOTIS selected so we can see what actions in the UI causes the memory usage to peek? At the moment I cannot reproduce this locally and also not with our demo server.

but it seems that in some situations, memory use rises a lot

Complex searches could potentially trigger the memory usage to go up at the moment. But with your datasets I couldn't find any query complex enough to trigger significant memory usage.

again with developer console opened

The networking tab that you showed looks good. Can you see anything in the "console" tab that looks suspicious?

[felixguendling]

I found a bug that loaded trips with block_id set twice (resulting in only one version being updated in the with GTFS-RT and then it sometimes showed the one without updates on the map, sometimes the one with updates). This is fixed in v0.8.4 which was just released.

target not found: /gbfs/info

This is triggered because you have the GBFS module not enabled. The UI asks the server for GBFS service providers. But it's not a problem if the response is "not found".

[kalon33]

Can you create a screencast with what you do in the UI vs htop with MOTIS selected so we can see what actions in the UI causes the memory usage to peek? At the moment I cannot reproduce this locally and also not with our demo server.

Sure I will once I found a pattern to reproduce it. But as the web interface is not loading properly, I can't for the moment.

The networking tab that you showed looks good. Can you see anything in the "console" tab that looks suspicious?

There is an error it seems.

I found a bug that loaded trips with block_id set twice (resulting in only one version being updated in the with GTFS-RT and then it sometimes showed the one without updates on the map, sometimes the one with updates). This is fixed in v0.8.4 which was just released.

Thanks for the fix and for the explanations!

[felixguendling]

There is an error it seems.

Seems like the server didn't respond with a railviz config which is a bit strange if you have the railviz module enabled. But even if the railviz module would be disabled (then this error would be expected), this should be fine.

We're using WebGL to render the trains. It could be that you're using an unstable graphics driver which crashes. Maybe Chrome or Chromium have more meaningful logs regarding this? Otherwise, I have no clue what's going wrong since I cannot reproduce the issue locally and your logs don't contain any relevant information.

[kalon33]

Ok I think I found what is happening. I started using a pretty big GTFS-RT feed generated for my "internal" use from the SIRI-Lite data provided by the Ile de France transportation authority (IDFM). RT data flushing step now takes about 70/80 seconds (for about 10000 GTFS-RT updates only for this feed) and user web interface seems to be unresponsive until this data is flushed or used (even if I dedicate 3 or 4 threads to MOTIS). Is there some way to enhance parsing or make MOTIS able to reply even if RT data is processing? If useful, I'm ready to provide the big GTFS-RT feed privately for tests, but I'm not sure I can share this modified data publicly because of licensing terms. Thanks for your help.

[felixguendling]

• I think I found the problem that caused MOTIS to use up all the memory when routing on your dataset.
• Also I improved the performance of the real-time updates.
• The UI resets the found connections when you clear one input field (from/to) and exit the input field

Would be great if you could checkout the new v0.8.5 release!
https://github.com/motis-project/motis/releases/tag/v0.8.5

[kalon33]

Dear Felix, thanks a lot, will test for some time but at least the map seems to be displayed instantly right now. The UI resets are also very useful, thanks!

But it seems that french translation doesn't work anymore when I pass ?lang=fr to the URL. Is that expected?

Also, it seems that trips matching using SNCF feeds doesn't work (I have the same feeds in OpenTripPlanner an matching is working as expected). Could you have a look?

Thanks for your work and for spending time on my feedbacks :)

[felixguendling]

But it seems that french translation doesn't work anymore when I pass ?lang=fr to the URL. Is that expected?

Was a mistake when working on the UI. Thank you for your hint!
It's fixed in v0.8.6

Could you have a look?

Yes, I'm looking into it.

[felixguendling]

sncf_ter: parsed 2968 GTFS-RT updates: 1351/2968 (45.5189%) successful
sncf_intercites: parsed 121 GTFS-RT updates: 45/121 (37.1901%) successful
sncf_tgv: parsed 373 GTFS-RT updates: 217/373 (58.1769%) successful
caee: parsed 6 GTFS-RT updates: 6/6 (100%) successful
bordeauxaero: parsed 0 GTFS-RT updates: 0/0 (-nan%) successful

To be sure: Is this also roughly what you are seeing?
I updated the timetables. Maybe you need to rm -rf data/schedule/* to force the schedule.raw files to be recreated.

I would guess that most of the mismatches are caused by trains getting real-time updates that are disabled for this traffic day in the GTFS-static schedule timetable data. Do you see other cases?

Map looks like maybe 40% of them have real-time information.

[felixguendling]

By the way, if you're not using only routing as intermoda.router option (which is the default) you can also set dataset.expand_footpaths=false. This should speed up the preprocessing when loading the timetable data.

[kalon33]

Done a few seconds ago. Thanks!