v9.8.3296 How do you log intermediate progress?

[jawbroken]

In v9.7.2996 I used to set log_frequency_in_seconds to 10 on CP-SAT to get an ongoing log of solver progress. This was removed in v9.8.3296. I've tried setting log_subsolver_statistics to true but I'm not sure it currently does anything (or at least GitHub doesn't show me any references to this field when I search the repository). I'm using the Python library, if that helps.

Edit: I do have log_search_progress set to True, to be clear.

[lperron]

what do you want to log ? The log is still there. log_frequency_in_seconds is just the frequency of some message (the # Model ones) .
log_subsolver_statistics prints subsolver specific info at the end.

[jawbroken]

log_frequency_in_seconds was removed in 9.8 (in this commit). It used to give periodic updates of internal progress e.g.

#Model 375.91s var:4490/4836 constraints:10043/10730
#Probe 382.27s #iterations:1 #literals fixed/probed:137/284 #bounds shaved/tried:0/260 #new_integer_bounds:293, #new_binary_clauses:3461
#TreeS 386.45s #nodes:575/909 #rc:66 #decisions:40228 #@root:19 #restarts:10
#Probe 393.96s #iterations:1 #literals fixed/probed:137/296 #bounds shaved/tried:0/272 #new_integer_bounds:293, #new_binary_clauses:3497
#TreeS 400.36s #nodes:581/917 #rc:66 #decisions:41019 #@root:21 #restarts:10
#Probe 405.46s #iterations:1 #literals fixed/probed:137/304 #bounds shaved/tried:0/280 #new_integer_bounds:293, #new_binary_clauses:3511
#TreeS 411.34s #nodes:906/1335 #rc:66 #decisions:43253 #@root:21 #restarts:10

but now I only get a message when there's a new solution and some irregular #Model and #Bound messages, so it's hard for me to tell if anything is happening. e.g.

#Model 285.14s var:4377/4730 constraints:9845/10550
#Model 289.17s var:4376/4730 constraints:9843/10550
#54    2893.44s best:3005028 next:[3005023,3005027] graph_var_lns (d=0.35 s=135047 t=0.50 p=0.50 stall=10 h=folio_rnd)
#Model 3278.89s var:4372/4730 constraints:9836/10550
#Model 3350.18s var:4369/4730 constraints:9830/10550

where I didn't get a message for over 40 minutes.

[lperron]

Yes, we have hidden these message beneath a vmodule (in c++ --vmodule=lb_tree_search=1) I would argue these messages are not informative, as no progress is really made. Laurent Perron | Operations Research | ***@***.*** | (33) 1 42 68 53 00 Le jeu. 16 nov. 2023 à 16:36, jawbroken ***@***.***> a écrit :

…

log_frequency_in_seconds was removed in 9.8 (in this commit <b8f224a>). It used to give periodic updates of internal progress e.g. #Model 375.91s var:4490/4836 constraints:10043/10730 #Probe 382.27s #iterations:1 #literals fixed/probed:137/284 #bounds shaved/tried:0/260 #new_integer_bounds:293, #new_binary_clauses:3461 #TreeS 386.45s #nodes:575/909 #rc:66 #decisions:40228 ***@***.***:19 #restarts:10 #Probe 393.96s #iterations:1 #literals fixed/probed:137/296 #bounds shaved/tried:0/272 #new_integer_bounds:293, #new_binary_clauses:3497 #TreeS 400.36s #nodes:581/917 #rc:66 #decisions:41019 ***@***.***:21 #restarts:10 #Probe 405.46s #iterations:1 #literals fixed/probed:137/304 #bounds shaved/tried:0/280 #new_integer_bounds:293, #new_binary_clauses:3511 #TreeS 411.34s #nodes:906/1335 #rc:66 #decisions:43253 ***@***.***:21 #restarts:10 but now I only get a message when there's a new solution and some irregular #Model and #Bound messages, so it's hard for me to tell if anything is happening. e.g. #Model 285.14s var:4377/4730 constraints:9845/10550 #Model 289.17s var:4376/4730 constraints:9843/10550 #54 2893.44s best:3005028 next:[3005023,3005027] graph_var_lns (d=0.35 s=135047 t=0.50 p=0.50 stall=10 h=folio_rnd) #Model 3278.89s var:4372/4730 constraints:9836/10550 #Model 3350.18s var:4369/4730 constraints:9830/10550 where I didn't get a message for over 40 minutes. — Reply to this email directly, view it on GitHub <#3989 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACUPL3LV6TB2UQVIRM3DXVDYEYXIPAVCNFSM6AAAAAA7OIY4ECVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TKOBZGU4TI> . You are receiving this because you commented.Message ID: ***@***.***>

[jawbroken]

Thanks for the responses. Is there any way to turn this on from the Python side? At least for my problem, the #Probe messages seem to be a reasonable indicator as to whether any progress was being made or if it was going to stall effectively forever.

[jawbroken]

According to https://github.com/google/glog setting the GLOG_vmodule environment variable to something like lb_tree_search=1,integer_search=1 should work, but I couldn't get it to. I'm not sure if that is reaching the or-tools module.

Edit: GLOG_v=2 doesn't seem to enable verbose logging either.

[lperron]

We dropped glog years ago. Le jeu. 16 nov. 2023, 17:34, jawbroken ***@***.***> a écrit :

…

According to https://github.com/google/glog setting the GLOG_vmodule environmental variable to something like lb_tree_search=1,integer_search=1 should work, but I couldn't get it to. I'm not sure if that is reaching the or-tools module. — Reply to this email directly, view it on GitHub <#3989 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACUPL3KAHMJONPP44RPNR33YEY57TAVCNFSM6AAAAAA7OIY4ECVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TKOJQGIYTE> . You are receiving this because you commented.Message ID: ***@***.***>

[jawbroken]

Ah, makes sense. Not sure you're reading any environmental variables then, from scanning the code. Might be impossible.

[lperron]

Is this another chatgpt nonsense? Worth a ban IMO. Le sam. 25 nov. 2023, 04:00, zzyyyy12138 ***@***.***> a écrit :

…

Based on the information you provide, you want to get a continuous log in the CP-SAT solver to see the solver progress. In earlier versions (v9.7.2996), you could achieve this by setting log_frequency_in_seconds to 10, but in later versions (v9.8.3296), this setting has been removed. You have tried setting log_subsolver_statistics to true, but are not sure if it works. In the v9.8 release, the log_subsolver_statistics you mentioned should be a valid option, but it might provide information in a different way. Typically, these logging options provide detailed information about how the solver performs the search, but may be presented in a different form or granularity. If you find that log_subsolver_statistics does not provide the information you need, you may want to review the available logging options based on the documentation for your specific version of the CPLEX or Gurobi Python libraries. Sometimes, these options may have changed or been added in a new version. In CPLEX and Gurobi's Python libraries, there are usually ways to get a detailed log of the solver. This may include intermediate solutions during the solution process, constraint addition/removal, or other search events. For more specific help, you can consult the relevant documentation or sample code to see what logging options are available in the particular version you are using. Please note that specific logging options and features may change depending on library versions and updates. Therefore, checking the latest documentation and sample code is a good way to get more accurate information. — Reply to this email directly, view it on GitHub <#3989 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACUPL3LHBKTBWLXV7KJMTM3YGFNMPAVCNFSM6AAAAAA7OIY4ECVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMNRUGUZTC> . You are receiving this because you commented.Message ID: ***@***.***>