Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement timeout for slow functions #3351

Open
mdouze opened this issue Apr 5, 2024 · 2 comments
Open

Implement timeout for slow functions #3351

mdouze opened this issue Apr 5, 2024 · 2 comments

Comments

@mdouze
Copy link
Contributor

mdouze commented Apr 5, 2024

In Faiss, slow functions periodically call InterruptCallback::check()

https://github.com/facebookresearch/faiss/blob/main/faiss/impl/AuxIndexStructures.h#L152

The callback is arbitrary. It does nothing by default.
In python there is an InterruptCallback that checks if the user pressed Ctrl-C.

It would be useful to implement an InterruptCallback that breaks out of a function that lasts too long, eg. a clustering.

@mdouze
Copy link
Contributor Author

mdouze commented Apr 11, 2024

  1. C++ only first
  2. support timeout from python

@mdouze
Copy link
Contributor Author

mdouze commented Apr 25, 2024

For the C++ version it can be done with just one additional test.
It requires subclassing InterruptCallback to implement want_interrupt to return true if more than x seconds elapsed since the counter was reset.

Something like:

struct TimeoutCallback: InterruptCallback {
    double t0 = 0; 
    double timeout = 0; 
    void set_timeout(double t) {
        t0 = get_time(); 
        timeout = t; 
    } 
   
   bool want_interrupt () override {
      if (timeout == 0) {
         return false; 
      }
      if (get_time() - t0 > timeout) {
         timeout = 0;  
         return true; 
      } 
     return false;
   }

};

can be tested by running a very long k-means clustering (see Clustering object).

asadoughi pushed a commit that referenced this issue Apr 29, 2024
Summary: #3351

Differential Revision: D56732720
asadoughi pushed a commit that referenced this issue Apr 30, 2024
Summary: #3351

Differential Revision: D56732720
asadoughi pushed a commit that referenced this issue May 1, 2024
Summary: #3351

Reviewed By: junjieqi

Differential Revision: D56732720
asadoughi added a commit to asadoughi/faiss that referenced this issue May 3, 2024
Summary: facebookresearch#3351

Differential Revision: D56732720

Reviewed By: junjieqi
asadoughi pushed a commit to asadoughi/faiss that referenced this issue May 3, 2024
Summary: facebookresearch#3351

Differential Revision: D56856744
asadoughi added a commit to asadoughi/faiss that referenced this issue May 3, 2024
Summary: facebookresearch#3351

Differential Revision: D56732720

Reviewed By: junjieqi
asadoughi pushed a commit to asadoughi/faiss that referenced this issue May 3, 2024
Summary:
Pull Request resolved: facebookresearch#3413

facebookresearch#3351

Differential Revision: D56856744
asadoughi pushed a commit that referenced this issue May 6, 2024
Summary:

#3351

Reviewed By: junjieqi

Differential Revision: D56732720
facebook-github-bot pushed a commit that referenced this issue May 6, 2024
Summary:

#3351

Reviewed By: junjieqi

Differential Revision: D56732720
facebook-github-bot pushed a commit that referenced this issue May 8, 2024
Summary: #3351

Differential Revision: D57120422
facebook-github-bot pushed a commit that referenced this issue May 9, 2024
Summary:
Pull Request resolved: #3417

#3351

Reviewed By: junjieqi

Differential Revision: D57120422

fbshipit-source-id: e2e446642e7be8647f5115f90916fad242e31286
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant