Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search Index Refactoring #3556

Draft
wants to merge 104 commits into
base: master
Choose a base branch
from
Draft

Search Index Refactoring #3556

wants to merge 104 commits into from

Commits on Dec 16, 2019

  1. coding style

    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    ddc452a View commit details
    Browse the repository at this point in the history
  2. encapsulate idx_cleanName()

    idx_cleanName() was called only from Doku_Indexer::addMetaKeys(), lookupKey(), getPages(), histogram()
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    0fb77e9 View commit details
    Browse the repository at this point in the history
  3. deprecated idx_indexLengths()

    This function is not called from elsewhere.
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    1dad69b View commit details
    Browse the repository at this point in the history
  4. encapsulate idx_listIndexLengths()

    Note:  idx_listIndexLengths() is used in inc/infoutils.php file
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    4316123 View commit details
    Browse the repository at this point in the history
  5. deprecated idx_getIndex()

    Note: idx_getIndex() is used in inc/infoutils.php file
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    861eb32 View commit details
    Browse the repository at this point in the history
  6. deprecated idx_tokenizer()

    This function is not called from elsewhere.
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    e15020b View commit details
    Browse the repository at this point in the history
  7. deprecated idx_lookup()

    This function is not called from elsewhere.
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    abfaea2 View commit details
    Browse the repository at this point in the history
  8. encapsulate idx_addPage()

    Note: idx_listIndexLengths() is used in iinc/TaskRunner.php and inc/Remote/ApiCore.php file
    
    Also used in _test files.
    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    56d1fe9 View commit details
    Browse the repository at this point in the history
  9. encapsulate idx_get_version()

    ssahara committed Dec 16, 2019
    Configuration menu
    Copy the full SHA
    8c01949 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    0af7b62 View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2019

  1. Configuration menu
    Copy the full SHA
    8896568 View commit details
    Browse the repository at this point in the history
  2. make Doku_Indexer singleton

    ssahara committed Dec 17, 2019
    Configuration menu
    Copy the full SHA
    b5daf9f View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2019

  1. encapsulate functions into Classes

    class FulltextSearch
    class MetaSearch
    ssahara committed Dec 19, 2019
    Configuration menu
    Copy the full SHA
    cd17dbd View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2019

  1. coding style PSR-12

    ssahara committed Dec 20, 2019
    Configuration menu
    Copy the full SHA
    83198f9 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d43b19d View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2019

  1. new namespace dokuwiki\Search

    ssahara committed Dec 21, 2019
    Configuration menu
    Copy the full SHA
    173bfbc View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2019

  1. Configuration menu
    Copy the full SHA
    c31af4f View commit details
    Browse the repository at this point in the history
  2. make public listIndex() and listIndexLengths() method

    because used in inc/infoutils.php file
    ssahara committed Dec 27, 2019
    Configuration menu
    Copy the full SHA
    fe21229 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    89b6193 View commit details
    Browse the repository at this point in the history

Commits on Dec 28, 2019

  1. Configuration menu
    Copy the full SHA
    0a3e25f View commit details
    Browse the repository at this point in the history
  2. updateTuple() 2nd parameter

    updateTuple() accept int for the second argument
    ssahara committed Dec 28, 2019
    Configuration menu
    Copy the full SHA
    115f491 View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2019

  1. Revert "updateTuple() 2nd parameter"

    This reverts commit 115f491.
    ssahara committed Dec 29, 2019
    Configuration menu
    Copy the full SHA
    f9c5d30 View commit details
    Browse the repository at this point in the history

Commits on Dec 30, 2019

  1. Configuration menu
    Copy the full SHA
    48b9265 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2020

  1. create QueryParser class

    provides convert($query), recert_simple(), and termParser().
    No needs to pass $Indexer in method's arguments.
    ssahara committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    3837ea9 View commit details
    Browse the repository at this point in the history
  2. move Quicksearch methods into MetaSearch class

    pageLookup() does not use fulltext index, but metadata index
    ssahara committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    677f78a View commit details
    Browse the repository at this point in the history
  3. bug fix and PHP74 warnings

    Warning: Parameter 1 to dokuwiki\Search\MetaSearch::callback_pageLookup() expected to be a reference, value given in /path/to/dokuwiki/inc/Extension/Event.php on line 135
    ssahara committed Jan 5, 2020
    Configuration menu
    Copy the full SHA
    6b6beca View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    fe2d1da View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2020

  1. Abstraction Index classes

    PageIndex, PagewordIndex, MetadataIndex inherit the AbstractIndex class
    ssahara committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    f076e3f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    86fc728 View commit details
    Browse the repository at this point in the history
  3. make $pidCache static, refactor getPID()

    all extending acstract classes should use a static pidCache array
    ssahara committed Jan 7, 2020
    Configuration menu
    Copy the full SHA
    5aa57cb View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2020

  1. Configuration menu
    Copy the full SHA
    5d3ae62 View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2020

  1. Configuration menu
    Copy the full SHA
    872fcdc View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2020

  1. Configuration menu
    Copy the full SHA
    7dcabbd View commit details
    Browse the repository at this point in the history
  2. obsolete deletePageNoLock()

    ssahara committed Jan 13, 2020
    Configuration menu
    Copy the full SHA
    9dd6e5f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    185796b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    780c140 View commit details
    Browse the repository at this point in the history
  5. Undefined methods dispatcher

    prepared for unit tests compatibility only
    ssahara committed Jan 13, 2020
    Configuration menu
    Copy the full SHA
    489a7bf View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    839557a View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2020

  1. fix $errors static

    ssahara committed Jan 14, 2020
    Configuration menu
    Copy the full SHA
    b47a51d View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9ad37f3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3f4a342 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0cba610 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2020

  1. remove global FT_SNIPPET_NUMBER

    it is defined as namespace constant in Search/FulltextSearch.php file, and used in Remote/ApiCore.php and Ui/Search.php files.
    ssahara committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    a991224 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    efe7080 View commit details
    Browse the repository at this point in the history
  3. cast int for addIndexKey()

    Does it solve scrutinizer inspection?
    The expression ``$this->addIndexKey('page', '', $page);`` of type ``false|integer|string`` adds the type ``string`` to the return on line 73 which is incompatible with the return type documented by ``dokuwiki\Search\AbstractIndex::getPID`` of type ``integer|false``.
    ssahara committed Jan 15, 2020
    Configuration menu
    Copy the full SHA
    166288e View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    196d73d View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2020

  1. Configuration menu
    Copy the full SHA
    2cbd2b9 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2020

  1. Configuration menu
    Copy the full SHA
    46b8351 View commit details
    Browse the repository at this point in the history
  2. move lookup() to PagewordIndex class, reduce term 'Indexer'

    make similar MetadataIndex::lookupKey() and PagewordIndex::lookup()
    ssahara committed Jan 19, 2020
    Configuration menu
    Copy the full SHA
    be5c1ea View commit details
    Browse the repository at this point in the history
  3. rewrite test files

    replace idx_* and ft_* functions with relevant class metods
    ssahara committed Jan 19, 2020
    Configuration menu
    Copy the full SHA
    bb38a88 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8bff05a View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c562873 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2020

  1. separate Tokenizer class

    ssahara committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    094ebf2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d756b1c View commit details
    Browse the repository at this point in the history
  3. define getPages() in AbstractIndex instead of PageIndex

    getPages() is inherited to each subclass of AbstractIndex, but MetadataIndex::getPages() will override the inherited method.
    ssahara committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    02361d2 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2020

  1. Configuration menu
    Copy the full SHA
    39cb9fc View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2020

  1. refactor getPID()

    getPID() : mark deleted page entry  in page.idx
    getPIDNoLock() : obsoleted
    getPages() : exclude deleted pages
    ssahara committed Jan 29, 2020
    Configuration menu
    Copy the full SHA
    f36fd30 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2020

  1. Configuration menu
    Copy the full SHA
    9a025b5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4027a91 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4f93610 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    8fb944f View commit details
    Browse the repository at this point in the history
  5. fix typo

    ssahara committed Jan 30, 2020
    Configuration menu
    Copy the full SHA
    a2f3916 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2020

  1. Configuration menu
    Copy the full SHA
    e9d278b View commit details
    Browse the repository at this point in the history
  2. histogram() change args order

    INdexer::histogram() is only used in indexer_histogram.test.php file.
    ssahara committed Jan 31, 2020
    Configuration menu
    Copy the full SHA
    4d04b7b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    5237d40 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    743c9a2 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2020

  1. obsolete fulltext.php and indexer.php files

    define deprecated functions in deprecated.php file.
    fix load.php,  loading fulltext.php and indexer.php is nolonger necessary.
    ssahara committed Feb 1, 2020
    Configuration menu
    Copy the full SHA
    6f58ff6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    653b91a View commit details
    Browse the repository at this point in the history

Commits on Feb 2, 2020

  1. Configuration menu
    Copy the full SHA
    4a90f94 View commit details
    Browse the repository at this point in the history
  2. fix set Stopwords property

    ssahara committed Feb 2, 2020
    Configuration menu
    Copy the full SHA
    f2e1d0b View commit details
    Browse the repository at this point in the history
  3. fix getPID()

    getPID() will be called only when the page exists.
    ssahara committed Feb 2, 2020
    Configuration menu
    Copy the full SHA
    db87936 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    9329b00 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2020

  1. remove unnecessary comment

    ssahara committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    fd9f989 View commit details
    Browse the repository at this point in the history
  2. simplify dispatch()

    ssahara committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    11d2e7d View commit details
    Browse the repository at this point in the history
  3. idx_get_indexer() returns MetadataIndex

    According to the development manual, for [Metadata index](https://www.dokuwiki.org/devel:metadata#metadata_index),  indexer object supports `addMetaKeys()`, `lookupKey()`, and `getpages()`. This means that `idx_get_indexer()` should return  MetadataIndex object.
    ssahara committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    b4c31a2 View commit details
    Browse the repository at this point in the history
  4. Revert "histogram() change args order"

    This reverts commit 4d04b7b.
    ssahara committed Feb 20, 2020
    Configuration menu
    Copy the full SHA
    5f9bd52 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    51ddbad View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2020

  1. Configuration menu
    Copy the full SHA
    abb227b View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2020

  1. Configuration menu
    Copy the full SHA
    22c4cfd View commit details
    Browse the repository at this point in the history
  2. Follow up #2985, fperm seetting

    Fixed inconsistent handling of falsy values on fperm setting
    ssahara committed Mar 14, 2020
    Configuration menu
    Copy the full SHA
    39f31b6 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2020

  1. Configuration menu
    Copy the full SHA
    e36bcee View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2020

  1. Configuration menu
    Copy the full SHA
    9de2ceb View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2020

  1. make FulltextIndex::getIndexLengths() public

    Third-party plugins may use this method. The [cloud plugin](https://github.com/dokufreaks/plugin-cloud) uses idx_indexLength().
    ssahara committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    558f089 View commit details
    Browse the repository at this point in the history
  2. fix method name

    ssahara committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    d42a607 View commit details
    Browse the repository at this point in the history
  3. fix deprecated.php

    ssahara committed Jun 24, 2020
    Configuration menu
    Copy the full SHA
    22df765 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2020

  1. Merge branch 'master' into Refactor_Fulltext

    * master: (111 commits)
      Update translation
      translation update
      don't crush tables too narrow. fixes #3250
      translation update
      Thorough tests for EO, DE, PT and ES
      translation update
      Optimized pageRestoreConfirm function
      Tests for Portuguese and Spanish
      Changes according to revisions in moisesbr-dw#2
      adjust callstack depth for deprecation message further
      better deprecation messages for self required plugin base files
      don't test on old PHP releases anymore
      increase minimum PHP version to 7.2
      fixed tests for cleanID and romanization for Greeklish
      Improved the transliteration from greek to latin.
      extension cli: do not try to upgrade bundled plugins
      Public access to patterns in external link parser
      test the collator fallback always
      cleanup for collator tests
      wrap sorting functions into their own class
      ...
    splitbrain committed Sep 10, 2020
    Configuration menu
    Copy the full SHA
    b9c8f03 View commit details
    Browse the repository at this point in the history
  2. replace user errors with exceptions

    Exceptions are better to handle than errors. What I don't like is that
    we now have an unfortunate mix of return code and exception signalling
    for errors. Some methods still return false for errors while others
    now throw exceptions (always returning true otherwise).
    splitbrain committed Sep 10, 2020
    Configuration menu
    Copy the full SHA
    15f699a View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2020

  1. Configuration menu
    Copy the full SHA
    265e2c9 View commit details
    Browse the repository at this point in the history
  2. remove unnecessary if blocks

    getPID(), saveIndex(), saveIndexKey(), getPageWords() return always true, otherwise exceptions.
    ssahara committed Sep 21, 2020
    Configuration menu
    Copy the full SHA
    a16bd54 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2020

  1. avoid null in addMetaKeys()

    Just ignore $value argument if $key argument is array .
    Ignore enpty key of $key argument.
    Ensure to treat any null value of $key array as empty string.
    ssahara committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    89e3dd3 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2020

  1. change Index objects to non-singleton

    Indexer, FulltextIndex, MetadataIndex uses common directory to store *.idx files, but this does not mean they should be singleton objects to avoid lock confrictions.
    ssahara committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    a32da6d View commit details
    Browse the repository at this point in the history
  2. instantiate *Index with numeric page id

    will reduce access to static $pidCache
    ssahara committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    725e8e5 View commit details
    Browse the repository at this point in the history
  3. fix scrutinizer claims

    ssahara committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    5792814 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2020

  1. change Tokenizer static utility

    frequently used in ajax call, singleton is not effective to reduce multiple instantiations.
    ssahara committed Sep 26, 2020
    Configuration menu
    Copy the full SHA
    1755450 View commit details
    Browse the repository at this point in the history
  2. change MetadataSearch and FulltextSearch to non-singleton

    singleton is not effective to reduce multiple instantiations, especially for MetadataSearch which is frequently used in ajax call.
    ssahara committed Sep 26, 2020
    Configuration menu
    Copy the full SHA
    cc3a3cd View commit details
    Browse the repository at this point in the history
  3. dbglog() for SearchException

    ssahara committed Sep 26, 2020
    Configuration menu
    Copy the full SHA
    72ebc99 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2020

  1. Configuration menu
    Copy the full SHA
    4d69838 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2020

  1. Configuration menu
    Copy the full SHA
    9e7aeeb View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bcd7722 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2021

  1. Configuration menu
    Copy the full SHA
    c1803f3 View commit details
    Browse the repository at this point in the history
  2. fix undefined array in FulltextSearch

    this was already fixed by 5afd958 on 2021-02-05
    ssahara committed Nov 29, 2021
    Configuration menu
    Copy the full SHA
    05606ae View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    fab81cc View commit details
    Browse the repository at this point in the history
  4. catch up #3115 Sort with collator

    ssahara committed Nov 29, 2021
    Configuration menu
    Copy the full SHA
    a02395a View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    3df1553 View commit details
    Browse the repository at this point in the history