Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

在官方最新源码基础上更改持久化存储对象为反向索引与评分字段 #35

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open

在官方最新源码基础上更改持久化存储对象为反向索引与评分字段 #35

wants to merge 23 commits into from

Commits on Oct 15, 2015

  1. 更新 

    DocId类型由uint64改为string
    @andeya
    andeya committed Oct 15, 2015
    Configuration menu
    Copy the full SHA
    5c44b93 View commit details
    Browse the repository at this point in the history

  2. 更新

    @andeya
    andeya committed Oct 15, 2015
    Configuration menu
    Copy the full SHA
    a676862 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2015

  1. 支持传出关键词分词及词频信息

    @andeya
    andeya committed Oct 18, 2015
    Configuration menu
    Copy the full SHA
    5a5b56a View commit details
    Browse the repository at this point in the history

  2. 支持传出关键词分词及词频信息

    @andeya
    andeya committed Oct 18, 2015
    Configuration menu
    Copy the full SHA
    681ca3b View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2015

  1. 全面更新 

    1. 持久存储的内容改为索引信息，从而加快重载速度；
2. 支持指定shard范围进行搜索；
3. 支持动态增加指定的shard工作协程；
    @andeya
    andeya committed Oct 22, 2015
    Configuration menu
    Copy the full SHA
    cf3b884 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2015

  1. shard类型改为uint64，便于指定任意值

    @andeya
    andeya committed Oct 23, 2015
    Configuration menu
    Copy the full SHA
    7967e5f View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2015

  1. 支持初始化时shards为空

    @andeya
    andeya committed Oct 30, 2015
    Configuration menu
    Copy the full SHA
    d961c60 View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2015

  1. 解决一些小bug

    @andeya
    andeya committed Nov 5, 2015
    Configuration menu
    Copy the full SHA
    4fffc3e View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2015

  1. 优化协程

    @andeya
    andeya committed Nov 6, 2015
    Configuration menu
    Copy the full SHA
    66d9a50 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2015

  1. 支持docker，添加bolt数据库存储

    @andeya
    andeya committed Nov 7, 2015
    Configuration menu
    Copy the full SHA
    650cfb9 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2015

  1. 添加单独分词方法Segment()(keywords []string)

    @andeya
    andeya committed Nov 9, 2015
    Configuration menu
    Copy the full SHA
    e5555be View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2015

  1. 添加单独分词方法Segment(content string)(keywords []string)

    @andeya
    andeya committed Nov 11, 2015
    Configuration menu
    Copy the full SHA
    1adda94 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2015

  1. 通道返回分词文档时，可以在外部程序读取全部数据后，任意写入与容量相等的数据量用以终止本批索引任务。

    @andeya
    andeya committed Nov 13, 2015
    Configuration menu
    Copy the full SHA
    defcc91 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2015

  1. 更新 

    1. 修复嵌入式获取分词信息的bug
2. 与官方包同步
    @andeya
    andeya committed Nov 24, 2015
    Configuration menu
    Copy the full SHA
    d513e5b View commit details
    Browse the repository at this point in the history

  2. 更新 

    1. 修复嵌入式获取分词信息的bug
2. 与官方包同步
    @andeya
    andeya committed Nov 24, 2015
    Configuration menu
    Copy the full SHA
    6994091 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2015

  1. 在官方最新源码基础上升级 

    1. 持久化存储对象，从原始文档改为反向索引文档与文档评分字段，从而避免程序重启后，需要重新分词、索引的麻烦；
2. 将持久数据库分片数与索引器、排序器的分片数保持一致，即实现一一对应关系，从而保证从数据库可以完美恢复；
3.依然存在的问题：悟空采用的这两中KV数据库，读写速率太慢，严重拖累高并发的特性。
    @andeya
    andeya committed Nov 28, 2015
    Configuration menu
    Copy the full SHA
    bd815fc View commit details
    Browse the repository at this point in the history

  2. 基于官方最新源码的较大改进： 

    1. 持久化存储对象，从原始文档改为反向索引文档与文档评分字段，从而避免程序重启后，需要重新分词、索引的麻烦；
2. 将持久数据库分片数与索引器、排序器的分片数保持一致，即实现一一对应关系，从而保证从数据库可以完美恢复；
3. 依然存在的问题：悟空采用的这两种KV数据库，读写速率太慢，严重拖累高并发的特性。
    @andeya
    andeya committed Nov 28, 2015
    Configuration menu
    Copy the full SHA
    8c1ea66 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2015

  1. Create ldb_storage.go 

    增加Goleveldb存储支持
    @insionng
    insionng committed Nov 29, 2015
    Configuration menu
    Copy the full SHA
    1402175 View commit details
    Browse the repository at this point in the history

  2. Update storage.go 

    增加Goleveldb存储支持
    @insionng
    insionng committed Nov 29, 2015
    Configuration menu
    Copy the full SHA
    96acd18 View commit details
    Browse the repository at this point in the history

  3. Update ldb_storage.go 

    增加Goleveldb存储支持（修正）
    @insionng
    insionng committed Nov 29, 2015
    Configuration menu
    Copy the full SHA
    7d8e053 View commit details
    Browse the repository at this point in the history

  4. Update storage.go 

    增加Goleveldb支持
    @insionng
    insionng committed Nov 29, 2015
    Configuration menu
    Copy the full SHA
    c150cc9 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #2 from insionng/patch-2 

    Update storage.go
    @andeya
    andeya committed Nov 29, 2015
    Configuration menu
    Copy the full SHA
    bd5be56 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #1 from insionng/patch-1 

    Create ldb_storage.go
    @andeya
    andeya committed Nov 29, 2015
    Configuration menu
    Copy the full SHA
    d12b490 View commit details
    Browse the repository at this point in the history