Skip to content

DIMAthe47/cbir_framework

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

29 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

cbir_framework

Framework represents CBIR as several steps:

  • Transformation (computing descriptors, any array manipultaions)
  • Sampling (frequent step before quantization)
  • Quantization
  • Search
  • Search evaluation, plotting

All steps (except plotting) are designed in flavour of pipeline: each step has input data store and output data store and in transformation step you can even imitate real pipeline chaining transformers.

To add new type of descriptor you need add your <transformer.py> which must implement method like 'transform_item', then you can easily pass it to transformation step and compute descriptors of new type you need.

Similar situtaion with data stores. There are already implementations like SQLiteDataStore, CSVFileDataStore, NumpyDataStore. You can add your <data_store.py> which must implement several methods like 'get_items_sorted_by_ids' and then pass it to any step in your cbir pipeline.

(Note that there is considerable mess with way of getting data, processing and saving it. There were intentions to process data in stream-like style, but its has led to unpleasant restrictions and time-perfomance issues.)

Framework depends on python modules in inverted_multi_index prjoect. It utilizes them for fast (I hope) vector operations, building inverted multi-index, perfoming inverted multi-index search, exhaustive search with SDC and ADC distance computations.

Here goes major steps and examples of them.

Descriptors computation

Quantization

Sampling

It`s often enough to quantize only sample from descriptors.

Example (sampling sifts)->.

Search

4 types of search are supported:

Exhaustive search

requires:

SDC exhaustive search

requires:

ADC exhaustive search

requires:

Inverted multi-index search

requires:

Evaluation

Step to evaluate search perfomance.

Example.

Plotting

  • compare descriptors for exhaustive search Example->
  • compare memory for descriptors for exhaustive search Example->
  • compare quantization parameters for pq search techniques(adc, sdc, imi) Example->
  • compare pq search types(adc, sdc, imi) Example->

UI

For UI of finding images by example, client-server architecture was implemented. Server needs to be configurated with one of "searcher" (like in examples), then he listens to requests on localhost. Client with GUI connects to server on startup, can choose images in filesystem, and then search for most similar images. (Here file pathes are transmitted to server, because all communication is on one machine, but its not difficult to change it to send/receive image bytes). When server recieves request for search, it searches using searcher it was configurated and send back to client file pathes to nearest images.

gui screenshot

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published