Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework storage to support fixed storage size and localized storage #160

Open
jrmadsen opened this issue Mar 3, 2021 · 0 comments
Open

Comments

@jrmadsen
Copy link
Collaborator

jrmadsen commented Mar 3, 2021

  • currently tim::storage<T> implements tim::impl::storage<T, bool> where the bool is whether the component collects data or not
  • tim::impl::storage<T, bool> should be updated to:
    • tim::impl::storage<T, true> should be renamed to tim::impl::dynamic_value_storage<T>
    • tim::impl::storage<T, false> should be renamed to tim::impl::void_storage<T>
    • A new storage type should be introduced with a fixed storage size using a ring-buffer, i.e. tim::impl::fixed_value_storage<T>
      • This new storage type will enable more complex usage of timemory component during sampling since memory must be pre-allocated within a signal handler
      • For components which support sampling, may actually want tim::storage<T> to hold both a fixed_value_storage instance and a dynamic_value_storage instance.
  • Singleton stuff w.r.t. storage should be migrated into tim::storage<T> instead of being held by the impl instance
    • This will help enable storage to be localized (see below)

Localized Storage

Currently, there is no way to do call-graph tracking locally, i.e. all call-graph tracking updates the same singleton. Therefore, it becomes a bit convoluted to do things like what happens when the compiler instrumentation is applied to code instrumented with timemory, i.e. you have two separate storage instances: one from compiler instrumentation and one from the manual instrumentation. A new storage_manager (non-templated) should be created which handles the storage instances of components. The default implementation should essentially provide the global "singletons". Here is a theoretical mock-up:

namespace tim
{
class storage_manger
{
public:
    storage_manager() = default;
    storage_manager(bool _use_global) : m_use_global{ _use_global } {}

    template <typename Tp>
    auto get_storage_instance()
    {
          using storage_type = tim::storage<Tp>;

          // return global instance
          if(m_use_global) return storage_type::instance();

          // return local instance
          auto& _val = m_storage_map[std::type_index{ typeid(Tp) }];
          if(!_val) _val = std::make_unique<storage_type>();
          return dynamic_cast<storage_type*>(_val.get());
    };

    void finalize(const std::string& _data_prefix)
    {
        if(m_use_global) return;

        auto _save = tim::settings::output_prefix(); // this isn't ideal but currently how it's done
        tim::settings::output_prefix() = _prefix;
        m_storage_map.clear();  // will trigger output
        tim::settings::output_prefix() = _save;
    };

private:
    bool m_use_global = false;
    std::unordered_map<std::type_index, std::unique_ptr<tim::base::storage>> m_storage_map;
};

namespace quirk
{
// template parameter to bundler which will require passing storage_manager to constructor
struct localized_storage : concepts::quirk_type
{};
}
}

namespace quirk = tim::quirk;

int main()
{
    tim::storage_manager _local_storage{};

    using local_bundle_t = tim::component_tuple<wall_clock, quirk::localized_storage>;

    // local_bundle_t _bundle{ "main" }; // Error!
    local_bundle_t _bundle{ "main", &_local_storage }; // Correct

    // local_bundle_t _bundle{ "main", quirk::config<quirk::localized_storage>{} }; // Error!
    local_bundle_t _bundle{ "main", &_local_storage, quirk::config<quirk::localized_storage>{} }; // Correct

    // ... etc. ...

   // generate output for 
   _local_storage.finalize("mydata");

   tim::timemory_finalize();
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant