Converting data formats to Caliper #329
-
|
I'm working with the RAJA Perfsuite, which outputs CSV data. I know how I want to group the data, and want to write a .cali file for use with Spot. How would you recommend a person go about this? My current idea is to write a program that reads the CSV, and then a Caliper service to process snapshots in a way that simulates the timestamps I want (also recording globals). That seems like a bad idea, but I don't see many good ones, I don't want to maintain a cali file writer if I can avoid it. Any thoughts? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 4 replies
-
|
Correct, that's a bad idea and there aren't many good ones ... the one and only good option is to put Caliper into RAJAPerf 😄 If you absolutely want to you can of course make a program that creates a bunch of attributes and records and puts them in a CaliWriter, or extend the Python caliper-reader to also write files (that's a bit of work though). Spot has a few extra constraints, too, like globals being recorded with adiak, which adds more complexity. We're also working on the DB backend for Spot, so you could probably write into the DB directly and avoid going through Caliper. However, the scheme isn't finalized yet; probably some weeks/months away from being ready. And you'd still need to maintain a second "spot data creator". So overall I don't think there is a great solution for producing Spot data without Caliper right now. I don't know why we ended up using .cali files as basis for Spot instead of an open JSON spec or something, but oh well. |
Beta Was this translation helpful? Give feedback.
-
|
We used a JSON spec for Spot in V1, it proved to be a bit of a mess. We'll see where everybody lands on SPOT in the Perfsuite |
Beta Was this translation helpful? Give feedback.
-
|
What were the issues with the open JSON spec? I just created a SPOT instance at NERSC for Kokkos and I'd like to use it for other things but restricting it to Caliper exclusively isn't something I am inclined to do... 😉 |
Beta Was this translation helpful? Give feedback.
-
|
The guy who wrote the spec was dumb as a post (haha it me). Jokes aside, the spec was something I made when I was an intern at Sandia, thinking it would only exist for three months. Two or three years later the thing refused to die... Anyway, when it came for V2, we had just done a lot of work getting people onto Caliper, so that's what we used |
Beta Was this translation helpful? Give feedback.
-
|
I'll mark this as answered, because for my purposes we can make the Perfsuite just output cali files (or whatever we use instead of those files). If others want a different answer, we can look at that then |
Beta Was this translation helpful? Give feedback.
Correct, that's a bad idea and there aren't many good ones ... the one and only good option is to put Caliper into RAJAPerf 😄
If you absolutely want to you can of course make a program that creates a bunch of attributes and records and puts them in a CaliWriter, or extend the Python caliper-reader to also write files (that's a bit of work though). Spot has a few extra constraints, too, like globals being recorded with adiak, which adds more complexity.
We're also working on the DB backend for Spot, so you could probably write into the DB directly and avoid going through Caliper. However, the scheme isn't finalized yet; probably some weeks/months away from being ready. And you'd still need…