Merge TemporalData with Data/HeteroData [2/n]

[DamianSzwichtenberg]

This PR attempts to merge TemporalData into HeteroData.
Currently, one can perform time-related modification/query using a specific node or edge type:

# bad: usage based on `BaseStorage` methods
for e in hetero_data.edge_types:
    hetero_data[e].up_to(10)
res = all([hetero_data[e].is_sorted_by_time() for e in hetero_data.edge_types])

This is possible because BaseStorage implements those functionalities. For better usability, however,
these functions should be exposed in the HeteroData API directly. Such an approach would also allow us to
perform operations on many node or edge types at once:

# good: usage based on `HeteroData` API
out = hetero_data.up_to(hetero_data.edge_types, 10)
res = hetero_data.is_sorted_by_time(hetero_data.edge_types)

Related: #3230, #8454

[codecov]

Codecov Report

Attention: Patch coverage is 86.84211% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 89.23%. Comparing base (af0f5f4) to head (5dcdc0b).

Files	Patch %	Lines
torch_geometric/data/hetero_data.py	85.71%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8739      +/-   ##
==========================================
- Coverage   89.86%   89.23%   -0.63%     
==========================================
  Files         468      468              
  Lines       30039    30070      +31     
==========================================
- Hits        26995    26834     -161     
- Misses       3044     3236     +192     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wsad1]

I have a question about snapshot which is relevant to this PR and #8454.
Assume nodes n1 and n2 with time equals 1 and 4 respectively, and an edge e1 (between n1 and n2 ) with time = 5.
On calling snapshot(start_time = 4, end_time = 5) what would the output look like, from the code it looks like the returned graph will have node n2 and edge e1, but e1 doens't have a source node.

[DamianSzwichtenberg]

In the current design, time cannot be an edge and node attribute at the same time, so this scenario is impossible. As these functionalities are moved directly from TemporalData I would consider underlying data rather as a list of events.

[wsad1]

Is there some part of the code that explicitly prevents this?
Does this also apply to hetero graphs?

[DamianSzwichtenberg]

2x Yes, for example at BaseStorage and NeighborSampler.

[wsad1]

For both up_to and snapshot , do we reindex edges. Like if we remove the 0th node from the graph, then does the 1st node become the 0th node in the edge index?

[DamianSzwichtenberg]

We don't. But that's a good point. When I integrated TemporalData into Data I kept the assumption that we represent events, so time as a node attribute is currently not well defined.