v0.35.0

What's new

• Aggregations - Declarative aggregations defined in the subgraph schema allow the developer to aggregate values on specific intervals using flexible aggregation functions. (#5082) (#5184) (#5209) (#5242) (#5208)
• Add pause and resume to admin JSON-RPC API - Adds support for explicit pausing and resuming of subgraph deployments with a field tracking the paused state in indexerStatuses. (#5190)
• Support eth_getBalance calls in subgraph mappings - Enables fetching the Eth balance of an address from the mappings using ethereum.getBalance(address). (#5202)
• Add parentHash to _meta query - Particularly useful when polling for data each block to verify the sequence of blocks. (#5232)
• Parallel execution of all top-level queries in a single query body (#5273)
• The ElasticSearch index to which graph-node logs can now be configured with the GRAPH_ELASTIC_SEARCH_INDEX environment variable which defaults to subgraph. (#5210)
• Some small prefetch simplifications. (#5132)
• Migration changing the type of health column to text. (#5077)
• Disable eth_call_execution_time metric by default. (#5164)
• Call revert_state_to whenever blockstream is restarted. (#5187)
• Pruning performance improvement: only analyze when rebuilding. (#5186)
• Disallow grafts within the reorg threshold. (#5135)
• Optimize subgraph synced check-less. (#5198)
• Improve error log. (#5217)
• Update provider docs. (#5216)
• Downgrade 'Entity cache statistics' log to trace. (#5241)
• Do not clone MappingEventHandlers in match_and_decode. (#5244)
• Make batching conditional on caught-up status. (#5252)
• Remove hack in chain_head_listener. (#5240)
• Increase sleep time in write queue processing. (#5266)
• Memoize Batch.indirect_weight. (#5276)
• Optionally track detailed indexing gas metrics in csv. (#5215)
• Do not use prefix comparisons for primary keys. (#5289)

Graphman

• Add ability to list removed unused deployment by id. (#5152)
• Add command to change block cache shard. (#5169)

Firehose and Substreams

• Add key-based authentication for Firehose/Substreams providers. (#5259)
• Increase blockstream buffer size for substreams. (#5182)
• Improve substreams error handling. (#5160)
• Reset substreams/firehose block ingestor backoff. (#5047)

Bug Fixes

• Fix graphiql issue when querying subgraph names with multiple path segments. (#5136)
• Fix change_health_column migration for sharded setup. (#5183)
• Fix conversion of BlockTime for NEAR. (#5206)
• Call revert_state_to to last good block instead of current block. (#5195)
• Fix Action::block_finished. (#5218)
• Fix runtime timeouts. (#5236)
• Remove panic from rewind and truncate. (#5233)
• Fix version stats for huge number of versions. (#5261)
• Fix _meta query failure due to incorrect selection set use. (#5265)

Major dependency upgrades

• Update to diesel v2. (#5002)
• Bump Rust version. (#4985)

Contributors

Thank you to all the contributors! @incrypto32, @mangas, @lutter, @leoyvens, @zorancv, @YaroShkvorets, @seem-less

Full Changelog: v0.34.1...v0.35.0

v0.34.1

$ docker pull graphprotocol/graph-node:v0.34.1

This a patch release that addresses a few issues

• Fixes an issue that caused an increase in data size of /metrics endpoint of graph-node. (#5161). Indexers are advised to skip v0.34.0 and use v0.34.1
• Fixes an issue that caused subgraphs with file data sources to skip non-deterministic errors that occurred in a file data source mapping handler.

v0.34.0

$ docker pull graphprotocol/graph-node:v0.34.0

What's New

• Substreams as Source of Triggers for Subgraphs - This update significantly enhances subgraph functionality by enabling substreams to act as a source of triggers for running subgraph mappings. Developers can now directly run subgraph mappings on the data output from substreams, facilitating a more integrated and efficient workflow.(#4887) (#4916)
• indexerHints in Manifest for Automated Pruning - This update introduces the ability for subgraph authors to specify indexerHints with a field prune in their manifest, indicating the desired extent of historical block data retention. This feature enables graph-node to automatically prune subgraphs when the stored history exceeds the specified limit, significantly improving query performance. This automated process eliminates the need for manual action by indexers for each subgraph. Indexers can also override user-set historyBlocks with the environment variable GRAPH_HISTORY_BLOCKS_OVERRIDE (#5032 (#5117)
• Initial Starknet Support - Introducing initial Starknet support for graph-node, expanding indexing capabilities to the Starknet ecosystem. The current integration is in its early stages, with notable areas for development including the implementation of trigger filters and data source template support. Future updates will also bring substream support. (#4895)
• endBlock Feature in Data Sources - This update adds the endBlock field for dataSources in subgraph manifest. By setting an endBlock, subgraph authors can define the exact block at which a data source will cease processing, ensuring no further triggers are processed beyond this point. (#4787
• Autogenerated Int8 IDs in graph-node - Introduced support for using Int8 as the ID type for entities, with the added capability to auto-generate these IDs, enhancing flexibility and functionality in entity management. (#5029)
• GraphiQL V2 Update - Updated GraphiQL query interface of graph-node to version 2. (#4677)
• Sharding Guide for Graph-Node - A new guide has been added to graph-node documentation, explaining how to scale graph-node installations using sharding with multiple Postgres instances. Sharding Guide
• Per-chain polling interval configuration for RPC Block Ingestors (#5066)
• Metrics Enhancements(#5055) (#4937)
• graph-node now avoids creating GIN indexes on array attributes to enhance database write performance, addressing the issue of expensive updates and underutilization in queries. (#4933)
• The subgraphFeatures endpoint in graph-node has been updated to load features from subgraphs prior to their deployment. (#4864)
• Improved log filtering performance in blockstream. (#5015)
• Enhanced GraphQL error reporting by including __schema and __type fields in the results during indexing errors (#4968)

Bug fixes

• Addressed a bug in the deduplication logic for Cosmos events, ensuring all distinct events are properly indexed and handled, especially when similar but not identical events occur within the same block. (#5112)
• Fixed compatibility issues with ElasticSearch 8.X, ensuring proper log functionality. (#5013)
• Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources 'from the future' could have been left behind. This release adds a database migration that fixes that. With very unlucky timing this migration might miss some subgraphs, which will later lead to an error assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block(). Should that happen, the migration script should be rerun against the affected shard. (#5083)
• Increased the base backoff time for RPC, enhancing stability and reliability under load. (#4984)
• Resolved an issue related to spawning offchain data sources from existing offchain data source mappings. (#5051)(#5092)
• Resolved an issue where eth-call results for reverted calls were being cached in call cache. (#4879)
• Fixed a bug in graphman's index creation to ensure entire String and Bytes columns are indexed rather than just their prefixes, resulting in optimized query performance and accuracy. (#4995)
• Adjusted SubstreamsBlockIngestor to initiate at the chain's head block instead of starting at block zero when no cursor exists. (#4951)
• Fixed a bug that caused incorrect progress reporting when copying subgraphs, ensuring accurate status updates. (#5075)

Graphman

• Graphman Deploy Command - A new graphman deploy command has been introduced, simplifying the process of deploying subgraphs to graph-node. (#4930)

Complete changelog: v0.33.0...v0.34.0

v0.34.0-rc.0

$ docker pull graphprotocol/graph-node:v0.34.0-rc.0

What's New

• Substreams as Source of Triggers for Subgraphs - This update significantly enhances subgraph functionality by enabling substreams to act as a source of triggers for running subgraph mappings. Developers can now directly run subgraph mappings on the data output from substreams, facilitating a more integrated and efficient workflow.(#4887) (#4916)
• indexerHints in Manifest for Automated Pruning - This update introduces the ability for subgraph authors to specify indexerHints with a field prune in their manifest, indicating the desired extent of historical block data retention. This feature enables graph-node to automatically prune subgraphs when the stored history exceeds the specified limit, significantly improving query performance. This automated process eliminates the need for manual action by indexers for each subgraph. Indexers can also override user-set historyBlocks with the environment variable GRAPH_HISTORY_BLOCKS_OVERRIDE (#5032 (#5117)
• Initial Starknet Support - Introducing initial Starknet support for graph-node, expanding indexing capabilities to the Starknet ecosystem. The current integration is in its early stages, with notable areas for development including the implementation of trigger filters and data source template support. Future updates will also bring substream support. (#4895)
• endBlock Feature in Data Sources - This update adds the endBlock field for dataSources in subgraph manifest. By setting an endBlock, subgraph authors can define the exact block at which a data source will cease processing, ensuring no further triggers are processed beyond this point. (#4787
• Autogenerated Int8 IDs in graph-node - Introduced support for using Int8 as the ID type for entities, with the added capability to auto-generate these IDs, enhancing flexibility and functionality in entity management. (#5029)
• GraphiQL V2 Update - Updated GraphiQL query interface of graph-node to version 2. (#4677)
• Sharding Guide for Graph-Node - A new guide has been added to graph-node documentation, explaining how to scale graph-node installations using sharding with multiple Postgres instances. Sharding Guide
• Per-chain polling interval configuration for RPC Block Ingestors (#5066)
• Metrics Enhancements(#5055) (#4937)
• graph-node now avoids creating GIN indexes on array attributes to enhance database write performance, addressing the issue of expensive updates and underutilization in queries. (#4933)
• The subgraphFeatures endpoint in graph-node has been updated to load features from subgraphs prior to their deployment. (#4864)
• Improved log filtering performance in blockstream. (#5015)
• Enhanced GraphQL error reporting by including __schema and __type fields in the results during indexing errors (#4968)

Bug fixes

• Addressed a bug in the deduplication logic for Cosmos events, ensuring all distinct events are properly indexed and handled, especially when similar but not identical events occur within the same block. (#5112)
• Fixed compatibility issues with ElasticSearch 8.X, ensuring proper log functionality. (#5013)
• Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources 'from the future' could have been left behind. This release adds a database migration that fixes that. With very unlucky timing this migration might miss some subgraphs, which will later lead to an error assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block(). Should that happen, the migration script should be rerun against the affected shard. (#5083)
• Increased the base backoff time for RPC, enhancing stability and reliability under load. (#4984)
• Resolved an issue related to spawning offchain data sources from existing offchain data source mappings. (#5051)(#5092)
• Resolved an issue where eth-call results for reverted calls were being cached in call cache. (#4879)
• Fixed a bug in graphman's index creation to ensure entire String and Bytes columns are indexed rather than just their prefixes, resulting in optimized query performance and accuracy. (#4995)
• Adjusted SubstreamsBlockIngestor to initiate at the chain's head block instead of starting at block zero when no cursor exists. (#4951)
• Fixed a bug that caused incorrect progress reporting when copying subgraphs, ensuring accurate status updates. (#5075)

Graphman

• Graphman Deploy Command - A new graphman deploy command has been introduced, simplifying the process of deploying subgraphs to graph-node. (#4930)

v0.33.0

What's New

• Arweave file data sources - Arweave file data sources allow subgraph developers to access offchain data from Arweave from within the subgraph mappings.(#4789)
• Major performance boost for substreams-based subgraphs - Significant performance improvements have been achieved for substreams-based subgraphs by moving substreams processing to the block stream.(#4851)
• Polling block handler - A new block handler filter polling for ethereum data sources which enables subgraph developers to run a block handler at defined block intervals. This is useful for use cases such as taking periodic snapshots of the contract state.(#4725)
• Initialization handler - A new block handler filter once for ethereum data sources which enables subgraph developers to create a handler which will be called only once before all other handlers run. This configuration allows the subgraph to use the handler as an initialization handler, performing specific tasks at the start of indexing. (#4725)
• DataSourceContext in manifest - DataSourceContext in Manifest - DataSourceContext can now be defined in the subgraph manifest. It's a free-form map accessible from the mapping. This feature is useful for templating chain-specific data in subgraphs that use the same codebase across multiple chains.(#4848)
• graph-node version in index node API - The Index Node API now features a new query, Version, which can be used to query the current graph-node version and commit. (#4852)
• Added a 'paused' field to Index Node API, a boolean indicating the subgraph’s pause status. (#4779)
• Proof of Indexing logs now include block number (#4798)
• subgraph_features table now tracks details about handlers used in a subgraph (#4820)
• Configurable SSL for Postgres in Dockerfile - ssl-mode for Postgres can now be configured via the connection string when deploying through Docker, offering enhanced flexibility in database security settings.(#4840)
• Introspection Schema Update - The introspection schema has been updated to align with the October 2021 GraphQL specification update.(#4676)
• trace_id Added to Substreams Logger (#4868)
• New apiVersion for Mapping Validation - The latest apiVersion 0.0.8 validates that fields set in entities from the mappings are actually defined in the schema. This fixes a source of non-deterministic PoI. Subgraphs using this new API version will fail if they try to set undefined schema fields in the mappings. Its strongly recommended updating to 0.0.8 to avoid these issues. (#4894)
• Substreams Block Ingestor Support - Added the ability to run a pure substreams chain by introducing a block ingestor for substreams-only chains. This feature allows users to run a chain with just a single substreams endpoint, enhancing support beyond RPC and firehose. Prior to this, a pure substreams chain couldn’t be synced.(#4839)

Bug fixes

• Fix for rewinding dynamic data source - Resolved an issue where a rewind would fail to properly remove dynamic data sources when using graphman rewind. This has been fixed to ensure correct behavior.(#4810)
• Improved Deployment Reliability with Retry Mechanism - A retry feature has been added to the block_pointer_from_number function to enhance the robustness of subgraph deployments. This resolves occasional failures encountered during deployment processes.(#4812)
• Fixed Cross-Shard Grafting Issue - Addressed a bug that prevented cross-shard grafting from starting, causing the copy operation to stall at 0% progress. This issue occurred when a new shard was added after the primary shard had already been configured. The fix ensures that foreign tables and schemas are correctly set up in new shards. For existing installations experiencing this issue, it can be resolved by running graphman database remap.(#4845)
• Fixed a Full-text search regression - Reverted a previous commit (ad1c6ea) that inadvertently limited the number of populated search indexes per entity.(#4808)
• Attestable Error for Nested Child Filters - Nested child filter queries now return an attestable ChildFilterNestingNotSupportedError, improving error reporting for users.(#4828)

Graphman

• Index on prefixed fields - The graphman index create command now correctly indexes prefixed fields of type String and Bytes for more query-efficient combined indexes. Note: For fields that are references to entities, the behavior may differ. The command may create an index using left(..) when it should index the column directly.
• Partial Indexing for Recent Blocks - The graphman index create command now includes a --after $recent_block flag for creating partial indexes focused on recent blocks. This enhances query performance similar to the effects of pruning. Queries using these partial indexes must include a specific clause for optimal performance.(#4830)

Full Changelog: incrypto32/v0.32.0...v0.33.0

v0.33.0-rc.0

$ docker pull graphprotocol/graph-node:v0.33.0-rc.0

What's New

• Arweave file data sources - Arweave file data sources allow subgraph developers to access offchain data from Arweave from within the subgraph mappings.(#4789)
• Major performance boost for substreams-based subgraphs - Significant performance improvements have been achieved for substreams-based subgraphs by moving substreams processing to the block stream.(#4851)
• Polling block handler - A new block handler filter polling for ethereum data sources which enables subgraph developers to run a block handler at defined block intervals. This is useful for use cases such as taking periodic snapshots of the contract state.(#4725)
• Initialization handler - A new block handler filter once for ethereum data sources which enables subgraph developers to create a handle which will be called only once before all other handlers run. This configuration allows the subgraph to use the handler as an initialization handler, performing specific tasks at the start of indexing. (#4725)
• DataSourceContext in manifest - DataSourceContext in Manifest - DataSourceContext can now be defined in the subgraph manifest. It's a free-form map accessible from the mapping. This feature is useful for templating chain-specific data in subgraphs that use the same codebase across multiple chains.(#4848)
• graph-node version in index node API - The Index Node API now features a new query, Version, which can be used to query the current graph-node version and commit. (#4852)
• Added a 'paused' field to Index Node API, a boolean indicating the subgraph’s pause status. (#4779)
• Proof of Indexing logs now include block number (#4798)
• subgraph_features table now tracks details about handlers used in a subgraph (#4820)
• Configurable SSL for Postgres in Dockerfile - ssl-mode for Postgres can now be configured via the connection string when deploying through Docker, offering enhanced flexibility in database security settings.(#4840)
• Introspection Schema Update - The introspection schema has been updated to align with the October 2021 GraphQL specification update.(#4676)
• trace_id Added to Substreams Logger (#4868)
• New apiVersion for Mapping Validation - The latest apiVersion 0.0.8 now validates that fields set in entities from the mappings are actually defined in the schema. Subgraphs using this new API version will fail if they try to set undefined schema fields in the mappings. (#4894)
• Substreams Block Ingestor Support - Added the ability to run a pure substreams chain by introducing a block ingestor for substreams-only chains. This feature allows users to run a chain with just a single substreams endpoint, enhancing support beyond RPC and firehose. Prior to this, a pure substreams chain couldn’t be synced.(#4839)

Bug fixes

• Fix for rewinding dynamic data source - Resolved an issue where a rewind would fail to properly remove dynamic data sources when using graphman rewind. This has been fixed to ensure correct behavior.(#4810)
• Improved Deployment Reliability with Retry Mechanism - A retry feature has been added to the block_pointer_from_number function to enhance the robustness of subgraph deployments. This resolves occasional failures encountered during deployment processes.(#4812)
• Fixed Cross-Shard Grafting Issue - Addressed a bug that prevented cross-shard grafting from starting, causing the copy operation to stall at 0% progress. This issue occurred when a new shard was added after the primary shard had already been configured. The fix ensures that foreign tables and schemas are correctly set up in new shards.(#4845)
• Fixed a Full-text search regression - Reverted a previous commit (ad1c6ea) that inadvertently limited the number of populated search indexes per entity.(#4808)
• Attestable Error for Nested Child Filters - Nested child filter queries now return an attestable ChildFilterNestingNotSupportedError, improving error reporting for users.(#4828)

Graphman

• Index on prefixed fields - The graphman index create command now correctly indexes prefixed fields of type String and Bytes, making combined indexes more query-efficient.
• Partial Indexing for Recent Blocks - The graphman index create command now includes a --after $recent_block flag for creating partial indexes focused on recent blocks. This enhances query performance similar to the effects of pruning. Queries using these partial indexes must include a specific clause for optimal performance.(#4830)

Full Changelog: v0.32.0...b1502a4

v0.32.0

$ docker pull graphprotocol/graph-node:v0.32.0

What's New

• Derived fields getter: Derived fields can now be accessed from within the mapping code during indexing. (#4434)
• Sorting interfaces by child entity: Interfaces can now be sorted by non-derived child entities. (#4058)
• File data sources can now be spawned from handlers of other file data sources: This enables the use of file data sources for scenarios where a file data source needs to be spawned from another one. One practical application of this feature is in handling NFT metadata. In such cases, the metadata itself is stored as a file on IPFS and contains embedded IPFS CID for the actual file for the NFT. (#4713)
• Allow redeployment of grafted subgraphs even when graft_base is not available: This will allow renaming of already synced grafted subgraphs even when the graft base is not available, which previously failed due to graft-base validation errors. (#4695)
• history_blocks is now available in the index-node API. (#4662)
• Added a new subgraph features table in primary to easily track information like apiVersion, specVersion, features, and data source kinds used by subgraphs. (#4679)
• subgraphFeatures endpoint now includes data from subgraph_features table.
• ens_name_by_hash is now undeprecated: This reintroduces support for fetching ENS names by their hash, dependent on the availability of the underlying Rainbow Table (#4751).
• Deterministically failed subgraphs now return valid POIs for subsequent blocks after the block at which it failed. (#4774)
• eth-call logs now include block hash and block number: This enables easier debugging of eth-call issues. (#4718)
• Enabled support for substreams on already supported networks. (#4767)
• Add new GraphQL scalar type Int8. This new scalar type allows subgraph developers to represent 8-bit signed integers. (#4511)
• Add support for overriding module params for substreams-based subgraphs when params are provided in the subgraph manifest. (#4759)

Breaking changes

• Duplicate provider labels are not allowed in graph-node config anymore

Bug fixes

• Fixed PublicProofsOfIndexing returning the error Null value resolved for non-null field proofOfIndexing when fetching POIs for blocks that are not in the cache (#4768)
• Fixed an issue where Block stream would fail when switching back to an RPC-based block ingestor from a Firehose ingestor. (#4790)
• Fixed an issue where derived loaders were not working with entities with Bytes as IDs (#4773)
• Firehose connection test now retries for 30 secs before setting the provider status to Broken (#4754)
• Fixed the nonFatalErrors field not populating in the index node API. (#4615)
• Fixed graph-node panicking on the first startup when both Firehose and RPC providers are configured together. (#4680)
• Fixed block ingestor failing to startup with the error net version for chain mainnet has changed from 0 to 1 when switching from Firehose to an RPC provider. (#4692)
• Fixed Firehose endpoints getting rate-limited due to duplicated providers during connection pool initialization. (#4778)
• Fixed a determinism issue where stale entities where being returned when using get_many and get_derived ([#4801]#4801)

Graphman

• Added two new graphman commands pause and resume: Instead of reassigning to a non-existent node these commands can now be used for pausing and resuming subgraphs. (#4642)
• Added a new graphman command restart to restart a subgraph. (#4742)

Full Changelog: v0.31.0...2891a76

v0.32.0-rc.0

v0.32.0-rc.0 is the first release candidate of the upcoming v0.32.0. This release candidate is intended for testnet use only.

$ docker pull graphprotocol/graph-node:v0.32.0-rc.0

What's New

• Derived fields getter: Derived fields can now be accessed from within the mapping code during indexing. (#4434)
• Sorting interfaces by child entity: Interfaces can now be sorted by non-derived child entities. (#4058)
• File data sources can now be spawned from handlers of other file data sources: This enables the use of file data sources for scenarios where a file data source needs to be spawned from another one. One practical application of this feature is in handling NFT metadata. In such cases, the metadata itself is stored as a file on IPFS and contains embedded IPFS CID for the actual file for the NFT. (#4713)
• Allow redeployment of grafted subgraphs even when graft_base is not available: This will allow renaming of already synced grafted subgraphs even when the graft base is not available, which previously failed due to graft-base validation errors. (#4695)
• history_blocks is now available in the index-node API. (#4662)
• Added a new subgraph features table in primary to easily track information like apiVersion, specVersion, features, and data source kinds used by subgraphs. (#4679)
• subgraphFeatures endpoint now includes data from subgraph_features table.
• ens_name_by_hash is now undeprecated: This reintroduces support for fetching ENS names by their hash, dependent on the availability of the underlying Rainbow Table (#4751).
• Deterministically failed subgraphs now return valid POIs for subsequent blocks after the block at which it failed. (#4774)
• eth-call logs now include block hash and block number: This enables easier debugging of eth-call issues. (#4718)
• Enabled support for substreams on already supported networks. (#4767)
• Add new GraphQL scalar type Int8. This new scalar type allows subgraph developers to represent 8-bit signed integers. (#4511)
• Add support for overriding module params for substreams-based subgraphs when params are provided in the subgraph manifest. (#4759)

Bug fixes

• Fixed PublicProofsOfIndexing returning the error Null value resolved for non-null field proofOfIndexing when fetching POIs for blocks that are not in the cache (#4768)
• Fixed an issue where Block stream would fail when switching back to an RPC-based block ingestor from a Firehose ingestor. (#4790)
• Fixed an issue where derived loaders were not working with entities with Bytes as IDs (#4773)
• Firehose connection test now retries for 30 secs before setting the provider status to Broken (#4754)
• Fixed the nonFatalErrors field not populating in the index node API. (#4615)
• Fixed graph-node panicking on the first startup when both Firehose and RPC providers are configured together. (#4680)
• Fixed block ingestor failing to startup with the error net version for chain mainnet has changed from 0 to 1 when switching from Firehose to an RPC provider. (#4692)
• Fixed Firehose endpoints getting rate-limited due to duplicated providers during connection pool initialization. (#4778)

Graphman

• Added two new graphman commands pause and resume: Instead of reassigning to a non-existent node these commands can now be used for pausing and resuming subgraphs. (#4642)
• Added a new graphman command restart to restart a subgraph. (#4742)

Full Changelog: v0.31.0...c350e4f

v0.31.0-rc.1

v0.31.0-rc.1 is the second release candidate of the upcoming v0.31.0.

$ docker pull graphprotocol/graph-node:v0.31.0-rc.1

Edit 2023-06-08: v0.31.0-rc.1 has been promoted to v0.31.0.

v0.31.0

$ docker pull graphprotocol/graph-node:v0.31.0

What's new

• Fulltext searches can now be combined with where filtering, further narrowing down search results. #4442
• Derived fields loader support. This feature requires subgraph authors to upgrade graph-cli to v0.51.0. #4434, graph-cli release notes
• Tweaked how RPC provider limiting rules are interpreted from configurations. In particular, node IDs that don't match any rules of a provider won't have access to said provider instead of having access to it for an unlimited number of subgraphs. Read the docs for more information. #4353
• Introduced WASM host function store.get_in_block, which is a much faster variant of store.get limited to entities created or updated in the current block. #4540
• The subgraph_deploy JSON-RPC method now accepts a history_blocks parameter, which indexers can use to set default amounts of history to keep. #4564
• IPFS requests for polling file data sources are not throttled anymore (also known as concurrency or burst limiting), only rate-limited. #4570, #4649
• Exponential requests backoff when retrying failed subgraphs is now "jittered", smoothing out request spikes. #4476
• Indexers can use thee new feature GRAPH_EXPERIMENTAL_SUBGRAPH_SETTINGS to define matching rules on subgraph names for reducing amounts of history stored. #4633
• RPC provider responses that decrease the chain head block number (non-monotonic) are now ignored, increasing resiliency against inconsistent provider data. #4354
• It's now possible to to have a Firehose-only chain with no RPC provider at all in the configuration. #4508, #4553
• The materialized views in the info schema (table_sizes, subgraph_sizes, and chain_sizes) that provide information about the size of various database objects are now automatically refreshed every 6 hours. #4461
• Subgraph error logs now contain the transaction hash when available. #4635
• Adapter selection now takes error rates into account, preferring adapters with lower error rates. #4468
• Removed support for GRAPH_ETHEREUM_IS_FIREHOSE_PREFERRED, REVERSIBLE_ORDER_BY_OFF, and GRAPH_STORE_CONNECTION_TRY_ALWAYS env. variables. #4375, #4436
• New env. var. GRAPH_ETH_CALL_NO_GAS which is a comma-separated list of chains' net_version for which to disable eth_call gas limits. The default value (421613) disables gas limits for Arbitrum Goerli. #4609

Performance improvements

• Batched writes: write operations to the database when indexing subgraphs are now accumulated into in-memory batches before persisting them, resulting in noticeably faster indexing and lower database CPU load. In case of unexpected issues, this feature can be turned off setting GRAPH_STORE_WRITE_BATCH_SIZE=0. #4606, #4632
• The entity cache that graph-node keeps around is much more efficient, meaning more cache entries fit in the same amount of memory resulting in a performance increase under a wide range of workloads. #4485, #4624
• Indexing speed improvements for subgraphs with more than 10k data sources. #4631
• BRIN indexes in PostgreSQL now use ..._minmax_multi_ops instead of _minmax_ops if available (PostgreSQL 14+), often resulting in more selective indexes. #4629
• GIN indexes are not created anymore for numeric arrays. #4628

Substreams

• The substreams protocol has been updated to sf.substreams.rpc.v2.Stream/Blocks. #4556
• Added support for adapter balancing a.k.a. provider failover to substreams. #4578
• Fixed faulty startBlock selection logic in substreams. #4463
• Several bug fixes were applied after breakage caused by an internal refactor of how entities are stored in-memory. #4604, #4572, #4612, #4664

Bug fixes

• Fixed a bug that would cause subgraphs to fail with a subgraph writer poisoned by previous error message following certain database errors. #4533
• Fixed a bug that would cause subgraphs to fail with a store error: no connection to the server message when database connection e.g. gets killed. #4435
• The subgraph_reassign JSON-RPC method doesn't fail anymore when multiple deployment copies are found: only the active copy is reassigned, the others are ignored. #4395
• Fixed a bug that would cause on_sync handlers on copied deployments to fail with the message Subgraph instance failed to run: deployment not found [...]. #4396
• Fixed a bug that would cause the copying or grafting of a subgraph while pruning it to incorrectly set earliest_block in the destination deployment. #4502
• Handler timeouts would sometimes be reported as deterministic errors with the error message Subgraph instance failed to run: Failed to call 'asc_type_id' with [...] wasm backtrace [...]; this error is now nondeterministic and recoverable. #4475
• Fixed faulty exponential request backoff behavior after many minutes of failed requests, caused by an overflow. #4421
• json.fromBytes and all BigInt operations now require more gas, protecting against malicious subgraphs. #4594, #4595
• Fixed a Rust panic on program startup when mixing Firehose and RPC providers. #4680

Graphman

• graphman rewind now requires block-number and block-hash to be passed as flags instead of arguments. #4400
• You can now use the new flag --start-block in graphman rewind to rewind a subgraph to the startBlock set in manifest, or to the genesis block if no startBlock is set. #4400
• The behavior for graphman prune has changed: running just graphman prune will mark the subgraph for ongoing pruning in addition to performing an initial pruning. To avoid ongoing pruning, use graphman prune --once (docs). #4429
• The env. var. GRAPH_STORE_HISTORY_COPY_THRESHOLD –which serves as a configuration setting for graphman prune– has been renamed to GRAPH_STORE_HISTORY_REBUILD_THRESHOLD. #4505
• You can now list all existing deployments via graphman info --all. #4347
• The command graphman chain call-cache remove now requires --remove-entire-cache as an explicit flag, protecting against accidental destructive command invocations. #4397
• graphman copy create accepts two new flags, --activate and --replace, which make moving of subgraphs across shards much easier. #4374
• The log level for graphman is now set via GRAPHMAN_LOG or command line instead of GRAPH_LOG. #4462
• graphman reassign now emits a warning when it suspects a typo in node IDs. #4377

Metrics and logging

• Subgraph syncing time metric deployment_sync_secs now stops updating once the subgraph has synced. #4489
• New endpoint_request metric to track error rates of different providers. #4490, #4504, [#4430](...