This tiny blockchain indexer scrapes Transfer
events emitted by the USDC contract and saves the data in a file-based dataset located at the Filebase S3-compatible data storage service. It is built with the Subsquid framework, hence the term "squid".
The squid uses @subsquid/file-store
, @subsquid/file-store-csv
and @subsquid/file-store-s3
packages to store the dataset. S3 connection parameters are partially hardcoded to illustrate how they can be supplied programmatically.
The files are stored in a simple CSV format, but a more performant Apache Parquet format is supported as well. See the documentation page and this example for more info.
Dependencies: NodeJS, Squid CLI.
To see it in action, clone the repository and install dependencies:
$ git clone https://github.com/subsquid-labs/file-store-s3-example
$ cd file-store-s3-example/
$ npm i
Next, create a subsquid-testing-bucket
bucket at your Filebase account and set your authentication credentials in .env
:
S3_ACCESS_KEY_ID=myKeyId
S3_SECRET_ACCESS_KEY=mySecretAccessKey
Finally, spin up a processor, a process that ingests the data from the Ethereum Archive:
$ sqd process
You should see a transfers-data
folder populated with indexer data appear in the bucket in a bit.