Iris compatibility

Main focus of this release was to get the middleware work with the latest AE Node v6.0.0. (candidate versions for Iris hardfork).

In addition, following minor features were implemented:

• allowing to specify a custom network id
• searching names by prefix

Internal transfers and statistics

This release adds following endpoints:

• transfers - showing internal transfers of the tokens which are not visible via transactions (txs) endpoints
• stats - showing statistics for a given generation
• totalstats - showing aggregated statistics - total number of block/dev rewards and total supply of tokens at given generation

Due to the fact that entities with lifecycles (names, oracles) prevent quick sync of only new data, full mdw sync is needed:

make reset-mdw-db
make shell # or any other way to start MDW

Indexing function calls inside contracts

News

Middleware can now index and query function calls which happened during contract execution. (examples: https://github.com/aeternity/ae_mdw#function-calls)

Migration

There is no migration which means that the middleware data need to sync from scratch.

Best way to achieve this is to reset middleware DB and let it sync:

$ make reset-mdw-db 

and

$ make shell  

Sync robustness

• final fix of AEX9 account presence endpoints - e.g. showing token balances for all AEX9 contracts given an account
• handling middleware syncing failures by stopping syncing and notifying of operator by mail, but continuing with responding to requests
• minor configuration change is needed, add these to :ae_mdw section:

  sync: true, # by default we want to sync middleware data
  operators: [], # email address where to send sync crash notification, requires `mail` binary 

For hosts which are not syncing from scratch, following migration allows only syncing of the aex9 account balance presence.
Run these in ae mdw elixir shell:

1.) define the migration module:

defmodule Aex9FixTables do

  alias AeMdw.Db.Stream, as: DBS
  alias AeMdw.Db.{Model, Format}
  alias AeMdw.Validate
  import AeMdw.Db.Util

  require Model

  def clear() do
    [:mnesia.clear_table(Model.Aex9AccountPresence),
     :mnesia.clear_table(Model.IdxAex9AccountPresence)]
  end

  def write(acc_pk, txi, ct_pk) do
    m_acc = Model.aex9_account_presence(index: {acc_pk, txi, ct_pk})
    m_idx = Model.idx_aex9_account_presence(index: {txi, acc_pk, ct_pk})
    :mnesia.write(Model.Aex9AccountPresence, m_acc, :write)
    :mnesia.write(Model.IdxAex9AccountPresence, m_idx, :write)
  end

  def bi_call_entries(contract_pk) do
    transfer_evt = AeMdw.Node.aex9_transfer_event_hash()
    Enum.flat_map(
      DBS.map(:forward, :raw, type: :contract_call, contract_id: contract_pk),
      fn %{tx: %{log: logs}} = x ->
        bi = {x.block_height, x.micro_index}
        idxed_logs = Enum.with_index(logs)
        for {%{topics: [^transfer_evt, from_pk, to_pk, <<amount::256>>]}, i} <- idxed_logs,
          do: {bi, {{contract_pk, x.tx_index, i}, {from_pk, to_pk}, amount}}
      end)
  end

  def idxed_bi_groups() do
    idxed_create_txis =
      Enum.map(:mnesia.dirty_all_keys(Model.RevAex9Contract), &elem(&1, 0))
      |> Enum.with_index(1)
    ct_count = Enum.count(idxed_create_txis)
    IO.puts("total contracts = #{ct_count}")

    all_entries =
      Enum.flat_map(idxed_create_txis,
        fn {create_txi, i} ->
          IO.puts("[#{i}/#{ct_count}] gathering activity for contract #{create_txi}")
          raw_tx = Format.to_raw_map(read_tx!(create_txi))
          ct_pk = Validate.id!(raw_tx.tx.contract_id)
          create_bi = {raw_tx.block_height, raw_tx.micro_index}
          create_entry = {{ct_pk, create_txi, -1}, <<>>, -1}
          [{create_bi, create_entry} | bi_call_entries(ct_pk)]
        end)

    bi_groups = Enum.group_by(all_entries, &elem(&1, 0))
    bi_count = Enum.count(bi_groups)
    IO.puts("microblock groups = #{bi_count}")

    {_, bi_groups} = bi_groups |> Enum.to_list |> Enum.sort |> Enum.unzip
    Enum.with_index(bi_groups, 1)
  end

  def fix(),
    do: fix(idxed_bi_groups())

  def fix(idxed_bi_groups) do
    :ets.delete_all_objects(:aex9_sync_cache)
    for {bi_group, i} <- idxed_bi_groups do
      # IO.puts("[#{i}/#{bi_count}] deriving account presence")

      [{bi, _} | _] = bi_group
      {_, bi_group} = Enum.unzip(bi_group)

      IO.inspect(bi_group, limit: :infinity, label: "BI[#{i}] #{inspect bi}")

      :ets.insert(:aex9_sync_cache, bi_group)
      {:atomic, _} = :mnesia.transaction(fn ->
        # creates
        AeMdw.Db.Sync.Contract.aex9_derive_account_presence!(bi)

        # calls
        for {{ct_pk, txi, _}, {_from_pk, to_pk}, _amount} <- bi_group,
          do: AeMdw.Db.Contract.aex9_write_presence(ct_pk, txi, to_pk)
      end)
    end
    :ok
  end
end

2.) temporarily stop syncing:
AeMdw.Application.sync(false)

3.) run the migration:

Aex9FixTables.clear()
Aex9FixTables.fix()  # the first "gathering of activity" takes a while, don't stop it

4.) resume syncing:
AeMdw.Application.sync(true)

AEX9 account balances, contract logs with literal constructor

This release includes following improvements:

• showing of all AEX9 balances for account - https://github.com/aeternity/ae_mdw/#aex-9-contract-balances-for-account
• querying of contract logs with event=EventConstructorName instead of event hash - https://github.com/aeternity/ae_mdw/#querying-of-contract-logs

Bugfix release

Fixes:

• syncing of AEX9 contracts

First public release.

This is a first public release of Indaex (Aeternity Middleware).