Releases: ArweaveTeam/arweave
Release 2.0.0.4
The release includes a bugfix, a new HTTP client, and an improvement.
A new HTTP client
The new client allows to strictly control how much data is downloaded and fetches big blobs of data more efficiently.
A bugfix
A bug is fixed which prevented fresh nodes from joining the network.
An improvement
Accepting data from peers is made more efficient.
Upgrade instructions
N.2.0.0.4 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.4.linux-x86_64.tar.gz
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.
If you want to run the miner from the existing Git folder, execute the following command to upgrade (note the submodule update and the cleaning step):
git fetch --all --tags && git checkout -f N.2.0.0.4
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.3
The release includes a bugfix, a cleanup made after the transition to 2.0, and support for specifying peers via DNS in the command line.
DNS peers
You can now use domain names of your trusted peers on startup. E.g. to use the Arweave peers, you may specify
peer sfo-1.na-west-1.arweave.net peer nyc-1.na-east-1.arweave.net peer tor-1.na-east-2.arweave.net peer ams-1.eu-central-1.arweave.net
A post-2.0 cleanup
There are changes to the miner's bookkeeping, which reduce the memory footprint and increase the reliability of the miner.
A bugfix
A bug was fixed which caused nodes under some very specific conditions to get stuck recovering from a fork.
Upgrade instructions
N.2.0.0.3 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.3.linux-x86_64.tar.gz
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.
If you want to run the miner from the existing Git folder, execute the following command to upgrade:
git fetch --all --tags && git checkout -f N.2.0.0.3
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.2
The release includes a bugfix, a major performance optimization, and a script for removing old wallet lists.
A performance optimization
After the 2.0 upgrade, some nodes were noticed to spend significant amount of time decoding recall data encoded in Base64URL and consequently lag behind.
The release introduces a new Base64URL decoding library, which is more than 20x faster than the previous one.
A script for removing old wallet lists
Starting from the version 2.0, the wallet lists are no longer required for block syncing and recall block verification. Therefore, the node will not download them again after removal. This allows to clean up a lot of disk space.
A syncing bugfix
A bug is fixed which prevented the v2 transaction data from syncing in a timely manner. The tooling for posting v2 transactions has not been published yet, so the bug had a mild effect on the network.
Upgrade instructions
N.2.0.0.2 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.2.linux-x86_64.tar.gz
It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.
If you want to run the miner from the existing Git folder, execute the following command to upgrade (note the submodule update and the cleaning step):
git fetch --all --tags && git checkout -f N.2.0.0.2
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.1
The release includes a RandomX upgrade and a memory usage optimization. Note that the upgrade instructions are slightly different this time if you are running the miner from Git.
RandomX is upgraded to v1.1.7
The new version contains major performance improvements, the increase is up to 20% on certain CPUs with the large pages enabled.
The updated RandomX commit is here.
It contains a few exra changes to make it compatible with the current Arweave miner.
The RAM usage is reduced significantly on big miners
The memory footprint is lowered by roughly 2 GB on the machines with ~20 mining threads.
Upgrade instructions
N.2.0.0.1 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it:
tar -xzf arweave-2.0.0.1.linux-x86_64.tar.gz
It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.
If you want to run the miner from the existing Git folder, install CMake (on Ubuntu, sudo apt install cmake) and execute the following command to upgrade (note the submodule update and the cleaning step):
git fetch --all --tags && git checkout -f N.2.0.0.1 && git submodule update && ./rebar3 clean
You can now run the miner using the arweave-server script.
Support
For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 2.0.0.0
The release is a hard fork that activates at height 422250, approximately 2020-04-09 10:00 UTC. You will need to make sure you have upgraded your miner before this time in order to be connected to the 2.0 network. The release introduces **Fast Write**: a major modification to the core Arweave protocol that removes relevant limits to the write speed of the network. The new _protocol_, enforced by this _client_ release, supports enormous volumes of data. However, the _client_ does not yet accept data separately from transaction headers, constraining the upload speeds not only by the network speeds, but also memory pool and transaction queue limits. The extended _client_ capabilities for managing arbitrarily huge volumes of data are planned for 2.1, but will not require a hard fork. Nonetheless, this upgrade still dramatically increases the practical block size of the network. In addition to increasing the write speed, the new protocol makes block validation more efficient. This is achieved through using a new Merkle tree-based Succinct Proofs of Access scheme, and an iterative block hashing mechanism. Other notable changes: - The memory footprint is reduced by about 1 GB. - Mining bookkeeping costs are reduced. - Data syncing speeds are improved. - The auto-update feature has been dropped. - No firewall checks are made during fork recovery. Since the data is distributed separately from the headers for the new transactions, it is not feasible to do firewall checks on the fly. - It is no longer necessary to store historical block metadata and wallet lists for mining. **Upgrade instructions** N.2.0.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. Download the corresponding archive and unpack it: ``` tar -xzf arweave-2.0.0.0.linux-x86_64.tar.gz ``` It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument. The archive contents look like this: ``` bin/ data/ erts-10.3/ lib/ releases/ ``` To run the miner, execute ./bin/start <arguments>. See the mining guide for more details. If your OS/platform architecture is not in the list, check README for how to build the miner from sources. If you want to run the miner from the existing Git folder, execute the following command to upgrade: ``` git fetch --all --tags && git checkout -f N.2.0.0.0 ``` You can now run the miner using the arweave-server script. **Support** For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 1.9.4.0
**Improvements & bug fixes** - A bottleneck was resolved in the core processing module which slowed down block application in the presence of a large number of transactions. - Transaction propagation parallelisation is made configurable, to allow the network to easily switch to a more conservative distribution rate. **Upgrade instructions** N.1.9.4.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. Download the corresponding archive and unpack it: ``` tar -xzf arweave-1.9.4.0.linux-x86_64.tar.gz ``` It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument. The archive contents look like this: ``` bin/ data/ erts-10.3/ lib/ releases/ ``` To run the miner, execute ./bin/start <arguments>. See the mining guide for more details. If your OS/platform architecture is not in the list, check README for how to build the miner from sources. If you want to run the miner from the existing Git folder, execute the following command to upgrade: ``` git fetch --all --tags && git checkout -f N.1.9.4.0 ``` You can now run the miner using the arweave-server script. **Support** For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release 1.9.3.0
**Improvements & bug fixes** The release includes a few improvements to transaction processing and gossiping mechanisms, including in the priority queue, plus miscellaneous bug fixes. - A bug was fixed in the priority queue which caused it to distribute transactions too quickly. - Improvements to the core gossiping mechanics of the network. - Miscellaneous bug fixes. **Upgrade instructions** N.1.9.3.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. Download the corresponding archive and unpack it: ``` tar -xzf arweave-1.9.3.0.linux-x86_64.tar.gz ``` It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument. The archive contents look like this: ``` bin/ data/ erts-10.3/ lib/ releases/ ``` To run the miner, execute ./bin/start <arguments>. See the mining guide for more details. If your OS/platform architecture is not in the list, check README for how to build the miner from sources. If you want to run the miner from the existing Git folder, execute the following command to upgrade: ``` git fetch --all --tags && git checkout -f N.1.9.3.0 ``` You can now run the miner using the arweave-server script. **Support** For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.
Release N.1.9.2.0
The release comes with the new build system. No need to install any dependencies anymore - simply download and run the binary! Please, read the upgrade upgrade instructions below for details - the upgrade instructions for the previous releases would not apply here.
Improvements and bugfixes, the miner:
- The polling mode was made more efficient. In addition to that, even outside the polling mode the node will ask its trusted peers for blocks if it has not received any in the last minute.
- The number of best peers to propagate transactions and blocks to can be specified in the command line.
- The number of file descriptors to use for incoming connections can be specified on the command line. After the limit is reached, the requests are queued.
- Collecting transactions during fork recovery now respects protocol limits to avoid abuse.
- Requests with valid transactions are excluded from the IP-based throttling to ensure timely propagation of excessive transaction volumes.
- Metrics for uploaded and downloaded bytes per endpoint were added.
Improvements and bugfixes, the gateway:
- The SQLite driver was replaced, the previous one had an issue where it occasionally would fail to restart after a crash.
- The missing CORS headers were added to the /:path and /:path/:subpath endpoints.
- The number of file descriptors necessary for incoming gateway connections was reduced.
- Metadata of all transactions of a block is inserted in a single database transaction now.
Upgrade instructions
N.1.9.2.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.
Download the corresponding archive and unpack it.
tar -xzf arweave-1.9.2.0.linux-x86_64.tar.gz
It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:
bin/ data/ erts-10.3/ lib/ releases/
To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.
If you have been already mining before this update, you can either download and unpack the archive in your Git arweave folder and run ./bin/start ... or execute the following to continue building the miner from sources:
git fetch --all --tags && git checkout -f N.1.9.2.0
git submodule update
rm -rf lib _build
You can now run the miner using the arweave-server script.
You can also create a new directory and unpack the archive there, but make sure you use the same data directory (data_dir argument) as you did before. If you used the default, it would be data_dir <path-to-your-previous-arweave-folder> in the mining command. Check this section of the guide if you would like to move your data elsewhere.
If you operate a gateway, note that the TLS certificates need to be put into the apps/arweave/priv/tls folder.
Support
For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us on team@arweave.org.
Release N.1.9.1.0
**Transaction priority queue** The feature seeks to optimize transaction propagation. The goals are to maximize the utility for miners and stay within the resource limits (network bandwidth, RAM). The task can be seen as a Knapsack auction, where miners need to collect transactions with the biggest total reward, subject to the given total size constraint. The choice of the allowed size, in turn, determines the latency. We use a greedy approximation algorithm, which prioritizes transactions with a higher per-byte reward. **Improvements** - Reduced network traffic. The number of outgoing connections for propagating transactions is now capped globally, not on a per-transaction basis. **Upgrade instructions** Shut down the miner. To make sure the shutdown was successful, run `ps aux | grep beam | grep -v grep`. The output should be empty. Run `git fetch --all --tags && git checkout N.1.9.1.0 && git reset --hard N.1.9.1.0`. Start arweave-server again with the same arguments as before. **Support** For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us on team@arweave.org.
Release N.1.9.0.0
Release N1.9.0.0 is out!
The release comes with a hard fork which activates at height 315700, approximately 2019-11-04 at 09:00 CET.
New features in the release:
- Improved RandomX hashing performance. The performance boost is up to 300%, depending on the setup.
- Transaction database upgrade. The transaction index backend has been upgraded to use SQLite, giving a huge performance boost to ArQL/GraphQL query speeds, more efficient indexing, and more flexibility to support future development. The feature is already live on arweave.net.
- The disk space limit feature has been updated to stop writing new files on disk when the disk partition with the Arweave data folder is 98% full, (this default behavior can be tweaked via the
disk_spacecommand line argument).
Other improvements and bugfixes:
- The general memory footprint for nodes has been reduced.
- Content-Type filtering has been tightened up, fixing an issue where header splitting with certain transaction tag values was possible.
- Pooling mechanics adjustments including updated benchmarks.
- Difficulty adjustment per retarget period will now be capped, in order to further stabilize block times.
Upgrade instructions
This is a big update, so comes with a slightly more involved upgrade process.
Shut down arweave-server. To make sure the shutdown was successful, run ps aux | grep beam | grep -v grep. The output should be empty.
Run git fetch --all --tags && git checkout N.1.9.0.0 && git reset --hard N.1.9.0.0
Now, we can update the dependencies and prepare your machine.
1. Erlang 21
Support for legacy Erlang 20 has been removed, Erlang 21 is now the minimum supported Erlang target. To check your current version, run erl -v, if you're running less than 21, run the following steps.
To upgrade Erlang on Ubuntu 18.04:
wget http://www.erlang.org/download/otp_src_21.3.tar.gz; tar -zvxf otp_src_21.3.tar.gz; cd otp_src_21.3; export ERL_TOP=`pwd`; ./configure --without-wx && make -j4 && sudo make install
2. SQLite
To install the SQLite dependency on Ubuntu 18.04, run the following:
sudo apt-get install libsqlite3-dev
3. Restart the node
Start arweave-server again with the same arguments as before, on the first run a database migration process will start, this process doesn't need to be supervised and can take an hour or so, depending on the specs of your machine. A ar_sqlite3: populated_db log message will be emitted when the upgrade is complete.
Your node will function as normal during this migration process with the exception of the /arql endpoint, which will return a 503 status code.
The install.sh script has been and the mining guide notes have been updated to reflect all of these changes.
Support
For more details on how to run the node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us on team@arweave.org.