Replies: 18 comments 32 replies
-
|
Bootloader and Programmer benchmark results -- using This will be the preferred method moving forward. To be updated. The following is with avrdude MSVC64 binary, Arduino Uno Rev 3 clone using ATmega16U2 as the USB to Serial chip (fastest USB to Serial chip so far).
|
Beta Was this translation helpful? Give feedback.
-
|
Bootloader benchmark results using the older methods -- this may not show the full picture and this method will be deprecated To be updated
Some benchmark results of urboot and @flyandancexo's optimized AVR109 bootloader FDxboot (no source codes available yet). Summary of results:
Note: please set FT232RL adapter latency timer to 1ms (Windows and Linux) before carrying out the benchmark test. ATmega328P, avrdude 7.3 release msvc64 binary urboot 1M baud rate (CH340) urboot 1M baud rate (Uno ATmega16U2) urboot 2M baud rate (Uno ATmega16U2) urboot 2M baud rate (Nano FT232RL) FDxboot optimized avr109 at 1M baud rate (1.61beta) (CH340) FDxboot optimized avr109 at 2M baud rate (1.62beta stable) (CH340) FDxboot optimized avr109 at 2M baud rate (1.62beta fast) (Uno ATmega16U2) FDxboot optimized avr109 at 2M baud rate (1.62beta fast) (Nano FT232RL) STK500v1 (from urboot) at 1M baud rate (CH340) More test results with u7.7 -eu-jPrac urboot hex file. urboot 1M baud rate (CH340), with chip erase (not supporting 2M baud) urboot 1M baud rate (FT232RL), with chip erase urboot 1M baud rate (ATmega16U2), with chip erase urboot 2M baud rate (FT232RL), with chip erase urboot 2M baud rate (ATmega16U2), with chip erase @flyandancexo's results are a bit better for the CH340, probably different CH340x adapter and different computer.
FDxboot 1.62@2Mbps with CH340G: Atmega16: Atmega128A: Programmers results summary. To be updated.
HW design info and firmware binaries here, no firmware source codes available as of now Speed info from the above repo: |
Beta Was this translation helpful? Give feedback.
-
|
Note that as of today FDxboot is unpublished and under development |
Beta Was this translation helpful? Give feedback.
-
|
If you got some time, please help on the following. Thanks.
|
Beta Was this translation helpful? Give feedback.
-
|
Dr Azy has some interesting data for serialupdi here for modern AVRs.
https://github.com/SpenceKonde/AVR-Guidance/blob/master/UPDI/jtag2updi.md megaTinyCore At 921600, only the CH340 worked anywhere without the delay. This should only be used when the voltage on the target during programming is 4.5V or more. DxCore |
Beta Was this translation helpful? Give feedback.
-
|
From @Ho-Ro
My timing shoot-outtiny85 - stk500v1tiny85 - scratchmonkeymega328p - stk500v1mega328p - scratchmonkeyFazit:
I will touch this function at some point (priority C) :)
I had no peace before I also optimised the writing of the EEPROM in my nanoSTK_V1 - now I write the EEPROM data page by page instead of single bytes
|
Beta Was this translation helpful? Give feedback.
-
|
I compared three programmers last year using ATmega328P as the target with a 32KB hex file, in Nov 2022.
|
Beta Was this translation helpful? Give feedback.
-
|
USBasp (based on ATmega88, firmware from https://github.com/dioannidis/usbasp.git) speed when programming
|
Beta Was this translation helpful? Give feedback.
-
|
Note: the so-called auto SCK is a fake. What it really does is pick a default fixed SCK frequency. The -B option is probably a way to change SCK based on a preset defined SCK frequency. Usbasp has a maximum defined 1.5 MHz SCK. With a 12MHz CPU clock, the maximum SCK is 6MHz but this will not work on a target MCU with only 16MHz; Max SCK <= target_CPU_clock/4; but next step 3MHz is still usable with target MCU clocked in at 16MHz. In short, when testing programmer SPI interface, make sure to use a board that has a 16MHz crystal oscillator on it or 20MHz if you can swap the crystal. Here is a test for a USBasp bootloader. The USBasp is built on top of Vusb, a software simulated USB controller, but its speed is acceptable. The USBasp has 2 jobs: Read/Write to the host; Read/write to the target MCU, so a bootloader is always faster than a programmer.
|
Beta Was this translation helpful? Give feedback.
-
|
Indeed AVR chip tested: ATmega2560, page size 256
click for detail run log |
Beta Was this translation helpful? Give feedback.
-
|
avrdue 7.3 Windows MSVC64 binary, AVRISP mkII clone.
|
Beta Was this translation helpful? Give feedback.
-
|
avrdude 7.3 Windows MSVC64 binary, PICkit 4 JTAG (ATmega32A).
|
Beta Was this translation helpful? Give feedback.
-
|
avrdude 7.3 Windows MSVC64 binary, STK500v2 programmer clone (CH340, ATmega8535 CPU, official firmware)
|
Beta Was this translation helpful? Give feedback.
-
|
From #1106, results from @MCUdude using STK600: reading the ATmega2560 in 12.8s. This beats my AVRISP mkii result of 14.52s. @MCUdude thinks that the reason is probably because of the use of a USB AVR in STK600 vs the use of an non-USB AVR plus a USB controller in AVRISP mkii. |
Beta Was this translation helpful? Give feedback.
-
|
The following page is a good summary of AVR programmers (in German but Google Translate seem to work well). It has some benchmarks as well with avrdude 5.5. The test file was 29704 bytes in size. Target is an ATmega6490 clocked at 8MHz by the RC oscillator. All this was tested with an AVRDUDE 5.5. |
Beta Was this translation helpful? Give feedback.
-
|
@flyandancexo has posted some good results with his AVR910 compatible programmer. HW design info and firmware binaries here, no firmware source codes available as of now Speed info from the above repo: |
Beta Was this translation helpful? Give feedback.
-
|
The fastest I see so far is from FTDI FT232H/FTDI2232H JTAG (8MHz): 128KB Flash reading at 0.40 seconds. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
It is good that you provide the HW info about your AVR910 compatible programmer in https://github.com/flyandancexo/FDxICSP. On the other hand, you only provide the binary of the firmware but not the source codes. avrdude project tends to support two types of programmers or bootloaders.
It probably does not matter too much for your AVR910 programmer as long as you make your firmware compatible with avrdude's AVR910 codes. Same for your FDxboot AVR109 compatible bootloader. However, we will not help much in case there are problems since the firmware source codes are not available. You can of course maintain your own fork of avrdude as well if you prefer to do that. |
Beta Was this translation helpful? Give feedback.
-
I am thinking that avrdude can do that. Maybe this is a worthy improment. Something like the following. Current output.
|
Beta Was this translation helpful? Give feedback.
-
|
@mcuee TLDR; I argue we should look at typical practical tasks for benchmarking. The suggestion above to add something like So, a suite of tasks takes 10.81 seconds in this example. I would always advise to verify written memories, so a more useful benchmark is the 2.008 s it took to write/verify our standard test file Fun fact: when you show in above table that urboot 1Mbit/s CH340 has an ATmega328P flash writing speed of 10738 Bytes/s this is only true for urboot bootloaders that provide a writepage function (
|
Beta Was this translation helpful? Give feedback.
-
|
Guess we do do arguing here after all? But I absolutely will not argue with anyone. That is my policy. Instead of this Easily change to this, and let the user interpret it however she or he wants. Adding a speed to fuse write is meaningless, but flash and EEPROM write and read is still useful and more modern. Again, I am not arguing. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the insight. Yes in principal I agree with you. And good tip for the urboot firmware. I usually use the following firmware for my Arduino Uno clones (or the similar one with ATmega328PB). I am involved in multiple Open Source projects and benchmark can be important sometimes. So yes it is important to present the correct data to the users. My idea mainly came from my older experiences from OpenOCD projects. I was quite involved in OpenOCD last time (2009-2013) as I was also heavily involved in libusb and libftdi. There were a lot discussions of libftdi vs FTDI D2XX due to perceived licensing issues with D2XX. In the end, the developers used libusb directly to replace libftdi to achieve faster results. Relevant discussions:
You can see outputs like this in the discussions (FT2232H based JTAG debugger) Another more recent example from libusbK project. |
Beta Was this translation helpful? Give feedback.
-
I agree with you that we need to consider the overheads like comms establishment and chip erasing as well as disengaging the chip. Instead of using external command like |
Beta Was this translation helpful? Give feedback.
-
Thanks for coming out with I still think it is worth explore the idea mentioned above inside avrdude.
For example, we can use the option For a typical CLI |
Beta Was this translation helpful? Give feedback.
-
This is what the environment var TIMEFORMAT is there for: |
Beta Was this translation helpful? Give feedback.
-
Indeed this is much faster. Previous results: New results: |
Beta Was this translation helpful? Give feedback.
-
|
This is fake fact for sure. 1, a bootloader can't issue a chip erase command. It erases each page either one by one, or in a batch, thus the chip erase is batch page erase, and it will add to the overall programming time, and for an atmega128, this erase time is significant, or about 2 seconds just to erase all pages, so I have figured out a way to scan all pages, and only erase the page that has content in it and I called it smart page eraser. 2, the page erase is only erasing the page before writing to it, so the overall time is same, but the output result appears faster because page erase takes about the same time as writing a page. Don't get fooled by @stefanrueger. Can you confirm if what I have pointed out is correct? The analogy for this is cook and eat. It takes you 1 minute to cook a meal in a microwave, and 1 minute to eat it. If you want to eat 5 meals at the same time, it would take you 5*2=10 minutes to eat 5 meals, but if you cook 5 meals first, it would take you 5 minutes to eat 5 meals, and it appears faster, but the actual time is the same. My bootloader do an all page erase first, so urboot is not too much behind. |
Beta Was this translation helpful? Give feedback.
-
|
urboot source codes are here, with clear documentation of the codes and the protocol. I admit I have not looking into the codes myself and only tested the bootloader and it works wonderfully. Please remember I do not really use AVR or knows very deep about AVR chips, I do mostly black-box testing. I do the same black box testing for projects I am involved (like libusb, hidapi, libusb-win32, libusbK, pyusb, libftdi, OpenOCD, libusbdotnet, etc). |
Beta Was this translation helpful? Give feedback.
-
|
More test results with this
New results: urboot 1M baud rate (FT232RL), with chip erase urboot 1M baud rate (ATmega16U2), with chip erase urboot 2M baud rate (FT232RL), with chip erase urboot 2M baud rate (ATmega16U2), with chip erase Run log: ATmega328P Uno Clone with FT232RLEven faster results with ATmega16U2 USB to Serial chip (original Arduino Uno Rev 3 design) ATmega328P Uno Clone with ATmega16U2 |
Beta Was this translation helpful? Give feedback.
-
Check out PR #1709 |
Beta Was this translation helpful? Give feedback.
-
|
PR #1709 seems to be a good enhancement. On the other hand, from avrdude itself (not the test-avrdude script), I think we may want to indicate the time of another potential big ticket item -- chip erase, since it can take quite a bit of time especially for bootloader. Right now we only print out the time for Flash/EEPROM write and read. As you mentions in PR #1709.
|
Beta Was this translation helpful? Give feedback.
-
Yes, it's a couple of seconds on medium-sized parts and will be more with large parts, eg, m2560. There is no way for avrdude to tell how long this might be. Issuing the CE command is an atomic process unlike writing flash, where you know how much you want to write and how far you have come. So, progress report isn't really possible for CE but eminently possible for writing to a memory (and makes sense, too, as some programmers take minutes to do things and you want the user to be in the picture). |
Beta Was this translation helpful? Give feedback.
-
|
We do not need to have the progress report for chip erase, I think just need to print the full time elapsed for chip erase. |
Beta Was this translation helpful? Give feedback.
-
|
@mcuee BTW, I get it that different projects use read/write speeds, eg, Ethernet, USB or communication chips. With microchip programming the typical user task is to upload & verify a sketch, upload and verify data to EEPROM or to backup a memory. Singling out memory write speed isn't helpful: for example, bootloaders in particular can decide whether to erase a page as part of the page write process (Optiboot does this) or they can decide to erase the flash (except obvs the bootloader) when avrdude issues a chip erase command before writing to flash. So, although the overall time is the same, in one example the write speed looks smaller than in the other approach. The other problem is that write time reported by avrdude is vvv small when the driver uses a cache (and the write speed would be phenomenal), but flushing the cache before closing the driver does all the hard work. So, all things considered, I have come to the conclusion that more often than not is is unhelpful or outright misleading to report memory write speed. The other bit with bootloaders is that there is, as virtually always, a trade-off between code space saving and time saving. The optiboot and urboot projects are firmly in the first corner, urboot actually more so than optiboot 😉. These projects know about tricks to save time, eg, the previously mentioned "smart" page erase or receiving data while erasing a page, but decided to save the needed code for more interesting features and/or leave flash for the application. See, for example, this comment by Bill Westfield a decade ago or so. |
Beta Was this translation helpful? Give feedback.
-
|
Actually OpenoCD is quite similar to avrdude when used not for debugging but rather flash writing/reading. No bootloader there though (JTAG/SWD). You can of course say that ARM MCUs can have larger memory but the JTAG/SWD programmers are typically much faster than AVR programmers. So in the end the timing may be comparable between an AVR with 64KB Flash and an ARM Cortex MCU with 512KB or even 1MB Flash (I have not done benchmarking for OpenOCD for a while though, so this may not be totally accurate). I understand your concerns about bootloader design trade-off, but we have less argument about programmers, where you have less trade-off about code size. |
Beta Was this translation helpful? Give feedback.
-
|
I have also added a note in the first post. |
Beta Was this translation helpful? Give feedback.
-
|
PR #1709
The reported times are realistic times with overhead of starting avrdude, resetting the board via DTR/RTS, establishing comms, including erasing the flash before writing/verifying the sketch and disengaging the chip. Once PR #1709 is merged, I will adopt it as well for benchmarking. Example test results under Linux, with ATmega328P (ATmega16U2 USB to Serial chip) and urboot. |
Beta Was this translation helpful? Give feedback.
-
|
I tried your FDxboot 1.62beta version (stable and fast) version, unfortunately EEPROM writing failed. You may want to investigate. Same for the 1.61beta 1M baud version. |
Beta Was this translation helpful? Give feedback.
-
|
Since FDxboot beta does not work with EEPROM, the following comparison is only for Flash and Chip Erase.
|
Beta Was this translation helpful? Give feedback.
-
|
It's called beta version not for no reason. EEPROM code has not been written for it yet. Testing EEPROM speed is probably not as important: 1, it's rarely written to; 2, it's more slow than flash. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the explanation. I agree EEPROM writing speed is less important. Still the bootloader needs to get it right to write/read EEPROM. Hopefully you will appreciate the new benchmark methodology with It is possible to skip EEPROM test with |
Beta Was this translation helpful? Give feedback.
-
|
Testing is relative to see improvement, or compare one to other, so it's up to you how to do the comparison. For me improving my bootloader, I only use byte-per-second because it's easier for me to see improvement. |
Beta Was this translation helpful? Give feedback.
-
|
The lack of response from @stefanrueger either means you don't want to argue with me or you admit the fact that there is no chip erase on a bootloader. Technically, you can call anything anything, but chip erase should only reserved for a programmer, and it's absolutely not about whom is right or wrong, but about not to spread false and dumb info, and make people confused with the info you have provided. By definition, chip erase erases the whole chip, including the EEPROM unless the fuse is programmed to preserve the EEPROM, and its lock fuse; so a bootloader that does chip erase would delete itself, and it's a very wrong term. Instead, it should be called: App-erase, app-clear or anything else of your choosing, but definitely not chip erase. I don't need to read your code to know how it works. It could be a bug not to do a full app section erase because for example, if previous upload is 5k, and now a 4k hex file is uploaded to the MCU. The 1k residue from previous upload is not cleaned, and is still in the chip. While normally this is not an issue, In a rarely case, either a bug in the code or a memory leak, the MCU could jump into the 1k residue and struck there forever. That means not cleaning all previous upload is considered a bug, if that is the case for your bootloader. So an "chip erase" or app-erase should be mandatory because it's the correct way. You don't need to erase every application section pages with the app-erase command, a smarter approach to that problem would be 1, do a quick scan on pages with data, and determine what pages are to be erased; 2, then erase those page with content in it; I have yet to perfect the algorithm to do that correctly, so my bootloader is still under development. You want to do read before erase, because reading is faster than erasing, so here is about speed efficiency, but of course it's totally okay to ignore this new feature if size efficiency of your bootloader is your main concern. |
Beta Was this translation helpful? Give feedback.
-
|
@stefanrueger mentioned in PR #1709.
|
Beta Was this translation helpful? Give feedback.
-
|
The key for good software is consistence. You don't want to call 2 different things with the same name. |
Beta Was this translation helpful? Give feedback.
-
|
It is called |
Beta Was this translation helpful? Give feedback.
-
|
@mcuee Yes, urprotocol can shift the burden of cleaning up to the uploader, eg,
@flyandancexo The AVRDUDE project expects comments to be civil. If something is unclear, please ask technical questions |
Beta Was this translation helpful? Give feedback.
-
|
Close this discussion and I will create a Wiki page to document new test results using |
Beta Was this translation helpful? Give feedback.
-
Discussions about Flash Read/Write benchmark, both programmers and bootloaders. EEPROM may be covered as well.
Note: speed is not the most important factor when choosing a programmer or bootloader. Stability may be more important. In terms of programmer, budget may play a part as well. In terms of bootloaders, code size can play a part as well.
This is the continuation from #1022.
Some general findings for avrdude 7.3 release (should be correct for older 7.0/7.1/7.2 release as well).
-b 2000000)-B 0.25vs-B 0.5). It may still help the reading speed.My own tests will be focusing on the programmers and bootloader recommended in the FAQ.
https://github.com/avrdudes/avrdude/wiki/FAQ
I am new to avrdude and I do not have a programmer, what are the recommended programmers to be used with avrdude?
avrdude supports many programmers. Here are some examples of low cost and well-supported programmers.
I have got the programmer working. Now I want to try using bootloader. Which bootloader should I use?
Survey of AVR bootloaders:
We recommend urboot for classic AVR chips.
For xmega, you may want to try xboot (AVR109 based)
For UPDI parts, you may want to try optiboot_x and optiboot_dx.
If you use USB AVRs, you may want to try Caterina (AVR109 based) or FLIP1/FLIP2 (Atmel extension to USB DFU).
Beta Was this translation helpful? Give feedback.
All reactions