Questions about Optiboot Flasher

[technoblogy]

I'm planning to use your Optiboot Flasher on the ATmega1284P, but there are a few things I don't understand:

• I'm not sure what you're referring to by 'buffer' in the documentation in the header of SerialReadWrite.ino. When you say:

  Buffer must be page aligned (see declaration of flash_buffer)
  

  There is no flash_buffer in the code. I assume this means the block of flash you're writing to, flashSpace[]?

• In:

  Writing to EEPROM destroys temporary buffer
  You can write only once into one location of temporary buffer
  

  By 'temporary buffer' do you mean:

  uint8_t ramBuffer[SPM_PAGESIZE];
  

  and by EEPROM do you mean flash? Why does it get destroyed by writing it to flash?

• Why might you want to do fill-erase-write rather than erase-fill-write?

• Instead of allocating the page(s) of flash in the middle of PROGMEM as you've done here, could I specify an explicit address for flashSpace?

  For example, if I understand correctly the bootloader occupies two pages from word addresses 0xFF00 to 0xFFFF. I would like to write to one page of flash just below that, from word addresses 0xFE80 to 0xFEFF. So could I do this?

  const *uint8_t flashSpace PROGMEM = 0xFE80;
  

• Is it essential to use a RAM buffer, ramBuffer[256], or if I'm short of RAM could I write the data to flash byte by byte, treating it more like an SD card? I realise I would have to provide my own alternative to optiboot_writePage().

• Finally, I'm puzzled by the code in optiboot_readPage() in optiboot.h:

  if(read_character != 0 && read_character != 255)                      
    storage_array[j] = read_character;
  

  Why is it skipping bytes if they are 0 or 255?

Sorry about the flood of questions!

[MCUdude]

Hi!

I wrote this crude sketch five years ago, so a lot has happened to my coding skills since since then. I haven't really used this functionality myself either, but I'll try to answer your questions as good as I can. Note that this sketch relies on optiboot.h, which is mostly written by @majekw. This is the sketch](https://github.com/majekw/optiboot/blob/spm/optiboot/examples/flash_program/flash_program.ino) SerialReadWrite.ino is based on. As you can see in this sketch, here's where I've copied all the "important things to know" text.

I'm not sure what you're referring to by 'buffer' in the documentation in the header of SerialReadWrite.ino. When you say:
Buffer must be page aligned (see declaration of flash_buffer)
There is no flash_buffer in the code. I assume this means the block of flash you're writing to, flashSpace[]?

Yes, I'm referring to flashSpace[] here.

In:
Writing to EEPROM destroys temporary buffer
You can write only once into one location of temporary buffer
By 'temporary buffer' do you mean:
uint8_t ramBuffer[SPM_PAGESIZE];
and by EEPROM do you mean flash? Why does it get destroyed by writing it to flash?

When looking at the code now, there is no reason why the ramBuffer should be destroyed when writing to EEPROM. ramBuffer is just an array living in RAM, and should not be affected. But it would be great if you could try this just to check if the statement from majekw still is true (for some reason).

Why might you want to do fill-erase-write rather than erase-fill-write?

I am doing erase-fill-write. See code here.

Instead of allocating the page(s) of flash in the middle of PROGMEM as you've done here, could I specify an explicit address for flashSpace? For example, if I understand correctly the bootloader occupies two pages from word addresses 0xFF00 to 0xFFFF. I would like to write to one page of flash just below that, from word addresses 0xFE80 to 0xFEFF. So could I do this?

I've not tried this myself, and I don't have a good answer to your question. However, you can always try this and use Avrdude to dump the flash and look at its content afterward to see if it did work? Perhaps maybe @majekw can give us a straight answer? 🙂

Is it essential to use a RAM buffer, ramBuffer[256], or if I'm short of RAM could I write the data to flash byte by byte, treating it more like an SD card? I realize I would have to provide my own alternative to optiboot_writePage().

As far as I know, you can only write a full page to the flash memory at once. You could of course create a wrapper that treats each page as a byte array, but you'll always have to "write" all the bytes, even though only one has changed.

Finally, I'm puzzled by the code in optiboot_readPage() in optiboot.h:
...
Why is it skipping bytes if they are 0 or 255?

The code is based on the read routine from the original sketch. It may be that 0x00 or 0xff represents an empty space? But I do agree that 0 and 255 should be read out as well. If not, it's pretty much useless for anything else than characters.

I can change the optiboot.h and the SerialReadWrite sketch based on what you figure out what works and what don't. Looking forward to seeing what you'll come up with! Is it something that you'll write about on your blog?

[technoblogy]

Thanks for the answers! I'll let you know what I find out after I've done some experiments.

The application I've got in mind is to improve my version of uLisp for the ATmega1284P. Currently it allows you to save the workspace from RAM to the EEPROM on the chip, which limits you to saving 4Kbytes of the 16Kbytes. By saving it to flash instead I could save the entire RAM.

[JAndrassy]

I use optiboot.h in the ArduinoOTA library to store the uploaded sketch binary in upper half of the flash memory. I don't use a memory buffer. The MCU has a buffer for the page.
https://github.com/jandrassy/ArduinoOTA/blob/master/src/InternalStorageAVR.cpp

btw: fill-erase-flash is only possible in last 4 flash pages to support advanced bootloader burning

[technoblogy]

@JAndrassy Thanks, I'll take a look at your code too.

[majekw]

Lot of questions :-)

1. flash_buffer is unfortunate name which conflicts with AVR's flash buffer used to store temporary flash page before writing it. 'flash_buffer' variable is just memory in flash where we want to write. That's also a reason for page aligning, because in fact we must write whole pages.
2. Flash vs. EEPROM note: during writing we must write to internal AVR flash buffer then 'commit' write to real flash location. This internal flash buffer could be erased during operations on EEPROM. It's somehow connected in chip and this limitation is described in AVR documentation.
3. fill-erase-write vs. erase-fill-write: when you want to preserve some contents of erased page, you don't need RAM to store old contents of erased page in first case. Preparing contents of page in RAM and write all in one go is just more convenient and easier to implement, so second way is more common. There also could be some magic with NRWW/RWW sections, but I don't remember all these quirks right now. For better understanding I recommend to read chip documentation about bootloader, NRWW, RWW sections and self programming.
4. regarding 'if' with 0 and 255: it was just to output some reasonable characters to serial console, so I was replacing them to dots :-) No other magic here :-)

[technoblogy]

OK, I've got it to work! Here is some feedback. This generally relates to the ATmega1284P and may need changing for the smaller chips.

• I couldn't get anything to work at first until I remembered that my ATmega1284P board had an old bootloader on it from Jack Christensen's Mighty 1284P core. After updating the bootloader from your MightyCore everything started to work!

  So I think it might be useful to provide a function like the one in Spence Konde's DxCore called Flash.checkWritable() which checks for obvious things. I'm not sure how to write this.

• I got very confused about whether I was working with byte addresses or word addresses. It doesn't help that the memory map in the Atmel datasheet shows the top of memory on an ATmega1284P as 0xFFFF. It would be worth saying that all the addresses used by the Optiboot Flash routines are byte addresses.

• The demo program SerialReadWrite.ino makes it appear that you have to allocate your own 256-byte RAM buffer, but this isn't necessary as there's a temporary buffer in the ATmega1284P that you can fill on the fly with optiboot_page_fill().

• I don't think the function optiboot_readPage() should be included in optiboot.h as it's more of a demo function, and it wouldn't be useful for reading real data as it ignores 0x00 and 0xFF bytes. Also it wouldn't work in the upper half of flash; you need to use pgm_read_byte_far() for that. It's not really needed anyway as you can just use pgm_read_byte() or pgm_read_byte_far().

• In the documentation there's a bit of confusion over the terminology. I suggest:

  Flash buffer: the temporary page in flash filled by optiboot_page_fill().
  Flash area: the page(s) of flash you are writing to (which must be page aligned). Don't call it a buffer (it's not temporary).
  RAM area: the area in RAM you're transferring to flash, although I suggest dropping this concept.

Finally, here are my demo programs which I've tried to keep as simple as possible.

Writing to an explicit page in far flash

/*
  Test Optiboot Flash on ATmega1284P - explicit page
*/

#include "optiboot.h"

// Bootloader takes from 0x1fc00 to 0x1ffff (byte addresses).
// Write in page immediately below that
uint32_t BaseAddress = 0x1fb00;

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.println("Erase");
  optiboot_page_erase(BaseAddress);
  
  Serial.println("Write data");
  uint32_t addr = BaseAddress;

  // Fill flash buffer. Page is 128 words
  for (unsigned int i=0; i<128; i++) {
    optiboot_page_fill(addr, i); addr++; addr++;
  }

  // Write flash buffer to flash area
  optiboot_page_write(BaseAddress);
  
  Serial.println("Read data");
  addr = BaseAddress;  
  for (unsigned int i=0; i<128; i++) {
    Serial.print(pgm_read_word_far(addr));
    addr++; addr++; Serial.print(' ');
  }
  Serial.println();
  
  delay(20000);
}

Writing to a page allocated in PROGMEM

/*
  Test Optiboot Flash on ATmega1284P - allocate page in PROGMEM
*/

#include "optiboot.h"

// Allocate one page of 256 bytes
const uint8_t flashSpace[256] __attribute__ (( aligned(256) )) PROGMEM = { };
uint32_t BaseAddress = (uint32_t)flashSpace;

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.println("Erase");
  optiboot_page_erase(BaseAddress);
  
  Serial.println("Write data");
  uint32_t addr = BaseAddress;

  // Fill flash buffer. Page is 128 words
  for (unsigned int i=0; i<128; i++) {
    optiboot_page_fill(addr, i); addr++; addr++;
  }

  // Write flash buffer to flash area
  optiboot_page_write(BaseAddress);
  
  Serial.println("Read data");
  addr = BaseAddress;  
  for (unsigned int i=0; i<128; i++) {
    Serial.print(pgm_read_word(addr));
    addr++; addr++; Serial.print(' ');
  }
  Serial.println();
  
  delay(20000);
}

[technoblogy]

I forgot to say: thanks for providing this really useful feature!

[MCUdude]

@technoblogy thanks for the feedback! I'll go through it later and apply some of the improvements you've suggested.

BTW you can write to the upper part of flash by using the PROGMEM1 attribute:

const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
  "This some default content stored on a page far away"
};

EDIT: Not, that this doesn't work for some reason. Haven't had the time to investigate why. I did change pgm_read_byte_far to pgm_read_byte_far in optiboot_readPage(), but still no success. For reference, PROGMEM1 is defined here.

[technoblogy]

Here's another example, which is what I'm using in uLisp. You simply keep calling FlashWriteInt() to write values to the flash (up to 16Kbytes in this example), and the only other thing you have to do is call FlashEndWrite() at the end. It takes care of the correct sequence of optiboot_page_erase, optiboot_page_fill, and optiboot_page_write calls for you:

const uint32_t BaseAddress = 0x1bc00;

void FlashWriteInt (uint32_t *addr, int data) {
  if (((*addr) & 0xFF) == 0) optiboot_page_erase(BaseAddress + ((*addr) & 0xFF00));
  optiboot_page_fill(BaseAddress + *addr, data);
  if (((*addr) & 0xFF) == 0xFE) optiboot_page_write(BaseAddress + ((*addr) & 0xFF00));
  (*addr)++; (*addr)++;
}

void FlashEndWrite (uint32_t *addr) {
  if (((*addr) & 0xFF) != 0x00) optiboot_page_write((BaseAddress + ((*addr) & 0xFF00)));
}
 
void loop() {
  uint32_t addr = 0;

  for (int word = 0; word<500; word++) FlashWriteInt(&addr, word);
  
  FlashEndWrite(&addr);
  
  for(;;);
}

[MCUdude]

Is there a "proper way" to read content that's stored in "far mem" (64-128kB)? I've tried to modify the optiboot_readPage function, but it seems like the pgm_get_far_address macro requires its parameter to be known at compile time. I'm getting this error:

/var/folders/6l/ypg6qbw172v1s4vtt6g990tw0000gn/T/arduino_build_544907/sketch/SerialReadWrite.ino.cpp.o: In function `optiboot_readPage(unsigned char const*, unsigned char*, unsigned int, char)':
/Users/hans/Documents/Arduino/hardware/MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.h:137: undefined reference to `r30'
/Users/hans/Documents/Arduino/hardware/MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.h:137: undefined reference to `r30'
/Users/hans/Documents/Arduino/hardware/MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.h:137: undefined reference to `r30'
collect2: error: ld returned 1 exit status
Using library Optiboot_flasher at version 1.0

// Function to read a flash page and store it in an array (storage_array[])
void optiboot_readPage(const uint8_t allocated_flash_space[], uint8_t storage_array[], uint16_t page, char blank_character)
{
  uint8_t read_character;
  for(uint16_t j = 0; j < SPM_PAGESIZE; j++) 
  {
    
    read_character = pgm_read_byte_far(pgm_get_far_address(allocated_flash_space) + j + SPM_PAGESIZE*(page-1));
    if(read_character != 0 && read_character != 255)
      storage_array[j] = read_character; 
    else
      storage_array[j] = blank_character;   
  }
}

[JAndrassy]

RAMZ register

[MCUdude]

The idea is to see if I can create a usable example where I utilize PROGMEM1.

// This array allocates the space you'll be able to write to
static const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
  "This some default content stored on page one"
};

I want to see if there's possible to read and write to the array when it's located in the far (64kB+) memory space. So far I've been able to read from it by "manually" passing it's memory address, but it's not very elegant. When I try to write to it I either corrupt the bootloader area or the program gets permanently corrupted.

Preferably, this is what I want to acheive:

// This array allocates the space you'll be able to write to
static const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
  "This some default content stored on page one but still far away!"
};

// ...

// Write content to page 1:
optiboot_writePage(flashSpace, ramBuffer, 1);

// Read page 1 of the allocated space:
optiboot_readPage(flashSpace, ramBuffer, 1); 
// Print page content
Serial.print(F("\nContent of page 1: "));
Serial.println((char*)ramBuffer);

[JAndrassy]

maybe my copy_flash_pages function's source code can help

MightyCore/avr/bootloaders/optiboot_flash/optiboot_flash.c

Lines 1190 to 1209 in 0990937

	void copy_flash_pages(uint32_t dest_page_addr, uint32_t src_page_addr, uint16_t page_count, uint8_t reset_mcu) {
	int i, j;
	for (i = 0; i < page_count; i++) { // do standard spm steps for every page
	do_spm_rampz(dest_page_addr, __BOOT_PAGE_ERASE, 0); // erase page
	for (j = 0; j < SPM_PAGESIZE; j += 2) { // fill the bytes for the page
	#ifdef RAMPZ // only devices with RAMPZ have pgm_read_word_far()
	do_spm_rampz(dest_page_addr + j, __BOOT_PAGE_FILL, pgm_read_word_far(src_page_addr + j));
	#else
	do_spm(dest_page_addr + j, __BOOT_PAGE_FILL, pgm_read_word(src_page_addr + j));
	#endif
	}
	do_spm_rampz(dest_page_addr, __BOOT_PAGE_WRITE, 0); // write the page
	dest_page_addr += SPM_PAGESIZE;
	src_page_addr += SPM_PAGESIZE;
	}
	if (reset_mcu) {
	watchdogConfig(WATCHDOG_16MS); // for a reset of the MCU
	while (1); // to prevent return to application in the 15MS to reset
	}
	}

[MCUdude]

Preferably, this is what I want to achieve:

OK, I got it working, but I still can't get around the fact that the address to the array has to be known at compile-time and thus passed to the optiboot_readPage and optiboot_writePage functions. But this isn't really a deal-breaker. I'll continue working on some of the suggestions @technoblogy came up with! There may be breaking changes to optiboot.h, but these will be minor, and absolutely for the best (like getting rid of the 0 and 255 guards in optiboot_read_page for instance).

[MCUdude]

In the SerialReadWrite example I've just updated, you can now easily place all your data in the far memory using PROGMEM1.
All you have to do is to add pgm_get_far_address when you're reading or writing:

const uint8_t flashSpace_near[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
  "This some default content stored on page one, really near!"
};

const uint8_t flashSpace_far[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__ (( aligned(SPM_PAGESIZE) )) PROGMEM1 = {
  "This some default content stored on page one, far, far away..."
};

// write to near mem
optiboot_writePage(flashSpace_near, ramBuffer, pageNumber);

// write to far mem
optiboot_writePage(pgm_get_far_address(flashSpace_far), ramBuffer, pageNumber);

// Read from near mem
optiboot_readPage(flashSpace_near, ramBuffer, page);

// Read from far mem
optiboot_readPage(pgm_get_far_address(flashSpace_far), ramBuffer, page);

I hope these changes to optiboot.h makes it more suitable for real-world applications 🙂 Feel free to bring some feedback! I'd love to further improve it.

[technoblogy]

Personally I find this a bit confusing, and would prefer it in a comment:

"This some default content stored on page one, really near!"

Why would you want to have some default content there? Also, it implies that you're only going to be storing text.

[MCUdude]

It's intended to show what the SerialReadWrite sketch is capable of. In the sketch, you can dump the entire allocated space, and one can see where the text is. As a demo, I think it helps to illustrate what the user is working with. It doesn't make sense in a read-world application, that's for sure.

I'm planning to create other examples as well that show how other data can be handled, with and without a buffer in RAM.

Do you have any ideas on what other examples could be useful to help users utilize the flash read/write feature in their own applications?

[technoblogy]

Perhaps my example?

#210 (comment)

[MCUdude]

@technoblogy I will include a modified version of your example to show how one can write 16-bit integers straight to the internal buffer. It's very fast and light-weight, but not as user-friendly as I want it to be.

That's why I've created a wrapper library that utilizes Optiboot.h but acts very much like the Arduino EEPROM library! This means that you can easily store all sorts of things like variables, structs, strings, etc, and the library will take care of the low-level stuff. It is template-based, so the compiled size is still very small.

The compiled size without LTO for the example below is 6056 bytes for an ATmega1284P. Approx. 1.7kB is due to printing floats.

Any thoughts? I will publish the library after I've fine-tuned it even more 👍

// Flash_put_get.ino
#include <Flash.h>

struct MyObject
{
  float field1;
  uint8_t field2;
  char name[10];
};

// RAM buffer needed by the Flash library
uint8_t ram_buffer[SPM_PAGESIZE] = {0x00};

// Allocate two flash pages for storing data
#define NUMBER_OF_PAGES 2
const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};

// Flash constructor
Flash flash(flashSpace, sizeof(flashSpace), ram_buffer);

void write_data()
{
  float f = 123.456f;
  uint8_t buffer_address = 0;

  // First, make sure there are no content in out buffer
  flash.clear_buffer();
 
  // One simple call, with the address first and the object second
  flash.put(buffer_address, f);

  Serial.println(F("Written float data type!"));

  // Data to store
  MyObject customVar =
  {
    3.14f,
    65,
    "MCUdude"
  };

  // Move address to the next byte after float 'f'
  buffer_address += sizeof(float);
  flash.put(buffer_address, customVar);

  // Write buffer to the first allocated flash page (page 0)
  flash.write_page(0);

  // Now let's set a flag on another flash page to indicate that the flash memory contains content
  // Here we're treating the object as an array
  flash.clear_buffer();
  flash[5] = 'X';
  flash.write_page(1);
  
  Serial.println(F("Written custom data type!\nReset your board to view the contents!\n"));
}

void read_data()
{
  Serial.println(F("Read float from flash: "));

  // fetch first flash page
  flash.fetch_page(0);

  float f = 0.00f; // Variable to store data read from flash
  uint8_t buffer_address = 0; // Buffer address to start from

  // Get the float data from flash at position 'buffer_address'
  flash.get(buffer_address, f);
  Serial.print(F("The value of f is now: "));
  Serial.println(f, 3);

  buffer_address += sizeof(float); // Move address to the next byte after float 'f'
  MyObject customVar; // Variable to store custom object read from flash.

  flash.get(buffer_address, customVar);

  Serial.println(F("Read custom object from flash: "));
  Serial.println(customVar.field1);
  Serial.println(customVar.field2);
  Serial.println(customVar.name);
}

void setup()
{
  delay(2000);
  Serial.begin(9600);

  // Fetch flash page 1, where we may have a flag
  flash.fetch_page(1);

  // Check if our flag is present
  if(flash[5] == 'X')
  {
    Serial.println(F("Content found! Content:"));
    read_data();
  }
  else
  {
    Serial.println(F("No content found! Writing new content..."));
    write_data();
  }
}

void loop()
{
}

Serial monitor output after upload:

No content found! Writing new content...
Written float data type!
Written custom data type!
Reset your board to view the contents!

(I press the reset button)

Content found! Content:
Read float from flash: 
The value of f is now: 123.456
Read custom object from flash: 
3.14
65
MCUdude

[technoblogy]

I'm not sure why you've provided:

uint8_t ram_buffer[SPM_PAGESIZE] = {0x00};

because it suggests that you need a RAM buffer to use the Optiboot Flash, which isn't the case; a point I made in my comments at:

#210 (comment)

[MCUdude]

One does not need a RAM buffer to read/write to flash, but it makes it much more convenient to have a pool you can read and write to, just like I've shown in the example.

If you have lots of structs and variables scattered around on a page, how would you work with it if you didn't have a buffer you can dump the content into before parsing the data? You may be able to reduce the RAM usage a little, but I bet the code is not going to be as pretty.

Optiboot.h does not require a RAM buffer, and I'm going to create an example showing this
Flash.h will most likely require a RAM buffer because It's supposed to be easy to use and act much like the EEPROM library. I'd love to get rid of the buffer, but I don't really see how that could be accomplished. Is it possible to read from the internal flash buffer, so this could be used as a "pool"?

[technoblogy]

Fair enough.

Is it possible to read from the internal flash buffer, so this could be used as a "pool"?

I'm not sure.

[technoblogy]

Do you think it will be possible to implement this (from my earlier comments):

So I think it might be useful to provide a function like the one in Spence Konde's DxCore called Flash.checkWritable() which checks for obvious things. I'm not sure how to write this.

It's pretty important, because currently if you try to use Optiboot.h with the wrong bootloader it just crashes your program in a major way. It should at least check that you've got the correct bootloader.

[MCUdude]

How about this?

MightyCore/avr/libraries/Optiboot_flasher/src/Flash.cpp

Lines 51 to 86 in e12eda5

	/**
	* @brief Checks if the microcontroller contains a bootloader that has flash
	* writing capabilities. It does so by checking if a spesific number is placed
	* at the very end of the flash memory.
	*
	* @return true if compatible bootloader is present
	* @return false if incompatible or no bootloader is present
	*/
	bool Flash::check_writable()
	{
	uint8_t content = 0;
	// 256kiB flash
	#if FLASHEND == 0x3FFFF
	content = pgm_read_byte_far(0x3FFFF);
	// 128kiB flash
	#elif FLASHEND == 0x1FFFF
	content = pgm_read_byte_far(0x1FFFF);
	//64kiB flash
	#elif FLASHEND == 0xFFFF
	content = pgm_read_byte(0xFFFF);
	// 32kiB flash
	#elif FLASHEND == 0x7FFF
	content = pgm_read_byte(0x7FFF);
	// 16kiB flash
	#elif FLASHEND == 0x3FFF
	content = pgm_read_byte(0x3FFF);
	// 8kiB flash
	#elif FLASHEND == 0x1FFF
	content = pgm_read_byte(0x1FFF);
	#endif
	if(content >= 8)
	return true;
	else
	return false;
	}

[technoblogy]

Yes, that looks perfect.

The only concern I have is that, in order to use Flash::check_writable(), I have to create a Flash instance which appears to carry an overhead of 10 bytes, which I don't need. I would prefer it not to be a C++ routine.

I assume you're leaving optiboot.h and optiboot.cpp unchanged?

[MCUdude]

I was already thinking about this. See my latest commit.

MightyCore/avr/libraries/Optiboot_flasher/src/optiboot.cpp

Lines 30 to 65 in 5b1dac8

	/**
	* @brief Checks if the microcontroller contains a bootloader that has flash
	* writing capabilities. It does so by checking if a spesific number is placed
	* at the very end of the flash memory
	*
	* @return true if compatible bootloader is present
	* @return false if incompatible or no bootloader is present
	*/
	bool optiboot_check_writable()
	{
	uint8_t content = 0;
	// 256kiB flash
	#if FLASHEND == 0x3FFFF
	content = pgm_read_byte_far(0x3FFFF);
	// 128kiB flash
	#elif FLASHEND == 0x1FFFF
	content = pgm_read_byte_far(0x1FFFF);
	//64kiB flash
	#elif FLASHEND == 0xFFFF
	content = pgm_read_byte(0xFFFF);
	// 32kiB flash
	#elif FLASHEND == 0x7FFF
	content = pgm_read_byte(0x7FFF);
	// 16kiB flash
	#elif FLASHEND == 0x3FFF
	content = pgm_read_byte(0x3FFF);
	// 8kiB flash
	#elif FLASHEND == 0x1FFF
	content = pgm_read_byte(0x1FFF);
	#endif
	if(content == 8)
	return true;
	else
	return false;
	}

BTW the reason why I created optiboot.cpp was because just having a header file made it a pain to include in other files. I constantly got "multiple definitions of..." errors. I also went with cpp since a few functions are overloaded, which is not supported in C.

[technoblogy]

OK, great. Now I can call optiboot_check_writable(). I assume optiboot_page_erase(), optiboot_page_fill(), and optiboot_page_write() haven't changed?

[MCUdude]

OK, great. Now I can call optiboot_check_writable(). I assume optiboot_page_erase(), optiboot_page_fill(), and optiboot_page_write() haven't changed?

Correct! Nothing has changed in these low-level functions, apart from me adding more comments in the code.

[MCUdude]

@technoblogy this is the example I'm going to provide that's based on your example code.
It isn't identical at all, but it borrows your principles. I'm also demonstrating how to write 8-bit data to flash:

/***********************************************************************|
| Optiboot Flash read/write                                             |
|                                                                       |
| Read_write_without_buffer.ino                                         |
|                                                                       |
| A library for interfacing with Optiboot Flash's write functionality   |
| Developed in 2021 by MCUdude                                          |
| https://github.com/MCUdude/                                           |
|                                                                       |
| In this low-level example we write 16-bit values to one flash page,   |
| and 8-bit values to another page. What's different about this         |
| example is that we do this without using a RAM buffer where the       |
| contents are stored before writing to flash. Instead, the internal,   |
| temporary buffer is used. By doing this we reduce RAM usage, but it   |
| is not nearly as user friendly as using the Flash library.            |
|                                                                       |
| IMPORTANT THINGS:                                                     |
| - All flash content gets erased after each upload cycle               |
| - Allocated flash space must be page aligned (it is in this example)  |
| - Writing to EEPROM destroys temporary buffer so make sure you call   |
|   optiboot_page_write before reusing the buffer or using EEPROM       |
| - You can write only once into one location of temporary buffer       |
|***********************************************************************/


#include <optiboot.h>

// Workaround for devices that has 64kiB flash or less
#ifndef pgm_read_byte_far
#define pgm_read_byte_far pgm_read_byte_near
#endif
#ifndef pgm_get_far_address
#define pgm_get_far_address
#endif


// Allocate one flash pages for storing data. If you want to allocate space in the high progmem (>64kiB)
// You can replace PROGMEM with PROGMEM1
#define NUMBER_OF_PAGES 2
const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};


// Function for writing 16-bit integers to a flash page
void flash_write_int(uint32_t base_addr, uint16_t offset_addr, int16_t data)
{
  // Start by erasing the flash page before we start writing to the buffer
  if ((offset_addr & 0xFF) == 0)
    optiboot_page_erase(base_addr + (offset_addr & 0xFF00));
    
  // Write the 16-bit value to the buffer
  optiboot_page_fill(base_addr + offset_addr, data);
  
  // Write the buffer to flash when the buffer is full
  if ((offset_addr & 0xFF) == 0xFE)
    optiboot_page_write(base_addr + (offset_addr & 0xFF00));
}


// Function to write bytes to a flash page
void flash_write_byte(uint32_t base_addr, uint16_t offset_addr, uint8_t data)
{
  static uint8_t low_byte = 0;

  if ((offset_addr & 0xFF) == 0)
    optiboot_page_erase(base_addr + (offset_addr & 0xFF00));
 
  if (!(offset_addr & 0x01)) // Address is 2, 4, 6 etc.
    low_byte = data;
  else                       // Address is 1, 3, 5 etc.
    optiboot_page_fill(base_addr + offset_addr, (data << 8) | low_byte);

  if ((offset_addr & 0xFF) == 0xFE)
    optiboot_page_write(base_addr + (offset_addr & 0xFF00));
}


// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
  // Write the buffer to flash if there are any contents in the buffer
  if ((offset_addr & 0xFF) != 0x00)
    optiboot_page_write(base_addr + (offset_addr & 0xFF00));
}


void setup()
{
  delay(2000);
  Serial.begin(9600);
  
  static uint16_t addr = 0;

  Serial.print(F("Filling up flash page 0 with 16-bit values...\n"));
  // Fill the first flash page (page 0) with 16-bit values (0x100 to 0x01FF)
  for(uint8_t data = 0; data < (SPM_PAGESIZE / 2); data++)
  { 
    flash_write_int(pgm_get_far_address(flashSpace), addr, data + 0x0100); // Write data
    addr += 2; // Increase memory address by two since we're writing 16-bit values
  }
  // Force an end write in case it hasn't already been done in flash_write_int
  flash_end_write(pgm_get_far_address(flashSpace), --addr);


  Serial.print(F("Filling up flash page 1 with 8-bit values...\n"));
  // Fill the second flash page (page 1) with 0-bit values (0x00 to 0x0FF)
  for(uint16_t data = 0; data < SPM_PAGESIZE; data++)
  {
    addr++; // Increase memory address by one since we're writing 8-bit values
    flash_write_byte(pgm_get_far_address(flashSpace), addr, data); // Write data
  }
  // Force an end write in case it hasn't already been done in flash_write_byte
  flash_end_write(pgm_get_far_address(flashSpace), addr);


  Serial.print(F("Flash pages filled. Reading back their content.\nPage 0:\n"));
  for(uint16_t i = 0; i < SPM_PAGESIZE; i += 2)
    Serial.printf(F("Flash mem addr: 0x%05lx, content: 0x%04x\n"), pgm_get_far_address(flashSpace) + i, (pgm_read_byte_far(pgm_get_far_address(flashSpace) + i + 1) << 8) + (pgm_read_byte_far(pgm_get_far_address(flashSpace) + i)));

  Serial.println(F("\n\nPage 1:"));
  for(uint16_t i = 0; i < SPM_PAGESIZE; i++)
    Serial.printf(F("Flash mem addr: 0x%05lx, content: 0x%02x\n"), pgm_get_far_address(flashSpace) + SPM_PAGESIZE + i, pgm_read_byte_far(pgm_get_far_address(flashSpace) + SPM_PAGESIZE + i));
}


void loop()
{

}

[technoblogy]

I think worth mentioning in the comment that you must call flash_end_write() when you've finished, otherwise you may end up with the last page not written.

[MCUdude]

I think worth mentioning in the comment that you must call flash_end_write() when you've finished, otherwise you may end up with the last page not written.

Thanks! I'll point this out

[MCUdude]

For some reason, the example sketch I just posted doesn't work with devices that has a flash page size of 128 bytes or less, which is all chips with 32kiB flash or less. It does compile just fine, but the data isn't written to flash properly. I guess I have a late night ahead of me...

[JAndrassy]

For some reason, the example sketch I just posted doesn't work with devices that has a flash page size of 128 bytes or less, which is all chips with 32kiB flash or less. It does compile just fine, but the data isn't written to flash properly. I guess I have a late night ahead of me...

don't use _far_ for those devices.

[MCUdude]

don't use far for those devices.

_far_ isn't even defined for these devices, that's why I added:

#ifndef pgm_read_byte_far
#define pgm_read_byte_far pgm_read_byte_near
#endif
#ifndef pgm_get_far_address
#define pgm_get_far_address
#endif

...at the beginning of the sketch. And the code works fine on an ATmega644P, which only has "near mem", but has 256-byte flash pages.

[technoblogy]

I think it might be the tests for the first and last word in the page:

if ((offset_addr & 0xFF) == 0)

if ((offset_addr & 0xFF) == 0xFE)

These assume the page size is 0xFF. For a page size of 0x7F the second test should be:

if ((offset_addr & 0xFF) == 0x7E)

Also, the test in the flash_write_byte() function for a page size of 0xFF should be:

if ((offset_addr & 0xFF) == 0xFF)

and for a page size of 0x7F it should be:

if ((offset_addr & 0xFF) == 0x7F)

[MCUdude]

@technoblogy thanks for the input, but for some reason, I'm still getting garbage in the Serial monitor.

In the first place, I'm writing 16-bit integers starting from 0x0100 to 0x17F. The output looks like this:

Page 0:
Flash mem addr: 0x00180, content: 0x0100
Flash mem addr: 0x00182, content: 0x0302
Flash mem addr: 0x00184, content: 0x0504
Flash mem addr: 0x00186, content: 0x0706
Flash mem addr: 0x00188, content: 0x0908
Flash mem addr: 0x0018a, content: 0x0b0a
Flash mem addr: 0x0018c, content: 0x0d0c
Flash mem addr: 0x0018e, content: 0x0f0e
Flash mem addr: 0x00190, content: 0x1110
Flash mem addr: 0x00192, content: 0x1312
Flash mem addr: 0x00194, content: 0x1514
...
...
Flash mem addr: 0x001fc, content: 0x7d7c
Flash mem addr: 0x001fe, content: 0x7f7e

For the second page I'm writing 8-bit values from 0x00 to 0x7F. This is the output:

Page1:
Flash mem addr: 0x00200, content: 0x00
Flash mem addr: 0x00201, content: 0x00
Flash mem addr: 0x00202, content: 0x00
Flash mem addr: 0x00203, content: 0x00
Flash mem addr: 0x00204, content: 0x00
Flash mem addr: 0x00205, content: 0x00
Flash mem addr: 0x00206, content: 0x00
Flash mem addr: 0x00207, content: 0x00
...
...
Flash mem addr: 0x0027e, content: 0x00
Flash mem addr: 0x0027f, content: 0x00

[technoblogy]

What chip (with what page size) are you using?

[MCUdude]

This test was done on an ATmega16 with a page size of 128 bytes. But I have tried with an ATmega324P (128 bytes page size), and I'm getting the exact same output. But I have every MightyCore compatible chip in my drawer if that could help...

[technoblogy]

Are there conditions under which it does work?

[tomaskovacik]

dump whole flash with avrdude and check if it is problem with writing or reading

[MCUdude]

Are there conditions under which it does work?

I have not been able to make it work on any chip with 64 or 128 bytes page size.

dump whole flash with avrdude and check if it is problem with writing or reading

Here are the relevant parts of the flash dump:

:200160002D6269742076616C7565732E2E2E0A0000000000000000000000000000000000CF
:20018000000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F6F (Addr: 0x0180 - first page)
:2001A000202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F4F
:2001C000404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F2F
:2001E000606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F0F
:200200000000000000000000000000000000000000000000000000000000000000000000DE (Addr: 0x0200 - second page)
:200220000000000000000000000000000000000000000000000000000000000000000000BE
:2002400000000000000000000000000000000000000000000000000000000000000000009E
:2002600000000000000000000000000000000000000000000000000000000000000000007E
:20028000700511241FBECFEFD8E0DEBFCDBF11E0A0E0B1E0E2E9F2E102C005900D92A631CB

[technoblogy]

Now I look at it again I realise that my example needs changing quite a bit for page sizes other than 256.

 optiboot_page_erase(base_addr + (offset_addr & 0xFF00));

and

optiboot_page_write(base_addr + (offset_addr & 0xFF00));

should be changed to this for 128-byte pages:

 optiboot_page_erase(base_addr + (offset_addr & 0xFF80));

and:

optiboot_page_write(base_addr + (offset_addr & 0xFF80));

[MCUdude]

Well, that certainly did the trick, thank you @technoblogy! I've tested it on an ATmega8535 (64 byte page), ATmega324P (128 byte page), ATmega644P (256 byte page) and ATmega1284 (256 byte page). I can also confirm that it works excellent in high progmem too. It can easilly be tested by defining the flash space in PROGMEM1 rather than PROGMEM.

Filling up flash page 0 with 16-bit values...
Filling up flash page 1 with 8-bit values...
Flash pages filled. Reading back their content.
Page 0:
Flash mem addr: 0x00180, content: 0x0100
Flash mem addr: 0x00182, content: 0x0101
Flash mem addr: 0x00184, content: 0x0102
Flash mem addr: 0x00186, content: 0x0103
Flash mem addr: 0x00188, content: 0x0104
Flash mem addr: 0x0018a, content: 0x0105
Flash mem addr: 0x0018c, content: 0x0106
Flash mem addr: 0x0018e, content: 0x0107
Flash mem addr: 0x00190, content: 0x0108
Flash mem addr: 0x00192, content: 0x0109
Flash mem addr: 0x00194, content: 0x010a
Flash mem addr: 0x00196, content: 0x010b
Flash mem addr: 0x00198, content: 0x010c
Flash mem addr: 0x0019a, content: 0x010d
Flash mem addr: 0x0019c, content: 0x010e
Flash mem addr: 0x0019e, content: 0x010f
Flash mem addr: 0x001a0, content: 0x0110
Flash mem addr: 0x001a2, content: 0x0111
Flash mem addr: 0x001a4, content: 0x0112
Flash mem addr: 0x001a6, content: 0x0113
Flash mem addr: 0x001a8, content: 0x0114
Flash mem addr: 0x001aa, content: 0x0115
Flash mem addr: 0x001ac, content: 0x0116
Flash mem addr: 0x001ae, content: 0x0117
Flash mem addr: 0x001b0, content: 0x0118
Flash mem addr: 0x001b2, content: 0x0119
Flash mem addr: 0x001b4, content: 0x011a
Flash mem addr: 0x001b6, content: 0x011b
Flash mem addr: 0x001b8, content: 0x011c
Flash mem addr: 0x001ba, content: 0x011d
Flash mem addr: 0x001bc, content: 0x011e
Flash mem addr: 0x001be, content: 0x011f
Flash mem addr: 0x001c0, content: 0x0120
Flash mem addr: 0x001c2, content: 0x0121
Flash mem addr: 0x001c4, content: 0x0122
Flash mem addr: 0x001c6, content: 0x0123
Flash mem addr: 0x001c8, content: 0x0124
Flash mem addr: 0x001ca, content: 0x0125
Flash mem addr: 0x001cc, content: 0x0126
Flash mem addr: 0x001ce, content: 0x0127
Flash mem addr: 0x001d0, content: 0x0128
Flash mem addr: 0x001d2, content: 0x0129
Flash mem addr: 0x001d4, content: 0x012a
Flash mem addr: 0x001d6, content: 0x012b
Flash mem addr: 0x001d8, content: 0x012c
Flash mem addr: 0x001da, content: 0x012d
Flash mem addr: 0x001dc, content: 0x012e
Flash mem addr: 0x001de, content: 0x012f
Flash mem addr: 0x001e0, content: 0x0130
Flash mem addr: 0x001e2, content: 0x0131
Flash mem addr: 0x001e4, content: 0x0132
Flash mem addr: 0x001e6, content: 0x0133
Flash mem addr: 0x001e8, content: 0x0134
Flash mem addr: 0x001ea, content: 0x0135
Flash mem addr: 0x001ec, content: 0x0136
Flash mem addr: 0x001ee, content: 0x0137
Flash mem addr: 0x001f0, content: 0x0138
Flash mem addr: 0x001f2, content: 0x0139
Flash mem addr: 0x001f4, content: 0x013a
Flash mem addr: 0x001f6, content: 0x013b
Flash mem addr: 0x001f8, content: 0x013c
Flash mem addr: 0x001fa, content: 0x013d
Flash mem addr: 0x001fc, content: 0x013e
Flash mem addr: 0x001fe, content: 0x013f


Page 1:
Flash mem addr: 0x00200, content: 0x00
Flash mem addr: 0x00201, content: 0x01
Flash mem addr: 0x00202, content: 0x02
Flash mem addr: 0x00203, content: 0x03
Flash mem addr: 0x00204, content: 0x04
Flash mem addr: 0x00205, content: 0x05
Flash mem addr: 0x00206, content: 0x06
Flash mem addr: 0x00207, content: 0x07
Flash mem addr: 0x00208, content: 0x08
Flash mem addr: 0x00209, content: 0x09
Flash mem addr: 0x0020a, content: 0x0a
Flash mem addr: 0x0020b, content: 0x0b
Flash mem addr: 0x0020c, content: 0x0c
Flash mem addr: 0x0020d, content: 0x0d
Flash mem addr: 0x0020e, content: 0x0e
Flash mem addr: 0x0020f, content: 0x0f
Flash mem addr: 0x00210, content: 0x10
Flash mem addr: 0x00211, content: 0x11
Flash mem addr: 0x00212, content: 0x12
Flash mem addr: 0x00213, content: 0x13
Flash mem addr: 0x00214, content: 0x14
Flash mem addr: 0x00215, content: 0x15
Flash mem addr: 0x00216, content: 0x16
Flash mem addr: 0x00217, content: 0x17
Flash mem addr: 0x00218, content: 0x18
Flash mem addr: 0x00219, content: 0x19
Flash mem addr: 0x0021a, content: 0x1a
Flash mem addr: 0x0021b, content: 0x1b
Flash mem addr: 0x0021c, content: 0x1c
Flash mem addr: 0x0021d, content: 0x1d
Flash mem addr: 0x0021e, content: 0x1e
Flash mem addr: 0x0021f, content: 0x1f
Flash mem addr: 0x00220, content: 0x20
Flash mem addr: 0x00221, content: 0x21
Flash mem addr: 0x00222, content: 0x22
Flash mem addr: 0x00223, content: 0x23
Flash mem addr: 0x00224, content: 0x24
Flash mem addr: 0x00225, content: 0x25
Flash mem addr: 0x00226, content: 0x26
Flash mem addr: 0x00227, content: 0x27
Flash mem addr: 0x00228, content: 0x28
Flash mem addr: 0x00229, content: 0x29
Flash mem addr: 0x0022a, content: 0x2a
Flash mem addr: 0x0022b, content: 0x2b
Flash mem addr: 0x0022c, content: 0x2c
Flash mem addr: 0x0022d, content: 0x2d
Flash mem addr: 0x0022e, content: 0x2e
Flash mem addr: 0x0022f, content: 0x2f
Flash mem addr: 0x00230, content: 0x30
Flash mem addr: 0x00231, content: 0x31
Flash mem addr: 0x00232, content: 0x32
Flash mem addr: 0x00233, content: 0x33
Flash mem addr: 0x00234, content: 0x34
Flash mem addr: 0x00235, content: 0x35
Flash mem addr: 0x00236, content: 0x36
Flash mem addr: 0x00237, content: 0x37
Flash mem addr: 0x00238, content: 0x38
Flash mem addr: 0x00239, content: 0x39
Flash mem addr: 0x0023a, content: 0x3a
Flash mem addr: 0x0023b, content: 0x3b
Flash mem addr: 0x0023c, content: 0x3c
Flash mem addr: 0x0023d, content: 0x3d
Flash mem addr: 0x0023e, content: 0x3e
Flash mem addr: 0x0023f, content: 0x3f
Flash mem addr: 0x00240, content: 0x40
Flash mem addr: 0x00241, content: 0x41
Flash mem addr: 0x00242, content: 0x42
Flash mem addr: 0x00243, content: 0x43
Flash mem addr: 0x00244, content: 0x44
Flash mem addr: 0x00245, content: 0x45
Flash mem addr: 0x00246, content: 0x46
Flash mem addr: 0x00247, content: 0x47
Flash mem addr: 0x00248, content: 0x48
Flash mem addr: 0x00249, content: 0x49
Flash mem addr: 0x0024a, content: 0x4a
Flash mem addr: 0x0024b, content: 0x4b
Flash mem addr: 0x0024c, content: 0x4c
Flash mem addr: 0x0024d, content: 0x4d
Flash mem addr: 0x0024e, content: 0x4e
Flash mem addr: 0x0024f, content: 0x4f
Flash mem addr: 0x00250, content: 0x50
Flash mem addr: 0x00251, content: 0x51
Flash mem addr: 0x00252, content: 0x52
Flash mem addr: 0x00253, content: 0x53
Flash mem addr: 0x00254, content: 0x54
Flash mem addr: 0x00255, content: 0x55
Flash mem addr: 0x00256, content: 0x56
Flash mem addr: 0x00257, content: 0x57
Flash mem addr: 0x00258, content: 0x58
Flash mem addr: 0x00259, content: 0x59
Flash mem addr: 0x0025a, content: 0x5a
Flash mem addr: 0x0025b, content: 0x5b
Flash mem addr: 0x0025c, content: 0x5c
Flash mem addr: 0x0025d, content: 0x5d
Flash mem addr: 0x0025e, content: 0x5e
Flash mem addr: 0x0025f, content: 0x5f
Flash mem addr: 0x00260, content: 0x60
Flash mem addr: 0x00261, content: 0x61
Flash mem addr: 0x00262, content: 0x62
Flash mem addr: 0x00263, content: 0x63
Flash mem addr: 0x00264, content: 0x64
Flash mem addr: 0x00265, content: 0x65
Flash mem addr: 0x00266, content: 0x66
Flash mem addr: 0x00267, content: 0x67
Flash mem addr: 0x00268, content: 0x68
Flash mem addr: 0x00269, content: 0x69
Flash mem addr: 0x0026a, content: 0x6a
Flash mem addr: 0x0026b, content: 0x6b
Flash mem addr: 0x0026c, content: 0x6c
Flash mem addr: 0x0026d, content: 0x6d
Flash mem addr: 0x0026e, content: 0x6e
Flash mem addr: 0x0026f, content: 0x6f
Flash mem addr: 0x00270, content: 0x70
Flash mem addr: 0x00271, content: 0x71
Flash mem addr: 0x00272, content: 0x72
Flash mem addr: 0x00273, content: 0x73
Flash mem addr: 0x00274, content: 0x74
Flash mem addr: 0x00275, content: 0x75
Flash mem addr: 0x00276, content: 0x76
Flash mem addr: 0x00277, content: 0x77
Flash mem addr: 0x00278, content: 0x78
Flash mem addr: 0x00279, content: 0x79
Flash mem addr: 0x0027a, content: 0x7a
Flash mem addr: 0x0027b, content: 0x7b
Flash mem addr: 0x0027c, content: 0x7c
Flash mem addr: 0x0027d, content: 0x7d
Flash mem addr: 0x0027e, content: 0x7e
Flash mem addr: 0x0027f, content: 0x7f

On the other side, the code now looks more cryptic than ever. But at least these functions will work on pretty much any classic AVR without the need for modification:

// Function for writing 16-bit integers to a flash page
void flash_write_int(uint32_t base_addr, uint16_t offset_addr, int16_t data)
{
  // Start by erasing the flash page before we start writing to the buffer
  if ((offset_addr & (SPM_PAGESIZE - 1)) == 0)
    optiboot_page_erase(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
    
  // Write the 16-bit value to the buffer
  optiboot_page_fill(base_addr + offset_addr, data);
  
  // Write the buffer to flash when the buffer is full
  if ((offset_addr & 0xFF) == (SPM_PAGESIZE - 2))
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}


// Function to write bytes to a flash page
void flash_write_byte(uint32_t base_addr, uint16_t offset_addr, uint8_t data)
{
  static uint8_t low_byte = 0;

  if ((offset_addr & (SPM_PAGESIZE - 1)) == 0)
    optiboot_page_erase(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
 
  if (!(offset_addr & 0x01)) // Address is 2, 4, 6 etc.
    low_byte = data;
  else                       // Address is 1, 3, 5 etc.
    optiboot_page_fill(base_addr + offset_addr, (data << 8) | low_byte);

  if ((offset_addr & 0xFF) == (SPM_PAGESIZE - 2))
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}


// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
  // Write the buffer to flash if there are any contents in the buffer
  if ((offset_addr & 0xFF) != 0x00)
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}

[technoblogy]

Great. Glad it works, and sorry it took me a couple of attempts to work out what the problem was!

Just one thing that people should be warned about: I'm not sure what would happen if flashSpace[] happens to overlap from near memory to far memory. Do you think that would work?

[MCUdude]

Great. Glad it works, and sorry it took me a couple of attempts to work out what the problem was!

No worries. To be honest I haven't been working on this the entire evening, we've also had some delicious easter lamb tonight that took a while to prepare.

Just one thing that people should be warned about: I'm not sure what would happen if flashSpace[] happens to overlap from near memory to far memory. Do you think that would work?

I'm not sure that could even happen as long as they work with a pre-allocated space and not just an arbitrary start address. If you stick with PROGMEM you can only allocate what's left in the near mem. If you use an AVR with 256kiB flash and you're using PROGMEM1, you can still only allocate an array with 0xFFFF places.

I modified the sketch to take a memory address instead, 0xFF00, and filled two pages from this address. This means that the first page will be in low mem, and the other one in high mem. The only "real" modification I had to do apart from commenting out the allocated flash space array and creating a constant with the same name that contains the start address, I had to remove pgm_get_far_address becuase it would fetch the pointer to the memory address I provided, not the address itself.

Anyways, here's the sketch and output if you're interested:

Sketch:

#include <optiboot.h>

// Workaround for devices that has 64kiB flash or less
#ifndef pgm_read_byte_far
#define pgm_read_byte_far pgm_read_byte_near
#endif

// "disable" pgm_get_far_address
#undef pgm_get_far_address
#define pgm_get_far_address

const uint32_t flashSpace = 0xFF00;
//#define NUMBER_OF_PAGES 2
//const uint8_t flashSpace[SPM_PAGESIZE * NUMBER_OF_PAGES] __attribute__((aligned(SPM_PAGESIZE))) PROGMEM = {};


// Function for writing 16-bit integers to a flash page
void flash_write_int(uint32_t base_addr, uint16_t offset_addr, int16_t data)
{
  // Start by erasing the flash page before we start writing to the buffer
  if ((offset_addr & (SPM_PAGESIZE - 1)) == 0)
    optiboot_page_erase(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
    
  // Write the 16-bit value to the buffer
  optiboot_page_fill(base_addr + offset_addr, data);
  
  // Write the buffer to flash when the buffer is full
  if ((offset_addr & 0xFF) == (SPM_PAGESIZE - 2))
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}


// Function to write bytes to a flash page
void flash_write_byte(uint32_t base_addr, uint16_t offset_addr, uint8_t data)
{
  static uint8_t low_byte = 0;

  if ((offset_addr & (SPM_PAGESIZE - 1)) == 0)
    optiboot_page_erase(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
 
  if (!(offset_addr & 0x01)) // Address is 2, 4, 6 etc.
    low_byte = data;
  else                       // Address is 1, 3, 5 etc.
    optiboot_page_fill(base_addr + offset_addr, (data << 8) | low_byte);

  if ((offset_addr & 0xFF) == (SPM_PAGESIZE - 2))
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}


// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
  // Write the buffer to flash if there are any contents in the buffer
  if ((offset_addr & 0xFF) != 0x00)
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}


void setup()
{
  delay(2000);
  Serial.begin(9600);
  
  static uint16_t addr = 0;

  Serial.print(F("Filling up flash page 0 with 16-bit values...\n"));
  // Fill the first flash page (page 0) with 16-bit values (0x100 to 0x01FF)
  for(uint8_t data = 0; data < (SPM_PAGESIZE / 2); data++)
  { 
    flash_write_int(pgm_get_far_address(flashSpace), addr, data + 0x0100); // Write data
    addr += 2; // Increase memory address by two since we're writing 16-bit values
  }
  // Force an end write in case it hasn't already been done in flash_write_int
  flash_end_write(pgm_get_far_address(flashSpace), --addr);


  Serial.print(F("Filling up flash page 1 with 8-bit values...\n"));
  // Fill the second flash page (page 1) with 0-bit values (0x00 to 0x0FF)
  for(uint16_t data = 0; data < SPM_PAGESIZE; data++)
  {
    addr++; // Increase memory address by one since we're writing 8-bit values
    flash_write_byte(pgm_get_far_address(flashSpace), addr, data); // Write data
  }
  // Force an end write in case it hasn't already been done in flash_write_byte
  flash_end_write(pgm_get_far_address(flashSpace), addr);


  Serial.print(F("Flash pages filled. Reading back their content.\nPage 0:\n"));
  for(uint16_t i = 0; i < SPM_PAGESIZE; i += 2)
    Serial.printf(F("Flash mem addr: 0x%05lx, content: 0x%04x\n"), pgm_get_far_address(flashSpace) + i, (pgm_read_byte_far(pgm_get_far_address(flashSpace) + i + 1) << 8) + (pgm_read_byte_far(pgm_get_far_address(flashSpace) + i)));

  Serial.println(F("\n\nPage 1:"));
  for(uint16_t i = 0; i < SPM_PAGESIZE; i++)
    Serial.printf(F("Flash mem addr: 0x%05lx, content: 0x%02x\n"), pgm_get_far_address(flashSpace) + SPM_PAGESIZE + i, pgm_read_byte_far(pgm_get_far_address(flashSpace) + SPM_PAGESIZE + i));
}


void loop()
{

}

Output:

Filling up flash page 0 with 16-bit values...
Filling up flash page 1 with 8-bit values...
Flash pages filled. Reading back their content.
Page 0:
Flash mem addr: 0x0ff00, content: 0x0100
Flash mem addr: 0x0ff02, content: 0x0101
Flash mem addr: 0x0ff04, content: 0x0102
Flash mem addr: 0x0ff06, content: 0x0103
Flash mem addr: 0x0ff08, content: 0x0104
Flash mem addr: 0x0ff0a, content: 0x0105
Flash mem addr: 0x0ff0c, content: 0x0106
Flash mem addr: 0x0ff0e, content: 0x0107
Flash mem addr: 0x0ff10, content: 0x0108
Flash mem addr: 0x0ff12, content: 0x0109
Flash mem addr: 0x0ff14, content: 0x010a
Flash mem addr: 0x0ff16, content: 0x010b
Flash mem addr: 0x0ff18, content: 0x010c
Flash mem addr: 0x0ff1a, content: 0x010d
Flash mem addr: 0x0ff1c, content: 0x010e
Flash mem addr: 0x0ff1e, content: 0x010f
Flash mem addr: 0x0ff20, content: 0x0110
Flash mem addr: 0x0ff22, content: 0x0111
Flash mem addr: 0x0ff24, content: 0x0112
Flash mem addr: 0x0ff26, content: 0x0113
Flash mem addr: 0x0ff28, content: 0x0114
Flash mem addr: 0x0ff2a, content: 0x0115
Flash mem addr: 0x0ff2c, content: 0x0116
Flash mem addr: 0x0ff2e, content: 0x0117
Flash mem addr: 0x0ff30, content: 0x0118
Flash mem addr: 0x0ff32, content: 0x0119
Flash mem addr: 0x0ff34, content: 0x011a
Flash mem addr: 0x0ff36, content: 0x011b
Flash mem addr: 0x0ff38, content: 0x011c
Flash mem addr: 0x0ff3a, content: 0x011d
Flash mem addr: 0x0ff3c, content: 0x011e
Flash mem addr: 0x0ff3e, content: 0x011f
Flash mem addr: 0x0ff40, content: 0x0120
Flash mem addr: 0x0ff42, content: 0x0121
Flash mem addr: 0x0ff44, content: 0x0122
Flash mem addr: 0x0ff46, content: 0x0123
Flash mem addr: 0x0ff48, content: 0x0124
Flash mem addr: 0x0ff4a, content: 0x0125
Flash mem addr: 0x0ff4c, content: 0x0126
Flash mem addr: 0x0ff4e, content: 0x0127
Flash mem addr: 0x0ff50, content: 0x0128
Flash mem addr: 0x0ff52, content: 0x0129
Flash mem addr: 0x0ff54, content: 0x012a
Flash mem addr: 0x0ff56, content: 0x012b
Flash mem addr: 0x0ff58, content: 0x012c
Flash mem addr: 0x0ff5a, content: 0x012d
Flash mem addr: 0x0ff5c, content: 0x012e
Flash mem addr: 0x0ff5e, content: 0x012f
Flash mem addr: 0x0ff60, content: 0x0130
Flash mem addr: 0x0ff62, content: 0x0131
Flash mem addr: 0x0ff64, content: 0x0132
Flash mem addr: 0x0ff66, content: 0x0133
Flash mem addr: 0x0ff68, content: 0x0134
Flash mem addr: 0x0ff6a, content: 0x0135
Flash mem addr: 0x0ff6c, content: 0x0136
Flash mem addr: 0x0ff6e, content: 0x0137
Flash mem addr: 0x0ff70, content: 0x0138
Flash mem addr: 0x0ff72, content: 0x0139
Flash mem addr: 0x0ff74, content: 0x013a
Flash mem addr: 0x0ff76, content: 0x013b
Flash mem addr: 0x0ff78, content: 0x013c
Flash mem addr: 0x0ff7a, content: 0x013d
Flash mem addr: 0x0ff7c, content: 0x013e
Flash mem addr: 0x0ff7e, content: 0x013f
Flash mem addr: 0x0ff80, content: 0x0140
Flash mem addr: 0x0ff82, content: 0x0141
Flash mem addr: 0x0ff84, content: 0x0142
Flash mem addr: 0x0ff86, content: 0x0143
Flash mem addr: 0x0ff88, content: 0x0144
Flash mem addr: 0x0ff8a, content: 0x0145
Flash mem addr: 0x0ff8c, content: 0x0146
Flash mem addr: 0x0ff8e, content: 0x0147
Flash mem addr: 0x0ff90, content: 0x0148
Flash mem addr: 0x0ff92, content: 0x0149
Flash mem addr: 0x0ff94, content: 0x014a
Flash mem addr: 0x0ff96, content: 0x014b
Flash mem addr: 0x0ff98, content: 0x014c
Flash mem addr: 0x0ff9a, content: 0x014d
Flash mem addr: 0x0ff9c, content: 0x014e
Flash mem addr: 0x0ff9e, content: 0x014f
Flash mem addr: 0x0ffa0, content: 0x0150
Flash mem addr: 0x0ffa2, content: 0x0151
Flash mem addr: 0x0ffa4, content: 0x0152
Flash mem addr: 0x0ffa6, content: 0x0153
Flash mem addr: 0x0ffa8, content: 0x0154
Flash mem addr: 0x0ffaa, content: 0x0155
Flash mem addr: 0x0ffac, content: 0x0156
Flash mem addr: 0x0ffae, content: 0x0157
Flash mem addr: 0x0ffb0, content: 0x0158
Flash mem addr: 0x0ffb2, content: 0x0159
Flash mem addr: 0x0ffb4, content: 0x015a
Flash mem addr: 0x0ffb6, content: 0x015b
Flash mem addr: 0x0ffb8, content: 0x015c
Flash mem addr: 0x0ffba, content: 0x015d
Flash mem addr: 0x0ffbc, content: 0x015e
Flash mem addr: 0x0ffbe, content: 0x015f
Flash mem addr: 0x0ffc0, content: 0x0160
Flash mem addr: 0x0ffc2, content: 0x0161
Flash mem addr: 0x0ffc4, content: 0x0162
Flash mem addr: 0x0ffc6, content: 0x0163
Flash mem addr: 0x0ffc8, content: 0x0164
Flash mem addr: 0x0ffca, content: 0x0165
Flash mem addr: 0x0ffcc, content: 0x0166
Flash mem addr: 0x0ffce, content: 0x0167
Flash mem addr: 0x0ffd0, content: 0x0168
Flash mem addr: 0x0ffd2, content: 0x0169
Flash mem addr: 0x0ffd4, content: 0x016a
Flash mem addr: 0x0ffd6, content: 0x016b
Flash mem addr: 0x0ffd8, content: 0x016c
Flash mem addr: 0x0ffda, content: 0x016d
Flash mem addr: 0x0ffdc, content: 0x016e
Flash mem addr: 0x0ffde, content: 0x016f
Flash mem addr: 0x0ffe0, content: 0x0170
Flash mem addr: 0x0ffe2, content: 0x0171
Flash mem addr: 0x0ffe4, content: 0x0172
Flash mem addr: 0x0ffe6, content: 0x0173
Flash mem addr: 0x0ffe8, content: 0x0174
Flash mem addr: 0x0ffea, content: 0x0175
Flash mem addr: 0x0ffec, content: 0x0176
Flash mem addr: 0x0ffee, content: 0x0177
Flash mem addr: 0x0fff0, content: 0x0178
Flash mem addr: 0x0fff2, content: 0x0179
Flash mem addr: 0x0fff4, content: 0x017a
Flash mem addr: 0x0fff6, content: 0x017b
Flash mem addr: 0x0fff8, content: 0x017c
Flash mem addr: 0x0fffa, content: 0x017d
Flash mem addr: 0x0fffc, content: 0x017e
Flash mem addr: 0x0fffe, content: 0x017f


Page 1:
Flash mem addr: 0x10000, content: 0x00
Flash mem addr: 0x10001, content: 0x01
Flash mem addr: 0x10002, content: 0x02
Flash mem addr: 0x10003, content: 0x03
Flash mem addr: 0x10004, content: 0x04
Flash mem addr: 0x10005, content: 0x05
Flash mem addr: 0x10006, content: 0x06
Flash mem addr: 0x10007, content: 0x07
Flash mem addr: 0x10008, content: 0x08
Flash mem addr: 0x10009, content: 0x09
Flash mem addr: 0x1000a, content: 0x0a
Flash mem addr: 0x1000b, content: 0x0b
Flash mem addr: 0x1000c, content: 0x0c
Flash mem addr: 0x1000d, content: 0x0d
Flash mem addr: 0x1000e, content: 0x0e
Flash mem addr: 0x1000f, content: 0x0f
Flash mem addr: 0x10010, content: 0x10
Flash mem addr: 0x10011, content: 0x11
Flash mem addr: 0x10012, content: 0x12
Flash mem addr: 0x10013, content: 0x13
Flash mem addr: 0x10014, content: 0x14
Flash mem addr: 0x10015, content: 0x15
Flash mem addr: 0x10016, content: 0x16
Flash mem addr: 0x10017, content: 0x17
Flash mem addr: 0x10018, content: 0x18
Flash mem addr: 0x10019, content: 0x19
Flash mem addr: 0x1001a, content: 0x1a
Flash mem addr: 0x1001b, content: 0x1b
Flash mem addr: 0x1001c, content: 0x1c
Flash mem addr: 0x1001d, content: 0x1d
Flash mem addr: 0x1001e, content: 0x1e
Flash mem addr: 0x1001f, content: 0x1f
Flash mem addr: 0x10020, content: 0x20
Flash mem addr: 0x10021, content: 0x21
Flash mem addr: 0x10022, content: 0x22
Flash mem addr: 0x10023, content: 0x23
Flash mem addr: 0x10024, content: 0x24
Flash mem addr: 0x10025, content: 0x25
Flash mem addr: 0x10026, content: 0x26
Flash mem addr: 0x10027, content: 0x27
Flash mem addr: 0x10028, content: 0x28
Flash mem addr: 0x10029, content: 0x29
Flash mem addr: 0x1002a, content: 0x2a
Flash mem addr: 0x1002b, content: 0x2b
Flash mem addr: 0x1002c, content: 0x2c
Flash mem addr: 0x1002d, content: 0x2d
Flash mem addr: 0x1002e, content: 0x2e
Flash mem addr: 0x1002f, content: 0x2f
Flash mem addr: 0x10030, content: 0x30
Flash mem addr: 0x10031, content: 0x31
Flash mem addr: 0x10032, content: 0x32
Flash mem addr: 0x10033, content: 0x33
Flash mem addr: 0x10034, content: 0x34
Flash mem addr: 0x10035, content: 0x35
Flash mem addr: 0x10036, content: 0x36
Flash mem addr: 0x10037, content: 0x37
Flash mem addr: 0x10038, content: 0x38
Flash mem addr: 0x10039, content: 0x39
Flash mem addr: 0x1003a, content: 0x3a
Flash mem addr: 0x1003b, content: 0x3b
Flash mem addr: 0x1003c, content: 0x3c
Flash mem addr: 0x1003d, content: 0x3d
Flash mem addr: 0x1003e, content: 0x3e
Flash mem addr: 0x1003f, content: 0x3f
Flash mem addr: 0x10040, content: 0x40
Flash mem addr: 0x10041, content: 0x41
Flash mem addr: 0x10042, content: 0x42
Flash mem addr: 0x10043, content: 0x43
Flash mem addr: 0x10044, content: 0x44
Flash mem addr: 0x10045, content: 0x45
Flash mem addr: 0x10046, content: 0x46
Flash mem addr: 0x10047, content: 0x47
Flash mem addr: 0x10048, content: 0x48
Flash mem addr: 0x10049, content: 0x49
Flash mem addr: 0x1004a, content: 0x4a
Flash mem addr: 0x1004b, content: 0x4b
Flash mem addr: 0x1004c, content: 0x4c
Flash mem addr: 0x1004d, content: 0x4d
Flash mem addr: 0x1004e, content: 0x4e
Flash mem addr: 0x1004f, content: 0x4f
Flash mem addr: 0x10050, content: 0x50
Flash mem addr: 0x10051, content: 0x51
Flash mem addr: 0x10052, content: 0x52
Flash mem addr: 0x10053, content: 0x53
Flash mem addr: 0x10054, content: 0x54
Flash mem addr: 0x10055, content: 0x55
Flash mem addr: 0x10056, content: 0x56
Flash mem addr: 0x10057, content: 0x57
Flash mem addr: 0x10058, content: 0x58
Flash mem addr: 0x10059, content: 0x59
Flash mem addr: 0x1005a, content: 0x5a
Flash mem addr: 0x1005b, content: 0x5b
Flash mem addr: 0x1005c, content: 0x5c
Flash mem addr: 0x1005d, content: 0x5d
Flash mem addr: 0x1005e, content: 0x5e
Flash mem addr: 0x1005f, content: 0x5f
Flash mem addr: 0x10060, content: 0x60
Flash mem addr: 0x10061, content: 0x61
Flash mem addr: 0x10062, content: 0x62
Flash mem addr: 0x10063, content: 0x63
Flash mem addr: 0x10064, content: 0x64
Flash mem addr: 0x10065, content: 0x65
Flash mem addr: 0x10066, content: 0x66
Flash mem addr: 0x10067, content: 0x67
Flash mem addr: 0x10068, content: 0x68
Flash mem addr: 0x10069, content: 0x69
Flash mem addr: 0x1006a, content: 0x6a
Flash mem addr: 0x1006b, content: 0x6b
Flash mem addr: 0x1006c, content: 0x6c
Flash mem addr: 0x1006d, content: 0x6d
Flash mem addr: 0x1006e, content: 0x6e
Flash mem addr: 0x1006f, content: 0x6f
Flash mem addr: 0x10070, content: 0x70
Flash mem addr: 0x10071, content: 0x71
Flash mem addr: 0x10072, content: 0x72
Flash mem addr: 0x10073, content: 0x73
Flash mem addr: 0x10074, content: 0x74
Flash mem addr: 0x10075, content: 0x75
Flash mem addr: 0x10076, content: 0x76
Flash mem addr: 0x10077, content: 0x77
Flash mem addr: 0x10078, content: 0x78
Flash mem addr: 0x10079, content: 0x79
Flash mem addr: 0x1007a, content: 0x7a
Flash mem addr: 0x1007b, content: 0x7b
Flash mem addr: 0x1007c, content: 0x7c
Flash mem addr: 0x1007d, content: 0x7d
Flash mem addr: 0x1007e, content: 0x7e
Flash mem addr: 0x1007f, content: 0x7f
Flash mem addr: 0x10080, content: 0x80
Flash mem addr: 0x10081, content: 0x81
Flash mem addr: 0x10082, content: 0x82
Flash mem addr: 0x10083, content: 0x83
Flash mem addr: 0x10084, content: 0x84
Flash mem addr: 0x10085, content: 0x85
Flash mem addr: 0x10086, content: 0x86
Flash mem addr: 0x10087, content: 0x87
Flash mem addr: 0x10088, content: 0x88
Flash mem addr: 0x10089, content: 0x89
Flash mem addr: 0x1008a, content: 0x8a
Flash mem addr: 0x1008b, content: 0x8b
Flash mem addr: 0x1008c, content: 0x8c
Flash mem addr: 0x1008d, content: 0x8d
Flash mem addr: 0x1008e, content: 0x8e
Flash mem addr: 0x1008f, content: 0x8f
Flash mem addr: 0x10090, content: 0x90
Flash mem addr: 0x10091, content: 0x91
Flash mem addr: 0x10092, content: 0x92
Flash mem addr: 0x10093, content: 0x93
Flash mem addr: 0x10094, content: 0x94
Flash mem addr: 0x10095, content: 0x95
Flash mem addr: 0x10096, content: 0x96
Flash mem addr: 0x10097, content: 0x97
Flash mem addr: 0x10098, content: 0x98
Flash mem addr: 0x10099, content: 0x99
Flash mem addr: 0x1009a, content: 0x9a
Flash mem addr: 0x1009b, content: 0x9b
Flash mem addr: 0x1009c, content: 0x9c
Flash mem addr: 0x1009d, content: 0x9d
Flash mem addr: 0x1009e, content: 0x9e
Flash mem addr: 0x1009f, content: 0x9f
Flash mem addr: 0x100a0, content: 0xa0
Flash mem addr: 0x100a1, content: 0xa1
Flash mem addr: 0x100a2, content: 0xa2
Flash mem addr: 0x100a3, content: 0xa3
Flash mem addr: 0x100a4, content: 0xa4
Flash mem addr: 0x100a5, content: 0xa5
Flash mem addr: 0x100a6, content: 0xa6
Flash mem addr: 0x100a7, content: 0xa7
Flash mem addr: 0x100a8, content: 0xa8
Flash mem addr: 0x100a9, content: 0xa9
Flash mem addr: 0x100aa, content: 0xaa
Flash mem addr: 0x100ab, content: 0xab
Flash mem addr: 0x100ac, content: 0xac
Flash mem addr: 0x100ad, content: 0xad
Flash mem addr: 0x100ae, content: 0xae
Flash mem addr: 0x100af, content: 0xaf
Flash mem addr: 0x100b0, content: 0xb0
Flash mem addr: 0x100b1, content: 0xb1
Flash mem addr: 0x100b2, content: 0xb2
Flash mem addr: 0x100b3, content: 0xb3
Flash mem addr: 0x100b4, content: 0xb4
Flash mem addr: 0x100b5, content: 0xb5
Flash mem addr: 0x100b6, content: 0xb6
Flash mem addr: 0x100b7, content: 0xb7
Flash mem addr: 0x100b8, content: 0xb8
Flash mem addr: 0x100b9, content: 0xb9
Flash mem addr: 0x100ba, content: 0xba
Flash mem addr: 0x100bb, content: 0xbb
Flash mem addr: 0x100bc, content: 0xbc
Flash mem addr: 0x100bd, content: 0xbd
Flash mem addr: 0x100be, content: 0xbe
Flash mem addr: 0x100bf, content: 0xbf
Flash mem addr: 0x100c0, content: 0xc0
Flash mem addr: 0x100c1, content: 0xc1
Flash mem addr: 0x100c2, content: 0xc2
Flash mem addr: 0x100c3, content: 0xc3
Flash mem addr: 0x100c4, content: 0xc4
Flash mem addr: 0x100c5, content: 0xc5
Flash mem addr: 0x100c6, content: 0xc6
Flash mem addr: 0x100c7, content: 0xc7
Flash mem addr: 0x100c8, content: 0xc8
Flash mem addr: 0x100c9, content: 0xc9
Flash mem addr: 0x100ca, content: 0xca
Flash mem addr: 0x100cb, content: 0xcb
Flash mem addr: 0x100cc, content: 0xcc
Flash mem addr: 0x100cd, content: 0xcd
Flash mem addr: 0x100ce, content: 0xce
Flash mem addr: 0x100cf, content: 0xcf
Flash mem addr: 0x100d0, content: 0xd0
Flash mem addr: 0x100d1, content: 0xd1
Flash mem addr: 0x100d2, content: 0xd2
Flash mem addr: 0x100d3, content: 0xd3
Flash mem addr: 0x100d4, content: 0xd4
Flash mem addr: 0x100d5, content: 0xd5
Flash mem addr: 0x100d6, content: 0xd6
Flash mem addr: 0x100d7, content: 0xd7
Flash mem addr: 0x100d8, content: 0xd8
Flash mem addr: 0x100d9, content: 0xd9
Flash mem addr: 0x100da, content: 0xda
Flash mem addr: 0x100db, content: 0xdb
Flash mem addr: 0x100dc, content: 0xdc
Flash mem addr: 0x100dd, content: 0xdd
Flash mem addr: 0x100de, content: 0xde
Flash mem addr: 0x100df, content: 0xdf
Flash mem addr: 0x100e0, content: 0xe0
Flash mem addr: 0x100e1, content: 0xe1
Flash mem addr: 0x100e2, content: 0xe2
Flash mem addr: 0x100e3, content: 0xe3
Flash mem addr: 0x100e4, content: 0xe4
Flash mem addr: 0x100e5, content: 0xe5
Flash mem addr: 0x100e6, content: 0xe6
Flash mem addr: 0x100e7, content: 0xe7
Flash mem addr: 0x100e8, content: 0xe8
Flash mem addr: 0x100e9, content: 0xe9
Flash mem addr: 0x100ea, content: 0xea
Flash mem addr: 0x100eb, content: 0xeb
Flash mem addr: 0x100ec, content: 0xec
Flash mem addr: 0x100ed, content: 0xed
Flash mem addr: 0x100ee, content: 0xee
Flash mem addr: 0x100ef, content: 0xef
Flash mem addr: 0x100f0, content: 0xf0
Flash mem addr: 0x100f1, content: 0xf1
Flash mem addr: 0x100f2, content: 0xf2
Flash mem addr: 0x100f3, content: 0xf3
Flash mem addr: 0x100f4, content: 0xf4
Flash mem addr: 0x100f5, content: 0xf5
Flash mem addr: 0x100f6, content: 0xf6
Flash mem addr: 0x100f7, content: 0xf7
Flash mem addr: 0x100f8, content: 0xf8
Flash mem addr: 0x100f9, content: 0xf9
Flash mem addr: 0x100fa, content: 0xfa
Flash mem addr: 0x100fb, content: 0xfb
Flash mem addr: 0x100fc, content: 0xfc
Flash mem addr: 0x100fd, content: 0xfd
Flash mem addr: 0x100fe, content: 0xfe
Flash mem addr: 0x100ff, content: 0xff

[technoblogy]

Mmm.. the lamb sounds good!

[MCUdude]

Mmm.. the lamb sounds good!

It absolutely was!

It turns out I've discovered another bug that seems to have always been there but not yet discovered.
When using the sketch I provided in this comment on an ATmega2561 (or an ATmega2560 I'm sure), I can write to any page in the first flash section (0x0000-0xFFFF) and the second section (0x10000-0x1FFFF). However, I try to write to the third section (0x20000-0x2FFFF) or fourth section (0x30000-0x3FFFF) the program crash and goes into a boot loop. I am able to upload new programs with the bootloader, but it has lost its flash writing capabilities, and I'll have to re-burn the bootloader to get to write to flash again. This one probably goes deeper than the other issues I've run into and is probably something @majekw knows more about.

EDIT: It turns out I'm able to erase a page and write to the buffer using optiboot_fill_page, it's only the write command that causes this:

// Function to force a flash page write operation
void flash_end_write(uint32_t base_addr, uint16_t offset_addr)
{
  // Write the buffer to flash if there are any contents in the buffer
  if ((offset_addr & 0xFF) != 0x00)
    optiboot_page_write(base_addr + (offset_addr & (0xFFFF - SPM_PAGESIZE + 1)));
}

[technoblogy]

I thought you might be interested to see what I'm using the Optiboot Flasher for. It's an AVR assembler, written in a combination of C and Lisp, that runs on an ATmega1284P and allows you to write machine-code programs using assembler mnemonics and run them on the processor:

AVR assembler overview

[MCUdude]

@technoblogy I'm not familiar with Lisp, but that is very impressive!

About the issue I was having regarding the 256kiB chips, it turned out that EIND has to be used:

void do_spm_cli(optiboot_addr_t address, uint8_t command, uint16_t data)
{
  uint8_t sreg_save;

  sreg_save = SREG;    // Save old SREG value
  asm volatile("cli"); // Disable interrupts
  #ifdef RAMPZ
    RAMPZ = (address >> 16) & 0xff;  // Address bits 23-16 goes to RAMPZ
    #ifdef EIND
      uint8_t eind = EIND;
      EIND = FLASHEND / 0x20000;
    #endif
    do_spm((address & 0xffff), command, data); // do_spm accepts only lower 16 bits of address
    #ifdef EIND
      EIND = eind;
    #endif
  #else
    do_spm(address, command, data);  // 16-bit address - no problems to pass directly
  #endif
  SREG = sreg_save; // Restore SREG
}

@JAndrassy I believe this might be relevant for your copy_flash_pages function as well.

[technoblogy]

Glad you solved it!

[JAndrassy]

I have EIND in my optiboot.h. and Optiboot repo has it too (cherry-picked from my PR).
I forgot about what it was as I saw you problem. I confused it with RAMPZ.
https://www.avrfreaks.net/forum/eicall-app-bootloader-atmega-2560-jumps-address-0x1fe01-without-eind-set
was it not in my PR for your Optiboot repo?

[MCUdude]

was it not in my PR for your Optiboot repo?

Nope, optiboot.h wasn't touched: MCUdude/optiboot_flash#5

BTW is there a good reason why you have chosen to not use EIND here and instead move it "outside", here?

[JAndrassy]

was it not in my PR for your Optiboot repo?

Nope, optiboot.h wasn't touched: MCUdude/optiboot_flash#5

BTW is there a good reason why you have chosen to not use EIND here and instead move it "outside", here?

EIND is for jump of instruction pointer to a higher flash bank.
without EIND the jump to bootloader for do_spm ended in first flash bank. (details are in my post in avrfreaks forum (link above))

I wrote to majekw too (majekw/optiboot#12)

bootloader on 2560 starts at word address 0x1FE00. the compiler doesn't set EIND, so call to do_spm at atmega 2560, jumps to word address 0xFE01 where FFFF is in empty flash. FFFF is executed as skip one word if bit set. it is set, so it executes every other FFFF in the rest of the flash until bootloader where it hits the rjmp to do_sdm. it 'works' unless something is written in the flash above word address 0xFE01.

it is what you see. if you write something into flash above 0xFE01, do_spm stops working. burrning boortloader clears the flash.

[obdevel]

I'm trying to get this working on a current project but I can't past optiboot_check_writable().

I'm using a 1284P as it's a large project, >64K (using LTO), currently around 70K, and will grow by maybe another 10K:

Sketch uses 69490 bytes (53%) of program storage space. Maximum is 130048 bytes.
Global variables use 3783 bytes (23%) of dynamic memory, leaving 12601 bytes for local variables. Maximum is 16384 bytes.

I'm using the Read_write_without_buffer.ino example as the basis for my test code.

If I use PROGMEM1, I get a linker error (as expected) because the lower section is already full:

Users/xxx/Library/Arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/../lib/gcc/avr/7.3.0/../../../../avr/bin/ld: section .FAR_MEM1 loaded at [0000000000010000,0000000000011fff] overlaps section .text loaded at [0000000000000000,0000000000010ca3]
/Users/xxx/Library/Arduino15/packages/arduino/tools/avr-gcc/7.3.0-atmel3.6.1-arduino7/bin/../lib/gcc/avr/7.3.0/../../../../avr/bin/ld: section .data loaded at [0000000000010ca4,000000000001126b] overlaps section .FAR_MEM1 loaded at [0000000000010000,0000000000011fff]
collect2: error: ld returned 1 exit status

If I add an 16K flash space in PROGMEM, it compiles successfully:

Sketch uses 86942 bytes (66%) of program storage space. Maximum is 130048 bytes.
Global variables use 3786 bytes (23%) of dynamic memory, leaving 12598 bytes for local variables. Maximum is 16384 bytes.

However, optiboot_check_writable() fails and the test data isn't read back as expected.

Any advice ? I'd like to use as much of the free space as possible for this data, maybe as much as 32K if possible, even if that means managing multiple 16K spaces. I'm happy to specify a load address if that helps (and recalculate this address as the program grows), but I need some guidance on how to do this.

If I can get this working I'll be able to drop the SD card reader and code from the project :)

All components (IDE, core, etc) are at the current version.

Thanks.