[Beta] Jump to system memory boot from user application #710

fpistm · 2019-10-18T09:39:11Z

This PR add support of automatically jump to system memory bootloader.

Currently, this is trigger only thanks a Serial over USB (USB CDC have to be enabled).
By setting the port at 1200 speed this request the restart by setting a magic number (0x515B) in a backup register then require a system reset.
Then when restart, this value is checked and jump is performed if match to go in bootloader mode.

STM32 Cube programmer scripts is currently under update.
See stm32duino/Arduino_Tools#44

Tested under Linux:

F3:
- BlackPill F303CC
F4:
- Black F407VE
- Nucleo F401RE
L4:
- SensorTile Box (L4)
WB:
- P-Nucleo WB55RG --> Reset in BL mode OK, programming start properly but failed. `Cube Programmer issue, not related to this PR.

Under Windows manually set speed at 1200 works as expected. Script under update.

Fixes #706

stas2z · 2019-10-18T12:09:18Z

Woohoo, it realy works) but i failed resetting it by 1200 baud and added simple mapple style reset (if "leaf" trigger recieved) so now it resets and uploads flawlessly using mapple's upload-reset and cubeprog in dfu mode after :)
p.s. tested it with F401

fpistm · 2019-10-18T12:17:08Z

@stas2z
It should be fine to find why it is not reset setting 1200.
Did you try manually without upload scripts?

stas2z · 2019-10-18T12:21:33Z

@stas2z
It should be fine to find why it is not reset setting 1200.
Did you try manually without upload scripts?

yes, ive tried to connect with arduino terminal using 1200 baudrate and also with putty, ive uploaded a simple echo fw to it and it works as expected at 1200 without reset

fpistm · 2019-10-18T12:22:43Z

I've just tested with a Nucleo-F401RE and a USB shield and this work fine...
You set the USB CDC port at 1200 not an HardwareSerial port ?

stas2z · 2019-10-18T12:24:49Z

I've just tested with a Nucleo-F401RE and a USB shield and this work fine...
You set the USB CDC port at 1200 not an HardwareSerial port ?

yes, sure
im using connection to ttyACM0 as my device appeared in dmesg (blackpill F401CC)

fpistm · 2019-10-18T12:37:57Z

Could you try with (several time if needed):
stty -F /dev/ttyACM0 1200

stas2z · 2019-10-18T13:22:43Z

stty -F /dev/ttyACM0 1200

No, its not working for my board

Also i found a problem i got when i tried to implement it by myself, it just reboots without entering a bootloader if any gpio pin initialized, just make pinMode for builtin led and it will stop loading boot.

resolved it by adding following code before remapping and jump to boot

    HAL_RCC_DeInit();
SysTick->CTRL = 0;
SysTick->LOAD = 0;
SysTick->VAL = 0;
__HAL_RCC_SYSCLK_CONFIG(RCC_SYSCLKSOURCE_HSE);
__HAL_RCC_SYSCFG_CLK_ENABLE();

fpistm · 2019-10-18T13:35:23Z

In fact I used the backup register to ensure having original config and no feature configured.
I'm surprise to see HSE in your code as it should be HSI.

stas2z · 2019-10-18T13:52:50Z

In fact I used the backup register to ensure having original config and no feature configured.
I'm surprise to see HSE in your code as it should be HSI.

Ok, it's my fault, HSI is also works and seems like will be better.

But it's not important, cuz SysTick reg values reset do the magic. Reiniting RCC i found by googling, but in my case it can be skipped.

Also ive tried reset by 1200 baud my another F4 board (black F407VE), and still no luck (maple reset works flawlessly), probably problem on my side, will try to find what i did wrong while i appleid this pull request to my local repo copy

fpistm · 2019-10-18T13:56:25Z

Ok. In fact I've made the minimum to work. Resetting systick an RCC could be added, I guess. Maybe also HAL_DeInit as it is init for backup access.

stas2z · 2019-10-18T14:05:36Z

Ok. In fact I've made the minimum to work. Resetting systick an RCC could be added, I guess. Maybe also HAL_DeInit as it is init for backup access.

Anyway, it works already, for me at least :) reset procedure is not so important, also usb is not only way to use it, cuz im going silently update slave devices built with chips with only serial dfu supported, so bootloader reboot requests can be implemented as hwserial commands or gpio pins.

fpistm · 2019-10-18T15:10:23Z

Right, this is not only for USB.

stas2z · 2019-10-18T16:49:45Z

its weird, but 1200 baud reset working under windows for me, without rebuilding
but for linux it's not, even with sudo (any terminal app works fine, but 1200 baudrate reset not)

fpistm · 2019-10-18T16:58:17Z

which Linux OS do you used?

stas2z · 2019-10-18T17:05:02Z

which Linux OS do you used?

mint 19.2 (ubuntu 18.04 based)

fpistm · 2019-10-18T17:11:45Z

Ok, I'v tested on 14.04 and 16.04 anyway this should be the same or a usb driver issue as the speed is not relevant for this kind of device

stas2z · 2019-10-18T17:29:29Z

Ok, I'v tested on 14.04 and 16.04 anyway this should be the same or a usb driver issue as the speed is not relevant for this kind of device

Seems like it was a usb driver glitch, ive rebooted back to linux and everything works as it should, sorry for this inconvinience

fpistm · 2019-10-18T17:37:57Z

OK so now it is fine on Windows and Linux ?

stas2z · 2019-10-18T17:43:34Z

@fpistm yep, both

stas2z · 2019-10-18T19:36:52Z

Good news, Ive successfully rebooted stm32f051c8t6 to bootloader mode using this PR. Will check stm32f030c8t6 a bit later.

ladyada · 2019-10-18T20:41:09Z

i 'tested' this with windows by manually opening the port at 1200 which did in fact kick into the DFU bootloader! then i uploaded. so we need an update to the tool, like linux has, and it should work. @fpistm im not great at bat files but i could try making the change

This now runs in early startup, so the normal delay will not work (systick is not running yet). Note: Seems that on F0, inline assembler is compiled using the classic rather than unified syntax (which *is* used on F4 it seems). This explicitly selects the unified syntax in the inline assembly block, to fix building for F0. TODO: There is a variant which also overrides the enumeration.

Since the bootloader does not touch these flags, but can start the sketch if instructed by the programmer (without a reset), clearing is needed to prevent a start from the bootloader looking like a software-reset. Otherwise, the startup code would be looking at the BootIntoBootloaderAfterReset variable to decide whether to jump to the bootloader. This variable should have been cleared before jumping to the bootloader the last time, but the bootloader might have overwritten it (or a sketch was uploaded with a different address for that variable). Additionally, clearing the reset flags allows a sketch to see that the they are started through the bootloader, when all reset flags are cleared.

On most CPUs, detecting the system flash address is not needed, since they can remap the system flash onto adress 0x0 and the bootloader reset vector can be read from there. This approach also more closely mimics a bootloader start using the BOOTx pins. For CPUs that cannot remap flash, this hardcodes the address of system flash and jumps there instead. For H7 and F7, this is also how it works when using the BOOT pins. For F1, remapping can be done using the BOOT pins, but not from software, resulting in a difference with a BOOT-activated bootloader. This puts the code that figures out where the bootloader lives back in a separate function. This makes the code easier to read, and makes it easier to jump to custom bootloaders (e.g. in normal flash instead of in system flash). TODO: Check value for H7, AN2606 and reference manual disagree

This ensures that the SP is not modified after loading it and ensures a jump instruction is used. The assembly code was based on micropython and another STM32 core: https://github.com/micropython/micropython/blob/f6375ac3ebac28656a0a757952d32c265b1ba7aa/ports/stm32/powerctrl.c#L71-L78 https://github.com/GrumpyOldPizza/arduino-STM32L4/blob/bacc184288a4644b2ee6a97672334983f3e788ab/system/STM32L4xx/Source/boot_stm32l4xx.c#L159-L166

This waits for 250ms, just like the Arduino SAM and SAMD cores do (AVR waits only 120ms), to allow the USB host and application a bit more time to close the port and clean up.

This function calls HAL_Delay, which relies on the systick timer to be running, so this results in an infinite loop. This should probably be fixed in HAL_Delay, but for now just remove the calls to USB_DevDisconnect and USB_DevConnect and replace it with the bit twiddles needed for some chip families (e.g. the F4), breaking compilation and/or functionality for others.

This adds support for the DFU runtime protocol, which allows resetting into the bootloader using a DFU command. This allows e.g. dfu-util to handle the complete firmware upload, including the needed reset. This consists of a number of changes: - An extra interface is added to the USB configuration descriptor. This descriptor has two parts (interface descriptor and functional descriptor) which together indicate to a host that this device supports DFU. - Control packets to this new interface are detected by the CDC code an forwarded to a new USBD_DFU_Runtime_Control() function. - This new function handles the DFU GET_STATE, GET_STATUS and DFU_DETACH commands. The former are optional, but simple enough, the latter is mandatory and handles resetting into the bootloader. - The CDC device descriptor is changed to become a composite device (CDC and DFU). This allows operating systems (in particular Windows, Linux did not really need this) to identify two different subdevices, and install different drivers for each (on Windows, this is serusb for the CDC part and WinUSB/libusb for the DFU part). Without this, dfu-util on Windows could not access the DFU commands when the serial driver was loaded. Because the CDC functionality already exposes two interfaces (which together form a single serial port), an IAD (Interface Association Descriptor) is inserted before these interfaces to group them together in a single subdevice. No IAD is needed for the DFU interface, since it is just a single interface. To become a composite device, the device class must be changed from CDC to a composite device class. This was originally class 0/0/0, but together with the IAD, a new EF/2/1 deviceclass was also introduced, which is used now. Note that this only adds descriptors and a command handler on the default control endpoint, so no extra (scarce) endpoints are used by this, just a bit of memory. This commit is still a bit rough, because: - The DFU descriptors and code are now pulled in directly by the CDC code (and HID is not supported yet). Ideally, there should be some kind of pluggable USB library where different interfaces can be registered independent of each other (see also stm32duino#687). - The interface number is hardcoded in the DFU descriptor. - The reset to bootloader happens immediately, while it might be better to wait a short while to allow the current USB transaction to complete. - DFU support is unconditionally advertised, while not all boards might support DFU.

Bootloader management is not applicable for this serie. Signed-off-by: Frederic Pillon <frederic.pillon@st.com>

rohitsam · 2020-11-04T12:46:13Z

@fpistm This is a very cool feature, I have tested this PR on my Windows10 machine for my 32f411Disco-boards and Blackpill works absolutely fine. this reduces lot of my efforts in uploading code.
It would be great if this feature could be merged.

fpistm · 2020-11-04T13:17:32Z

It would be great if this feature could be merged.

It will be when ready. 😉

rohitsam · 2020-11-17T19:43:10Z

There is an issue with the upload.use_1200bps_touch=true

The first time you want to flash a board thanks DFU, there is no com port available, so no port displayed in the menu (port is grayed). The board is manually start in STM32 BOOTLOADER mode thanks the BOOT0 pin and the upload is OK. The 1200 touch does nothing.

@fpistm Could you please tell if this is the only issue with this PR that still remains or there any others?
Thanks.

matthijskooijman · 2020-11-17T19:50:21Z

AFAIK there's still a few other bits of code that work but are too hardcoded or break on some boards, see for example #710 (comment). Also, I think the DFU-descriptor and handling might also need to be slightly more generalized. There might be other issues too.

matthijskooijman · 2021-04-19T18:48:43Z

I worked on another board with custom bootloader running my version of this PR and ran into an issue, related to what I previously commented:

I also wondered if this should support jumping to custom bootloaders (e.g. not the system bootloader, but a bootloader in flash). However, I just realized that if these are used, they will likely live at the start of flash, so just a normal reset will start them (possibly combined with writing some RTC value as, e.g. see here). So, the jump-to-bootloader code from this PR does not need to handle jumping to those bootloaders, (...)

It seems that a custom bootloader at the start of flash is not compatible with the noinit variable approach I used. The bootloader runs before the sketch, so when it does a software reset, does not clear the reset reason and writes to the noinit BootIntoBootloaderAfterReset variable in RAM (i.e. by just using that location for something else), the jumpToBootloaderIfRequested function will jump to system ROM bootloader, even when it is not requested (and even more, makes no sense). In general, jumping to the ROM bootloader makes no sense when combined with a custom bootloader, so this probably needs a board-dependent define to disable all the jump-to-bootloader handling (or maybe automatically disabled if LD_FLASH_OFFSET and/or VECT_TAB_OFFSET is set, which effectively means a reset does not reach us. Or maybe it would be better to disable this explicitly and check against these defines as a sanity check).. This might be part of some refactoring to include the jump-to-custom bootloader (HID and the other one I think) in the same structure as jump-to-custom bootloader.

Also, I noticed this previous comment from @fpistm:

If for any reason the Serial monitor is opened and the board is disconnected then the port still selected in the menu and in this case the 1200 touch failed and the upload is ended. I think it should only report that the COM port is not found and try to upload anyway. In my case I've simply restart my board manually in STM32 bootloader mode so the upload would work properly.

There has been some work in arduino-cli recently that touches upon this, though I don't think it actually fixes this, though. For reference: arduino/arduino-cli#1263 and arduino/arduino-cli#1267

adragomir · 2021-10-15T17:45:04Z

@fpistm if it helps, I rechecked that this works, applying the patches on 2.0.0, with an STM32F405RGT flavor board (like Adafruit Feather Express STM32F405, or Sparkfun Thing Plus STM32F405)

Also had success getting it to work on a platform.io project, using an STM32F405RGT board with a build hook that bounces the port manually and then waits for dfu-util -l

jiri-jirus · 2022-02-03T03:25:37Z

Oh, all 3d printers based on STM32F446 would need this feature badly.

Kamoool · 2022-03-13T22:22:51Z

Hello,

Any news on that feature? I'm waiting so badly for that to use in my STM32L432 projects.

Kamoool · 2022-07-07T09:37:08Z

Hey,
Has anyone checked that with 2.3.0?

fpistm · 2022-07-07T11:23:41Z

Hey, Has anyone checked that with 2.3.0?

I have to rebase and finish it. When I will have time...

kingovchouffe · 2024-01-09T17:03:31Z

Hi, is there any news on this feature ? I have a custom F446 devellopment board I can test on my side if this help.

fpistm self-assigned this Oct 18, 2019

fpistm added Enhancement New feature On going labels Oct 18, 2019

fpistm added this to In progress in STM32 core based on ST HAL via automation Oct 18, 2019

fpistm added this to the 1.8.0 milestone Oct 18, 2019

fpistm changed the title ~~Jump to system memory boot from user application …~~ [Beta] Jump to system memory boot from user application Oct 18, 2019

fpistm mentioned this pull request Oct 18, 2019

New board definition - Adafruit Feather STM32F405 #701

Merged

fpistm force-pushed the jumpSysBL branch from 0ed4b83 to 3bc114c Compare October 18, 2019 16:17

matthijskooijman and others added 10 commits September 18, 2020 17:54

Forward declare Original_Reset_handler

0871fee

Delay autoreset to bootloader

8e5ad31

This waits for 250ms, just like the Arduino SAM and SAMD cores do (AVR waits only 120ms), to allow the USB host and application a bit more time to close the port and clean up.

HACK: Extend HACK to boards with DPPU-based pullup control (e.g. F0)

2f148bf

Do not build for STM32MP1xx

a37e822

Bootloader management is not applicable for this serie. Signed-off-by: Frederic Pillon <frederic.pillon@st.com>

fpistm force-pushed the jumpSysBL branch from 99ba762 to a37e822 Compare September 18, 2020 15:55

rohitsam mentioned this pull request Oct 21, 2020

request reset to bootloader mode in Windows fpistm/Arduino_Tools#1

Closed

rohitsam mentioned this pull request Nov 23, 2020

issue with the upload.use_1200bps_touch=true arduino/arduino-cli#1083

Open

fpistm added this to the 2.x.x milestone Dec 14, 2020

fpistm mentioned this pull request Apr 17, 2021

[variant] Improve variant support and genericity #1091

Merged

27 tasks

matthijskooijman mentioned this pull request Apr 23, 2021

Pluggable discovery arduino/tooling-rfcs#2

Merged

fpistm removed this from the 2.1.0 milestone Jul 13, 2021

fpistm mentioned this pull request Aug 23, 2022

Continuous flashing in dfu mode is not working #1782

Closed

blsamad12 approved these changes Aug 1, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Beta] Jump to system memory boot from user application #710

[Beta] Jump to system memory boot from user application #710

fpistm commented Oct 18, 2019 •

edited

stas2z commented Oct 18, 2019 •

edited

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019 •

edited

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019 •

edited

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019 •

edited

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

stas2z commented Oct 18, 2019

ladyada commented Oct 18, 2019

rohitsam commented Nov 4, 2020

fpistm commented Nov 4, 2020

rohitsam commented Nov 17, 2020 •

edited

matthijskooijman commented Nov 17, 2020

matthijskooijman commented Apr 19, 2021 •

edited

adragomir commented Oct 15, 2021

jiri-jirus commented Feb 3, 2022

Kamoool commented Mar 13, 2022

Kamoool commented Jul 7, 2022

fpistm commented Jul 7, 2022

kingovchouffe commented Jan 9, 2024

[Beta] Jump to system memory boot from user application #710

Are you sure you want to change the base?

[Beta] Jump to system memory boot from user application #710

Conversation

fpistm commented Oct 18, 2019 • edited

stas2z commented Oct 18, 2019 • edited

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019 • edited

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019 • edited

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019 • edited

stas2z commented Oct 18, 2019

fpistm commented Oct 18, 2019

stas2z commented Oct 18, 2019

stas2z commented Oct 18, 2019

ladyada commented Oct 18, 2019

rohitsam commented Nov 4, 2020

fpistm commented Nov 4, 2020

rohitsam commented Nov 17, 2020 • edited

matthijskooijman commented Nov 17, 2020

matthijskooijman commented Apr 19, 2021 • edited

adragomir commented Oct 15, 2021

jiri-jirus commented Feb 3, 2022

Kamoool commented Mar 13, 2022

Kamoool commented Jul 7, 2022

fpistm commented Jul 7, 2022

kingovchouffe commented Jan 9, 2024

fpistm commented Oct 18, 2019 •

edited

stas2z commented Oct 18, 2019 •

edited

stas2z commented Oct 18, 2019 •

edited

fpistm commented Oct 18, 2019 •

edited

fpistm commented Oct 18, 2019 •

edited

rohitsam commented Nov 17, 2020 •

edited

matthijskooijman commented Apr 19, 2021 •

edited