Moddable SDK 4.2

[mkellner]

This release contains improvements to the Moddable SDK made between September 9, 2023 and October 2, 2023.

Moddable SDK 4.2 is a substantial update. Highlights of this release include:

• ESP-IDF v5 is here! After a major effort with great community support, we've merged ESP-IDF v5 support to the main branch. (more)
  • Developers working with ESP32 will need to update their ESP-IDF (update instruction links below).
• xsbug, our JavaScript debugger, has received a major update to speed your development. (more)
  • New breakpoints - conditional, counter, trace, and function name - give you more control.
  • A new interactive console lets you evaluate expressions, set values, and call functions -- making it much faster to gain insights into running code without having to change the source code, rebuild, and re-deploy.
• mcpack, our new embedded packager based to make embedded JavaScript development more accessible to developers working on the web platform. (more)
  • Uses package.json to drive build, allowing easy use of npm and other JavaScript ecosystem tools
  • Analyzes project code to automatically generate a Moddable manifest.json optimized for the project
  • Experimental! Please share your experiences to help evolve mcpack

News

• Save the date! 🎉Ecma TC53 is holding a workshop to discuss proposals and consider options for Embedded JavaScript standardization including new APIs, interoperability, packaging, and Hardened JavaScript. 🎉
  • 📆December 6
  • 📍Apple Park
  • 🧑🏽‍💻Everyone with relevant expertise welcome
• GitHub Copilot – Members of the community have shared their experiences and opinions about using Github Copilot for embedded JavaScript. Check out the discussion to see how it might help your development.

October Community Update

The monthly Moddable Community Call is now a monthly Moddable Community Update! We'll share more about this soon. Sign up to receive an email notification when new community updates are published.

Release Details

• Devices
  • ESP32
    • ESP-IDF v5 support is here!
      • You must upgrade your ESP-IDF, as builds with ESP-IDF v4 stop with an error message.
      • Update instructions are available for macOS, Windows, and Linux.
      • The Moddable SDK currently targets ESP-IDF v5.1.1 for all ESP32 family microcontrollers
      • There are no changes to the JavaScript APIs on ESP32, so no script changes should be required(!!).
      • This release supports the following models in the ESP32 family: ESP32, ESP32-S2, ESP32-S3, ESP32-C3, ESP32-C6, and ESP32-H2.
      • The Moddable SDK has been updated to use the latest APIs in ESP-IDF v5 (deprecated APIs are nearly all removed)
      • Improvements in the RMT APIs allow the Neopixels module to support multiple strands (on separate GPIO pins)
    • New device breadboard build platform targets with displays (mostly ILI9341 and one GC9A01). Great for breadboard prototyping.
      • esp32/qtpyc3_ili9341
      • esp32/qtpyc2_ili9341
      • esp32/qtpys3_ili9341
      • esp32/xiao_esp32c3_ili9341
      • esp32/xiao_esp32s3_ili9341
      • esp32/esp32h2_ili9341
      • esp32/esp32c6_gc9a01
  • nRF52
    • $(UF2_VOLUME_NAME) now handled correctly on on Windows (reported by @NextWorldLab) Unable to write to nRF52(Windows, xiao) #1207
    • Include pwm in Windows build
    • Improved build messages when building for targets that aren't Moddable Four
    • Build improvements to support nRF52 builds of Node-RED MCU using the MCU plug-in (contributed by @NextWorldLab)
  • Raspberry Pi Pico
    • New device breadboard build platform targets. Great for prototyping.
      • pico/ili9341_i2s (includes speaker output)
      • pico/qtpy_ili9341
  • ESP8266
    • modGPIO.h needs to be included when build ECMA-419 IO. (reported by @dtex) Is/was there an "esp" platform identifier with no subplatform? #1212
• Tools
  • xsbug has received a major update
    • Enhancements to breakpoints for faster debugging
      • Conditional breakpoints - break only when a JavaScript expression evaluates to rue (e.g this.dx > 3)
      • Counter breakpoints - break after a counter reaches a specified threshold (e.g. on the 10th time, every 60th time, the first 3 times)
      • Function name breakpoints - break on all calls to a specified function name across all source files (e.g. onReadable)
      • Trace breakpoints - instead of breaking, these trace the evaluation of a specified JavaScript expression to the debug console (e.g. `x = ${x}`)
      • Breakpoints may now be disabled
    • Interactive debug console (REPL)
      • Expressions can be evaluated in-scope when stopped in the debugger
      • View properties
      • Set values
      • Call functions
    • See the xsbug documentation for more information
  • mcpack is our new embedded packaging tool for embedded devices. It makes standard, modern embedded JavaScript development more accessible to JavaScript developers working with npm and Node.js.
    • mcpack is a front-end to existing Moddable build tools like mcconfig and mcrun
    • mcpack starts from package.json and analyzes your project's dependencies to automatically generate a Moddable SDK manifest.json
    • mcpack detects the use of Moddable SDK built-in modules and automatically adds them to the generated manifest.
    • mcpack detects the use of well-known globals -- setTimeout, fetch, URL, Worker, etc -- and automatically includes the necessary modules in the generated manifest and initializes the global variables.
    • mcpack detects the use of top-level await (TLA) and automatically generates the needed manifest settings to enable TLA
    • mcpack introduces the "moddable:" namespace for built-in modules. The prefix is optional, so built-in modules may be accessed as "timer" or "moddable:timer".
    • mcpack uses the runtime key "moddable" (registered with WinterCG) to allow package.json to include runs that apply only to Moddable SDK builds.
    • mcpack allows Moddable SDK manifest fragments to be embedded in package.json or to be referenced externally, allowing full access to all capabilities of manifest.json.
    • There is a suite of example projects available that demonstrate how to use mcpack. Check out the readme to learn how to get started with mcpack.
    • mcpack is experimental. We expect it to evolve based on community feedback. Please give it a try!
    • mcpack is not yet fully documented. If you have questions or suggestions, please ask them in our Discussions forum on GitHub.
  • New flags to control debugger launch behavior with mcconfig and mcrun. -d for default debugger (launch xsbug), -dn to not launch debugger, -dl to launch xsbug-log, and -dx to launch xsbug. -l is still supported for xsbug-log, but will eventually be removed. New flags are available on macOS and Linux; Windows is coming soon.
  • mcrun improvements to help installing to ESP32 USB-native devices
  • Simulator (mcsim)
    • Would sometimes launch the project twice, confusing debuggers. Fixed to only launch target project once.
    • Release build fix (breakOnStartFlag only available in debug builds)
  • Rename DISPLAY and ROTATION build variables to COMMODETTOBITMAPFORMAT and POCOROTATION to eliminate conflicts on some Linux distributions
  • nRF52 builds can debug over serial too (in addition to existing USB support)
  • ESP32 installation improvements to add prompts for button presses when necessary (devices without auto-programming capability)
  • Raspberry Pi Pico
    • xsbug-log now supported for Raspberry Pi Pico Linux builds
    • mcconfig now able to build PIO source files included in a manifest. This simplifies integration of PIO support and minimizes binary size. (inspired by @linfan in discussion RPI Pico (RS2040) PIO supprt? #1220)
      • Build neopixel and i2s_audio pio modules only if included in project
      • pio and supporting source files compiled directly from external directories (i.e. pico-extras)
• Modules
  • Manifests
    • To more thoroughly support automatic manifest includes by mcpack, many existing module manifests were updated and a number of new module manifests were added. The goal is to ensure that every module can be built by simply including its manifest (rather than directly including the modules in a project manifest). This is now true, with just a few exceptions.
      • In making the changes several errors were corrected, resulting in smaller binaries in some cases.
      • While the changes were made with care, it is possible that some uses were overlooked. If you run into problems building existing projects, please report an issue.
    • BLE client and server manifests now have error case for unsupported platforms to generate earlier failure with clearer message
  • ECMA-419
    • HTTP client
      • Don't require input HTTP headers to be normalized to lowercase (reported by @tve) embedded:network/http/client requires headers in lower case #1218
      • Handle HTTP 204 and 205 (no content) responses (reported by @tve) embedded:network/http/client hangs if server sends no response body #1219
    • Digital bank
      • Fix deadlock in close() uncovered by ESP-IDF v5
  • Poco refactored for better compatibility. The commodetto/PocoCore module is now the foundation and the commodetto/Poco module contains the drawing primitives. This revises a change in Moddable SDK 4.0 made to reduce the firmware image size for projects using exclusively Piu APIs for rendering.
  • nRF52 flash partition fixes
    • Don't pin mod partition ("xs") to size of currently installed mod
    • Respect LFS partition size from manifest
  • Base64, TextEncoder, and TextDecoder globals (primarily used in xst) are now non-enumerable, as are the methods on their prototype. This matches JavaScript conventions. (reported by @kriskowal)
  • Outline renderer
    • Forces garbage collection on allocation failures
    • Eliminate type coercion on number arguments
  • Neopixels close() and destructor fixed on Pico to eliminate crash
  • Creating a socket with tcp_new() on ESP32 when the network is uninitialized would crash in lwip. The Moddable SDK socket constructor now detects that the network is uninitialized and throws instead.
• XS
  • Support for new xsbug debugging features
  • Dead stripping fixed so using Compartments no longer always include eval (JavaScript parser, etc) to be included in engine
  • Eliminate incorrect mapping of @ and default symbols in linker. (reported by @linfan - very good find!) "0" and "@" hash value conflicted? #1209
  • Fix memory leak of the->pathValue introduced with recent debugging improvements. (contributed by @raphdev)
  • Fix handling of XS_STRING_X type in corner cases when mxSnapshot enabled
  • Fixes for issues found by Fuzzilli and oss-fuzz fuzz testing (thank you @Agoric)
• Examples
  • Random static graphics example now includes support for varying brightness.
  • Outline Figures example mods now all include manifest_mod.json so they can be deployed to all devices. (contributed by @stc1988) examples/piu/outline/figures works on ESP32 devices #1215
  • QR Code example manifest fixed to track Poco refactoring. (reported by @kairos0ne) Error when compiling QR code example on mac - file missing  #1216
  • Adjust manifests for outline examples to give more memory to outlines.
• Documentation
  • Update XS Differences to describe optional CESU-8 (motivated by @warner)
• Tests
  • digitalBank/readwrite test incorrectly assumed -1 would not write to any pins. Add assert to make sure output test pin is not GPIO 1 (test will fail)
  • spi/read and spi/transfer tests updated to avoid loopback that caused tests to fail

Contact Us

If you have questions or suggestions about anything here, please reach out:

• Start a new Discussion on our GitHub repository
• Drop by our Gitter to chat
• Contact us on Twitter at @moddabletech

---

This discussion was created from the release Moddable SDK 4.2.