8000 Release Moddable SDK 5.5.0 · Moddable-OpenSource/moddable · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Moddable SDK 5.5.0
Compare
Choose a tag to compare
@mkellner mkellner released this 06 Mar 00:53
· 172 commits to public since this release
5.5.0
b11c527

Moddable SDK 5.5.0 contains improvements made between January 20, 2025 and March 4, 2025.

Realtime AI

You may have noticed that everyone seems to be either talking about or talking with AI. Moddable is joining the party.

The rapid rise of realtime AI service APIs has created the need for fundamental realtime AI communication services on embedded devices. The goal is to deliver an amazingly interactive experience for users and a streamlined experience for developers. Moddable delivers on both with a new lightweight, low-latency, and high-performance architecture built on industry-standard APIs. Getting this right is tough. It requires balancing network communication, parsing complex service messages efficiently, capturing and playing audio seamlessly, and meeting the demands of a highly interactive user interface. Moddable has implemented that using industry best practices to free developers to focus on what makes their project unique.

A New Architecture

Moddable SDK 5.5.0 introduces our Realtime AI architecture. It works with multiple realtime AI services, and we're launching with support for two of the biggest, OpenAI and Google Gemini. The architecture supports both services with the same JavaScript API. The implementation uses Web Workers to cleanly separate the AI services from the application and keep the UI responsive.

A New Application

To show what our new architecture is capable of, we've created Conversational AI, an application with eighteen voice assistants. The UI is slick and smooth, especially on our own Moddable Six with its high-speed display bus. Conversational AI runs on macOS, Linux, and Windows too, using our simulator, so you can start exploring even before your Moddable Six arrives!

Amazing Results

We're achieving realtime voice chats with latencies that are indistinguishable from desktop, even when combined with a mobile-style user interface. All of that is implemented in modern, standard JavaScript using standard embedded JavaScript APIs from ECMA-419 3rd Edition, running on extraordinarily low-cost hardware.

More to Come

And this is just the first step. We have big plans to enhance the architecture with new features and new services. You can expect cool new applications to show those off.

Microphone for Moddable Hardware

The best way to experience our beautiful new Conversational AI application is on Moddable Six, where the software is fine tuned to get the best performance from every hardware component. You'll need to add a microphone. To make that easy, Moddable just launched a plug-in microphone for just $4.99. This high quality PDM microphone from TDK plugs into Moddable Six and Moddable Display 6. It also works with Moddable Two.

ESP-IDF v5.4

We've migrated to ESP-IDF v5.4.0 from v5.3.1. The update should be transparent to all JavaScript code (and the vast majority of C code). The update is recommended for reliability and security, and to stay current with the latest from Espressif. Updating takes just a couple minutes.

  • If you used xs-dev to install the Moddable SDK, execute xs-dev update --device esp32
  • Otherwise, follow the update instructions for macOS and Linux, or Windows.

NOTE: OneWire support is broken in ESP-IDF v5.4.0, though it worked in ESP-IDF v5.3.*. Moddable has reported this issue to Espressif.

Release Details

  • Examples
    • Added Conversational AI
  • XS JavaScript engine
    • Marshall and demarshall (IPC mechanism used by workers)
      • Enhanced to keep garbage collector enabled when demarshalling. This significantly reduces IPC overhead by eliminating an explicit collection before demarshalling each message received
      • Hardened to safely report errors for unsupported data
      • Detects stack overflows
      • Supports sparse arrays
    • Long-standing warning in BigInt GCC optimization resolved
    • Eliminate unnecessary misaligned opcode retrieval from runloop on ESP32
    • decodeURI and decodeURIComponent detect invalid UTF-8 codepoints #1464 (reported by @ayuan0828)
  • Devices
    • ESP32
      • Divide esp32/m5atom_s3r into esp32/m5atom_s3r_cam and esp32/m5atom_s3r_m12. (Contributed by @stc1988) #1456
      • ESP32-C3 has only one I²C port (reported by @mshioji)
      • Task queue default timeout reduced to 20 ms (from 1000) to detect message queue blocking sooner (can override in manifest with MODDEF_TASK_QUEUEWAIT)
      • Fully initialize ledc_channel_config_t structure (ESP-IDF v5.4.0 compatibility)
      • CONFIG_FREERTOS_HZ no longer required to be 1000 (but 1000 is still recommended). Useful for compatibility with ESP-IDF default of 100 Hz ticks.
      • Web Workers using FreeRTOS task once again behave correctly on all ESP32 family devices
      • Web Workers given lower priority than main task, consistent with other platforms
      • Eliminate memory allocation for small IPC message payloads
  • Modules
    • Piu Port fixed so computed styles are not garbage collected while still in use
  • ECMA-419
    • Digital Bank class on ESP32 supports native onReadable callback
    • TLS socket
      • Reduce memory overhead in packet buffers
      • Reduce allocations in GCM module
      • Validate format when setting
    • ESP32 Camera Image In reliably matches requested image dimensions to available frame sizes
    • TCP socket on macOS no longer invokes pending callbacks after close()
    • HTTP server
      • Properly handles all cases of unknown server response length phoddie/node-red-mcu#129
      • read() works when passed no arguments
    • Audio In on ESP32
      • read on ESP32 returns undefined (instead of 0) when nothing available
      • Option to set PDM slot mask in manifest "defines"
    • Audio Out on ESP32 no longer blocks on write (buffer size calculation improvements)
    • Web Socket Client after connecting frees properties no longer needed to reduce memory
    • File storage modules (POSIX and littlefs) default export reworked to conform with ECMA-419 3rd Edition
    • ESP32 IO modules only preload themselves (previously some preloaded embedded:*)
    • Update module specifiers of new storage modules to match 419 3rd Edition
    • TCP, UDP, and Listener sockets on ESP32 check if network interface is available before invoking lwip (lwip calls abort() when no active network interface)
    • Serial on ESP32 removes interrupt handler on close #1462 (reported by @mauroForlimpopoli)
  • Tools
    • JSON modules supported in manifest preload section
    • xst supports fuzzing of marshall and demarshall
    • xsl
      • Reports details of non-frozen array elements correctly
      • Correct Atomics table when mxFloat16 is defined
    • compileDataView updated to latest with support for float16 (and much more)
    • mcconfig and mcrun can transform JSON files (resolves conflict between Node-RED MCU Edition and ES2025 JSON module support)
  • Testing
    • Add 419 Audio In unit tests
    • Add add Map, Set, and BigInt tests to worker IPC test
    • File storage tests updated to latest ECMA-419 3rd Edition specification
    • Consistently apply naming convention for test fixtures
  • TypeScript
    • Correct type declaration of Bytes in Bluetooth Utilities modules (contributed by @stc1988)

Contact Us

If you have questions or suggestions about anything here, please reach out:

Contributors

stc1988, mshioji, and 2 other contributors
Assets 20
Loading
0