8000 [camera] Use common configs to create a GStreamer pipeline by yufengwangca · Pull Request #39009 · project-chip/connectedhomeip · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[camera] Use common configs to create a GStreamer pipeline #39009

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 16, 2025

Conversation

yufengwangca
Copy link
Contributor

The camera settings were previously hard‑coded to a single configuration that fails on all three of the test cameras I have. This commit begins our move toward auto‑detecting device capabilities and building the GStreamer pipeline dynamically.
As a transitional step we switch to a “safe‑default” configuration that should work with the majority of off‑the‑shelf webcams.

  1. Most UVC cameras can output MJPEG, so the new pipeline:
  2. Captures MJPEG frames from the camera.
  3. Decodes them to raw video.
  4. Re‑encodes the frames to H.264.
  5. Packages the H.264 stream in RTP and sends it over UDP to the configured port.

This approach gives broad compatibility today while paving the way for full hardware‑driven configuration in the next phase.

Testing

  1. Start video from camera -controller
    > cameraavstreammanagement video-stream-allocate 3 0 30 120 '{"width": 320, "height": 240}' '{"width": 640, "height": 480}' 10000 10000 1 10 1 1

  2. View video stream from another terminal using gst-launch-1.0

Copy link
github-actions bot commented May 16, 2025

PR #39009: Size comparison from 7e868e7 to 248d586

Full report (73 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 7e868e7 248d586 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1103312 1103312 0 0.0
RAM 179074 179074 0 0.0
bl702 lighting-app bl702+eth FLASH 656542 656542 0 0.0
RAM 135041 135041 0 0.0
bl702+wifi FLASH 834052 834052 0 0.0
RAM 124589 124589 0 0.0
bl706+mfd+rpc+littlefs FLASH 1066592 1066592 0 0.0
RAM 117429 117429 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 895528 895528 0 0.0
RAM 105724 105724 0 0.0
lighting-app bl702l+mfd+littlefs FLASH 979616 979616 0 0.0
RAM 109924 109924 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 821764 821764 0 0.0
RAM 120296 120296 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 831640 831640 0 0.0
RAM 125440 125440 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 779200 779200 0 0.0
RAM 113844 113844 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 763420 763420 0 0.0
RAM 114052 114052 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 549650 549650 0 0.0
RAM 205256 205256 0 0.0
lock CC3235SF_LAUNCHXL FLASH 581066 581066 0 0.0
RAM 205456 205456 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 662909 662909 0 0.0
RAM 77300 77300 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 682761 682761 0 0.0
RAM 79940 79940 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 682761 682761 0 0.0
RAM 79940 79940 0 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 639693 639693 0 0.0
RAM 72368 72368 0 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 624069 624069 0 0.0
RAM 73604 73604 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 643697 643697 0 0.0
RAM 76156 76156 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 643697 643697 0 0.0
RAM 76156 76156 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 643413 643413 0 0.0
RAM 76604 76604 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 663129 663129 0 0.0
RAM 79156 79156 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 663129 663129 0 0.0
RAM 79156 79156 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 619593 619593 0 0.0
RAM 70708 70708 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 639445 639445 0 0.0
RAM 73340 73340 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 639445 639445 0 0.0
RAM 73340 73340 0 0.0
efr32 lock-app BRD4187C FLASH 946164 946164 0 0.0
RAM 132092 132092 0 0.0
BRD4338a FLASH 772984 772984 0 0.0
RAM 173320 173320 0 0.0
window-app BRD4187C FLASH 1038808 1038808 0 0.0
RAM 128228 128228 0 0.0
esp32 all-clusters-app c3devkit DRAM 103400 103400 0 0.0
FLASH 1808076 1808076 0 0.0
IRAM 83846 83846 0 0.0
m5stack DRAM 122012 122012 0 0.0
FLASH 1774146 1774146 0 0.0
IRAM 117043 117043 0 0.0
linux air-purifier-app debug unknown 4848 4848 0 0.0
FLASH 2793980 2793980 0 0.0
RAM 115256 115256 0 0.0
all-clusters-app debug unknown 5656 5656 0 0.0
FLASH 6360090 6360090 0 0.0
RAM 533816 533816 0 0.0
all-clusters-minimal-app debug unknown 5528 5528 0 0.0
FLASH 5462590 5462590 0 0.0
RAM 225592 225592 0 0.0
bridge-app debug unknown 5560 5560 0 0.0
FLASH 4770786 4770786 0 0.0
RAM 204976 204976 0 0.0
camera-app debug unknown 8856 8864 8 0.1
FLASH 6896331 6896459 128 0.0
RAM 226648 226648 0 0.0
camera-controller debug unknown 9240 9240 0 0.0
FLASH 14391979 14391979 0 0.0
RAM 661552 661552 0 0.0
chip-tool debug unknown 6256 6256 0 0.0
FLASH 14674707 14674707 0 0.0
RAM 648288 648288 0 0.0
chip-tool-ipv6only arm64 unknown 40328 40328 0 0.0
FLASH 12625376 12625376 0 0.0
RAM 694576 694576 0 0.0
fabric-admin debug unknown 5936 5936 0 0.0
FLASH 12751047 12751047 0 0.0
RAM 647648 647648 0 0.0
fabric-bridge-app debug unknown 4808 4808 0 0.0
FLASH 4587388 4587388 0 0.0
RAM 191056 191056 0 0.0
fabric-sync debug unknown 5064 5064 0 0.0
FLASH 5733541 5733541 0 0.0
RAM 487168 487168 0 0.0
lighting-app debug+rpc+ui unknown 6264 6264 0 0.0
FLASH 5651073 5651073 0 0.0
RAM 207544 207544 0 0.0
lock-app debug unknown 5496 5496 0 0.0
FLASH 4824444 4824444 0 0.0
RAM 194840 194840 0 0.0
ota-provider-app debug unknown 4848 4848 0 0.0
FLASH 4446286 4446286 0 0.0
RAM 184144 184144 0 0.0
ota-requestor-app debug unknown 4800 4800 0 0.0
FLASH 4575890 4575890 0 0.0
RAM 188048 188048 0 0.0
shell debug unknown 4272 4272 0 0.0
FLASH 3095276 3095276 0 0.0
RAM 148632 148632 0 0.0
thermostat-no-ble arm64 unknown 9624 9624 0 0.0
FLASH 4234160 4234160 0 0.0
RAM 231256 231256 0 0.0
tv-app debug unknown 5832 5832 0 0.0
FLASH 6092341 6092341 0 0.0
RAM 611640 611640 0 0.0
tv-casting-app debug unknown 5416 5416 0 0.0
FLASH 12873917 12873917 0 0.0
RAM 769072 769072 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 921588 921588 0 0.0
RAM 167398 167398 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 913296 913296 0 0.0
RAM 145642 145642 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 858692 858692 0 0.0
RAM 141161 141161 0 0.0
nxp contact k32w0+release FLASH 592976 592976 0 0.0
RAM 72920 72920 0 0.0
mcxw71+release FLASH 625832 625832 0 0.0
RAM 63260 63260 0 0.0
light k32w0+release FLASH 619100 619100 0 0.0
RAM 72216 72216 0 0.0
k32w1+release FLASH 710552 710552 0 0.0
RAM 72108 72108 0 0.0
lock mcxw71+release FLASH 775632 775632 0 0.0
RAM 67692 67692 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1669252 1669252 0 0.0
RAM 212400 212400 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1573356 1573356 0 0.0
RAM 208584 208584 0 0.0
light cy8ckit_062s2_43012 FLASH 1447556 1447556 0 0.0
RAM 197296 197296 0 0.0
lock cy8ckit_062s2_43012 FLASH 1477444 1477444 0 0.0
RAM 225016 225016 0 0.0
qpg lighting-app qpg6105+debug FLASH 668340 668340 0 0.0
RAM 105236 105236 0 0.0
lock-app qpg6105+debug FLASH 626536 626536 0 0.0
RAM 99872 99872 0 0.0
stm32 light STM32WB5MM-DK FLASH 466004 466004 0 0.0
RAM 141488 141488 0 0.0
telink bridge-app tl7218x FLASH 694618 694618 0 0.0
RAM 101908 101908 0 0.0
light-app-ota-compress-lzma-factory-data tl3218x FLASH 774328 774328 0 0.0
RAM 50060 50060 0 0.0
light-app-ota-shell-factory-data tl7218x FLASH 781434 781434 0 0.0
RAM 109232 109232 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 703580 703580 0 0.0
RAM 62624 62624 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 741534 741534 0 0.0
RAM 85796 85796 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 607756 607756 0 0.0
RAM 119448 119448 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 816744 816748 4 0.0
RAM 107500 107500 0 0.0
tizen all-clusters-app arm unknown 5224 5224 0 0.0
FLASH 1822464 1822464 0 0.0
RAM 95164 95164 0 0.0
chip-tool-ubsan arm unknown 20600 20600 0 0.0
FLASH 20893806 20893806 0 0.0
RAM 9084072 9084072 0 0.0

@yufengwangca yufengwangca enabled auto-merge (squash) May 16, 2025 15:19
Copy link
github-actions bot commented May 16, 2025

PR #39009: Size comparison from 7e868e7 to b833540

Full report (73 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, cyw30739, efr32, esp32, linux, nrfconnect, nxp, psoc6, qpg, stm32, telink, tizen)
platform target config section 7e868e7 b833540 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1103312 1103246 -66 -0.0
RAM 179074 179082 8 0.0
bl702 lighting-app bl702+eth FLASH 656542 656476 -66 -0.0
RAM 135041 135041 0 0.0
bl702+wifi FLASH 834052 833986 -66 -0.0
RAM 124589 124597 8 0.0
bl706+mfd+rpc+littlefs FLASH 1066592 1066526 -66 -0.0
RAM 117429 117429 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 895528 895462 -66 -0.0
RAM 105724 105732 8 0.0
lighting-app bl702l+mfd+littlefs FLASH 979616 979550 -66 -0.0
RAM 109924 109932 8 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 821764 821700 -64 -0.0
RAM 120296 120296 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 831640 831560 -80 -0.0
RAM 125440 125448 8 0.0
pump-app LP_EM_CC1354P10_6 FLASH 779200 779120 -80 -0.0
RAM 113844 113844 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 763420 763332 -88 -0.0
RAM 114052 114060 8 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 549650 549574 -76 -0.0
RAM 205256 205264 8 0.0
lock CC3235SF_LAUNCHXL FLASH 581066 580982 -84 -0.0
RAM 205456 205456 0 0.0
cyw30739 light CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 662909 662829 -80 -0.0
RAM 77300 77308 8 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 682761 682681 -80 -0.0
RAM 79940 79948 8 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 682761 682681 -80 -0.0
RAM 79940 79948 8 0.0
CYW930739M2EVB-02 unknown 2040 2040 0 0.0
FLASH 639693 639613 -80 -0.0
RAM 72368 72376 8 0.0
light-switch CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 624069 623981 -88 -0.0
RAM 73604 73604 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 643697 643609 -88 -0.0
RAM 76156 76156 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 643697 643609 -88 -0.0
RAM 76156 76156 0 0.0
lock CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 643413 643325 -88 -0.0
RAM 76604 76604 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 663129 663041 -88 -0.0
RAM 79156 79156 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 663129 663041 -88 -0.0
RAM 79156 79156 0 0.0
thermostat CYW30739B2-P5-EVK-01 unknown 2040 2040 0 0.0
FLASH 619593 619505 -88 -0.0
RAM 70708 70708 0 0.0
CYW30739B2-P5-EVK-02 unknown 2040 2040 0 0.0
FLASH 639445 639357 -88 -0.0
RAM 73340 73340 0 0.0
CYW30739B2-P5-EVK-03 unknown 2040 2040 0 0.0
FLASH 639445 639357 -88 -0.0
RAM 73340 73340 0 0.0
efr32 lock-app BRD4187C FLASH 946164 946084 -80 -0.0
RAM 132092 132096 4 0.0
BRD4338a FLASH 772984 772912 -72 -0.0
RAM 173320 173320 0 0.0
window-app BRD4187C FLASH 1038808 1038728 -80 -0.0
RAM 128228 128232 4 0.0
esp32 all-clusters-app c3devkit DRAM 103400 103408 8 0.0
FLASH 1808076 1808016 -60 -0.0
IRAM 83846 83846 0 0.0
m5stack DRAM 122012 122028 16 0.0
FLASH 1774146 1774094 -52 -0.0
IRAM 117043 117043 0 0.0
linux air-purifier-app debug unknown 4848 4848 0 0.0
FLASH 2793980 2793796 -184 -0.0
RAM 115256 115288 32 0.0
all-clusters-app debug unknown 5656 5656 0 0.0
FLASH 6360090 6359874 -216 -0.0
RAM 533816 533848 32 0.0
all-clusters-minimal-app debug unknown 5528 5528 0 0.0
FLASH 5462590 5462406 -184 -0.0
RAM 225592 225624 32 0.0
bridge-app debug unknown 5560 5560 0 0.0
FLASH 4770786 4770570 -216 -0.0
RAM 204976 205008 32 0.0
camera-app debug unknown 8856 8864 8 0.1
FLASH 6896331 6896235 -96 -0.0
RAM 226648 226680 32 0.0
camera-controller debug unknown 9240 9240 0 0.0
FLASH 14391979 14391787 -192 -0.0
RAM 661552 661552 0 0.0
chip-tool debug unknown 6256 6256 0 0.0
FLASH 14674707 14674549 -158 -0.0
RAM 648288 648288 0 0.0
chip-tool-ipv6only arm64 unknown 40328 40328 0 0.0
FLASH 12625376 12625232 -144 -0.0
RAM 694576 694576 0 0.0
fabric-admin debug unknown 5936 5936 0 0.0
FLASH 12751047 12750857 -190 -0.0
RAM 647648 647648 0 0.0
fabric-bridge-app debug unknown 4808 4808 0 0.0
FLASH 4587388 4587172 -216 -0.0
RAM 191056 191088 32 0.0
fabric-sync debug unknown 5064 5064 0 0.0
FLASH 5733541 5733349 -192 -0.0
RAM 487168 487200 32 0.0
lighting-app debug+rpc+ui unknown 6264 6264 0 0.0
FLASH 5651073 5650881 -192 -0.0
RAM 207544 207576 32 0.0
lock-app debug unknown 5496 5496 0 0.0
FLASH 4824444 4824260 -184 -0.0
RAM 194840 194872 32 0.0
ota-provider-app debug unknown 4848 4848 0 0.0
FLASH 4446286 4446070 -216 -0.0
RAM 184144 184176 32 0.0
ota-requestor-app debug unknown 4800 4800 0 0.0
FLASH 4575890 4575674 -216 -0.0
RAM 188048 188080 32 0.0
shell debug unknown 4272 4272 0 0.0
FLASH 3095276 3095068 -208 -0.0
RAM 148632 148632 0 0.0
thermostat-no-ble arm64 unknown 9624 9624 0 0.0
FLASH 4234160 4234000 -160 -0.0
RAM 231256 231264 8 0.0
tv-app debug unknown 5832 5832 0 0.0
FLASH 6092341 6092165 -176 -0.0
RAM 611640 611672 32 0.0
tv-casting-app debug unknown 5416 5416 0 0.0
FLASH 12873917 12873773 -144 -0.0
RAM 769072 769072 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 921588 921508 -80 -0.0
RAM 167398 167398 0 0.0
nrf7002dk_nrf5340_cpuapp FLASH 913296 913232 -64 -0.0
RAM 145642 145642 0 0.0
all-clusters-minimal-app nrf52840dk_nrf52840 FLASH 858692 858616 -76 -0.0
RAM 141161 141161 0 0.0
nxp contact k32w0+release FLASH 592976 592880 -96 -0.0
RAM 72920 72928 8 0.0
mcxw71+release FLASH 625832 625752 -80 -0.0
RAM 63260 63268 8 0.0
light k32w0+release FLASH 619100 619020 -80 -0.0
RAM 72216 72224 8 0.0
k32w1+release FLASH 710552 710480 -72 -0.0
RAM 72108 72116 8 0.0
lock mcxw71+release FLASH 775632 775560 -72 -0.0
RAM 67692 67692 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1669252 1669172 -80 -0.0
RAM 212400 212408 8 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1573356 1573284 -72 -0.0
RAM 208584 208592 8 0.0
light cy8ckit_062s2_43012 FLASH 1447556 1447468 -88 -0.0
RAM 197296 197296 0 0.0
lock cy8ckit_062s2_43012 FLASH 1477444 1477364 -80 -0.0
RAM 225016 225016 0 0.0
qpg lighting-app qpg6105+debug FLASH 668340 668252 -88 -0.0
RAM 105236 105244 8 0.0
lock-app qpg6105+debug FLASH 626536 626448 -88 -0.0
RAM 99872 99872 0 0.0
stm32 light STM32WB5MM-DK FLASH 466004 465940 -64 -0.0
RAM 141488 141496 8 0.0
telink bridge-app tl7218x FLASH 694618 694572 -46 -0.0
RAM 101908 101908 0 0.0
light-app-ota-compress-lzma-factory-data tl3218x FLASH 774328 774282 -46 -0.0
RAM 50060 50060 0 0.0
light-app-ota-shell-factory-data tl7218x FLASH 781434 781388 -46 -0.0
RAM 109232 109232 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 703580 703534 -46 -0.0
RAM 62624 62624 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 741534 741488 -46 -0.0
RAM 85796 85796 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 607756 607710 -46 -0.0
RAM 119448 119448 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 816744 816702 -42 -0.0
RAM 107500 107500 0 0.0
tizen all-clusters-app arm unknown 5224 5224 0 0.0
FLASH 1822464 1822336 -128 -0.0
RAM 95164 95164 0 0.0
chip-tool-ubsan arm unknown 20600 20600 0 0.0
FLASH 20893806 20893358 -448 -0.0
RAM 9084072 9083856 -216 -0.0

@yufengwangca yufengwangca merged commit 137e0a6 into project-chip:master May 16, 2025
70 of 71 checks passed
@yufengwangca yufengwangca deleted the pr/camera/config branch May 17, 2025 16:27
heinsaar pushed a commit to heinsaar/connectedhomeip that referenced this pull request May 19, 2025
…hip#39009)

* Use common configs to create a GStreamer pipeline

* Don't mix aspect ratios between the min and max resolutions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0