PTB beta update Psychtoolbox 3.0.20 "Appletizer" #855
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Resync with public beta 3.0.19.15 release.
To avoid deprecation/unsupported warnings now, and failure in the future.
These are for the LexActivator SDK and runtimes.
To make troubleshooting easier for user.
… Windows. Tested against Octave 7.3 64-Bit Intel on Windows-10 and Windows-11.
Used for license checking and management for prebuilt mex files which are included in the Psychtoolbox distribution and require paid software licensing on target platforms like Octave and Matlab on macOS and Windows. Uses Cryptlex as license management provider, and their LexActivator SDK and client libraries. See: https://cryptlex.com This code is implemented and currently used on behalf of Psychtoolbox commercial host company, the Medical Innovations Incubator GmbH, in Tübingen, Germany. Website: https://psychtoolbox.net Any questions wrt. the software licensing should be directed to them under the following current address, as of November 2024: Medical Innovations Incubator GmbH Eisenbahnstr. 63 72070 Tübingen Germany Commercial register: HRB 751684 Register court: Local court Stuttgart, Germany Contact E-Mail address for inquiries: info@mi-incubator.com Signed-off-by: Mario Kleiner <kleiner@mi-incubator.com>
Add path to PsychPlugins subfolder for 64-Bit Intel or ARM plugin dll's, depending on processor build architecture. This is future prep for a Windows-on-ARM variant, although no immediate plans exist for such a variant.
Make sure mex files that aren't license managed and don't use PTB core infrastructure still build in the future. Currently these are moglcore, moalcore, pnet and SRAnipalMex. Suppress some pointless compiler warnings under Octave's POSIX like MinGW build system on Windows, for moglcore and PsychOculusVRCore1.
Caused by using a POSIX/Unix MinGW build system for Octave on Windows, where EWOULDBLOCK is a defined - but pointless for our purpose here - error code, so redefining it to something meaningful without first undefining it caused a compiler warning.
…-open-error Fix 'AttributeError' raised when creating a stream fails.
…ure drivers. AMDVLK v-2024.Q4.2 will have a suitable bug fix, so the workaround ain't needed anymore on Navi+ / RDNA gpu's supported by that driver. The fix is untested and not testable by myself as I don't have a RDNA+ gpu, but I performed a full code review and conclude that the bug should be fixed by that fix, which now implements dynamic allocation of suitable data structures, so no hard mode count limit anymore and future proof. Bug report: GPUOpen-Drivers/xgl#179 Bugfix merged 5.11.2024: GPUOpen-Drivers/xgl#180
Also add unrelated PsychColorCorrection to contents.m file for PsychGLImageProcessing.
Define preliminary target folder for mex files built for/on 64-Bit ARM. Not neccessary the final word on this, but will do for current experiments. Enable basic building on Fedora Linux and spins/remixes for 64-Bit cpu architectures. Build WaitSecs to identify as a Wayland mex file when building on a Wayland system. This mostly for the benefit of LM stats collection about Linux WSI type.
The target folder for these files may not be the last word, but it will do for now, ie. current experimentation.
Wayland lacks all required standardized protocol and functionality to establish identity pixel passthrough, e.g., no gamma table control or dithering control for regular non-privileged applications. For now, simply turn this into a no-op, and user has to configure identity pixel passthrough manually via a Wayland compositor specific means, if such a thing exists.
…hw cursors. Some 64-Bit ARM platforms display engines or drivers do not support hardware cursors, so displaying a mouse cursor will enforce fallback cursor rendering that may impair visual timing significantly, which is not what we want in these test scripts. As a workaround, HideCursor() the cursor on 64-Bit ARM for that reason. When we are at it, convert PerceptualVBLSyncTestFlipInfo.m from DOS CR+LF to Unix LF line endings to make things less disgusting.
…d sample rate. Some recent sound chips don't like our hard-coded 44100 Hz, so leave it to the audio system to choose what it can handle. Doesn't matter if sound sounds weird during this test, as it only fills the purpose of creating some processing load in the audio system, not to please the ears of the listener.
…switcher trouble.
PsychVideoSwitcher('SwitchMode') is unreliable when used under Wayland
on some 64-Bit ARM gpu's. Skip it when running timing tests that utilize
the Videoswitcher on such hardware + software setups.
…scaling. These are hacks, and only half of them work halfways, but better than nothing, and if only to document what doesn't work well so far, so we don't repeat past mistakes again out of forgetfulness... Especially the hack in PsychWindowGlueWayland.c was totally useless, and is therefore coded to be a no-op due to bscale = 1; as other bscale values do more harm than good. The hack in PsychScreenGlueWayland.c is implemented in the wrong place, but shows some promise, just needs a more careful implementation for future production use.
Wrong include path for libfreenect headers, dead and unused since many years, but not a problem on Debian/Ubuntu for some reason. A very real problem on some other distros though.
Suppress some pointless compiler warnings for moglcore, moalcore on recent gcc compiler versions. Make PsychOpenXRCore and PsychCV build conditional on availability of OpenXR SDK and Apriltags library. Only avoid PsychOpenXRCore build on 32-Bit ARM, but allow on 64-Bit ARM. Allow PsychCV build on ARM systems if Apriltags library is available on those systems.
Name is libptbdrawtext_ftgl_arm64.so.1
Same ones we get on Octave, lets declutter our lifes.
… for format 7. pixelFormat 8 used the FOURCC 'Y800' until December 2024, but it turns out that Y800 is no longer supported as a FOURCC, already since GStreamer 1.0 (removed 29th May 2012), so selecting format 8 in the old days didn't yield any performance benefits - in fact it caused a slight degradation, and selecting it in recent years just caused playback failure. Going forward, we will treat pixelFormat 8 as an alias for pixelFormat 7, for working playback with full compatibility and no known downsides.
A dumb missing if-elseif ladder caused PsychCreateTexture() to be called twice per movie texture creation, the 2nd call with wrong parameters and a NULL pointer as win->textureMemory, which causes glTexSubImage2D() on at least Linux/Mesa to harmlessly no-op (everything works by accident), whereas the macOS OpenGL implementation crashes with that NULL pointer. -> With the fix, pixelFormat 6 also works on macOS.
…ader. A reimplementation of isnan() for macOS GLSL 1.20 lack of isnan().
… on maOS. On macOS disable use of hardware accelerated video decoding for movie playback when pixelFormat 11 for WCG/HDR/deep color playback is requested. Turns out that Apples applemedia hardware decoder plugins vtdec and vtdec_hw can only decode to YUV I420 8 bpc if our pixelFormat 11 / HDR decode path is used, not to any of our supported other higher precision (semi-)planar formats. This would degrade color precision to at most 8 bpc, which is bad for precise color playback, playback of HDR or wide color gamut WCG content, with very visible banding especially for HDR playback! Long term we would need to implement dedicated decode shader to decode from the only supported high precision packed pixel 16 bpc formats AYUV64 or RGBA64, or wait for potential improvments on the GStreamer side. For now we simply enforce software decoding with non macOS specific codecs like libav codecs, e.g., avdec_h264, avdec_h265 etc. where we get good quality and precision. Also fix some status output messages and update some copyrights.
Extract video frame width x height from video frame during playback if frame size could not be found during 'OpenMovie'. Rarely needed, but redundancy is good.
…e it. If the pixel row stride of a video frame can be detected from GStreamer, use it during texture creation for pixelFormat 6 and 11, ie. for (semi-)planar decode. Fall back to simple video frame width as before if stride info is unavailable. This fixes movie playback for two movies on macOS for Apple Silicon, which failed there, but worked fine on macOS Intel, Linux and Windows. Some of the macOS Apple Silicon hardware accelerated decoders have unusual stride requirements.
The new win->textureStridePixels window record field allows setting a stride in pixels as GL_UNPACK_ROW_LENGTH. The init default of zero means to just use texture source image width as stride, as in the past. This way we don't need shady tricks with texture size during PsychCreateTexture() to select specific stride for source memory image buffers which need this.
Apply it to define stride for pixelFormat 6 and 11 texture creation in a more efficient way.
Screenshot from backBuffer, not from (default) frontBuffer. The backBuffer screen grab behaves also correct with external display backends like Vulkan/Metal, as auto-enabled on macOS for Apple Silicon, whereas a standard frontBuffer screen grab goes wrong there, capturing the welcome screen logo.
Since GStreamer 1.20, which is the minimum required GStreamer version for Apple Silicon macOS, video recording was broken. This also on macOS for Intel Macs, but there users could work around the problem with a downgrade to GStreamer 1.18. Now we need a better fix for Apple Silicon. Debugging and reading GStreamer source code suggests roughly the following problem: When video recording is used, the element after avfvideosrc exposes caps that allow use of GLMemory as image backing storage, ie. OpenGL textures, but the plugin can not actually handle that for some reason and crashes. So we need to try to prevent that, and do so by adding a capsfilter element behind the avfvideosrc, which limits allowable src pad caps to standard video/x-raw, disallowing use of GLMemory, thereby preventing the crash. This seems to work, as tested with GStreamer 1.24.10 on macOS 14.5 on an Apple M2 Pro with builtin FaceTime HD camera. VideoRecordingDemo.m now works and writes a valid movie, with and without live video feed. VideoCaptureDemo.m continues to work fine. What still doesn't work on macOS for Apple Silicon is sound recording though. Enabling that will freeze the video recording after a few seconds of recording. When we're at it, update VideoRecordingDemo.m to use the default codec for movie recording, as that now seems to work again.
-> macOS Sonoma 14.5 build system. XCode 15.1. -> Octave 9.2 and Matlab R2024b. -> Various GStreamer related fixes and improvement. -> Better video playback. Working video recording, no sound recording though.
GStreamer 1.24.10 on macOS fixes playback of some movies.
-> macOS Ventura 13.7.1 build system. XCode 14.0.3. -> Octave 9.2 and Matlab R2024b. -> 4th rebuild for upcoming Psychtoolbox 3.0.20. -> Various GStreamer related fixes and improvement. -> Better video playback. Working video and sound recording. Tested GStreamer 1.22.0.
-> Ubuntu 20.04.6-LTS build system for 64-Bit Intel machines. -> Octave 5.2, Matlab R2024b. -> Some GStreamer related improvements.
-> Windows 10 22H2 build system. MSVC 2019 and MinGW 64. -> Octave 7.3 and Matlab R2024b. -> 2nd rebuild for upcoming Psychtoolbox 3.0.20. -> Screen GStreamer and status output refinements. -> PsychVulkanCore refinements.
The f$%§$ bug is back on AMD graphics on Windows-10, for no apparent reason. Disable fs exclusive for worse presentation timing reliability, but at least a picture instead of a black screen and hang. Fracking hell!!
-> Ubuntu 20.04.6-LTS build system for 64-Bit Intel machines. -> Octave 5.2, Matlab R2024b. -> Resync with other OS platforms.
Ubuntu 22.04.5-LTS build system, Octave 6.4.
RaspberryPi OS 11.10 build system for 32-Bit ARM. -> Syncs RaspberryPi with the desktop Linux variants for Intel.
-> macOS Sonoma 14.5 build system. XCode 15.1. -> Octave 9.2 and Matlab R2024b. -> Tone down timestamping timeout debug messages a bit to be less scary.
…pported. This does not yet work on the Vulkan display backend, the VR/OpenXR display backends for VR/AR/MR/XR, and on other external display backends. Especially it does not work on Apple Silicon Macs for fullscreen windows, where Vulkan will be used as display backend. It would crash hard! Therefore, if one tries to call 'AsyncFlipBegin', fall back to a safe standard sync Flip, and issue a warning message every 500 attempted async flips, ie. a throttled performance warning. Other modes requiring the background flipper thread have the same limitation, e.g., frame sequential stereo mode 11 (or fallback from mode 1 -> 11), and VRR modes, but those already protect themselves against such unsupported use. TODO: Remove these limitations at least for async flips and similar with Vulkan backend soon.
-> macOS Sonoma 14.5 build system. XCode 15.1. -> Octave 9.2 and Matlab R2024b. -> Protect against "deadly" AsyncFlipBegin and fall back to Flip, with warning.
-> Ubuntu 20.04.6-LTS build system for 64-Bit Intel machines. -> Octave 5.2, Matlab R2024b. -> Psychtoolbox 3.0.20 should be ready for initial release!
Resync with stuff from public master...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the first release with beta quality macOS Apple Silicon Mac support for native use on
Apple Macintosh computers with 64-Bit ARM Apple proprietary M1, M2, M3, ... SoC's.
We thank the labs of Keith Schneider at the University of Delaware and of
Bas Rokers at NYU Abu Dhabi for substantial sponsorship to push Apple
Silicon support over the finish line.
The software license management code has been added on behalf of
Psychtoolbox commercial host company, the Medical Innovations Incubator GmbH,
in Tübingen, Germany:
Compatibility changes wrt. Psychtoolbox 3.0.19.16:
Psychtoolbox 3.0.20 and later on Apple macOS and Microsoft Windows
will require purchase of a paid software license key, once the
initial time-limited free trial period after first installation and
use has expired. Psychtoolbox will not work after the end of the trial
period without such a paid license key. The Psychtoolbox variants for
Linux on Intel processors and 32-Bit RaspberryPi will remain free to
use for the time being.
The free time-limited trial period will last until the launch of
our new online shop, ie. until you will be able to buy software
license keys. The estimated date of launch, and end of the free
trial, will be likely around early to mid January 2025.
Follow the news on http://psychtoolbox.org and on our Discourse
user forum or our social media feeds @psychtoolbox on X / Twitter,
https://neuromatch.social/@psychtoolbox on Mastodon, or on BlueSky
@psychtoolbox.bsky.social for announcements of the launch of the
software license key sale early in the year 2025, or go to
https://psychtoolbox.net to find out how to buy a key, once sales
has started.
Highlights:
Initial "Beta" Apple Silicon support for running Psychtoolbox on macOS
for Apple Silicon Macs. Psychtoolbox can now be used with native Octave
and Matlab for Apple Silicon ARM Macs, ie. machines with Apple M1,
M2, M3, M4, ... SoC's. Most basic functionality should work reasonably
well, and substantially better than when using older Psychtoolbox
versions via Matlab for IntelMacs, which had various severe limitations
which would make them painful to use and deeply hazardous for any real
data collection! This versions Apple Silicon support was tested by myself
on a Apple MacBookPro early 2023 with 16 inch Liquid Retina XDR display
and Apple M2 Pro SoC under macOS 14.5 Sonoma with 64-Bit native Octave 9.2
from HomeBrew, and with Matlab R2024a and with Matlab R2024b. It was also
more lightly tested by others on a similar 16 inch MacBookPro late 2023 with
M3 Pro SoC under the macOS 15.2 Sequoia beta under Matlab R2024b, and a
MacBookAir 2020 with M1 SoC, and a Mac Studio 2023 M2 Ultra and MacBookAir
M3 13 inch early 2024, the latter two with macOS 15.2 beta and Matlab R2024b.
Thanks to our volunteer testers.
While surely some bugs or limitations still exist, this should be a first good
release for basic use and evaluation.
Known limitations are lack of frame-sequential stereo support in stereo modes
1 and 11, Screen Async flips, ie. subfunctions 'AsyncFlipBegin', 'AsyncFlipEnd'
and 'AsyncFlipCheckEnd' are unsupported. Certain glitches in frame presentation
timing are still observed under certain conditions at least on macOS 14, e.g.,
stalls / short freezes after the stimulus image was static for a while, e.g,
observable with MouseTraceDemo if not moving the mouse for a while. These may
also trigger printing of some warning messages into the Matlab or Octave Window
about "...Failed to retrieve ... stimulus onset timestamp! Timed out.". It is
not yet clear if these issues are still present on macOS 15 Sequoia as well,
testing to be done. These remaining timing glitches are almost certainly due to
Apple macOS operating system bugs, which will either need proper fixing by Apple,
or need invention of new creative workarounds on our side, if that is needed and
possible.
Apples "ProMotion" display mode on some builtin Retina displays, e.g., of the
MacBookPro models, and variable refresh rate modes in general, are not supported
and will cause more erratic behaviour. Make sure to switch to a fixed refresh rate
mode in display settings instead on such machines.
Movie playback, movie writing and video capture works well with GStreamer 1.24.10,
but due to what seem to be shortcomings or bugs of current GStreamer releases,
video recording on Apple Silicon only works for video, not when also trying to
record audio - otherwise hangs will occur. On Intel Macs, video recording does
not work at all with GStreamer 1.24.10, but does for both video and audio with
GStreamer 1.22, however with the downside that playback of some movies fails.
So both GStreamer 1.22 and 1.24 have some limitations, pick the variant with
limitations you can better live with, until GStreamer is properly fixed in some
future GStreamer version.
You should probably not yet use this release for real data collection if
highest reliability of visual stimulus presentation timing is required. Use
for less demanding scenarios may be fine, but tread carefully with this early
release!
All:
Screen: Fix PsychNormalizeTextureOrientation() for multi-window use.
Use of multiple onscreen windows could cause the wrong OpenGL context
of the wrong onscreen window to be in use during operation. Fix this.
Cfe. https://psychtoolbox.discourse.group/t/unknown-problem-playing-movies-with-gstreamer-on-ubuntu-linux/4876/8
Screen: Various improvements to movie playback. E. 8000 g., improve reliability of
playback with pixelFormat 6 and 8.
Screen: Reduce chattiness of some status/debug output in general. Generally
refine status output a bit.
AlphaRotateDemo.m: Small fixups for weirdly sized displays and formatting fixes.
Especially useful for the Retina displays of 16 inch Apple Silicon MacBookPro,
with uneven horizontal resolution.
PsychPython/audio.py: Fix 'AttributeError' raised when creating a stream fails.
Contributed by GitHub user @mdcutone
PsychColorimetric/LMSToMacBoyn(): "Add an example and save the correct version",
some poorly documented fixes or improvements by the Brainard lab with mostly
void of information commit message.
Various other smaller refinements, bug fixes and documentation updates.
Linux:
Psychtoolbox was built and tested against Matlab R2024b and Octave 5.2 and later.
PsychVulkan: Only use workaround against an AMDVLK bug that causes crash if the
used Vulkan display monitor has more than 64 video modes on AMDVLK driver versions
before v-2024.Q4.2, as the bug has been fixed in the upstream driver v-2024.Q4.2,
released early December 2024.
Windows:
Psychtoolbox was built and tested against Matlab R2024b and Octave 7.3.
Screen: Drop GStreamer DirectShow video capture. Prioritize Mediafoundation.
DirectShow video capture is unmaintained and strongly discouraged by GStreamer
folks since the year 2022, and indeed testing shows it is quite broken now.
WDM-KS kernel streaming is also deprecated since the year 2022. MediaFoundation
capture is the only recommended way forward. Limited testing on Windows 10 22H2
also shows it to be the most capable and stable right now (tested GStreamer 1.22).
Drop support for DirectShow dshowvideosrc completely, and prefer Mediafoundation
mfvideosrc over the deprecated Kernel Streaming ksvideosrc.
PsychVulkan/Windows: Disable fullscreen exclusive mode on shoddy AMD Vulkan drivers
again. This still does not work reliably and often leads to black screens instead
of stimulus display.
macOS:
Psychtoolbox was built and tested against native Matlab R2024b and against native
Octave 9.2 from HomeBrew, on macOS 13.7.1 Ventura for Intel Macs, and on macOS 14.5
Sonoma for Apple Silicon Macs.
Initial support for running Psychtoolbox on macOS for Apple Silicon Macs with ARM
native Matlab and Octave.
Bump minimum required macOS version to macOS 10.13 "High Sierra". Effective this
means macOS 10.13 for IntelMacs, and macOS 11 "Big Sur" for Apple Silicon Macs
are required to run Psychtoolbox 3.0.20.
Of course this new minimum is a bit redundant, because all macOS versions older
than macOS 13 are completely unsupported by Apple since October 2024, so running
anything older is at least a high computer security risk, and has lots of unfixed
remaining bugs and is therefore not a good idea. Additionally macOS 13 is needed
at a minimum on Apple Silicon Macs to allow proper operation of visual stimulators
from VPixx and Cambridge Research Systems.
Fix video recording on macOS. Works fully with video + audio on macOS for Intel
Macs with GStreamer 1.22. Works with video only, no audio, on macOS for Apple
Silicon Macs.
Implement initial pixelFormat 11 movie playback support on macOS for playback of
Deep color, Wide color gamut (WCG), and High Dynamic Range (HDR) movies. This
provides 10 bpc deep color, WCG and HDR, but at a lower performance than on other
operating systems, as Apples prehistoric OpenGL 2.1 implementation forces us to
use a less efficient decoding shader, and hardware accelerated video decoding can
not be used as of GStreamer 1.24.10, as it would reduce color precision to only 8
bpc which is utterly insufficient for WCG and HDR playback, causing significant
banding and HDR artefacts.
Make pixelFormat 6 playback work without crashing.
Screen: Make Intel iGPU pageflip checking on recent macOS for Intel graphics chips
more robust. This may improve robustness of correctness checks on Intel integrated
graphics on Intel Macs with PsychtoolboxKernelDriver installed. Problem seen and
fixed on Intel Kabylake Gen 9.5 graphics of MacBookPro 2017.
Fix some crashes on Octave for macOS when using Psychtoolbox after issuing a
"clear all", or "clear mex", or "clear Screen", or "clear PsychVulkanCore" command.
Enjoy!