DRAFT: add a CMake based makefile to build #1601
Conversation
|
Hi @robUx4 I have merged 8 out of 10 PR's that you separately posted and which are present in this one. To avoid one additional merge commit per PR, I rebased them first. This has the unfortunate side effect that GitHub is unable to flag those PR as "merged" and I was obliged to manually close them. However, don't worry, your commits are here (the contributors sections on the code page already accounts 9 commits from you). For the two additional PR's (inline methods referencing private inner classes), I have used another solution and didn't merge your commits. Please check if it works for you. Concerning this draft PR about CMake, the hardly maintainable list of source files is exactly what I don't want to see in TSDuck.
That is precisely what the "Modern CMake" zealots considered as a no-go. They explained that this "does not work" but their explanations were not very convincing for a CMake newbie like me. So, I am a bit perplex. I see two difficulties with CMake. I don't know if they can be solved.
So, if we keep the GNU Make and MSBuild systems, using CMake is only there to use clang on Windows and I don't see any benefit. This is just added complexity, and just for a "non-natural" build combination which is not used outside your project. |
Yes this is frowned upon. For one thing if you add or remove a file, CMake (and in the underlying build system) won't notice and will not update the build accordingly. This is not a big deal in the CI, it becomes a problem if you switch between branches locally to develop. But running CMake is quite fast so it's not a major problem.
The complexity about the varying toolchains is usually hidden in a CMAKE_TOOLCHAIN_FILE. In VLC we build for a lot of different platforms and run CMake and meson on all of them for our "contribs". Once you have a setup working for a platform, the rest is straightforward. For Android the NDK provides the toolchain file so there is nothing special to do (apart from picking the Android version you target). Android is one of the targets we will probably use tsduck with. For the dependencies, there are standard ways of finding them. This is even easier when they were built with CMake (like SRT). Alternatively it's possible to plug a package manager into it like vcpkg which uses CMake too. You can use the existing dependencies or add your own and build exactly with what you want, from source. CMake can also download tarballs to build "contribs", so it could download SRT, RIST, etc if you don't have them and build with them.
I think it generates one . |
I assume that having a common platform for many targets makes the integration of new projects easier and faster.
In the latter case, a basic spec would be:
If you prefer to stay with a specific integration in your VLC build system - which is perfectly acceptable - I can only recommend to avoid hardcoded lists of source files. TSDuck is very modular. The DTV standards include lots of signalization tables and descriptors. New ones appear on a regular basis. Each of them is a C++ class, meaning 2 source files + an XML configuration file + an asciidoc file + an optional "names" configuration file. So, you'd better implement a logic of assigning a build strategy to each source file depending on its location but never assume a predefined list of files. And, BTW, I am still curious about the reasons for a VLC / TSDuck integration and how you plan to implement that. Do you plan to keep the standard TSDuck modularity (libtscore + libtsduck + configuration files + plugin shared objects + executable tools) or do you plan to build a big library of what you need only? |
From what I've seen both from the makefiles and the VS project that is doable. I haven't looked at your PowerShell scripts to add the third party code. Even these could probably be merged in the CMake part in a more portable way.
The VLC integration would be through another library that uses tsduck, VLC itself would not know about tsduck (apart from the linking parts it needs to build). All the contribs we build are built as static library so we don't even care too much about the DLL aspect/fixes. Also we have patches for a lot of contribs although we usually try to "upstream" as much as possible so we don't have to maintain local patches and ensure it's correct according to the upstream project. One of the thing we often have to patch is the build system that can't handle all the various targets we use. So it would not be uncommon to have our own CMake file aside from your code. But if it can be "upstreamed" that would be a lot easier for everyone. |
|
I removed the parts to build with mingw-w64 so there's only the CMake project and test of that project in the CI (winget seems to fail on some of them, not sure why). |
This is a recurrent issue on GitHub CI runners. They do not integrated winget by default. The installation of winget som
8000
etimes fails for some unknown reasons. I try to make the script |
|
I added a first |
|
I did the rest of the file globbing to be closer to the other build systems and be more flexible. There's still a lot missing but at least it helps checking it build with clang-cl, in addition of the existing CI targets. |
I agree with that. About the C++ code, I am ready to merge any patch which improves the code. This is good for the project. Feel fr 8000 ee to submit PR's when necessary. About the build system, I am open to consider a migration from GNU Make / MSBuild to CMake if we can get the same features, flexibility, and list of targets as the current TSDuck build system. If it covers only a subset of the needs, then I prefer that you keep it and maintain it in your project. The problem with separate build systems is that I cannot reproduce your builds, cannot run non-regression tests in your configuration and cannot guarantee that some future code change will not break your build. |
I understand and I will do that. The problem is that we only need a subset of what tsduck does. So putting extra work to support every single thing that tsduck can do is not on the map for now.
As said earlier, we could have the CMake build system on the side, as not officially supported, but nightly (or weekly) and updated/improved gradually. From what I've seen CMake should be capable of handling of these your 2 build systems do, including Python and Java integration. I will keep updating this branch when I can, it should at least build and pass the tests. |
There is one pitfall you should be aware of when putting TSDuck code into static libraries. Given the constraints that 1) I don't have much time and 2) there is a lot of code into the project with regular additions, I pay attention to remove all redundancies because they need extra work and are error prone. No hard-coded lists of files is one example: Adding a new source file shall be self-sufficient. Updating a file list is redundant extra work which will be easily overlooked. Another example is self-registration of C++ classes. New plugins, tables, descriptors, extension features, are implemented as separate C++ classes. Adding the class in the code base shall be sufficient. No redundant list of initialization calls, tables of pointers, or external init sequence shall be necessary. We achieve this using self-registration macros (explained here). As soon as a new C++ class for a descriptor is added, the implicit initialization code of the compilation unit automatically registers it into the signalization repository. This is very convenient, no risk to forget some external initialization requirement. The consequence of this system is that new features are pushed into the code base. They are not pulled by the application. In the standard TSDuck build, all classes are automatically included in the relevant shared libraries, depending on the location of the source file. When the OS loader activates the shared library, all modules are initialized and all features are self-registered. However, if all modules are placed in a static library, the linker will only include the objects which are explicitly pulled, using symbol references. Concerning the signalization, the application may explicitly reference a few common stuff such as a PAT, PMT or CA descriptor. However, the vast majority of the signalization will not be pulled. If your application analyzes the signalization of a stream, most of if won't be handled properly. In the makefile of libtsduck, when building the static library, we build intermediate big object files gathering all tables, all descriptors, etc. So, from the application, as long as you reference one table for instance, you pull the big object which contains all of them. Using static libraries in TSDuck is not very common. We use for test only and because a few people wanted it to build on small embedded systems. If you create your own build system based on static libraries only, you should keep this in mind. |
ecf09c0 to
af7b964
Compare
The build either generates DLLs or static builds depending if BUILD_SHARED_LIBS is defined [^1]. The libraries are tsduckdll and tsducklib as with the MSVC build. [^1]: https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html
This is an initial draft of something working for me for Windows and Linux targets.
It doesn't include the whole binaries and doesn't handle tests for now.
file(GLOB) could probably be used rather than hardcoding every file.