In the beginning, we built code with custom shell scripts.
The problem with shell scripts was they tended to be dumb, knowing only how to build the entire project from scratch.
Then along came Make, which was a particularly great tool for building code.
The good parts of make were very good:
-
incremental builds
-
suffix rules
By deductions based off the modification times of source and object files, make would quickly identify where work was needed to be done, and go ahead and do it. Unlike shell scripts, make was optimised around the goal of building code incrementally.
Make was built around the concept of dependencies:
a.o: b.c c.c
cc -o a.o b.c c.c
Here, b.c and c.c might be source files. If either of them were newer
than a.o, or if a.o didn’t exist, then the command cc -o a.o b.c c.c
would be run.
You could rewrite this make target using 'automatic variables' like this:
a.o: b.c c.c
cc -o $@ $?
Make goes downhill rapidly from here, forcing you to learn a ton of stuff that nowadays looks and feels antiquated.
But Make was very flexible, and could be used to build almost anything that could be expressed in dependency and target files.
Unfortunately, make syntax was baroque. Make relied on unintuitive 'features' whenever it needed to compute something, manipulate strings or lists, and even required tabs in the right places, and might silently fail if you forgot the rules.
Make was also not designed for the hierarchical file system used by Java and others, and for this reason other tools such as Ant, Ivy, Maven and Gradle came along and supplanted Make. However, these tools tended to go back to the idea that it was good to build code from scratch, and people got used to the idea that it was OK to wait ages for something to build. Instead of fixing the problem, they swept the problem away under an out-of-sight build machine or 'continuous integration' server, turning adversity into virtue.
Then Clojure came along and people built things like Leiningen and boot. These suffer from painfully slow start-up times. Leiningen is really a Clojure launch tool rather than a build tool. Boot’s name at least indicates what it’s best for: booting a powerful development environment.
Mach is a remake of make, striving to keep the good parts.
Since you ask, the name is from the German verb, machen (to do, to make), used in the imperative. Mach is as much about 'doing' as 'making', which the German verb captures well.
Mach is in alpha status. You are encouraged to use it if you are prepared to accept some changes if you upgrade. The design of Mach is likely to change in the near future.
Create a simple project and add a Machfile.edn file.
Your very first Machfile.edn might look like this:
{
hallo (println "Guten Tag, Welt!")
}You can invoke Mach with the following:
$ mach hallo
to get the following output:
Guten Tag, Welt!
Clone the mach project.
$ git clone https://github.com/juxt/mach $ cd mach/test
Try the following:
$ mach target
This creates a directory called target.
Try again.
$ mach target
Since target already exists, you should get the following output:
Nothing to do!
Now try this:
$ mach css
If you have the SASS compiler installed, sassc, this will compile the
sass files in sass to target/app.css.
Examine the file Machfile.edn in test/.
This contains a map target entries, indexed by name.
A target can be either an expression or map.
Machfiles are data, and we have chosen the EDN format. If the only reason EDN was better than JSON was because it allows comments, that would be reason enough to choose it. It has many other advantages though.
The Machfile is a map, modelled in a similar fashion to the original Makefile, with targets as keys and dependencies/actions as values.
In Make, targets were intended to be files. However, you’d often want a
task such as 'clean'. To ensure Make wouldn’t confuse 'clean' with a
target file, you would have to declare clean in a .PROXY declaration,
one of many of Makes esoteric conventions.
With mach, consistency is favoured over terseness. All keys are names, not target files. Target files are specified in the values, where necessary.
Using symbols for targets, rather than files, solves another problem with Make: support for directories.
In Unix, a directory’s modification time reflects the last time a directory was modified (files were added or removed), rather than the last time anything in the directory (or sub-directories) changed (which is what you usually want for determining novelty).
Mach allows you to write functions (in ClojureScript!) for determining novelty (i.e. whether a target is fresh or stale, and how so).
Machfile entries have target names (the keys) and targets (actions or maps which describe when and how to build the target).
If you use a map, you can put anything you like in this map, but a few of these keys are special and are described below. Try to avoid using lowercase non-namespaced symbols, since these are reserved by Mach and some of these are discussed below.
A short string of descriptive text
{css {summary "CSS for the website, compiled from sass sources"}}The depends entry contains a list of targets that must be updated (if
stale) prior to this target being considered for update.
{jar {description "A jar file containing Java classes and CSS"
depends [classes css]}}A depends is simply a sequence of targets to check.
The product of a target is the file (or files) that it produces. If you
declare this with the special symbol product it will assist the
'auto-clean' feature of Mach.
Deciding whether a target is stale (requiring a re-build) or fresh (no re-build is necessary) might be a simple procedure or a complex computation. Regardless, Mach asks you to provide a predicate, written in ClojureScript to determine whether or not a target is stale.
Mach provides a built-in predicate to determine if any files in a given directory have been modified with respect to a given file.
The signature of this function is:
(mach.core/modified-since [file dir])The first argument is the file with which all files in the second argument (directory) are compared against. If any file in the directory has been modified more recently than the file in the first argument, the predicate returns true.
For example:
{css {novelty (mach.core/modified-since "target/app.css" "sass")}}It is also possible to express this target like this:
{css {target "target/app.css"
novelty (mach.core/modified-since target "sass")}}This illustrates that symbols in ClojureScript expressions are resolved with respect to the given map, which defines the scope for all expressions. This allows you to surface key values as declarations, accessible from within local ClojureScript expressions and also exposed to other tools. These values are also accessible by other targets, via references (see below).
If novelty is detected, a target is updated by calling the update!
function. The terminology here is intended to align with our
skip project.
The update! expression must do whatever is necessary to rebuild
(freshen) the target.
{css {target "target/app.css"
novelty (mach.core/modified-since target #ref [sass dir])
update! (apply mach.core/sh (concat ["sassc"] novelty [">" target]))}}In the update! expression can be side-effecting (and should be!).
Often, an update! expression will reference the value of novelty to
reduce work.
One of the best design decisions in the original Make tool was to
integrate closely with the Unix shell. There are countless operations
that are accessible via the shell, and Mach strives to encourage this
usage via its custom EDN tag literal #$.
clojure {hello-internal (println "Hello World!") hello-external #$ ["echo Hello!"]}
The #$ tag literal is a short-cut to the built-in Mach function
mach.core/sh.
Make makes heavy use of variables, in the spirit of DRY (Don’t Repeat Yourself). Often, this leads to obfuscation, variables are defined in terms of other variables, and so on.
Mach achieves DRY without endless indirection by using references (the
same way Aero does it) - key values can be
declared in a target and referenced from other parts of the Machfile,
via the #ref tag literal.
{
src {dir "src"}
classes {update! (compile #ref [src dir])}
}The #ref tag must be followed by a vector of symbols which target the
required value.
A target can optionally be called with a verb.
For example:
mach pdf:clean
This calls the pdf target with the clean verb, which removes any
files created by the target (declared in product).
This calls the update! (or produce) expressions, regardless of
whether the target if fresh or not. No dependencies are called.
Since derived files are declared with product, Mach is able to
automatically determine how to clean a target. Therefore, you don’t need
to specify a special rule, conventionally called clean, to clean up
derived files.
Mach can read Aero (EDN) config files. Aero is an ideal solution for storing the configuration data that drives build processes because it allows comments, supports the specification of multiple environments together in version control and has extension points for powerful customisation.
(mach.core/read-config "config.edn" {:profile :prod})
This opens up the possibillity to keep all your data in Aero and use the same data in your application, build and (dev)ops. Don’t repeat yourself (DRY) for configuration up and down your stack.
Status: experimental and incomplete
Mach sits on the extensive nodejs eco-system. If you need to do anything remotely complex, feel free to pick from the quarter-million-plus modules available to you.
Mach is built on lumo by António Nuno Monteiro.
The goal of Mach is to create something that is capable of building
complex systems as well as running them. One option is to use Mach to
generate a classpath from a project.clj (lein classpath) and use that
to run Clojure applications with java directly, avoiding the use of lein
and its associated memory costs. It might also be possible to make more
judicious use of AOT to speed things are further - by utilising
file-system dates, it is possible to detect staleness and fix it when
necessary - say if a project.clj is determined to be newer then the
classpath can be regenerated.