Makefiles

The make utility is way to manage compute graphs; you encode the graph in a file, usually named Makefile (case-sensitive).

Makefiles are a sequence of recipes, each of the form:

target: dependency1 dependency2 ...
	<script>

Here target is the output of a build command, dependency* are the immediate inputs to that build process. The shell script must be indented by exactly one tab symbol \t. Recipes can be multi-line, but each line is executed in a separate shell by default, so shell variables and context is not preserved.

The command make builds the first target that appears.

Basic example

We first generate some text; save to wikipedia.txt

Wikipedia was launched on January 15, 2001, by Jimmy Wales and Larry
Sanger. Sanger coined its name^[5]^[6] as a portmanteau of "wiki"
and "encyclopedia". It was initially an English-language encyclopedia,
but versions in other languages were quickly developed. With
6.2 million articles, the English Wikipedia is the largest of the 317
Wikipedia encyclopedias. Overall, Wikipedia comprises more than
55 million articles,^[7] attracting 1.7 billion unique visitors per
month.^[8]^[9]

We will use make to build a list of words (with repetition) and a frequency histogram:

wikipedia.txt.lower: wikipedia.txt
	tr A-Z a-z wikipedia.txt > wikipedia.txt.lower
wikipedia.words: wikipedia.txt.lower
	grep -o '[a-zA-Z]\+' wikipedia.txt.lower > wikipedia.words
wikipedia.words.hist: wikipedia.words
	sort wikipedia.words | uniq -c > wikipedia.words.hist

Enter the command

make wikipedia.words.hist

The make program will read Makefile, and build a graph which will produce wikipedia.words.hist, and execute it.

Add the following paragraph to wikipedia.txt:

Wikipedia has been criticized for its uneven accuracy and for
exhibiting systemic bias, including gender bias, with the majority of
editors being male.^[4] Edit-a-thons have been held to encourage
female editors and increase the coverage of women's topics.^[10] In
2006, Time magazine stated that the open-door policy of allowing
anyone to edit had made Wikipedia the biggest and possibly the best
encyclopedia in the world, and was a testament to the vision of Jimmy
Wales.^[11] The project's reputation improved further in the
2010s as it increased efforts to improve its quality and reliability,
based on its unique structure, curation and absence of
commercial bias.^[4] In 2018, Facebook and YouTube announced that they
would help users detect fake news by suggesting links to
related Wikipedia articles.^[12]

Re-run make wikipedia.words.hist and watch as make detects that wikipedia.txt has changed and thus re-making wikipedia.txt.lower, causing a re-make of wikipedia.words, which then causes wikipedia.words.hist to be re-made.

You can set the default target using the .DEFAULT_GOAL variable:

.DEFAULT_GOAL := wikipedia.words.hist
wikipedia.txt.lower: wikipedia.txt
	tr A-Z a-z wikipedia.txt > wikipedia.txt.lower
wikipedia.words: wikipedia.txt.lower
	grep -o '[a-zA-Z]\+' wikipedia.txt.lower > wikipedia.words
wikipedia.words.hist: wikipedia.words
	sort wikipedia.words | uniq -c > wikipedia.words.hist

Now just typing make is the same as make wikipedia.words.hist.

Magic variables

Each recipe script can refer to the magic variable $@ (the target), $^ (the list of dependencies), $< (the first dependency).

Here's the example rewritten with these:

.DEFAULT_GOAL := wikipedia.words.hist
wikipedia.txt.lower: wikipedia.txt
	tr A-Z a-z $< > $@
wikipedia.words: wikipedia.txt.lower
	grep -o '[a-zA-Z]\+' $< > $@
wikipedia.words.hist: wikipedia.words
	sort $< | uniq -c > $@

Pattern recipes

If you have multiple similar recipes, you may be able to write a pattern recipe to handle them. Pattern recipes break the target into stem + suffix; the stem (denoted by % in the target) can then be referred to in the dependencies (also as %) and in the build script (as $*). Here's the example generalised to work for any language:

%.txt.lower: %.txt
	tr A-Z a-z $< > $@
%.words: %.txt.lower
	grep -o '[a-zA-Z]\+' $< > $@
%.words.hist: %.words
	sort $< | uniq -c > $@

Now there is no default target; you have to type explicitly make wikipedia.words.hist; assuming all necessary dependencies exist, make will follow the rules correctly. But if you create other text files in the same directory, say twitter.txt, you will also be able to run make twitter.words.hist.