4.5.2 Examining and Using Environment Variables

One way to see the value of an environment variable is to use the printenv command:

At any point in an interactive command or in a shell script, you can tell the shell that you want the value of the environment variable by prefixing its name with the $ character:

Here, echo is a standard Unix command that copies its arguments to its output, in this case the screen.

By convention, environment variables are virtually always written in all capital letters² .

There may be times when the Workbook instructions tell you to set an environment variable to some value. To do so, type the following at the command prompt:

If you read bash scripts written by others, you may see the following variant, which accomplishes the same thing:

4.6 Paths and $PATH

Path (or PATH) is an overloaded word in computing. Here are the ways in which it is used:

Lowercase path

can refer to the location of a file or a directory; a path may be absolute or relative, e.g.
/absolute/path/to/mydir/myfile or
relative/path/to/mydir/myfile or
../another/relative/path/to/mydir/myfile

PATH

refers to the standard Unix environment variable set by your login scripts and updated by other scripts that extend your environment; it is a colon-separated list of directory names, e.g.,
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin.
It contains the list of directories that the shell searches to find programs/files required by Unix shell commands (i.e., PATH is used by the shell to “resolve” commands).

“path”

generically, refers to any environment variable whose value is a colon-separated list of directory names e.g.,
/abs/path/a:/abs/path/b:rel/path/c

In addition, art defines a fourth idea, also called a path, that is unrelated to any of the above; it will be described as you encounter it in the Workbook, e.g., Section 9.8.8.

All of these path concepts are important to users of art. In addition to PATH itself, there are three PATH-like environment variables (colon-separated list of directory names) that are particularly important:

   LD_LIBRARY_PATH

(Linux only) used by art to resolve dynamic libraries

   DYLD_LIBRARY_PATH

(OS X only) used by art to resolve dynamic libraries

   PRODUCTS

used by UPS to resolve external products

   FHICL_FILE_PATH

use by FHiCL to resolve #include directives.

When you source the scripts that setup your environment for art, these will be defined and additional colon-separated elements will be added to your PATH. To look at the value of PATH (or the others), enter:

To make the output easier to read by replacing all of the colons with newline characters, enter:

In the above line, the vertical bar is referred to as a pipe and tr is a standard Unix command. A pipe takes the output of the command to its left and makes that the input of the command to its right. The tr command replaces patterns of characters with other patterns of characters; in this case it replaces every occurrence of the colon character with the newline character. To learn why a double back slash is needed, read bash documentation to learn about escaping special characters.

4.7 Scripts: Part 2

There are two ways to run a bash script (actually three, but two of them are the same). Suppose that you are given a bash script named file.sh. You can run any of these commands:

The first version, file.sh, starts a new bash shell, called a subshell, and it executes the commands from file.sh in that subshell; upon completion of the script, control returns to the parent shell. At the startup of a subshell, the environment of that subshell is initialized to be a copy of the environment of its parent shell. If file.sh modifies its environment, then it will modify only the environment of the subshell, leaving the environment of the parent shell unchanged. This version is called executing the script.

The second and third versions are equivalent. They do not start a subshell; they execute the commands from file.sh in your current shell. If file.sh modifies any environment variables, then those modifications remain in effect when the script completes and control returns to the command prompt. This is called sourcing the script.

Some shell scripts are designed so that they must be sourced and others are designed so that they must be executed. Many shell scripts will work either way.

If the purpose of a shell script is to modify your working environment then it must be sourced, not executed. As you work through the Workbook exercises, pay careful attention to which scripts it tells you to source and which to execute. In particular, the scripts to setup your environment (the first scripts you will run) are bash scripts that must be sourced because their purpose is to configure your environment so that it is ready to run the Workbook exercises.

Some people adopt the convention that all bash scripts end in .sh; others adopt the convention that only scripts designed to be sourced end in .sh while scripts that must be executed have no file-type ending (no “.something” at the end). Neither convention is uniformly applied either in the Workbook or in HEP in general.

If you would like to learn more about bash, some references are listed in Section 4.10.

4.8 bash Functions and Aliases

The bash shell also has the notion of a bash function. Typically bash functions are defined by sourcing a bash script; once defined, they become part of your environment and they can be invoked as if they were regular commands. The setup product “command” that you will sometimes need to issue, described in Chapter 7, is an example. A bash function is similar to a bash script in that it is just a collection of bash commands that are accessible via a name; the difference is that bash holds the definition of a function as part of the environment while it must open a file every time that a bash script is invoked.

You can see the names of all defined bash functions using:

The bash shell also supports the idea of aliases; this allows you to define a new command in terms of other commands. You can see the definition of all aliases using:

You can read more about bash shell functions and aliases in any standard bash reference.

When you type a command at the command prompt, bash will resolve the command using the following order:

1. Is the command a known alias?
2. Is the command a bash keyword, such as if or declare?
3. Is the command a shell function?
4. Is the command a shell built-in command?
5. Is the command found in $PATH?

To learn how bash will resolve a particular command, enter:

4.9 Login Scripts

When you first login to a computer running the Unix operating system, the system will look for specially named files in your home directory that are scripts to set up your working environment; if it finds these files it will source them before you first get a shell prompt. As mentioned in Section 4.5, these scripts modify your PATH and define bash functions, aliases and environment variables. All of these become part of your environment.

When your account on a Fermilab computer was first created, you were given standard versions of the files .profile and .bashrc; these files are used by bash³ . You can read about login scripts in any standard bash reference. You may add to these files but you should not remove anything that is present.

If you are working on a non-Fermilab computer, inspect the login scripts to understand what they do.

It can be useful to inspect the login scripts of your colleagues to find useful customizations.

If you read generic Unix documentation, you will see that there are other login scripts with names like, .login, .cshrc and .tcshrc. These are used by the csh family of shells and are not relevant for the Workbook exercises, which require the bash shell.

4.10 Suggested Unix and bash References

The following cheat sheet provides some of the basics:

• http://mu2e.fnal.gov/atwork/computing/UnixHints.shtml

A more comprehensive summary is available from:

• http://www.tldp.org/LDP/.../GNU-Linux-Tools-Summary.html

Information about writing bash scripts and using bash interactive features can be found in:

• BASH Programming - Introduction HOW-TO
  http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
• Bash Guide for Beginners
  http://www.tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html
• Advanced Bash Scripting Guide
  http://www.tldp.org/LDP/abs/html/abs-guide.html

The first of these is a compact introduction and the second is more comprehensive.

The above guides were all found at the Linux Documentation Project: Workbook:

• http://www.tldp.org/guides.html

Books about Unix are numerous, of course. Examples include Mark Sobell’s A practical guide to the UNIX system and Graham Glass’ UNIX for programmers and users: a complete guide, both of which are in the Fermilab library along with many others (http://ccd.fnal.gov/library/).

Chapter 5
Site-Specific Setup Procedure

Section 4.5 discussed the notion of a working environment on a computer. This chapter answers the question: How do I make sure that my environment is configured so that I can run the Workbook exercises or my experiment’s code?

This chapter will explain how to do this in several different situations:

1. If you are logged in to one of your experiment’s computers.
2. If you are logged in to one of the machines supported for the August 2015 art/LArSoft course; at this writing there are two machines named alcourse.fnal.gov and alcourse2.fnal.gov; more may be added.
3. If you install art and its tool chain to your own computer.

On every computer that hosts the Workbook, a procedure must be established that every user is expected to follow once per login session. In most cases (NOνA being a notable exception), the procedure involves only sourcing a shell script (recall the discussion in Section 4.7). In this documentation, we refer to this procedure as the “site-specific setup procedure.” It is the responsibility of the people who maintain the Workbook software for each site(γ) to ensure that this procedure does the right thing on all the site’s machines.

As a user of the Workbook, you will need to know what the procedure is and you must remember to follow it each time that you log in.

For all of the Intensity Frontier experiments at Fermilab, the site-specific setup procedure defines all of the environment variables that are necessary to create the working environment for either the Workbook exercises or for the experiment’s own code.

Table 5.1 lists the site-specific setup procedure for each experiment. You will follow the procedure when you get to Section 9.6.

---

Experiment	Site-Specific Setup Procedure
ArgoNeut	See the instructions for MicroBoone
Darkside	source /ds50/app/ds50/ds50.sh
LArIAT	Will be available in a future release of the workbook
DUNE	source /grid/fermiapp/lbne/software/setup_lbne.sh
MicroBoone	source /cvmfs/uboone.opensciencegrid.org/products/setup_uboone.sh
	also, the following will work on the Fermilab site
	source /grid/fermiapp/products/uboone/setup_uboone.sh
Muon g-2	source source /grid/fermiapp/gm2/setup
Mu2e	setup mu2e
NOνA	See Listing 5.1
art LArSoft Course	source /products/course_setup.sh
Private machine	See Appendix B

---

NOνA users should check that their login scripts do not setup any of the UPS products related to art. Remove any lines that do; then log out and log in again. In particular, make sure that nothing in your login scripts, either directly or indirectly, executes the following line:

Once you have a clean login, follow the procedure given in Listing 5.1.

Chapter 6
Get your C++ up to Speed

6.1 Introduction

This change is for topic one.

There are two goals for this chapter. The first is to provide an overview of the features of C++ that will be important for users of art, especially those features that will be used in the Workbook exercises. It does not attempt to cover C++ comprehensively.

You will need to consult standard documentation to learn about any of the features that you are not already familiar with. The examples and exercises in this chapter will in many cases only skim the surface of C++ features that you will need to know how to manipulate as you work through the Workbook exercises and then use C++ code with art in your work.

The second goal is to explain the process of turning source code files into an executable program. The two steps in this process are compiling and linking. In informal writing, the word build is sometimes used to mean just compiling or just linking, but usually it refers to the two together.

This chapter is designed around a handful of exercises, each of which you will first build and run, then “pick apart” to understand how the results were obtained.

6.2 File Types Used and Generated in C++ Programming

A typical program consists of many source code files, each of which contains a human-readable description of one or more components of the program. In the Workbook, you will see source code files written in the C++ computer language; these files have names that end in .cc.¹ In C++, there is a second sort of source code file, called a header file. These typically have names that end in .h² ; in most cases, but not all, the source file has an associated header file with the same base name but with the different suffix. A header file can be thought of as the “parts list” for its corresponding source file; you will see how these are used in Section 6.5.

In the compilation step each source file is translated into machine code, also called binary code or object code, which is a set of instructions, in the computer’s native language, to do the tasks described by the source code. The output of the compilation step is called an object file; in the examples you will see in the Workbook, object files always end in .o.³ But an object file, by itself, is not an executable program. It is not executable primarily because it lacks the instructions that tell the operating system how to start executing the instructions in the file.

It is often convenient to collect related groups of object files and put them into libraries. There are two kinds of library files, static libraries and dynamic libraries. Static libraries are not used by art, and we do not discuss them further; when this document refers to a library, it means a dynamic library. Putting many object files into a single library allows you to use them as a single coherent entity. We will defer further discussion of libraries until more background information has been provided.

The job of the linking step is to read the information found in the various libraries and object files and form them into either a dynamic library or an executable program. When you run the linker, you tell it the name of the file it is to create. It is a common, but not universal, practice that the filename of an executable program has no extension (i.e., no .something at the end). Dynamic libraries on Linux typically have the extension .so, and on OS X they typically have the extension .dylib.

After the linker has finished, you can run your executable program typing the filename of the program at the bash command prompt. If you do not have the current directory on the PATH, you need to preface the filename of the program with ./ (a dot followed by a forward slash). At this time, the loader does the final steps of linking the program, to allow the use of instructions and data in the dynamic libraries to which your program is linked.

A typical program links both to libraries that were built from the program’s source code and to libraries from other sources. Some of these other libraries might have been developed by the same programmer as general purpose tools to be used by his or her future programs; other libraries are provided by third parties, such as art or your experiment. Many C++ language features are made available to your program by telling the linker to use libraries provided by the C++ compiler vendor. Other libraries are provided by the operating system.

Now that you know about libraries, we can give a second reason why an object file, by itself, is not an executable program: until it is linked, it does not have access to the functions provided by any of the external libraries. Even the simplest program will need to be linked against some of the libraries supplied by the compiler vendor and by the operating system.

The names of all of the libraries and object files that you give to the linker is called the link list.

6.3 Establishing the Environment

6.3.1 Initial Setup

To start these exercises for the first time, do the following:

After these steps, you are ready to begin the exercise in Section 6.4.

6.3.2 Subsequent Logins

If you log out and log back in again, reestablish your environment by following these steps:

6.4 C++ Exercise 1: Basic C++ Syntax and Building an Executable

6.4.1 Concepts to Understand

This section provides a program that illustrates the concepts in C++ that are assumed knowledge for the Workbook material. Brief explanations are provided, but in many cases you will need to consult other sources to gain the level of understanding that you will need. Several C++ references are listed in Section 6.9.

This sample program will introduce you to the following C++ concepts and features:

• how to indicate comments
• what is a main program
• how to compile, link and run the main program
• how to distinguish between source, object, library, and executable files
• how to print to standard output, std::cout
• what is a type
• how to declare and define variables(γ) of some of the frequently used built-in types: int, float, double, bool
• the {} initializer syntax (in addition to other forms)
• assignment of values to variables

• what are arrays, and how to declare and define them
• several forms of looping
• comparisons: ==, !=, <, >, >=, <=
• if and if-else
• what are pointers, and how to declare and define them
• what are references, and how to declare and define them
• std::string (a type from the C++ Standard Library (std(γ))
• what is the class template from the standard library, std::vector<T>⁴

The above list explicitly does not include classes, objects and inheritance, which will be discussed in Sections 6.7 and a future section on inheritance.

6.4.2 How to Compile, Link and Run

In this section you will learn how to compile, link and run the small C++ program that illustrates the features of C++ that are considered prerequisites to the Workbook exercises.

Run the following procedure. The idea here is for you to get used to the steps and see what results you get. Then in Section 6.4.3 you will examine the source file and output.

To compile, link and run the sample C++ program, called t1:

Just to see how the exercise was built, look at the script BasicSyntax/v1/build that you ran to compile and link t1.cc; the following command was issued:

c++ -Wall -Wextra -pedantic -std=c++11 -o t1 t1.cc

This turned the source file t1.cc into an executable program, named t1 (the argument to the -o (for “output”) option). We will discuss compiling and linking in Section 6.5.

6.4.3 Discussion

Look at the file t1.cc, in particular the relationship between the lines in the program and the lines in the output, and see how much you understand. Remember, you will need to consult standard documentation to learn about any of the features that you are not already familiar with; some are listed in Section 6.9. Note that some questions may be answered in Section 6.4.3.

In the source file, it is important to first point out the function called the main program. Every program needs one, and execution of the program takes place within the braces of this function, which is written

 
int main() { 
     ...executable code... 
}
   

Compare your output with the standard example:

There will almost certainly be a handful of differences, which we will discuss in Section 6.4.3.1.

The following sections correspond to sections of the code in BasicSyntax/v1/t1.cc and provide supplementary information.

6.4.3.1 Primitive types, Initialization and Printing Output

All variables, parameters, arguments, and so on in C++ need to have a type, e.g., int, float, bool, or another so-called primitive (or built-in) type, or a more complicated type defined by a class or structure. The code in this exercise introduces the primitive types.

Now, about the handful of differences in the output of one run versus another. There are two main sources of the differences: (1) an uninitialized variable and (2) variation in object addresses from run to run.

In t1.cc, the line int k; declares that k is a variable whose type is int but it does not initialize the variable. Therefore the value of the variable k is whatever value happened to be sitting in the memory location that the program assigned to k. Each time that the program runs, the operating system will put the program into whatever region of memory makes sense to the operating system; therefore the address of any variable, and thus the value returned, may change unpredictably from run to run.

This line is also the source of the warning message produced by the build script. This line was included to make it clear what we mean by initialized variables and uninitialized variables. Uninitialized variables are frequent sources of errors in code and therefore you should always initialize your variables. In order to help you establish this good coding habit, the remaining exercises in this series and in the Workbook include the compiler option -Werror. This tells the compiler to promote warning messages to error level and to stop compilation without producing an output file.

See Section 6.4.3.6 for other output that may vary between program runs.

6.4.3.2 Arrays

The next section of the example code introduces arrays, sometimes called C-style arrays to distinguish them from std::array, a class template element of the C++ Standard Library. Classes will be discussed in Section 6.7, and class templates will first be used in Chapter 6.7.

While you might find use of arrays in existing code, we recommend avoiding them in new code arrays, and using either std::vector or std::array. See Section 6.4.3.5 for an introduction to these types.

6.4.3.3 Equality testing

Two variables which refer to different objects that contain the same value (either by design or by coincidence) are equal. Equality is tested using the equality testing operator, ==. It is important to distinguish between the assignment operator (=) and the equality testing operator. Using = where == is intended is a common mistake.

Another distinction to be made is that of two variables being identical versus equal. In contrast to equality, two variables are identical if they refer to the same object, and thus have the same memory address. How can two variables be identical? One common case can be seen in the call to a function like maxSize:

 
#include <algoritm> 
#include <string> 
std::size_t maxSize(std::string const& a, 
                    std::string const& b) { 
  return std::max(a.size(), b.size()); 
}
   

If we consider the call:

 
std::string s(~cow~); 
auto sz = maxSize(s, s);
   

then, in the body of the function maxSize, and for this call, the variables a and b refer to the same object—so they are identical.

6.4.3.4 Conditionals

The primary conditional statements in C++ are if and if-else. There is also the ternary operator, ?:. The ternary operator, which evaluates an expression as true or false then chooses a value based on the result, can replace more cumbersome code with a single line. It is especially useful in places where an expression, not a statement, is needed. It works this way (shown here on three lines to emphasize the syntax):

 
// Note: this is pseudocode, not C++ 
type variable-to-initialize  (expression-to-evaluate) ? 
                                      value-if-true : 
                                      value-if-false;
   

An example is shown in the code.

6.4.3.5 Some C++ Standard Library Types

The C++ Standard Library is quite large, and contains many classes, functions, class templates, and function templates. Our sample code introduces only three: the class std::string, and the templates std::vector<T> and std::array.

A std::vector<T> behaves much like is an array of objects of some type T (e.g., int or std::string). It has the extra capability that its size can change as needed, unlike a C-style array, whose size is fixed.

The std::array is new in C++, and should be used in preference to the older C-style array discussed in Section 6.4.3.2 due to its greater capabilities. Unlike the C-style array, std::array knows its own size, can be copied, and can be returned from a function.

6.4.3.6 Pointers

A pointer is a variable whose value is the memory address of another object. The type of pointer must correspond to the type of the object to which it points.

In addition to the sources of difference in the program output between runs discussed in Section 6.4.3.1, another stems from the line:

 
float* pa = &a;
   

This line declares a variable pa and initializes it to be the memory address of the variable a. The variable a is of type float; therefore pa must be declared as type pointer(γ) to float.

Note that this line could have been written with the asterisk next to pa:

 
float *pa = &a;
   

This latter style is common in the C community. In the C++ community, the former style is preferred, because it emphasizes the type of the variable pa, rather than the type of the expression *pa.

Since the address may change from run to run, so may the printout that starts pa =.

The next line,

 
std::cout << ~*pa = ~ << *pa << std::endl;
   

shows how to access the value to which a pointer points. This is called defererencing the pointer, or following the pointer, and is done with the dereferencing operator, *. The expression *pa dereferences the pointer pa, yielding the value of the object to which pa points, in this case, the value of a.

In Section 6.7 you will learn about classes. One attribute of classes is that they have separately addressable parts, called members. Members of a class are selected using syntax like classname.membername. The combination of dereferencing a pointer and selecting a member of the pointed-to object (the pointee) can be done in two steps: first dereferencing then selecting, or in one step using the member selection operator operator->(). The following two expressions are equivalent:

 
(*panimal).size() 
panimal->size()
   

In the example code, the lines

 
std::cout << ~The size of animal is: ~ 
          << (*panimal).size() << std::endl; 
 
std::cout << ~The size of animal is: ~ 
          << panimal->size() << std::endl;
   

do exactly the same thing. Note that the parentheses in the first line are necessary because the precedence of . is higher than that of *.

Note that in many situations, the compiler is free to convert an array-of-T into a pointer-to-T. In such cases, the value of the pointer-to-T is the address of the initial element in the array.

6.4.3.7 References

A reference is a variable that acts as an alias for another object, and it can not be re-seated to reference a different object. It is not an object itself, and thus a reference does not have an address. The address-of operator operator&, when used on a reference, yields the address of the referent:

 
float  a; 
float& ra = a; 
float* p = &a; 
float* q = &ra;
   

The values of p and q will be the same. Because they print memory addresses determined by the compiler and linker, the lines in the printout that start &a = and &ra = may also change from run to run.

6.4.3.8 Loops

Loops, also called iteration statements, appear in several forms in C++. The most prevalent is the for loop. New in C++11 is the range-based for loop; this is the looping construction that should be preferred for cases to which it applies.

6.5 C++ Exercise 2: About Compiling and Linking

6.5.1 What You Will Learn

In the previous exercise, the user code was found in a single file and the build script performed compiling and linking in a single step. For all but the smallest programs, this is not practical. It would mean, for example, that you would need to recompile and relink everything when you made even the smallest change anywhere in the code; generally this would take much too long. To address this, some computer languages, including C++, allow you to break up a large program into many smaller files and rebuild only a small subset of files when you make changes in one.

There are two exercises in this section. In the first one the source code consists of three files. This example has enough richness to discuss the details of what happens during compiling and linking, without being overwhelming. The second exercise introduces the idea of libraries.

6.5.2 The Source Code for this Exercise

The source code for this exercise is found in Build/v1, relative to your working directory. The relevant files are t1.cc, times2.cc and times2.h. Open these files and read along with the discussion below.

The file t1.cc contains the source code for the function main for this exercise. Every C++ program must have one and only one function named main, which is where the program actually starts execution. Note that the term main program sometimes refers to this function, but other times refers to the source file that contains it. In either case, main program refers to this function, either directly or indirectly. For more information, consult any standard C++ reference. The file times2.h is a header file that declares a function named times2. The file times2.cc is another source code file; it provides the definition of that function.

Look at t1.cc: it both declares and defines the program’s function main, with the signature int main(): it takes no arguments, and returns an int. A function with this signature(γ) has special meaning to the complier and the linker: they recognize it as a C++ main program. There are other signatures that the compiler and linker will recognize as a C++ main program; consult the standard C++ documentation.

To be recognized as a main program, there is one more requirement: main must be declared in the global namespace.

The body of the main program (between the braces), declares and defines a variable a of type double and initializes it to the value of 3.0; it prints out the value of a. Then it calls a function that takes a as an argument and prints out the value returned by that function.

You, as the programmer using that function, need to know what the function does but the C++ compiler doesn’t. It only needs to know the name, argument list and return type of the function — information that is provided in the header file, times2.h. This file contains the line

 
double times2(double);
   

This line is called the declaration(γ) of the function. It says (1) that the identifier times2 is the name of a function that (2) takes an argument of type double (the “double” inside the parentheses) and (3) returns a value of type double (the “double” at the start of the line). The file t1.cc includes this header file, thereby giving the compiler these three pieces of information it needs to know about function.

The other three lines in times2.h make up an include guard, described in Appendix F. In brief, they deal with the following scenario: suppose that we have two header files, A.h and B.h, and that A.h includes B.h; there are many scenarios in which it makes good sense for a third file, either .h or .cc, to include both A.h and B.h. The include guards ensure that, when all of the includes have been expanded, the compiler sees exactly one copy of B.h.

Finally, the file times2.cc contains the source code for the function named times2:

 
double times2(double i) { 
  return 2 * i; 
}
   

It names its argument i, multiplies this argument by two and returns that value. This code fragment is called the definition of the function or the implementation(γ) of the function. (The C++ standard uses the word definition but implementation is in common use.)

We now have a rich enough example to discuss a case in which the same word is frequently used for two different things — instead of two words used for the same thing.

Sometimes people use the phrase “the source code of the function named times2” to refer collectively to both times2.h and times2.cc; sometimes they use it to refer exclusively to times2.cc. Unfortunately the only way to distinguish the two uses is from context.

The phrase header file always refers unambiguously to the .h file. The term implementation file is used to refer unambiguously to the .cc file. This name follows from the its contents: it describes how to implement the items declared in the header file.

Based on the above description, when this exercise is run, we expect it to print out:

6.5.3 Compile, Link and Run the Exercise

To perform this exercise, first log in and cd to your working directory if you haven’t already, then

This matches the expected printout.

Look at the file build that you just ran. It has three steps:

1. It compiles the main program, t1.cc, into the object file (with the default name) t1.o (which will now be the thing that the term main program refers to):
   c++ -Wall -Wextra -pedantic -Werror -std=c++11 -c t1.cc
2. It (separately) compiles times2.cc into the object file times2.o:
   c++ -Wall -Wextra -pedantic -Werror -std=c++11 -c times2.cc
3. It links t1.o and times2.o (and some system libraries) to form the executable program t1 (the name of the main program is the argument of the -o option):
   c++ -std=c++11 -o t1 t1.o times2.o

You should have noticed that the same command, c++, is used both for compiling and linking. The full story is that when you run the command c++, you are actually running a program that parses its command line to determine which, if any, files need to be compiled and which, if any, files need to be linked. It also determines which of its command line arguments should be forwarded to the compiler and which to the linker. It then runs the compiler and linker as many times as required.

If the -c option is present, it tells c++ to compile only, and not to link. If -c is specified, the source file(s) to compile must also be specified. Each of the files will be compiled to create its corresponding object file and then processing stops. In our example, the first two commands each compile a single source file. Note that if any object files are given on the command line, c++ will issue a warning and ignore them.

The third command (with no -c option) is the linking step. Even if the -c option is missing, c++ will first look for source files on the command line; if it finds any, it will compile them and put the output into temporary object files. In our example, there are none, so it goes straight to linking. The two just-created object files are specified (at the end, here, but the order is not important); the -o t1 portion of the command tells the linker to write its output (the executable) to the file t1.

As it is compiling the main program, t1.cc, the compiler recognizes every function that is defined within the file and every function that is called by the code in the file. It recognizes that t1.cc defines a function main and that main calls a function named times2, whose definition is not found inside t1.cc. At the point that main calls times2, the compiler will write to t1.o all of the machine code needed to prepare for the call; it will also write all of the machine code needed to use the result of times2. In between these two pieces, the compiler will write machine code that says “call the function whose memory address is” but it must leave an empty placeholder for the address. The placeholder is empty because the compiler does not know the memory address of that function.

The compiler also makes a table that lists all functions defined by the file and all functions that are called by code within the file. The name of each entry in the table is called a linker symbol and the table is called a symbol table. When the compiler was compiling t1.cc and it found the definition of the main program, it created a linker symbol for the main program and added a notation to say the this file contains the definition of that symbol. When the compiler was compiling t1.cc and it encountered the call to times2, it created a linker symbol for this function; it marked this symbol as an undefined reference (because it could not find the definition of times2 within t1.cc). The symbol table also lists all of the places in the machine code of t1.o that are placeholders that must be updated once the memory address of times2 is known. In this example there is only one such place.

When the compiler writes an object file, it writes out both the compiled code and the table of linker symbols.

The symbol table in the file times2.o is simple; it says that this file defines a function named times2 that takes a single argument of type double and that returns a double. The name in the symbol table encodes not only the function name, but also the number and types of the function arguments. These are necessary for overload resolution(γ).

The job of the linker (also invoked by the command c++) is to play match-maker. First it inspects the symbol tables inside all of the object files listed on the command line and looks for a linker symbol that defines the location of the main program. If it cannot find one, or if it finds more than one, it will issue an error message and stop. In this example

1. The linker will find the definition of a main program in t1.o.
2. It will start to build the executable (output) file by copying the machine code from t1.o to the output file.
3. Then it will try to resolve the unresolved references listed in the symbol table of t1.o; it does this by looking at the symbol tables of the other object files on the command line. It also knows to look at the symbol tables from a standard set of compiler-supplied and system-supplied libraries.
4. It will discover that times2.o resolves one of the external references from t1.o. So it will copy the machine code from times2.o to the executable file.
5. It will discover that the the other unresolved references in t1.o are found in the compiler-supplied dynamic libraries. It will put into the executable the necessary information to resolve these references at the time when the program is run.
6. Once all of the machine code has been copied into the executable, the compiler knows the memory address of every function, or where to find them at run-time. The compiler can then go into the machine code, find all of the placeholders and update them with the correct memory addresses.

Sometimes resolving one unresolved reference will generate new ones. The linker iterates until (a) all references are resolved and no new unresolved references appear (success) or (b) the same unresolved references continue to appear (error). In the former case, the linker writes the output to the file specified by the -o option; if no -o option is specified the linker will write its output to a file named a.out. In the latter case, the linker issues an error message and does not write the output file.

After the link completes, the files t1.o and times2.o are no longer needed because everything that was useful from them was copied into the executable t1. You may delete the object files, and the executable will still run.

6.5.4 Alternate Script build2

The script build2 shows an equivalent way of building t1 that is commonly used for small programs; it does it all on one line. To exercise this script:

Look at the script build2; it contains only one command:

c++ -Wall -Wextra -pedantic -Werror -std=c++11 -o t1 t1.cc times2.cc

This script automatically does the same operations as build but it knows that the object files are temporaries. Perhaps the command c++ kept the contents of the two object files in memory and never actually wrote them out as disk files. Or, perhaps, the command c++ did explcitly create disk files and deleted them when it was finished. In either case you don’t see them when you use build2.

6.5.5 Suggested Homework

It takes a bit of experience to decipher the error messages issued by a C++ compiler. The three exercises in this section are intended to introduce you to them so that you (a) get used to looking at them and (b) understand these particular errors if/when you encounter them later.

Each of the following three exercises is independent of the others. Therefore, when you finish with each exercise, you will need to undo the changes you made in the source file(s) before beginning the next exercise.

1. In Build/v1/t1.cc, comment out the include directive for times2.h; rebuild and observe the error message.
2. In Build/v1/times2.cc, change the return type to float; rebuild and observe the error message.
3. In Build/v1/t1.cc, change double a = 3. to {cppcodefloat a = 3.; rebuild and run. This will work without error and will produce the same output as before.

The first homework exercise will issue the diagnostic:

When you see a message like this one, you can guess that either you have not included a required header file or you have misspelled the name of the function.

The second homework exercise will issue the diagnostic (second and last lines split into two here):

times2.cc: In function ’float␣times2(double)’: 
times2.cc:3:22: error: new declaration ’float␣times2(double)’ 
 float times2(double i) { 
                      ^ 
In file included from times2.cc:1:0: 
times2.h:4:8: error: ambiguates old declaration ’double␣times2(double)’ 
 double times2(double); 
        ^

This error message says that the compiler has found two functions that have the same signature but different return types. The compiler does not know which of the two functions you want it to use.

The bottom line here is that you must ensure that the definition of a function is consistent with its declaration; and you must ensure that the use of a function is consistent with its declaration.

The third homework exercise illustrates the C++ idea of implicit (type) conversion; in this case the compiler will make a temporary variable of type double and set its value to that of a, as if the code included:

 
double tmp = a; 
... 
std::cout << ~times2(a) ~ << times2(tmp) << std::endl;
   

Consult the standard C++ documentation to understand when implicit type conversions may occur; see Section 6.9.

6.6 C++ Exercise 3: Libraries

Multiple object files can be grouped into a single file known as a library, obviating the need to specify each and every object file when linking; you can reference the libraries instead. This simplifies the multiple use and sharing of software components. Libraries were introduced in Section 6.1); here we introduce the building of libraries.

6.6.1 What You Will Learn

In this section you will repeat the example of Section 6.5 with a variation. You will create a library from times2.o and use that library in the link step. This pattern generalizes easily to the case that you will encounter in your experiment software, where object libraries will typically contain many object files.

6.6.2 Building and Running the Exercise

To perform this exercise, do the following:

This matches the expected printout. Now let’s look at the script build. It has four parts which do the following things:

1. Compiles times2.cc; the same as the previous exercise:
   c++ -Wall -Wextra -pedantic -Werror -std=c++11 -c times2.cc
2. Creates the library named libpackage1.so (on OS X, the standard suffix for a dynamic library is different, and the the library is called libpackage1.dylib) from times2.o:
   c++ -o libpackage1.so -shared times2.o
   or
   c++ -o libpackage1.dylib -shared times2.o
   Note that the name of the library must come before the name of the object file. The flag -shared directs the linker to create a dynamic library rather than an executable image; without this flag, this command would produce an error complaining about the lack of a main function.
3. Compiles t1.cc; the same as the previous exercise:
   c++ -Wall -Wextra -pedantic -Werror -std=c++11 -c t1.cc
4. Links the main program against the dynamic library (either libpackage1.so or libpackage1.dylib) and, implicitly, the necessary system libraries:
   c++ -o t1 t1.o libpackage1.so
   or
   c++ -o t1 t1.o libpackage1.dylib

Note that from this point on, in order to reduce the verbosity of some library descriptions, we will use the Linux form of library names (e.g. libpackage1.so). If you are working on OS X, you will need to translate all these to the OS X form (e.g. libpackage1.dylib).

The two new features are in step 2, which creates the dynamic library, and step 4, in which times2.o is replaced in the link list with dynamic library. If you have many object files to add to the library, you may add them one at a time by repeating step 2 or you may add them all in one command. When you do the latter you may name each object file separately or may use a wildcard:

c++ -o libpackage1.so -shared *.o

In the filename libpackage1.so the string package1 has no special meaning; it was an arbitrary name chosen for this exercise.

The other parts of the name, the prefix lib and the suffix .so, are part of a long-standing Unix convention. Some Unix tools presume that libraries are named following this convention, so you should always follow it. The use of this convention is illustrated by the scripts build2 and build3.

To perform the exercise using build2, stay in the same directory and clean up and rebuild as follows:

The only difference between build and build2 is the link line. The version from build is:

c++ -o t1 t1.o libpackage1.so

while that from build2 is:

c++ -o t1 t1.o -L. -lpackage1

In the script build, the path to the library, relative or absolute, is written explicitly on the command line. In the script build2, two new elements are introduced. The command line may contain any number of -L options; the argument of each option is the name of a directory. The ensemble of all of the -L options forms a search path to look for named libraries; the path is searched in the order in which the -L options appear on the line. The names of libraries are specified with the -l options (this is a lower case letter L, not the numeral one); if a -l option has an argument of XXX (or package1), then the linker will search the path defined by the -L options for a file with the name libXXX.so (or libpackage1.so).

In the above, the dot in -L. is the usual Unix pathname that denotes the current working directory. And it is important that there be no whitespace after a -L or a -l option and its value.

This syntax generalizes to multiple libraries in multiple directories as follows. Suppose that the libraries libaaa.so, libbbb.so and libccc.so are in the directory L1 and that the libraries libddd.so, libeee.so and libfff.so are in the directory L2. In this case, the link list would look like:

-Lpath-to-L1 -laaa -lbbb -lccc -Lpath-to-L2 -lddd -leee -lfff

The -L -l syntax is in common use throughout many Unix build systems: if your link list contains many object libraries from a single directory then it is not necessary to repeatedly specify the path to the directory; once is enough. If you are writing link lists by hand, this is very convenient. In a script, if the path name of the directory is very long, this convention makes a much more readable link list.

To perform the exercise using build3, stay in the same directory and clean up and rebuild as follows:

The difference between build2 and build3 is that build3 compiles the main program and links it all one one line instead of two.