How to Troubleshoot a Failed Easybuild Compilation

Problem

If you've been using Easybuild for self-service software, you may have encountered trouble with one of your builds, such as the one pictured below. When Easybuild doesn't build easily, what do you do? Keep reading for some tips and suggestions!

Step 1: Build Dependencies Separately, in Sequence

The first thing to try, is to build each dependency separately, in the sequence that they appear in a dry-run output. Run a dry-run of your software, as shown bellow, so that you can see which dependencies your software needs. It is recommended to store the output from your dry-run in a file for later viewing, as shown with the appended "2>&1|tee output.dry-run.log".

eb --robot --dry-run <package_to_build> 2>&1|tee output.dry-run.log

Example dry run output (for the Spyder-3.3.1-foss-2018a-Python-3.6.4.eb package) is shown below. Packages marked with "X" are already installed. Packages that need to be installed are blank.

In the screenshot, you can see I've selected the name of the next package that eb needs to build. From your dry-run output, select and copy the name of the next necessary package, and try to build it by itself. You may also do a dry-run of that dependency as well to see what packages it requires. Example:

eb --robot OpenPGM-5.2.122-GCCcore-6.4.0.eb 2>&1|tee OpenPGM.build.out

Continue building each needed dependency in this same way, working down the list in the dry-run output, until you encounter an error.

Sometimes building dependencies one-at-a-time like this will resolve the issue. If not, you should at least know exactly which dependency is causing the issue by the end of this step.

What you do next depends on the particular error:

If you are seeing an error similar to: "build failed (first 300 chars): Couldn't find file .tar.gz anywhere, and downloading it didn't work either..." Then you are dealing with a downloading issue. Go to the next section for tips to resolve it.

If the error is something else, there are some things you can do to obtain more information about the error:

1) Check the easybuild log file. It should give you the location of the log file after the error output ("Results of the build can be found in ..."), as shown below:

2) Try building the package manually to obtain more error information. You should be able to download the package without building it by using the eb "--fetch" option, and use "–sourcepath=./" to download it to the current working directory. An example of this is shown below.

eb --fetch Downloads All Dependencies!

eb --fetch will download the source tarballs for all dependencies of the specified package, whether they have been already installed or not. To download an individual package, follow the instructions in the Downloading Issue

section, to go to the source_url of the package.

eb --robot --sourcepath=./ --fetch OpenPGM-5.2.122-GCCcore-6.4.0.eb

Once the package is downloaded, you can untar it and try to compile it manually. (There is usually a README or some file with installation instructions with the package.) Even if the manual build fails, the error that it gives may provide clues as to why eb couldn't build it, or maybe the installation instructions themselves give helpful information. For example, maybe you need to set an environment variable to specify the CPU architecture, or some other configuration setting. If so, then set the variable with "export =" and then try building it with eb again.

Downloading Issue

If you are dealing with a downloading issue. The next step is to check the download URL in the easyconfig file of the failed dependency.

Use "cp" to copy the easyconfig file directly to your home directory (or whatever directory you'd like to use as a work space). You'll need the full path to the easyconfig file given in the dry-run output, as shown below:

(If your terminal doesn't let you "right-click → Copy", the try just right-clicking, and then right-clicking where you want to paste it.)

Copy the easyconfig using "cp" as shown below:

cp /mnt/data/EasyBuild-Intel/software/EasyBuild/4.5.1/easybuild/easyconfigs/o/OpenPGM/OpenPGM-5.2.122-GCCcore-6.4.0.eb ./

Now open it with your preferred text editor, and you'll see something similar to the file shown below:

Notice the "source_urls" property. That is the website from which Easybuild tries to download the package to build. You should try to go to that URL in your web browser and investigate. Maybe the repository was moved, or that particular version of the package is no longer available, or maybe the website now requires you to create an account and sign in to download the package. You can build the full download URL by appending what is given in "sources" to the "source_urls", and replacing "%(version)s" with the package version, as show below:

https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/openpgmhttps://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/openpgm/libpgm-%(version)s.tar.gzhttps://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/openpgm/libpgm-5.2.122.tar.gz

If the full URL doesn't allow you to download the file (which it probably won't, since it failed in the first place), then try going to the source URL itself (e.g. https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/openpgm ) or going up one level in the URL (e.g. to https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com ), and see if you can access the website to find any clues.

Check the checksum!

If you see that the package you want is available on the website, and if the website lists the "checksums" of the files. Make sure the checksum for the package matches the "checksums" field in the easyconfig. If not, update it with the one from the website and try again.

If you are unable to access the source website at all, consider searching for the package somewhere else online–maybe you can find it from another source repository.

Using a Manually Downloaded Package

If you find that you need to downloaded the package manually, you need to make sure that eb uses the downloaded package, instead of trying to download it automatically from the "source_urls" once again, and failing. To do this add the "–sourcepath" option to your eb command, and enter the path to the package that you downloaded. The example below shows this if the package is in your working directory:

eb --robot --sourcepath=./ OpenPGM-5.2.122-GCCcore-6.4.0.eb

Editing the Easyconfig to Use a Different Version

If the version of the package you are looking for is no longer available on the website, maybe you can use a newer version. Copy the easyconfig of the original package you were trying to build (in this example Spyder-3.3.1-foss-2018a-Python-3.6.4.eb) into your home directory or working directory, as we did with the failing dependency's easyconfig. Now edit it, and try changing the version of the failing package in the "dependencies" or "builddependencies" list.

Once you've edited the easyconfig, and have it in your home directory, you must build it by specifying the path to that edited easyconfig, like so, if you have it in your working directory:

eb --robot ./Spyder-3.3.1-foss-2018a-Python-3.6.4.eb

Verify!

You may want to verify that you program works as expected, if you're trying to force it to use a different version of a dependency.

Update EasyBlock, If Necessary

If modifying the easyconfig file, and other surface-level techniques to correct the problem prove insufficient, consider that the underlying easyblock may need to be modified. For more information on this see: Going Deeper into EasyBuild: EasyBlocks