What is the gold linker?
Has anyone used the
gold linker before? To link a fairly large project, I had to use this as opposed to the GNU
ld, which threw up a few errors and failed to link.
How is the
gold linker able to link large projects where
ld fails? Is there some kind of memory trickery somewhere?
gold linker was designed as an ELF-specific linker, with the intention of producing a more maintainable and faster linker than BFD
ld (the “traditional” GNU binutils linker). As a side-effect, it is indeed able to link very large programs using less memory than BFD
ld, presumably because there are fewer layers of abstraction to deal with, and because the linker’s data structures map more directly to the ELF format.
I’m not sure there’s much documentation which specifically addresses the design differences between the two linkers, and their effect on memory use. There is a very interesting series of articles on linkers by Ian Lance Taylor, the author of the various GNU linkers, which explains many of the design decisions leading up to
gold. He writes that
The linker I am now working, called gold, on will be my third. It is exclusively an ELF linker. Once again, the goal is speed, in this case being faster than my second linker. That linker has been significantly slowed down over the years by adding support for ELF and for shared libraries. This support was patched in rather than being designed in.
(The second linker is BFD
The gold linker was written to make the link process considerably faster. According to the gold auther Ian Lance Taylor
At the moment gold has only one significant advantage over the
existing linker: it is faster. On large C++ programs, I have measured
it as running five times faster.
He is comparing gold linker performance with the traditional GNU linker. gold (unlike the GNU linker) does not use the BFD library to process object files.
The limitation of gold is that (unlike GNU linker which can process many object file types) it can only link ELF format object files.
Regarding the issues you faced with when using GNU linker, here is an interesting answer to a similar question on SO from Michael Adam:
The gold linker even found some dependency problems in our code, since
it seems to be more correct than the classical one with respect to
some details. See, e.g. this Samba commit.
I have published a concrete synthetic benchmark of ld vs gold at: https://stackoverflow.com/questions/3476093/replacing-ld-with-gold-any-experience/53921263#53921263
Summary of results: gold was 2x to 3x times faster than ld.
This time gain can be a huge game changer on complex C++ projects with out-of-control templates and code generation, because the link step involves all files from the project, and unlike compilation, it has to be done always, even if you change just a single .cpp file.
So a slow link time makes the development cycle unbearable, and is likely the main reason Google sank resources into it. Just imagine the gains of waiting 10s instead of 30s for every trivial file change.
The time gains of the synthetic benchmark also agreed with actual gains I had on a complex real world project (gem5), as also mentioned on that answer.
There are three linkers available on modern GNU/Linux systems:
- ld, maintained by GNU binutils,
- gold, maintained by GNU binutils, “still in beta test”,
- lld, developed as part of the LLVM project.
For speed benchmarks, see: https://www.phoronix.com/scan.php?page=article&item=lld4-linux-tests&num=2 TL, DR:
lld is fastest, followed by
gold, followed by
Some sources say the gold project has been stagnating, and the package structure in Fedora reflects this.