I want to create a shared library that uses functions from a 3rd-party static library. For example, foo and bar from libfoobar.a. I know that my main application is also using foo and will be exporting that symbol. So I simply want to link in bar to save code size and leave 'foo' unresolved (as it will be provided by main application). If I include libfoobar.a, the linker ld will include both functions in my shared library. If I don't include libfoobar.a, my library will not have access to function bar because the application itself is not linking in bar. Questions:
我想创建一个使用第三方静态库中的函数的共享库。例如,来自libfoobar.a的foo和bar。我知道我的主应用程序也在使用foo,并将导出该符号。所以我只想在栏中链接以保存代码大小并保持'foo'未解析(因为它将由主应用程序提供)。如果我包含libfoobar.a,链接器ld将在我的共享库中包含这两个函数。如果我不包含libfoobar.a,我的库将无法访问功能栏,因为应用程序本身没有链接到栏中。问题:
- Is there a way to tell ld to only resolve certain symbols when building the shared library?
- Turn
libfoobar.ainto a shared library? - Extract file containing function
barfromlibfoobar.aand specify that on the linker line? - Don't worry about it, the run-time loader will use
barfrom your application so the copy ofbarin the shared library will not be loaded?
有没有办法告诉ld在构建共享库时只解析某些符号?
将libfoobar.a变成共享库?
从libfoobar.a中提取包含功能栏的文件并在链接器行上指定它?
不用担心,运行时加载器将使用您应用程序中的bar,因此不会加载共享库中的bar副本?
3 个解决方案
#1
4
The following points attempt to answer the questions I had posed:
以下几点试图回答我提出的问题:
-
ld does not seem to allow you to omit linking in certain symbols from a static library. The usage of
--just-symbolsor--undefined(or theEXTERNlinker script command) will not prevent ld from linking the symbols. -
To convert a static library, libfoobar.a, into a shared one, libfoobar.so.1.0, and exporting all visible symbols. You can also use
--version-scriptand other methods to export only a subset of symbols.要将静态库libfoobar.a转换为共享库libfoobar.so.1.0,并导出所有可见符号。您还可以使用--version-script和其他方法仅导出符号子集。
ld -shared -soname libfoobar.so.1 -o libfoobar.so.1.0 --whole-archive libfoobar.a --no-whole-archiveld -shared -soname libfoobar.so.1 -o libfoobar.so.1.0 --whole-archive libfoobar.a --no-whole-archive
-
It is better to delete archive members from a copy of your static library than it is to extract them because there may be internal dependencies you have to manage. For example, assuming you are exporting all symbols, you can generate a map file from your main executable. You can then grep for all the archive members that the executable pulled in from the copy of the static library and delete them from the copy. So when your DSO is linking in the static library, it will leave the same symbols unresolved.
从静态库的副本中删除存档成员比删除它们更好,因为可能存在必须管理的内部依赖项。例如,假设您要导出所有符号,则可以从主可执行文件生成映射文件。然后,您可以从静态库的副本中获取可执行文件所引入的所有存档成员,并从副本中删除它们。因此,当您的DSO在静态库中进行链接时,它将保留相同的符号未解析。
-
It is possible to specify your main executable as a shared library for your DSO if you compile the executable with the
--pieoption. Your DSO will link first to your executable if it preceded the static library in the link command. The caveat is that the main executable must be available viaLD_LIBRARY_PATHor-rpath. Furthermore, using strace reveals that, since the executable is a dependency of your library, it is loaded again when your DSO loads.如果使用--pie选项编译可执行文件,则可以将主可执行文件指定为DSO的共享库。如果DSO在链接命令中的静态库之前,它将首先链接到您的可执行文件。需要注意的是,主要可执行文件必须通过LD_LIBRARY_PATH或-rpath可用。此外,使用strace会发现,由于可执行文件是库的依赖项,因此在加载DSO时会再次加载它。
ld -shared -rpath '$ORIGIN' -L. -lc -ldl -o DSO.so DSO.o app libfoobar.ald -shared -rpath'$ ORIGIN'-L。 -lc -ldl -o DSO.so DSO.o app libfoobar.a
-
The dynamic linker will use the executable's version of foo first unless you call dlopen() with the
RTLD_DEEPBINDflag. Using strace reveals that the entire DSO is file mapped mmap2() into memory. However, Wikipedia claims that for mmap "The actual reads from disk are performed in "lazy" manner, after a specific location is accessed." If this is true, then the duplicate foo will not be loaded. Note that the override only happens if your DSO exported the function foo. Otherwise, the function foo that was statically linked into your DSO will be used whenever your DSO calls foo.除非使用RTLD_DEEPBIND标志调用dlopen(),否则动态链接器将首先使用可执行文件的foo版本。使用strace显示整个DSO是将mmap2()文件映射到内存中。然而,*声称对于mmap“在访问特定位置后,从磁盘的实际读取以”懒惰“的方式执行。如果是这样,则不会加载重复的foo。请注意,只有在DSO导出函数foo时才会发生覆盖。否则,只要您的DSO调用foo,就会使用静态链接到DSO的函数foo。
ld似乎不允许您省略静态库中某些符号的链接。使用--just-symbols或--undefined(或EXTERN链接器脚本命令)不会阻止ld链接符号。
In conclusion, if mmap() uses a lazy read, then the best solution is to link your DSO in the normal manner and let the dynamic linker and linux take care of the rest.
总之,如果mmap()使用延迟读取,那么最好的解决方案是以正常方式链接您的DSO,让动态链接器和linux负责其余部分。
#2
1
I'm not the biggest expert on shared libraries, so I may be wrong here!
我不是共享库的最大专家,所以我可能错了!
If I'm guessing right about what you're trying to do, just link your shared lib against libc.so. You don't want an extra copy of sscanf embedded in your library.
如果我正在猜测你正在尝试做什么,只需将你的共享库链接到libc.so.您不希望在库中嵌入额外的sscanf副本。
I answered your questions before I had quite figured out what you were getting at, in case you're interested in the answers.
在我弄清楚你得到什么之前,我已经回答了你的问题,以防你对答案感兴趣。
Is there a way to tell ld to only resolve certain symbols when building the shared library?
有没有办法告诉ld在构建共享库时只解析某些符号?
only extern, not static, functions and variables go in the shared library's symbol table.
只有extern,而不是static,函数和变量才会进入共享库的符号表。
When you build your shared library, any symbols not found in objects on the linker command line will remain unresolved. If the linker complains about that, you probably need to link your shared lib against shared libc. You can have shared libs that depend on other shared libs, and ld.so can deal with the dependency chains.
构建共享库时,链接器命令行上的对象中找不到的任何符号将保持未解析状态。如果链接器抱怨这一点,您可能需要将共享库链接到共享libc。您可以拥有依赖于其他共享库的共享库,而ld.so可以处理依赖关系链。
If I had more rep, I'd ask this as a comment: Do you have a customized version of sprintf/sscanf, or would it be ok for your shared lib to use the implementation in -lc? If -lc is fine, then my answer probably solves your problem. If not, then you need to build your shared lib out of objects that only have the functions you need. i.e. don't link it against /usr/lib/libc.a.
如果我有更多的代表,我会问这个评论:你有自定义版本的sprintf / sscanf,或者你的共享库可以在-lc中使用实现吗?如果-lc很好,那么我的回答可能解决了你的问题。如果没有,那么您需要使用仅具有所需功能的对象构建共享库。即不要将它与/usr/lib/libc.a链接。
Maybe I'm getting confused by your
也许我对你感到困惑
libc.a (not actually the "real" libc) line. /usr/lib/libc.a is really glibc (on linux). It's a statically linked copy of the same code in libc.so. Unless you're talking about your own libc.a (which is what I was thinking at first)...
libc.a(实际上不是“真正的”libc)行。 /usr/lib/libc.a真的是glibc(在Linux上)。它是libc.so中相同代码的静态链接副本。除非你在谈论自己的libc.a(这是我一开始的想法)......
Turn libc.a into a shared library? You probably can, but don't, because it's probably not compiled as position-independent code, so it would be require a lot of relocations by ld.so at run time.
将libc.a变成共享库?您可能,但不是,因为它可能不会编译为与位置无关的代码,因此在运行时需要ld.so进行大量重定位。
Extract sscanf from libc.a and specify that on the linker line?
从libc.a中提取sscanf并在链接器行上指定它?
May be possible. ar t /usr/lib/libc.a to list contents. (ar's args are similar to tar. tar was ar for tapes.... Old school Unix here.) Probably not that easy, because sscanf probably depends on symbols in other .o files in the .a.
可能是可能的。 ar t /usr/lib/libc.a列出内容。 (ar的args类似于tar。tar是ar for tapes ....旧学校unix。)可能不那么容易,因为sscanf可能依赖于.a中其他.o文件中的符号。
#3
1
Answering your revised more-clear question.
回答你修改过的更明确的问题。
Keep in mind that normally the point of a shared lib is that multiple programs can link against it. So your optimization of using the main program's symbol for a function you need will only work if the main program always provides that symbol (via a static lib or otherwise). This is not usually what people want to do.
请记住,通常共享库的重点是多个程序可以链接它。因此,只有主程序始终提供该符号(通过静态库或其他方式)时,才能优化使用主程序的符号来实现所需的功能。这通常不是人们想要做的。
If it's just a couple small functions, probably you should let it be. You'll probably end up with two copies of the code for the functions, one in your shlib, and one in the main program. If they're small (or at least not huge), or not called often and not performance-critical, then the code-size / I-cache hit from having two copies isn't something to worry about. (translation: I don't know how to avoid it it off the top of my head, so I might not take the time to look it up and make a more complex Makefile to avoid it.)
如果它只是一些小功能,可能你应该放手。您可能最终会得到两个函数代码副本,一个在shlib中,一个在主程序中。如果它们很小(或者至少不是很大),或者不经常调用而不是性能关键,那么拥有两个副本的代码大小/ I-cache就不用担心了。 (翻译:我不知道如何避免它在我的头顶,所以我可能不会花时间查找它并制作一个更复杂的Makefile来避免它。)
See my other answer for some comments on messing around with ar to extract stuff from a static library. summary: probably non-trivial, since you don't know the dependencies between the various .o files in the .a.
请参阅我的另一个答案,了解有关使用ar从静态库中提取内容的一些注意事项。总结:可能非平凡,因为你不知道.a中各种.o文件之间的依赖关系。
It may be possible to do what you're hoping for by having your shared library export the symbols that it pulls in from the static library. Then, when you link the main app, put your shared library before the static lib on the linker command line. ld will find "foo" in your shlib, and use that copy (if this re-exporting trick is possible), but for "bar" it will have to include a copy from the static lib.
通过让共享库导出从静态库中提取的符号,可以执行您希望的操作。然后,当您链接主应用程序时,将您的共享库放在链接器命令行上的静态库之前。 ld会在你的shlib中找到“foo”,然后使用该副本(如果可以重新导出这种技巧),但是对于“bar”,它必须包含来自静态lib的副本。
ld --export-dynamic may be what you need to export all symbols in the dynamic symbol table. Try that. And search for "export" in the docs/man page. "export" is the jargon for making a symbol visible in a library. --export-all-symbols is in the i386 PE (windows DLL) section, otherwise it would probably do the trick.
ld --export-dynamic可能是导出动态符号表中所有符号所需的内容。试试吧。并在docs / man页面中搜索“export”。 “export”是使符号在库中可见的术语。 --export-all-symbols在i386 PE(windows DLL)部分中,否则它可能会起作用。
#1
4
The following points attempt to answer the questions I had posed:
以下几点试图回答我提出的问题:
-
ld does not seem to allow you to omit linking in certain symbols from a static library. The usage of
--just-symbolsor--undefined(or theEXTERNlinker script command) will not prevent ld from linking the symbols. -
To convert a static library, libfoobar.a, into a shared one, libfoobar.so.1.0, and exporting all visible symbols. You can also use
--version-scriptand other methods to export only a subset of symbols.要将静态库libfoobar.a转换为共享库libfoobar.so.1.0,并导出所有可见符号。您还可以使用--version-script和其他方法仅导出符号子集。
ld -shared -soname libfoobar.so.1 -o libfoobar.so.1.0 --whole-archive libfoobar.a --no-whole-archiveld -shared -soname libfoobar.so.1 -o libfoobar.so.1.0 --whole-archive libfoobar.a --no-whole-archive
-
It is better to delete archive members from a copy of your static library than it is to extract them because there may be internal dependencies you have to manage. For example, assuming you are exporting all symbols, you can generate a map file from your main executable. You can then grep for all the archive members that the executable pulled in from the copy of the static library and delete them from the copy. So when your DSO is linking in the static library, it will leave the same symbols unresolved.
从静态库的副本中删除存档成员比删除它们更好,因为可能存在必须管理的内部依赖项。例如,假设您要导出所有符号,则可以从主可执行文件生成映射文件。然后,您可以从静态库的副本中获取可执行文件所引入的所有存档成员,并从副本中删除它们。因此,当您的DSO在静态库中进行链接时,它将保留相同的符号未解析。
-
It is possible to specify your main executable as a shared library for your DSO if you compile the executable with the
--pieoption. Your DSO will link first to your executable if it preceded the static library in the link command. The caveat is that the main executable must be available viaLD_LIBRARY_PATHor-rpath. Furthermore, using strace reveals that, since the executable is a dependency of your library, it is loaded again when your DSO loads.如果使用--pie选项编译可执行文件,则可以将主可执行文件指定为DSO的共享库。如果DSO在链接命令中的静态库之前,它将首先链接到您的可执行文件。需要注意的是,主要可执行文件必须通过LD_LIBRARY_PATH或-rpath可用。此外,使用strace会发现,由于可执行文件是库的依赖项,因此在加载DSO时会再次加载它。
ld -shared -rpath '$ORIGIN' -L. -lc -ldl -o DSO.so DSO.o app libfoobar.ald -shared -rpath'$ ORIGIN'-L。 -lc -ldl -o DSO.so DSO.o app libfoobar.a
-
The dynamic linker will use the executable's version of foo first unless you call dlopen() with the
RTLD_DEEPBINDflag. Using strace reveals that the entire DSO is file mapped mmap2() into memory. However, Wikipedia claims that for mmap "The actual reads from disk are performed in "lazy" manner, after a specific location is accessed." If this is true, then the duplicate foo will not be loaded. Note that the override only happens if your DSO exported the function foo. Otherwise, the function foo that was statically linked into your DSO will be used whenever your DSO calls foo.除非使用RTLD_DEEPBIND标志调用dlopen(),否则动态链接器将首先使用可执行文件的foo版本。使用strace显示整个DSO是将mmap2()文件映射到内存中。然而,*声称对于mmap“在访问特定位置后,从磁盘的实际读取以”懒惰“的方式执行。如果是这样,则不会加载重复的foo。请注意,只有在DSO导出函数foo时才会发生覆盖。否则,只要您的DSO调用foo,就会使用静态链接到DSO的函数foo。
ld似乎不允许您省略静态库中某些符号的链接。使用--just-symbols或--undefined(或EXTERN链接器脚本命令)不会阻止ld链接符号。
In conclusion, if mmap() uses a lazy read, then the best solution is to link your DSO in the normal manner and let the dynamic linker and linux take care of the rest.
总之,如果mmap()使用延迟读取,那么最好的解决方案是以正常方式链接您的DSO,让动态链接器和linux负责其余部分。
#2
1
I'm not the biggest expert on shared libraries, so I may be wrong here!
我不是共享库的最大专家,所以我可能错了!
If I'm guessing right about what you're trying to do, just link your shared lib against libc.so. You don't want an extra copy of sscanf embedded in your library.
如果我正在猜测你正在尝试做什么,只需将你的共享库链接到libc.so.您不希望在库中嵌入额外的sscanf副本。
I answered your questions before I had quite figured out what you were getting at, in case you're interested in the answers.
在我弄清楚你得到什么之前,我已经回答了你的问题,以防你对答案感兴趣。
Is there a way to tell ld to only resolve certain symbols when building the shared library?
有没有办法告诉ld在构建共享库时只解析某些符号?
only extern, not static, functions and variables go in the shared library's symbol table.
只有extern,而不是static,函数和变量才会进入共享库的符号表。
When you build your shared library, any symbols not found in objects on the linker command line will remain unresolved. If the linker complains about that, you probably need to link your shared lib against shared libc. You can have shared libs that depend on other shared libs, and ld.so can deal with the dependency chains.
构建共享库时,链接器命令行上的对象中找不到的任何符号将保持未解析状态。如果链接器抱怨这一点,您可能需要将共享库链接到共享libc。您可以拥有依赖于其他共享库的共享库,而ld.so可以处理依赖关系链。
If I had more rep, I'd ask this as a comment: Do you have a customized version of sprintf/sscanf, or would it be ok for your shared lib to use the implementation in -lc? If -lc is fine, then my answer probably solves your problem. If not, then you need to build your shared lib out of objects that only have the functions you need. i.e. don't link it against /usr/lib/libc.a.
如果我有更多的代表,我会问这个评论:你有自定义版本的sprintf / sscanf,或者你的共享库可以在-lc中使用实现吗?如果-lc很好,那么我的回答可能解决了你的问题。如果没有,那么您需要使用仅具有所需功能的对象构建共享库。即不要将它与/usr/lib/libc.a链接。
Maybe I'm getting confused by your
也许我对你感到困惑
libc.a (not actually the "real" libc) line. /usr/lib/libc.a is really glibc (on linux). It's a statically linked copy of the same code in libc.so. Unless you're talking about your own libc.a (which is what I was thinking at first)...
libc.a(实际上不是“真正的”libc)行。 /usr/lib/libc.a真的是glibc(在Linux上)。它是libc.so中相同代码的静态链接副本。除非你在谈论自己的libc.a(这是我一开始的想法)......
Turn libc.a into a shared library? You probably can, but don't, because it's probably not compiled as position-independent code, so it would be require a lot of relocations by ld.so at run time.
将libc.a变成共享库?您可能,但不是,因为它可能不会编译为与位置无关的代码,因此在运行时需要ld.so进行大量重定位。
Extract sscanf from libc.a and specify that on the linker line?
从libc.a中提取sscanf并在链接器行上指定它?
May be possible. ar t /usr/lib/libc.a to list contents. (ar's args are similar to tar. tar was ar for tapes.... Old school Unix here.) Probably not that easy, because sscanf probably depends on symbols in other .o files in the .a.
可能是可能的。 ar t /usr/lib/libc.a列出内容。 (ar的args类似于tar。tar是ar for tapes ....旧学校unix。)可能不那么容易,因为sscanf可能依赖于.a中其他.o文件中的符号。
#3
1
Answering your revised more-clear question.
回答你修改过的更明确的问题。
Keep in mind that normally the point of a shared lib is that multiple programs can link against it. So your optimization of using the main program's symbol for a function you need will only work if the main program always provides that symbol (via a static lib or otherwise). This is not usually what people want to do.
请记住,通常共享库的重点是多个程序可以链接它。因此,只有主程序始终提供该符号(通过静态库或其他方式)时,才能优化使用主程序的符号来实现所需的功能。这通常不是人们想要做的。
If it's just a couple small functions, probably you should let it be. You'll probably end up with two copies of the code for the functions, one in your shlib, and one in the main program. If they're small (or at least not huge), or not called often and not performance-critical, then the code-size / I-cache hit from having two copies isn't something to worry about. (translation: I don't know how to avoid it it off the top of my head, so I might not take the time to look it up and make a more complex Makefile to avoid it.)
如果它只是一些小功能,可能你应该放手。您可能最终会得到两个函数代码副本,一个在shlib中,一个在主程序中。如果它们很小(或者至少不是很大),或者不经常调用而不是性能关键,那么拥有两个副本的代码大小/ I-cache就不用担心了。 (翻译:我不知道如何避免它在我的头顶,所以我可能不会花时间查找它并制作一个更复杂的Makefile来避免它。)
See my other answer for some comments on messing around with ar to extract stuff from a static library. summary: probably non-trivial, since you don't know the dependencies between the various .o files in the .a.
请参阅我的另一个答案,了解有关使用ar从静态库中提取内容的一些注意事项。总结:可能非平凡,因为你不知道.a中各种.o文件之间的依赖关系。
It may be possible to do what you're hoping for by having your shared library export the symbols that it pulls in from the static library. Then, when you link the main app, put your shared library before the static lib on the linker command line. ld will find "foo" in your shlib, and use that copy (if this re-exporting trick is possible), but for "bar" it will have to include a copy from the static lib.
通过让共享库导出从静态库中提取的符号,可以执行您希望的操作。然后,当您链接主应用程序时,将您的共享库放在链接器命令行上的静态库之前。 ld会在你的shlib中找到“foo”,然后使用该副本(如果可以重新导出这种技巧),但是对于“bar”,它必须包含来自静态lib的副本。
ld --export-dynamic may be what you need to export all symbols in the dynamic symbol table. Try that. And search for "export" in the docs/man page. "export" is the jargon for making a symbol visible in a library. --export-all-symbols is in the i386 PE (windows DLL) section, otherwise it would probably do the trick.
ld --export-dynamic可能是导出动态符号表中所有符号所需的内容。试试吧。并在docs / man页面中搜索“export”。 “export”是使符号在库中可见的术语。 --export-all-symbols在i386 PE(windows DLL)部分中,否则它可能会起作用。