如何创建静态链接共享库

For my master's thesis i'm trying to adapt a shared library approach for an ARM Cortex-M3 embedded system. As our targeted board has no MMU I think that it would make no sense to use "normal" dynamic shared libraries. Because .text is executed directly from flash and .data is copied to RAM at boot time I can't address .data relative to the code thus GOT too. GOT would have to be accessed through an absolute address which has to be defined at link time. So why not assigning fixed absolute addresses to all symbols at link time...?

对于我的硕士论文,我正在尝试为ARM Cortex-M3嵌入式系统调整共享库方法。由于我们的目标板没有MMU,我认为使用“普通”动态共享库是没有意义的。因为.text是直接从flash执行的,并且.data在启动时被复制到RAM,所以我无法解决相对于代码的.data,因此GOT也是如此。必须通过必须在链接时定义的绝对地址来访问GOT。那么为什么不在链接时为所​​有符号分配固定的绝对地址......?

From the book "Linkers and Loaders" I got aware of "static linked shared libraries, that is, libraries where program and data addresses in libraries are bound to executables at link time". The linked chapter describes how such libraries could be created in general and gives references to Unix System V, BSD/OS; but also mentions Linux and it's uselib() system call. Unfortunately the book gives no information how to actually create such libraries such as tools and/or compiler/linker switches. Apart from that book I hardly found any other information about such libraries "in the wild". The only thing I found in this regard was prelink for Linux. But as this operates on "normal" dynamic libraries thats not really what I'm searching for.

从“链接器和加载器”一书中我了解到“静态链接共享库,即库中程序和数据地址在链接时绑定到可执行文件的库”。链接的章节描述了如何创建这样的库,并提供对Unix System V,BSD / OS的引用;还提到了Linux和它的uselib()系统调用。不幸的是,本书没有提供如何实际创建诸如工具和/或编译器/链接器开关之类的库的信息。除了那本书之外,我几乎没有找到任何关于这些“野外”图书馆的信息。我在这方面唯一发现的是Linux的预链接。但是,由于这是在“正常”动态库上运行,而不是我正在寻找的。

I fear that the use of these kind of libaries is very specific, so that no common tools exists to create them. Although the mentioned uselib() syscall in this context makes me wondering. But I wanted to make sure that I haven't overlooked anything before starting to hack my own linker... ;) So could anyone give me more information about such libraries?

我担心这些库的使用非常具体,因此不存在创建它们的常用工具。虽然在这种情况下提到的uselib()系统调用让我感到疑惑。但是我想确保在开始破解我自己的链接器之前我没有忽略任何东西......;)那么有人能给我更多关于这些库的信息吗?

Furthermore I'm wondering if there is any gcc/ld switch which links and relocates a file but keeps the relocation entries in the file - so that it could be re-relocated? I found the "-r" option, but that completely skips the relocation process. Does anyone have an idea?

此外,我想知道是否有任何gcc / ld开关链接和重定位文件,但保留文件中的重定位条目 - 以便它可以重新重新定位?我找到了“-r”选项,但完全跳过了重定位过程。有没有人有想法?

edit:

Yes, I'm also aware of linker scripts. With gcc libfoo.c -o libfoo -nostdlib -e initLib -Ttext 0xdeadc0de I managed to get some sort of linked & relocated object file. But so far I haven't found any possibility to link a main program against this and use it as shared library. (The "normal way" of linking a dynamic shared library will be refused by the linker.)

是的,我也知道链接器脚本。使用gcc libfoo.c -o libfoo -nostdlib -e initLib -Ttext 0xdeadc0de我设法得到某种链接和重定位的目标文件。但到目前为止,我还没有找到任何可能将主程序与此链接并将其用作共享库的可能性。 (链接动态共享库的“正常方式”将被链接器拒绝。)

3 个解决方案

#1

Concepts

Minimum concept of what such a shared library maybe about.

这种共享库可能的最小概念。

• same code
• different data

There are variations on this. Do you support linking between libraries. Are the references a DAG structure or fully cyclic? Do you want to put the code in ROM, or support code updates? Do you wish to load libraries after a process is initially run? The last one is generally the difference between static shared libraries and dynamic shared libraries. Although many people will forbid references between libraries as well.

这有变化。你支持图书馆之间的链接吗?参考文献是DAG结构还是完全循环?您想将代码放入ROM中,还是支持代码更新?您是否希望在最初运行进程后加载库?最后一个通常是静态共享库和动态共享库之间的区别。虽然许多人也会禁止图书馆之间的引用。

Facilities

Eventually, everything will come down to the addressing modes of the processor. In this case, the ARM thumb. The loader is generally coupled to the OS and the binary format in use. Your tool chain (compiler and linker) must also support the binary format and can generate the needed code.

最终,一切都将归结为处理器的寻址模式。在这种情况下,ARM拇指。加载器通常耦合到OS和使用中的二进制格式。您的工具链(编译器和链接器)还必须支持二进制格式并可以生成所需的代码。

Support for accessing data via a register is intrinsic in the APCS (the ARM Procedure calling standard). In this case, the data is accessed via the sb (for static base) which is register R9. The static base and stack checking are optional features. I believe you may need to configure/compile GCC to enable or disable these options.

通过寄存器访问数据的支持是APCS(ARM过程调用标准)中固有的。在这种情况下,通过sb(对于静态基础)访问数据,该寄存器是R9。静态基础和堆栈检查是可选功能。我相信您可能需要配置/编译GCC以启用或禁用这些选项。

The options -msingle-pic-base and -mpic-register are in the GCC manual. The idea is that an OS will initially allocate separate data for each library user and then load/reload the sb on a context switch. When code runs to a library, the data is accessed via the sb for that instances data.

选项-msingle-pic-base和-mpic-register在GCC手册中。想法是OS最初将为每个库用户分配单独的数据,然后在上下文切换上加载/重新加载sb。当代码运行到库时,通过sb访问该实例数据的数据。

Gcc's arm.c code has the require_pic_register() which does code generation for data references in a shared library. It may correspond to the ARM ATPCS shared library mechanics.^{See Sec 5.5}

Gcc的arm.c代码有require_pic_register(),它为共享库中的数据引用执行代码生成。它可能对应于ARM ATPCS共享库机制。参见第5.5节

You may circumvent the tool chain by using macros and inline assembler and possibly function annotations, like naked and section. However, the library and possibly the process need code modification in this case; Ie, non-standard macros like EXPORT(myFunction), etc.

您可以通过使用宏和内联汇编程序以及可能的函数注释来绕过工具链,例如裸和部分。但是,在这种情况下,库和可能的进程需要修改代码;即,像EXPORT(myFunction)等非标准宏。

One possibility

If the system is fully specified (a ROM image), you can make the offsets you can pre-generate data offsets that are unique for each library in the system. This is done fairly easily with a linker script. Use the NOLOAD and put the library data in some phony section. It is even possible to make the main program a static shared library. For instance, you are making a network device with four Ethernet ports. The main application handles traffic on one port. You can spawn four instances of the application with different data to indicate which port is being handled.

如果系统已完全指定(ROM映像),则可以设置偏移量,以便预先生成系统中每个库唯一的数据偏移量。使用链接描述文件可以很容易地完成此操作。使用NOLOAD并将库数据放在某些虚假部分中。甚至可以使主程序成为静态共享库。例如,您正在制作具有四个以太网端口的网络设备。主应用程序处理一个端口上的流量。您可以使用不同的数据生成应用程序的四个实例,以指示正在处理哪个端口。

If you have a large mix/match of library types, the foot print for the library data may become large. In this case you need to re-adjust the sb when calls are made through a wrapper function on the external API to the library.

如果库类型的混合/匹配很大,则库数据的占地面积可能会变大。在这种情况下,您需要在通过库的外部API上的包装函数进行调用时重新调整sb。

  void *__wrap_malloc(size_t size)  /* Wrapped version. */
  {
       /* Locals on stack */
       unsigned int new_sb = glob_libc; /* accessed via current sb. */
       void * rval;
       unsigned int old_sb;

       volatile asm(" mov %0, sb\n" : "=r" (old_sb);
       volatile asm(" mov sb, %0\n" :: "r" (new_sb);
       rval = __real_malloc(size);
       volatile asm(" mov sb, %0\n" :: "r" (old_sb);
       return rval;
  }

See the GNU ld --wrap option. This complexity is needed if you have a larger homogenous set of libraries. If your libraries consists of only 'libc/libsupc++', then you may not need to wrap anything.

请参阅GNU ld --wrap选项。如果您拥有更大的同类库集,则需要这种复杂性。如果您的库只包含'libc / libsupc ++',那么您可能不需要包装任何东西。

The ARM ATPCS has veneers inserted by the compiler that do the equivalent,

ARM ATPCS有编译器插入的代码,它们具有相同的功能,

LDR a4, [PC, #4] ; data address
MOV SB, a4
LDR a4, [PC, #4] ; function-entry
BX a4
DCD data-address
DCD function-entry

The size of the library data using this technique is 4k (possibly 8k, but that might need compiler modification). The limit is via ldr rN, [sb, #offset], were ARM limits offset to 12bits. Using the wrapping, each library has a 4k limit.

使用这种技术的库数据的大小是4k(可能是8k,但可能需要编译器修改)。限制是通过ldr rN,[sb,#offset],ARM限制偏移到12位。使用包装,每个库都有4k的限制。

If you have multiple libraries that are not known when the original application builds, then you need to wrap each one and place a GOT type table via the OS loader at a fixed location in the main applications static base. Each application will require space for a pointer for each library. If the library is not used by the application, then the OS does not need to allocate the space and that pointer can be NULL.

如果原始应用程序构建时有多个未知的库,则需要将每个库包装起来并通过OS加载程序在主应用程序静态库中的固定位置放置GOT类型表。每个应用程序都需要空间用于每个库的指针。如果应用程序未使用该库,则操作系统不需要分配空间,并且该指针可以为NULL。

The library table can be accessed via known locations in .text, via the original processes sb or via a mask of the stack. For instance, if all processes get a 2K stack, you can reserve the lower 16 words for a library table. sp & ~0x7ff will give an implicit anchor for all tasks. The OS will need to allocate task stacks as well.

可以通过.text中的已知位置,通过原始进程sb或通过堆栈的掩码访问库表。例如,如果所有进程都获得2K堆栈,则可以为库表保留低16个字。 sp&~0x7ff将为所有任务提供隐式锚点。操作系统也需要分配任务堆栈。

Note, this mechanism is different than the ATPCS, which uses sb as a table to get offsets to the actual library data. As the memory is rather limited for the Cortex-M3 described it is unlikely that each individual library will need to use more than 4k of data. If the system supports an allocator this is a work around to this limitation.

请注意,此机制与ATPCS不同,后者使用sb作为表来获取实际库数据的偏移量。由于内存对Cortex-M3的描述相当有限,因此每个库不太可能需要使用超过4k的数据。如果系统支持分配器,则可以解决此限制。

References

• Xflat technical overview - Technical discussion from the Xflat authors; Xflat is a uCLinux binary format that supports shared libraries. A very good read.

Xflat技术概述 - Xflat作者的技术讨论; Xflat是一种支持共享库的uCLinux二进制格式。一个非常好的阅读。

• Linkage table and GOT - SO on PLT and GOT.

PLT和GOT上的联动表和GOT - SO。

• ARM EABI - The normal ARM binary format.

ARM EABI - 正常的ARM二进制格式。

• Assemblers and Loader, by David Solomon. Especially, pg262 A.3 Base Registers

汇集者和装载者,David Solomon。特别是,pg262 A.3基本寄存器

• ARM ATPCS, especially Section 5.5, Shared Libraries, pg18.

ARM ATPCS,特别是第5.5节,共享库,第18页。

• bFLT is another uCLinux binary format that supports shared libraries.

bFLT是另一种支持共享库的uCLinux二进制格式。

#2

How much RAM do you have attached? Cortex-M systems have only a few dozen kiB on-chip and for the rest they require external SRAM.

你附加了多少内存? Cortex-M系统片上只有几十个kiB,其余的则需要外部SRAM。

I can't address .data relative to the code

我无法解决相对于代码的.data问题

You don't have to. You can place the library symbol jump table in the .data segment (or a segment that behaves similarly) at a fixed position.

你不必。您可以将库符号跳转表放在固定位置的.data段(或行为相似的段)中。

thus GOT too. GOT would have to be accessed through an absolute address which has to be defined at link time. So why not assigning fixed absolute addresses to all symbols at link time...?

因此也是GOT。必须通过必须在链接时定义的绝对地址来访问GOT。那么为什么不在链接时为所​​有符号分配固定的绝对地址......?

Nothing prevents you from having a second GOT placed at a fixed location, that's writable. You have to instruct your linker where and how to create it. For this you give the linker a so called "linker script", which is kind of a template-blueprint for the memory layout of the final program.

没有什么能阻止你在固定位置放置第二个GOT,这是可写的。您必须指示链接器在何处以及如何创建它。为此,您为链接器提供了一个所谓的“链接器脚本”,它是最终程序内存布局的模板蓝图。

#3

I'll try to answer your question before commenting about your intentions.

在评论您的意图之前,我会尝试回答您的问题。

To compile a file in linux/solaris/any platform that uses ELF binaries:

要在linux / solaris /任何使用ELF二进制文件的平台上编译文件:

gcc -o libFoo.so.1.0.0 -shared -fPIC foo1.c foo2.c foo3.c ... -Wl,-soname=libFoo.so.1

I'll explain all the options next:

我将在下面解释所有选项:

-o libFoo.so.1.0.0

is the name we are going to give to the shared library file, once linked.

是链接后我们要给共享库文件的名称。

-shared

means that you have a shared object file at end, so there can be unsolved references after compilation and linked, that would be solved in late binding.

意味着您在末尾有一个共享对象文件,因此在编译和链接后可能会有未解决的引用,这将在后期绑定中解决。

-fPIC

instructs the compiler to generate position independent code, so the library can be linked in a relocatable fashion.

指示编译器生成与位置无关的代码,因此库可以以可重定位的方式链接。

-Wl,-soname=libFoo.so.1

has two parts: first, -Wl instructs the compiler to pass the next option (separated by comma) to the linker. The option is -soname=libFoo.so.1. This option, tells the linker the soname used for this library. The exact value of the soname is free style string, but there's a convenience custom to use the name of the library and the major version number. This is important, as when you do static linking of a shared library, the soname of the library gets stuck to the executable, so only a library with that soname can be loaded to assist this executable. Traditionally, when only the implementation of a library changes, we change only the name of the library, without changing the soname part, as the interface of the library doesn't change. But when you change the interface, you are building a new, incompatible one, so you must change the soname part, as it doesn't get in conflict with other 'versions' of it.

有两部分:第一,-Wl指示编译器将下一个选项(用逗号分隔)传递给链接器。选项是-soname = libFoo.so.1。此选项告诉链接器用于此库的soname。 soname的确切值是*样式字符串,但是有一个方便的自定义来使用库的名称和主要版本号。这很重要,因为当您对共享库进行静态链接时,库的soname会粘在可执行文件上,因此只能加载具有该soname的库来协助此可执行文件。传统上,当只更改库的实现时,我们只更改库的名称,而不更改soname部分,因为库的接口不会更改。但是当您更改界面时,您正在构建一个新的,不兼容的界面,因此您必须更改soname部分,因为它不会与其他“版本”冲突。

To link to a shared library is the same than to link to a static one (one that has .a as extension) Just put it on the command file, as in:

链接到共享库与链接到静态库(具有.a作为扩展名的库)相同,只需将其放在命令文件中,如下所示:

gcc -o bar bar.c libFoo.so.1.0.0

Normally, when you get some library in the system, you get one file and one or two symbolic links to it in /usr/lib directory:

通常,当您在系统中获得某个库时,您会在/ usr / lib目录中获得一个文件和一个或两个符号链接:

/usr/lib/libFoo.so.1.0.0
/usr/lib/libFoo.so.1 --> /usr/lib/libFoo.so.1.0.0
/usr/lib/libFoo.so --> /usr/lib/libFoo.so.1

The first is the actual library called on executing your program. The second is a link with the soname as the name of the file, just to be able to do the late binding. The third is the one you must have to make

第一个是执行程序时调用的实际库。第二个是使用soname作为文件名的链接,只是为了能够进行后期绑定。第三个是你必须要做的

gcc -o bar bar.c -lFoo

work. (gcc and other ELF compilers search for libFoo.so, then for libFoo.a, in /usr/lib directory)

工作。 (gcc和其他ELF编译器搜索libFoo.so,然后搜索libFoo.a,在/ usr / lib目录中)

After all, there's an explanation of the concept of shared libraries, that perhaps will make you to change your image about statically linked shared code.

毕竟,对共享库的概念有一个解释,这可能会让您改变关于静态链接共享代码的图像。

Dynamic libraries are a way for several programs to share the functionalities of them (that means the code, perhaps the data also). I think you are a little disoriented, as I feel you have someway misinterpreted what a statically linked shared library means.

动态库是多个程序共享它们功能的一种方式(即代码,也可能是数据)。我认为你有点迷失方向,因为我觉得你有点误解了静态链接共享库的意思。

static linking refers to the association of a program to the shared libraries it's going to use before even launching it, so there's a hardwired link between the program and all the symbols the library has. Once you launch the program, the linking process begins and you get a program running with all of its statically linked shared libraries. The references to the shared library are resolved, as the shared library is given a fixed place in the virtual memory map of the process. That's the reason the library has to be compiled with the -fPIC option (relocatable code) as it can be placed differently in the virtual space of each program.

静态链接是指程序与它在启动之前将要使用的共享库的关联,因此程序与库中的所有符号之间存在硬连线。启动程序后,链接过程开始,您将获得一个运行所有静态链接共享库的程序。解析对共享库的引用,因为共享库在进程的虚拟内存映射中被赋予固定位置。这就是必须使用-fPIC选项(可重定位代码)编译库的原因,因为它可以在每个程序的虚拟空间中以不同方式放置。

On the opposite, dynamic linking of shared libraries refers to the use of a library (libdl.so) that allows you to load (once the program is executing) a shared library (even one that has not been known about before), search for its public symbols, solve references, load more libraries related to this one (and solve recursively as the linker could have done) and allow the program to make calls to symbols on it. The program doesn't even need to know the library was there on compiling or linking time.

相反,共享库的动态链接是指使用一个库(libdl.so),它允许您加载(一旦程序执行)一个共享库(甚至是之前未知的那个),搜索它的公共符号,解决引用,加载更多与此相关的库(并且可以像链接器那样递归地求解)并允许程序对其上的符号进行调用。该程序甚至不需要知道库是否存在编译或链接时间。

Shared libraries is a concept related to the sharing of code. A long time ago, there was UNIX, and it made a great advance to share the text segment (whit the penalty of not being able for a program to modify its own code) of a program by all instances of it, so you have to wait for it to load just the first time. Nowadays, the concept of code sharing has extended to the library concept, and you can have several programs making use of the same library (perhaps libc, libdl or libm) The kernel makes a count reference of all the programs that are using it, and it just gets unloaded when no other program is using it.

共享库是与代码共享相关的概念。很久以前,有UNIX,并且通过它的所有实例共享文本段(不能让程序修改自己的代码的惩罚)取得了很大的进步,所以你必须等待它第一次加载。如今,代码共享的概念已扩展到库概念,您可以使用相同的库(可能是libc,libdl或libm)使用多个程序。内核对所有使用它的程序进行计数引用,以及它只是在没有其他程序正在使用它时被卸载。

using shared libraries has only one drawback: the compiler must create relocatable code to generate a shared library as the space used by one program for it can be used for another library when we try to link it to another program. This imposes normally a restriction in the set of op codes to be generated or imposes the use of one/several registers to cope with the mobility of code (there's no mobility but several linkings can make it to be situated at different places)

使用共享库只有一个缺点:编译器必须创建可重定位代码以生成共享库,因为当我们尝试将其链接到另一个程序时,一个程序使用的空间可用于另一个库。这通常会对要生成的操作码集产生限制,或者强制使用一个/多个寄存器来处理代码的移动性(没有移动性,但有几个链接可以使它位于不同的位置)

Believe me, using static code just derives you to making bigger executables, as you cannot share effectively the code, but with a shared library.

相信我,使用静态代码只会导致您制作更大的可执行文件,因为您无法有效地共享代码,但使用共享库。