Embedding, in the context of this article, is the process of hosting a web rendering engine inside another application. This engine displays content exactly as a regular web browser would, but allows the application author to customize the user’s experience further than what is possible in the typical display of a normal website. Development time can be reduced by keeping part of the content of an application in web-related languages due to the relative ease of writing web content and the widespread knowledge of HTML5.
The technique of embedding web content is used in many places by many companies, including:
- Popular Linux applications such as Rhythmbox, Kate, Eclipse, and Evolution have support for embedding web content.
- Microsoft Entourage and Apple Mail for Mac OS both utilize embedding for displaying web content and parts of the UI.
- Adobe products, including their constantly-running updater, embed full web runtimes.
- Valve’s Steam client also depends heavily on embedding web content, as they reuse the content from their online store to display in their native application.
- Many applications in mobile app stores are simply a native wrapper around a web browser control and HTML5 resources.
HTML5 has been gaining significant traction in the mobile and embedded world. According to a recent survey HTML5 is a leading platform for application development in all markets, and this figure is likely to increase as the number of mobile platforms grows in the future. Finally, HTML5 is a widely known cross-platform solution which guarantees portability.
The Need for A Common, Embeddable API
There are very few common APIs for embedding content into applications, and until a couple years ago, the most prevalent API was WebKit. This was problematic because WebKit’s API is not considered stable, forcing developers to rewrite parts of their application in order to keep up with changes in the underlying browser engine. In recent years, Blink was forked from WebKit, creating yet another embeddable API. Unfortunately, in addition to being confined to C++, Blink’s embedding API is also not stable, perpetuating the increased workload for application developers.
One solution for these problems is to use the Chromium Embedded Framework. Launched in 2008, the project is based on the Blink browser engine but aims to completely insulate its users from underlying engine changes. It provides a base C API with C++ extensions; allowing support at the lowest levels of software development. Currently, this API is used extensively by Valve for their Steam client, and by Adobe, for various products.
The Servo Solution
The Servo browser engine aims to be embeddable in order to provide maximum flexibility to developers. To this end, it must provide a stable API and ABI for developers to rely on. It is written in Rust because it provides an API usable from C. Designing yet another embeddable web API is an extremely complex task which requires a tremendous amount of consideration and review. It also requires extensive documentation to be written, frameworks to be tested, and the resulting product to be promoted and accepted into general usage. Based on this, we decided to take a different approach.
Servo implements the Chromium Embedded Framework API: if it’s good enough for Valve, it’s good enough for us. This means developers who currently use CEF will not need to make any application changes to compare performance between the Blink and Servo engines, and prospective developers will not have yet another browser API to review and consider. Hopefully, this also means Servo will reach an embeddable state much more quickly.
The methodology for this implementation is twofold: ensure full symbol coverage, and attempt to mimic the exact functionality of each method call as closely as possible. Since this is not a common practice, let’s break it down a little further. Symbol coverage refers to the externally visible symbols provided by a library. Rust, using the extern “C” keywords, allows any function written in Rust to be directly accessible from C with no extra work needed. So in this regard, full symbol coverage would mean that every CEF function call is able to be hooked by Servo’s embedding library. It also requires that any time an externally-used struct is allocated, it must match the size and member positioning as a similar struct allocated by the real CEF.
pub struct cef_string_utf8 {
pub str: *mut u8,
pub length: size_t,
pub dtor: extern "C" fn(str: *mut u8),
}
Mimicking functionality is a bit trickier. This requires full understanding of each CEF function and how it ties into the browser engine. Then, the same order of operations must be replicated using the underlying Servo browser engine. This can be quite a complex process, as CEF makes internal function calls back and forth between itself and Blink, sometimes requiring functions to operate recursively.
The method for implementing and testing the success of this endeavor has been a bit complex. First, a symbol list was needed. Using standard utilities, specifically the nm tool on Linux, we copied the undefined symbols from some of the example CEF applications. This provided an easy starting point to begin implementing functions and functionality. With the list of target symbols, it became much easier to track the execution of the application through the CEF library and determine what was happening in each function call. From here it became a case of using Rust/Servo to replicate the expected and observed behavior.
A Few Hitches Along The Way
The process of writing this implementation for embedding has not come very far because it was only recently begun; at present, there is only a very small team focusing on it. However, there have been some difficulties that may be interesting to other developers. One such difficulty that was immediately apparent was tracking the C API’s execution through the C++ extensions provided by CEF. The example applications all use the C++ extensions which required a bit of legwork to understand how they utilize and wrap the C API and what was going on internally. The solution here was extremely rudimentary, just read a whole bunch of source code. Eventually, even the most high-level wrapper would come down to one or two function calls which could then be written and tested more thoroughly.
Another difficulty that took quite a while to track down is a byproduct of the testing environment. In order to run the CEF examples with the Servo backend, we needed to use some library preloading hacks (LD_PRELOAD and friends on Linux) so the running environment would inject the Servo embedding symbols for use in the application, effectively overriding the CEF-provided functions. However, as long as the CEF library is used in any function call it will invoke the underlying Blink runtime, which uses the tcmalloc allocator. The reason this is problematic is because tcmalloc uses different function calls for allocating memory between C and C++ than what is expected, and it complains quite loudly when it detects any deviation from this behavior. As a result, the Servo embedding library needed to provide wrappers to allow calling these functions directly for memory allocation, but only when using the preload hack:
pub fn new(size: size_t) -> *mut c_void {
unsafe {
tc_new(size)
}
}
The embedding support is now sufficient to run Servo within the cefsimple application, and a number of tasks are currently open to improve support in other directions. For more information check out the GitHub page.
Guest Author: Lars Bergstrom
Lars Bergstrom is a researcher at Mozilla, currently working on the servo parallel web browser project.He received his Ph.D. from the University of Chicago‘s Computer Science department, studying under Dr. John Reppy. His Master’s paper was on the implementation of analysis and optimization passes in our parallel compiler, Manticore. His Ph.D. research was on how to add mutation safely and efficiently into a functional parallel programming language. He has also been doing work on a runtime, garbage collector, and most recently some extensions to control-flow analysis. Before that, he was a manager and a developer at Microsoft in the Visual Studio organization, working on next-generation software development tools technology out at the Redmond, WA offices.