C++ Actor Framework — Dev Blog

An Open Source implementation of the actor model in C++

Version 0.18.0-rc.1 Released!

Today, we are more than happy to announce the first pre-release of CAF 0.18!

As outlined in our previous post, this is the first time since the inception of CAF that we raise the required C++ standard version. We took this opportunity to modernize many, many aspects of CAF. This includes the build system, which now requires CMake in version ≥ 3.4. Please take a look at the available build options in CMake directly.

The most notable changes with this release are:

  • The type inspection API received a complete overhaul. Please refer to the updated manual section for the new DSL. One of the upsides of the new inspection API is to enable the configuration framework to pick up user-defined types. So once you provide an inspect overload, you can read your types directly from a CAF configuration. This also enables more formats such as JSON and XML, although we currently ship no default serializers for such formats.
  • All types that the application sends in a CAF message now require a type ID. This is also a breaking change. The API for assigning type IDs is an optional part of the API since version 0.17.5, but became mandatory with this release. The new type IDs require more boilerplate code than before, but they enable improved compile-time checks and faster serialization/deserialization.
  • CAF now ships with an API for runtime metrics that applications can export directly to Prometheus. Have a look at the brand new manual section covering this feature!
  • The library now compiles with default visibility hidden on all platforms and we finally support generating shared libraries on Windows (i.e., building .dll files).

For the full list of changes, please head over to the release notes or take a look at our changelog.

We encourage everyone to try the new release and look forward to any feedback (bug reports in particular). After publishing a pre-release, we no longer make changes to the API except for bugs that we cannot fix otherwise.

C++17, Breaking Changes, and Roadmap Update

Switching to C++17

The last time we have dropped support for a compiler was back in 2015 when we removed support for GCC 4.7. While C++14 had several quality-of-life features, we preferred stability over what C++14 had to offer. With C++17, the story is different. Fold expressions and if constexpr alone are game changers for CAF. That being said, we want to stay as conservative as possible when it comes to our dependencies. So, we are not rushing towards latest GCC or Clang versions.

Compiler vendors have gotten a lot better at catching up to new language versions and so did package maintainers and distributors. We did a survey on the current landscape and found that all major platforms give their users easy access to C++17 by now.

Microsoft has C++17 support in their Visual Studio Community Edition, Apple ships very recent Clang versions with XCode, major BSD distributions such as FreeBSD and OpenBSD come with C++17-ready versions of Clang, and, last but not least, all major Linux distributors include at least one C++17-capable compiler in their packages. On RHEL/CentOS, devtoolset gives you access to GCC 7, SLES provides access to recent GCC versions and even the oldest (still supported) LTS versions for Ubuntu and Debian have Clang 4.

This makes us confident that we are not leaving many (if any) users behind if we raise our minimal required compiler version to one of:

  • GCC ≥ 7
  • Clang ≥ 4 (Apple Clang ≥ 9)
  • Visual Studio ≥ 2019.

Breaking Changes

With access to C++17, we are going to drop our replacement types such as caf::optional and caf::variant. Rather than slowly fading in C++17 features and deprecating individual parts one by one, we decided to make use of this opportunity to reshape CAF as a whole. Since we are breaking the API by switching to STL types and updating clumsy C++11 parts with more robust C++17 counterparts, we believe it is the right time to also apply lessons learned from previous CAF iterations.

Ideally, CAF 0.18 breaks many things at once now, so that we can resume incrementally evolving CAF after that point.

Messaging

Over the course of the last years of developing CAF, we came to understand message as a variant-like type able to hold any tuple type used for actor messaging. The current design can play that role, but the implementation is a highly flexible container that holds arbitrary data, offers views and even enables composing messages in tree-like structures. This flexibility comes at a cost. Obvious costs through virtual dispatch, but also runtime overhead for matching message handler signatures to the content of incoming messages. Further, because a message can essentially hold anything, CAF has to include a lot of meta data in each message on the wire. This increases size overhead on the wire but also makes deserializing messages costly.

Streamlining the messaging layer in CAF has the potential to improve performance significantly. First prototypes for a new messaging layer speed up the matching of messages to handlers as well as deserializing messages by a factor of 3. The downside of that design is that users have to enumerate all allowed types in the system. We did not reach full agreement whether we want to go down that route, but it would also allow CAF to better target embedded systems, as the new messaging layer would no longer require RTTI and stores less meta data.

Serialization

Fold expressions and other C++17 features allow us to reimplement the serializers in CAF more efficiently. Some of the optimizations are going to require changes to the class hierarchy. We are most likely going to drop the central data_processor and remove classes such as stream_serializer that CAF no longer uses internally.

OpenCL

A vendor-neutral API for GPGPU programming sure sounds great. Unfortunately, OpenCL did not catch on in the way we had hoped. At this point, we can call OpenCL dead and gone. There is only legacy support available and recent versions of the standard were never implemented in the first place. Consequently, we are going to drop caf_opencl.

Networking

Unrelated to CAF 0.18, we are working on a new networking layer in CAF. While we generally like the actor-based Broker API of caf_io, the module struggles to integrate exchangeable transport layers. Ideally, caf_openssl would not have to come as separate module.

With caf_net, we are currently working on a clean slate design that gives more flexibility in deployment while at the same time improving the user experience. We based the new API on URIs, which makes is easy to select a transport via URI scheme. We are also working on name-based access to remote actors that replaces the somewhat quirky publish/remote_actor function pair of caf_io.

However, caf_net is not going to replace caf_io overnight. Once the caf_net module reached a stable version, we will move it into the main repository where it eventually replaces caf_io.

Spotlight: Lumicks

For this spotlight, we dive into the world of molecular research and talk to the software team of the highly ambitious startup Lumicks at their Amsterdam HQ.

Background: Lumicks

Official Lumicks Logo

Lumicks manufactures dynamic single-molecule and cell avidity analysis equipment. Their breakthrough technologies enable visualization of molecular interactions and acoustic manipulation of biomolecules. With their products, Lumicks helps scientists to understand life to the smallest detail, which is critical for cancer research and drug development.

Founded in 2014, Lumicks quickly became a success story. Today, research sites all around the world have adopted Lumicks technology: from UC Berkeley at the U.S. west coast all the way to ShanghaiTech University in far east China.

The Interview

CAF Team: Dear Lumicks team, thank you for taking the time speaking to us! Lumicks works hard to deliver unique insights into biological processes at a molecular level. A recent article in The Scientist speaks about using sound waves to capture cells, which is a process you use for some of your products, while other products mention optical tweezers. A bit exaggerated: How can lasers and music help us to cure cancer? Without going into too much detail, how would you explain what your products do for people like us that don’t have degrees in molecular biology?

Lumicks Team: The key thing about both the lasers and the sounds waves is that they exert tiny forces. With the music during a pop concert, you can really feel those forces in your stomach. With light, it’s perhaps a bit harder to imagine: after all, you don’t experience a “thump” whenever you turn on the light, nor do you feel the sunlight pushing you back on a sunny day. Yet even light pushes against things a tiny bit, and this is actually used in the vacuum of space to drive spacecrafts with “solar sails”. At Lumicks, we harness those tiny forces to push and pull on the cells and molecules that make up your body. We can, for instance, “grab” a DNA molecule by its ends and stretch it, and then watch how other molecules land on it to repair damage on the DNA. Researchers learn invaluable things from such experiments about the processes that lead to diseases, including cancer.

CAF Team: You are working with very small scales but very high resolutions. We can imagine it takes quite a lot of data processing in real-time in order to visualize the experiments and allow scientists to manipulate individual proteins or cells while studying their behavior. What are the biggest software engineering challenges that you faced during initial product development?

Lumicks Team: Perhaps surprisingly, data volume is not our biggest challenge. Our sensor data comes in at hundreds of MB per second, tops. The real challenge lies in the large variety of hardware we need to talk to from the software, and orchestrating all the hardware in a coordinated way, often with strict timing requirements. We also have both a GUI and Python interface, which can run simultaneously, and so need some careful synchronisation. Plus each researcher using our software has their own specific needs for the experiments they are doing, so we need a lot of flexibility in putting the various modules together.

CAF Team: A big part of software engineering is constant learning about the problem domain and iterating on a prototype until a robust solution emerges. However, we can’t iterate endlessly. Time to market is very crucial, even critical in a startup like Lumicks. What brings the actor model, and CAF in particular, to the table that helps you iterating faster and reaching a sophisticated solution in reasonable time?

Lumicks Team: Our application does a lot of parallel processing and multi-threading: there is all sorts of hardware I/O running in the background, plus data processing, storage management, etc. CAF allows us to stay sane amidst all the threads: the actor framework enforces a clear model of data ownership and saves us from a lot of mutex-related headaches. Not having to worry about a whole class of subtle, hard-to-reproduce bugs is a life saver. On top of that, the availability of compile-time checking of messaging interfaces is very useful, and something where CAF has a tangible advantage over actor implementations we’ve seen in other frameworks and languages.

CAF Team: How did you learn about CAF and what convinced you to use it over other frameworks?

Lumicks Team: As a team we have used actors in other languages, such as LabVIEW and Erlang, and found it to be a good model for parallel processing. During the development of our control software it became clear that actors would be a great fit and and so we undertook a review of C++ frameworks. CAF came out as a clear winner and we have been using it ever since. The active development of the library was a major factor for us, as well as its flexibility and level of compile-time safety.

CAF Team: How would you summarize your experience with CAF so far? What was the first version you’ve used? Were there any unexpected obstacles in adopting CAF or aha moments?

Lumicks Team: CAF has been working very well for us. We started on version 0.15.5, in the Fall of 2017. We had some small issues in the beginning, mostly related to the lack of official shared library support on Windows, but otherwise have been closely tracking the official releases. We haven’t had any major unexpected obstacles, so far, but finding the Qt mix-in was a pleasant surprise. It made integrating with our UI components much easier! 

CAF Team: You develop all of your software in house with an international team of software engineers. How do you introduce new team members to CAF and how would you describe the learning curve?

Lumicks Team: We haven’t found the learning curve for new team members to be too challenging. As a team we have built a number of abstractions around CAF and so this helps to reduce the surface area that new team members need to learn. We also work with CAF on a single node, and use typed interfaces as much as possible to make intentions clear and explicit. Processes within the team also help. All of our code is thoroughly reviewed before merging and we do lots of internal knowledge sharing, from working together on tricky problems to fortnightly show and tell sessions on new and interesting developments.

CAF Team: What could CAF do better to smooth out the learning curve and help developers being more productive?

Lumicks Team: Although the CAF documentation has improved since we first started using the library, it is still an area where more work would help to smooth out the learning curve, along with some more complex examples of CAF usage. Improved tooling for debugging would also make a big impact on productivity. 

CAF Team: Final question: if you were to decide what the next feature of CAF would be, what would you have in mind?

Lumicks Team: One feature that would really help us as a team is better tooling to understand message flows, for example to visualize the flow of messages between actors. It would help not only to debug issues during development but also to optimise performance. We are also particularly interested in the possibilities of the new streaming API, so seeing that move out of its experimental state would also be great!

CAF Team: Thank you very much for this interview! We wish you and your team all the great success in helping scientists around the world to understand—and hopefully cure!—diseases such as cancer and Alzheimer’s!

Lumicks Team: Thank you! If any of your readers are interested in working with CAF to help scientists unlock new types of experiments in a small dynamic team based in Amsterdam then check out our careers page!