Introduction to Conan Package Manager - Kuba Sejdak (2023)

In the previous article in this series, I listed some issues that we as developers face when using external code dependencies in our projects. These range from version control to cross-compilation issues to the horror of integrating into our build system. Linux users are used to the convenience of working with package managers that download and install applications and libraries on our host system with easy-to-use command-line utilities. It would be very useful to have a tool that does something similar for libraries in the C++ world. Among the many package managers mentioned above, one stole my heart and attention forever:Conan.

Summary at a high level

Conan is a C/C++ package manager written in Python. That doesn't just mean that your binary is compiled from Python. Conan is distributed as a Python package. Also, the packages provided by this manager are defined in recipes in the form of Python classes. This may seem like a small thing, but using one of the world's most popular programming languages ​​as the package definition language offers great flexibility and a very low entry level for new users or packagers. I experienced this myself when creating recipes for packages that were missing from the existing Conan repository. I was able to focus on describing how to build a particular library instead of struggling with application-specific vocabulary for configuration formats. However, Python is only needed if you want to create new packages. If you just want to download and use one of the existing packages, there are several handy and easy ways to do it (including Python if you want).

Normally, when we want to download a Conan package, we call the package manager with the name of the package. Conan first checks registered package repositories for those that specify a specific package name. We can register multiple package repositories, which can be public or private. We can also use one of the existing repositories or build the entire infrastructure on our own private servers. Of course, this repository must be compatible with the Conan package protocol.

There are several ways to install Conan. The most recommended is as a Python package. To do this you need to run the following command (assuming you have bothFelshakenjlumpFurnished):

pip3 installs conan

Once that's done, you can search for packages with the following command:

suche conan <package_name> -r=all

or list the registered package repositories:

Conan Remote-Liste

and many more.

Example: usefmtjnlohmann_json

To quickly introduce you to the typical flow of using Conan, let's try to create a simple C++ application that usesfmtjnlohmann_jsonaliasJSON for modern C++. Both libraries are just headers, but that's not the point now. Our current situation is that we haven't downloaded it anywhere and we don't want to install it as a regular package on our system. Let's say the reason for this is that we want to use the latest possible version of both dependencies and our distribution is missing "some" versions.

In this example we will try to compile the following file (principal.cpp):

#include <fmt/printf.h>#include <nlohmann/json.hpp>int main(){ nlohmann::json json = { {"pi", 3.14}, {"happy", true}, {"name" , "cuba"}, {"nothing", nullptr}, {"answer", { {"all", 42} }}, {"list", {1, 2, 3}}, {"object", { {"currency", "PLN"}, {"value", 100.0} }} }; fmt::print("JSON: {}\n", json); returns 0 ;}

As we can see, first we create a JSON object usingnlohmann_jsonlibrary and then we print it with thefmtLibrary. We need access to 2 header files of the mentioned libraries. To get it, we'll create a Conan Requirements file, which simply lists the names of the required dependencies along with their versions, and also how we want to integrate them into our build system.

Here it isconanfile.txtFile saved in the root of the project:

[required]nlohmann_json/3.9.1fmt/7.1.3[generators]cmake

inside[required]section we list our dependencies and in the[The Generator]In the section we specify for which build system we want Conan to generate the integration layer.

Of course, since "our" preferred build system is CMake, we need a simple setup for it:

cmake_minimum_required(VERSIÓN 3.15)proyecto(conan_example LANGUAGES CXX)add_compile_options(-std=c++17)include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)conan_basic_setup()add_executable(conan_example main.cpp)target_link_libraries(conan_example PRIVATE ${ANCONLIBS} )

The final structure of the project looks like this:

conan_example/ ├── build ├── CMakeLists.txt ├── conanfile.txt └── main.cpp

To create the project we must follow the following steps:

1. install dependencies with Conan,
2. generate build system with CMake,
3. Compile the project with make (on a Linux machine).

1. Install dependencies with Conan

Install dependencies withconanfile.txtThe method is very simple: Runconan installation <path>from the build directory.<ruta>should indicate the locationconanfile.txtFile (in our example in the root of the project (one level up).

kuba@chimera:~/projects/conan_example/build$ conan install ..Config:[configuration]arch=x86_64arch_build=x86_64build_type=Releasecompiler=gcccompiler.libcxx=libstdc++11compiler.version=10os=Linuxos_build=Linux[options][build ][env]AR=gcc-ar-10AS=gcc-10CC=gcc-10CXX=g++-10LD=ldOBJCOPY=objcopyRANLIB=gcc-ranlib-10SIZE=sizeSTRIP=stripnlohmann_json/3.9.1: not found in local cache, look in remotes... nlohmann_json/3.9.1: Test with 'conan-center'... Download of conanmanifest.txt completed [0.10k] Download of conanfile.py completed [2.43k] Download of conan_export.tgz completed [0.24k] ] Conan_export. tgz decompression complete [0.00k] nlohmann_json/3.9.1: Downloaded recipe revision 0conanfile.txt: PackageRequirements installation fmt/7.1.3 from 'conan-center' - Cache nlohmann_json/3.9.1 from 'conan-center' - Downloaded packages fmt /7.1.3:b173bbda18164d49a449ffadc1c9e819d49e – Cache nlohmann_json/3.9.1:d1091b2ed420e6d287293709a907ae8 24d5de508 - DownloadInstall (herunterladen, erstellen) Binärdateien…nlohmann_json/3.9.1: Paket d1091b2ed420e6d287293709a907ae824d5de508 wird vom entfernten „conan-center“ abgerufen kpresstconzan_package] Descompresión [0.00k] nlohmann_json/3.9.1: Paquete instalado d1091b2ed420e6d287293709a907ae824d5de508nlohmann_json/3.9.1: Paquete descargado revisión 0fmt/7.1.3: ¡Ya instalado!conanfile.txt: Generador cmake creado conanbuildinfo.cmakeconanfile .txt created contxtx txt: Generiert mitaninfo.txtconanfile.txt: Generated graph info

Here we can see some interesting things.

At the beginning (line 2-23) Conan shows the configuration used to create a specific set of dependencies. In the case of header-only libraries, of course, it doesn't matter, but the same scheme is used for any other library. This configuration includes the architecture and operating system for both the host and target platforms (must be identical if not cross compiled). We can also see environment variables and paths/names to the toolchain binaries. This configuration dump is very useful when debugging problems. In fact, this setting is exactly what the user specifies in ConanProfile.

Later, Conan will install the dependencies one by one. First it tries to find a specific package in the local cache. in case ofnlohmann_json/3.9.1library, I've never installed it, so there was no such entry in the cache. In such a situation, Conan goes through the registered remotes one by one to find the one that provides the specified package. Here thiscenter-conanRemote control (the main and default registered Conan remote control) is the selected one. After that, the package will be downloaded and cached. By package I mean the metadata (recipe), likeconanfile.py(line 27) This is the main description of how a given library is created.

Just before running the build, Conan again explicitly shows which packages were selected (along with their unique IDs) and where they were located: cache or remotes (lines 32-37). Finally, it starts building, which in the case of header-only libraries results in files simply being downloaded and placed in the appropriate cache location.

In the end, Conan will generate an integration layer for our build system as specified in the[The Generator]section ofconanfile.txtArchive. In our case, it is about creating files that we must include in oursCMakeLists.txtfor CMake to correctly find installed libraries in the conan cache (line 47-50). In particular, the most important file for us will be theconanbuildinfo.cmake(see excerpt withCMakeLists.txtlines 7-8).

2. Creating the build system with CMake

Now it's time to start CMake. We do this in typical fashion, nothing fancy here.

kuba@chimera:~/projects/conan_example/build$ cmake ..-- CXX compiler ID is GNU 10.2.0-- Detecting CXX compiler ABI information-- Detecting CXX compiler ABI information - done-- Make sure the CXX compiler works : /usr/bin/c++ - skipped - CXX compiler feature detection - CXX compiler feature detection - done - Conan: Set output directories - Conan: Use cmake global settings - Conan: Set default RPATH Conan policies -- Conan: Set language default-- Current conanbuildinfo.cmake directory: /home/kuba/projects/conan_example/build-- Conan: GCC compiler>=5, checking for major version 10- - Conan: Check for correct version: 10-- Configuration complete-- Build complete: Build files have been written to: /home/kuba/projects/conan_example/build

Assuming we installed the dependencies correctly with Conan, including theconanbuildinfo.cmakefile and calledconan_basic_setup()(defined inconanbuildinfo.cmake) there should be some commit logs about Conan in the CMake output (line 8-14).

Note that the CMake generator creates a variableCONAN_LIBSA, which groups all CMake targets that represent installed dependencies. All we have to do is use this helper variable in thelibrary_target_link()domain (CMakeLists.txt– lines 14-16).

3. Build the project with make (on a Linux machine)

Finally, we can build the app using the native toolset:

kuba@chimera:~/projects/conan_example/build$ makeScanning target dependencies conan_example[50%] Build CXX object CMakeFiles/conan_example.dir/main.cpp.o[100%] Bind executable CXX file bin/conan_example[100% ] Constructed target conan_example

The output of the program is as expected:

JSON: {"response":{"all":42},"happy":true,"list":[1,2,3],"name":"Kuba","nothing":null,"object" :{"currency":"PLN","value":100.0},"pi":3.14}

custom settings

Conan's advantage over many existing package managers is that we can customize many different settings that can be related to the tools we want to use, the target platform or even where to look for and download packages. Everything that can be customized by the user is neatly organized in specific configuration files. They're usually in an easy-to-understand text format (like JSON or INI-ish). You can edit it manually (if you know the syntax) or use Conan's commands to change it for you. The last option is very useful when you need to automatically adjust settings in the continuous integration system.

There are two types of customization options that you typically want to customize: Profiles and Remote Controls. Below is more information about each of them.

Profile

Conan-ProfileThey are the heart of Conan's user build. They specify many different variables that Conan uses during the build process:

• toolchain names/paths,
• metadata of the host and target platforms,
• environment variables,
• package-specific options (options provided by the creator of each package, such as "Include Tests" or "Enable Feature A").

Many of the above options can be set globally or for the specified package (e.g. use further optimization forspdlogPackage).

This feature itself is very powerful because you can create as many profiles as you like and give them a meaningful name that reflects the content. For example, you can have a profile that allows compilation with GCC, Clang, with full debug mode, with high optimization, a profile that is used for cross-compilation for the Raspberry Pi, etc.

Even more possibilities can be achieved by compiling profiles. So if you have a profile that only specifies the compiler (e.g. GCC or Clang), a separate profile that selects the build mode (debug or release), and finally an OS-specific profile (e.g. Windows and Linux) , you can combine these together just by listing their names on the command line, for example:

installiere conan .. -pr gcc-10 -pr debug -pr linux

Personally, I define a profile per specific toolchain (including the target OS). Here is my profile for the GCC 10 compiler (calledgcc-10):

[configuración] os=Linux os_build=Linux arch=x86_64 arch_build=x86_64 compiler=gcc compiler.version=10 compiler.libcxx=libstdc++11 build_type=Release [opciones] [build_requires] [env] AR=gcc-ar-10 AS=gcc-10 CC=gcc-10 CXX=g++-10 OBJCOPY=objcopy RANLIB=gcc-ranlib-10 SIZE=tamaño STRIP=tira

As you can see in the above[Ideas]Section I define metadata for the platform (operating system, architecture, build type and compiler type). In[options]j[build_requires]I am not specifying anything related to the package as this profile is generic, just select the appropriate toolchain). inside[env]In the section, I specify well-known variables used by all major build systems. That way, it doesn't matter if the given package was compiled with automated tools or CMake, it just uses specific variables.

Below is a list of my personal profiles:

kuba@chimera:~$ Liste der Conan-Perfiles arm-linux-gnueabihf-clang-11 arm-linux-gnueabihf-gcc-10 arm-none-eabi-gcc-10 clang-11 default gcc-10

Each profile is saved as a separate file in~/.conan/profiles/<Profilname>. On first use, Conan will detect your current operating system settings and generate oneStandardprofile for youStandardThe profile is used by default unless you specify a different one on the command line (or otherwise).

remote controls

remote controlsare other user-specific settings that you may want to change. They are simply the names and URLs of the Conan servers used to look up the packages. As demonstrated withfmtjnlohmann_jsonFor example, each time a package is installed, Conan will search the registered remotes one by one and check if any of them are deploying a specific package in a specific version.

By default, the only registered remote control is thecenter-conan, the main official repository for Conan. Every day Conan maintainers add new packages and update existing ones.

However, you can register remote controls that belong to other people, organizations, or even your company. It doesn't matter if the specified server is public or private. As long as you have access rights, you're good to go. So in other words, Conan is the perfect choice for companies that need to keep their stuff private but still want to provide a central location for their packages. I registered my own repository to store packages that may not be generic enough to upload to the official repository but are very useful to me. I also use it as a proving ground for packages that may end up on thecenter-conanas soon as I try them enough.

Here is the list of remote controls registered on my computer:

kuba@chimera:~$ conan remote list conan-center: https://conan.bintray.com [Verify SSL: True] bincrafters: https://api.bintray.com/conan/bincrafters/public-conan [Verify SSL : True] kubasejdak: https://api.bintray.com/conan/kubasejdak/public-conan [Confirm SSL: True]

Conan's servers contain two types of data: package recipes (metadata) and prebuilt binaries. In a typical workflow, Conan first checks the local cache for the package, and then the registered remotes. However, downloading a package from a registry should not always result in a build on the client machine. Package maintainers sometimes create precompiled binaries for the most common configurations and upload them to remotes. Conan will check if the currently selected profile (mentioned above) has a corresponding prebuilt package on the server. If yes, it downloads the binary and saves your time. However, you can force Conan to always compile dependencies on your computer.

install packages

Conan packages can be installed in at least three different ways:

1. a man,
2. automatically withconanfile.txt,
3. automatically with the CMake wrapper.

Each of them has its purpose, but in general you will use each oneconanfile.txtor CMake wrapper. This is because you should typically automate the installation of dependencies as you build your project. Let's see a quick summary of what you can do in each of these ways.

Manual installation from the command line

Manual package installation means you explicitlySummon Conanto install a set of libraries explicitly defined on the command line. A simple example would be the following:

conan installs opencv/4.5.1@

Here we want to installopencvPaket in Version4.5.1. We don't specify profiles, so Conan uses the default. We also don't specify a generator, so no build system integration is generated. That seems easy. It's an easy way to experiment, test your own packages, or try different libraries.

This command can be expanded to include settings specific to the package, required profile, or generator we want to use:

conan install opencv/4.5.1@ -o opencv:with_png=Falsch -o opencv:with_tiff=Falsch -o opencv:with_gtk=Falsch -pr gcc-10 -g cmake

This line can be quickly expanded if we put together several profiles, libraries and options. Not to mention that you can override certain settings of selected profiles. You will quickly realize that this is not production code. However, if you're experimenting or looking for a suitable library to include in your project, this is the quickest way to get started.

Automatic installation withconanfile.txt

Conan supports the idea, borrowed from Python's package manager (pip), that allows you to put all the requirements and options in a single file: theconanfile.txt. The example above withopencvcould look something like thisconanfile.txt:

[requires]opencv/4.5.1[generators]cmake[options]opencv:with_png=Falschopencv:with_tiff=Falschopencv:with_gtk=Falsch

Here we have a slightly more readable format than the command line due to the separate sections for each setting. However, the biggest advantage of this is that you can store all these settings in one file and have them under the control of the VCS. This way you always have the right dependencies for the currently extracted code.

You can also have a bunch of files that contain different libraries. So by giving Conan another way to do thatconanfile.txtYou can install a different set of packages. This can be useful when building many applications on the same code base. Usually each application has its own dependencies.

Automatic installation with CMake container

The canonical (recommended) way to use Conan is through theconanfile.txtAttitude. This is fine as long as your dependencies don't change dynamically. For example, this can happen when you have a library that can be built in different configurations. And each configuration might require a separate set of dependencies. Let's look at this simple example:

mylib/ ├── CMakeLists.txt ├── module_a │ └── NetworkBackend.cpp # requires library "libcurl" └── module_b └── DatabaseBackend.cpp # requires library "sqlite3"

Here we have a directory structure for themilibLibrary consisting of two submodules:ModulatejModule_b. The first relates to some network stuff, so you need itlibcurlin its implementation. The second works on a local database and requiressqlite3Library. User application being linked tomilib, you can compile it withModulate, ÖModule_bor both. How can we instruct Conan to actually work with CMake and only install what is necessary in the given configuration?

If this example still doesn't convince, imagine that there are hundreds of such modules, which internally can also be built from different parts depending on other configurations. Also, we may have some pieces of code in our library that are platform specific and need libraries only for that platform. We don't want to get into a situation where Conan is trying to install Linux libraries on Windows.

At the very top is the application (which only has themiliblibrary) may have its own dependencies. So we need to install many potential source libraries. And the final set of packages depends on the build system. If you remember the first example withfmtjnlohmann_json, you know that the order of steps is as follows:

1. install dependencies,
2. start cmake,
3. compile.

In our situation, CMake needs to be started first. But even that isn't enough, because CMake might want to check that the libraries we want to use are actually available. Therefore, Conan needs to install the libraries before CMake finishes. Impossible to solve? No, there is a practical solution:cmake-conanEnvelope.

This script is a CMake file that contains helper functions to run Conan from CMake. This allows you to collect dependencies in a list while CMake parses your project. Then at the end you can call Conan from the build system and install all the libraries at once. Alternatively, you can install ad hoc dependencies on theCMakeLists.txtfile, you really need it. The second approach is much better since you can guarantee that you're only installing what's actually being used, and you're using well-known CMake mechanisms (such asfind_library) that require a prior installation of the library.

Here is a very simple example of howcmake-conanWrapper can be used:

include(conan.cmake) # This is the actual wrapper file for cmake-conan.conan_cmake_run(REQUIRES OPTIONS opencv/5.4.1 opencv:with_png=False opencv:with_tiff=False opencv:with_gtk=False)

conan_cmake_run()It actually invokes Conan's command and waits for the result. Because you can use CMake variables in this call, this mechanism gives you great flexibility in defining which packages to install.

hidden from Conan

As I mentioned earlier, Conan creates a local cache for all packages downloaded and created for future reuse. It is usually found in$HOME/.conan/dataDirectory. Below is an example cache status from my computer:

kuba@chimera:~$ tree .conan/data/ -L 5 -d .conan/data/ ├── catch2 │ ├── 2.13.2 │ │ └── _ │ │ └── _ │ │ ├── Export │ │ └── package │ └── 2.13.3 │ └── _ │ └── _ │ ├── dl │ ├── export │ ├ ├ ├ ├ ├ ├ ├ ├ ├ ├ ├ ├ ├ │ │ " └ 1.3 │ └ ─ ─ KubasejDAK │ └──¨ Stable │ ├── Build │ ├─´ export │ ├─´ export │ │ │ Source └iqu. _ └── _ ├── dl ├── export ├── sperrt └── Paket

Each package that is installed in each version has its own subdirectory there. Conan first checks if the specified dependency is already in the cache before downloading the recipe. Knowledge of the cache and its structure is very useful when creating your own packages.

Summary

Conan is a powerful package manager for C and C++. It is true that many things can seem complicated or overwhelming at first. But that's the price of introducing something new. All I can say is that using this tool will become very easy over time and you will quickly be tempted to add new packages yourself as soon as you find one that isn't already provided by the official repository.

In the next article I will focus more on cross-compiling and show how I use Conan in my CMake projects. For this I use a slightly modified version of thecmake-conanEnvelope. Stay tuned!