TLDR: How to Reduce Known CVEs (Common Vulnerabilities and Exposures) to Zero by Building Your Own Keycloak Distribution*.
Introduction
Keycloak (verInternet site)it will get easier and more powerful with the move to Quarkus, at least that's the promise. We've already shown you how to approach a productive setup step by step in the blog postFrom Keycloak to Keycloak.Xwith an older version of Keycloak.X. Meanwhile, editing18.0.0was also releasedroadmap for the Keycloak projectwas specified in more detail. Among other things, it states that the last Wildfly distribution will be released in September 2022 – from then on there will only be the Keycloak distribution based on Quarkus.
This article describes an approach to improving the performance and security of a Keycloak system by creating a custom Keycloak distribution. This requires full control over creating your own Keycloak distribution.
Aspects of a custom Keycloak distribution
Creating your own custom Keycloak distribution can improve the security and/or performance of your current Keycloak system. As a counter-argument, we often hear that having your own distribution leads to unnecessary and increasing complexity. Furthermore, there seems to be a general recommendation to use official images so that part of the responsibility does not fall on him. We defend here the explicit assumption of this responsibility in the case of Keycloak and we see great advantages in this step.
A custom distribution can support the following:
- Use optimized configuration for fast server startup
- Support for custom extensions and themes
- Only Quarkus extensions enabled
- Additional required Quarkus extensions are supported
- Libraries can be updated to a current patch level
Normal distribution properties
To view the properties of the default Keycloak distribution, we use the following default Keycloak Docker image:quay.io/keycloak/keycloak:18.0.0.
A Docker container with the image can then be started as follows:
1docker run --rm -it quay.io/keycloak/keycloak:18.0.0 start\2--autobuild \3--http-enabled=TRUE\4--hostname-strict=false\5--hostname-strict-https=false
we use it--auto buildparameter to tell Keycloak to apply the configuration at build time.
Default image enabled extensions
The previous command generates the following list of enabled Quarkus extensions (Keycloak features) when starting the Keycloak server:
12022-05-07 10:44:39,393 INFO [io.quarkus] (main) Installed features:2[agroal, cdi, hibernate-orm, infinispan-client, jdbc-h2, jdbc-mariadb,3jdbc-mssql, jdbc-mysql, jdbc-oracle, jdbc-postgresql, keycloak,4narayana-jta, reactive routes, resteasy, resteasy-jackson,5smallrye context propagation, smallrye health, smallrye metrics, vault,6vertex]
We see here that Keycloak supports several databases by default: MSSQL, Oracle, MySQL, MariaDB, H2 (old version 1.x with many CVEs). We'd like to narrow it down to the single database needed in the next lesson: PostgreSQL.
Extensions are missing from the default image
Quarkus offers a wide range of features that can be enabled through Quarkus extensions. It is already standard in the Keycloak distribution.
A way to enable these functions has already been requested in atalk about the keyand there was already a community solution. The process described in the Keycloak discussion works, but may deter users due to its complexity.
Vulnerabilities found in the default image
In our example, we use the toolCuriosities of Aquasecurityto check Docker images for known CVEs. You can easily run the tool as a Docker container.
We use a small Java CLI wrapper here to perform the Trivy scan:
1java bin/scanImage.java --image-name=quay.io/keycloak/keycloak:18.0.0
Trivial scan result with default Keycloak Docker image assubstance.
1quay.io/keycloak/keycloak:18.0.0 (redhat 8.5)2=============================================3Total: 104 (UNKNOWN: 0, LOW: 37, MEDIUM: 65, HIGH: 2, CRITICAL: 0)4 5java (jar)6===========7Total: 5 (UNKNOWN: 1, LOW: 0, MEDIUM: 0, HIGH: 1, CRITICAL: 3)
Observation: These results change over time:
- New vulnerabilities are detected
- Overall CVE ranking changes due to new discoveries
- There is a re-release of the Docker image with updated operating system components
Create your own Keycloak distribution
To create our own Keycloak distribution with the customizations mentioned above, we combined parts of the Keycloak.X server distribution with the Keycloak server application Quarkus, which is also used by the Keycloak project in its own distribution. To do this, we created our own Maven project. Using Maven Dependency Management, we include the Keycloak Quarkus distribution as a.zipperfile.
This file is then unzipped with themaven dependency plugininsidetargetlist, where we explicitly exclude itlibrarydistribution directory. The next step is to include thekeycloak-quarkus serverMaven dependency, which allows us to customize the Keycloak Quarkus Server application dependencies.
In order to be able to store other configurations in the generated Keycloak distribution, its contentsrc/main/copy to keycloakThe directory is copied via Keycloak's unzipped distribution through itmaven resource plugin.
We can build our own distribution with the following command:
1mvn clean package
After that we found our own Keycloak distribution in the directory
target/keycloak-18.0.0, which can now be used.
Add extensions and themes
This approach also allows for the use of custom extensions and themes. In the example, we used our own event listener provider and a custom theme.
Testing with Keycloak Testcontainers
Our own extensions can be tested automatically with your helpKeycloak test containerslibrary in the form of integration tests. For simplicity, we use the default Keycloak Docker image for testing. With a bit of additional configuration and build orchestration, the custom image created earlier can be used here as well.
Create a custom Docker image
Our own Keycloak.X distribution can be ported to our own Docker image in the same way as the standard Keycloak.X Docker image. In our example we use fabric8Plug-in Maven Dockerfor this.
Next, we start building the Docker Image using the following command:
1mvn clean package docker:build2-Ddocker.image=thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT
Remove unnecessary Quarkus extensions
Keycloak uses numerous libraries built in through Quarkus extensions. Depending on the environment, some of these extensions are not required, e.g. if only one PostgreSQL database is used, support for Oracle and other databases is not required. In this case, Quarkus extensions can be removed through the appropriateMaven dependency exceptions. For example, if we want to remove support for the Oracle database, we can apply the following exceptions to theorg.keycloak:keycloak-quarkus-server:18.0.0Maven dependency:
123 org.keycloak 4keycloak-quarkus server 5${keycloak.version}67 8 9...1011 12 com.oracle.database.jdbc 13ojdbc11 141516 eu.quarkus 17quarkus-jdbc-oracle 181920 eu.quarkus 21quarkus-jdbc-oracle deployment 2223...2425
This technique can also be used to remove vulnerable libraries that are not needed. For example, Keycloak currently uses an old version 1.x of the H2 database by default, which is affected by various CVEs (note: once Keycloak is updated to new Quarkus version >2.8.2, H2 will also will be updated to a new version Version 2.x with no known CVEs). However, if you only use Keycloak with another database like PostgreSQL, you can also remove the H2 extension.
To remove support for the H2 database, we can implement the
NEXTMaven dependency exceptions:
1256 com.h2database 7h2 8910 eu.quarkus 11quarkus-jdbc-h2 121314 eu.quarkus 15quarkus-jdbc-h2 deployment 16
Furthermore, an input likedatabase=postgresmust be added to the file
src/main/resources/META-INF/keycloak.conf. You need to add an entry like
database=postgres, otherwise the distribution version of Keycloak will complain about the missing H2 library.
Let's start our distribution created this way as a Docker container (see below) with the following command:
1docker run --rm -it\2-p 8080:8080\3-e KEYCLOAK_ADMIN=keycloak\4-e KEYCLOAK_ADMIN_PASSWORD=keycloak\5thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT \6to start \7--autobuild \8--http-enabled=TRUE\9--http-relative-path=auth\10--hostname-strict=false\11--hostname-strict-https=false\12--db=postgres\13--db-url-host=172.17.0.1\14--db-url-database=keycloak \15--db-username=keycloak\16--db-password=keycloak
We see in the container log output that the unnecessary database extensions are gone and justjdbc-postgresqlremains.
12022-05-07 14:27:09.161 INFO [io.quarkus] (main) Installed features:2[agroal, cdi, hibernate-orm, jdbc-postgresql, keycloak, narayana-jta,3reactive routes, resteasy, resteasy-jackson, smallrye context propagation,4micronilli-health, metric rye, vault, peak]
Integration of additional Quarkus extensions
This approach also allows us to use new Quarkus extensions for Keycloak.
For example, we want to enable support for centralized registration using GELF in Keycloak.
To do this, we add the following dependencies to the Maven project:
123 eu.quarkus 4quarkus-logging-gelf 5${quarkus.version}67 8 eu.quarkus 9quarkus-logging-gelf deployment 10${quarkus.version}1112 13
When we built our custom distribution, the newQuarkus GELF Extensionsit will be recognized and activated accordingly.
These Quarkus-specific extensions can then be configured using the
quarks.propertiesfile inconditionKeycloak installation directory.
An example configuration inquarks.propertiesfor GELF:
1# Configure log stream via gelf2quarkus.log.handler.gelf.enabled=TRUE3quarkus.log.handler.gelf.host=localhost4quarkus.log.handler.gelf.port=122015quarkus.log.handler.gelf.facility=iam
Let's restart our distribution created this way as a Docker container:
1docker run --rm -it\2-p 8080:8080\3-e KEYCLOAK_ADMIN=keycloak\4-e KEYCLOAK_ADMIN_PASSWORD=keycloak\5thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT \6to start \7--autobuild \8--http-enabled=TRUE\9--http-relative-path=auth\10--hostname-strict=false\11--hostname-strict-https=false\12--db=postgres\13--db-url-host=172.17.0.1\14--db-url-database=keycloak \15--db-username=keycloak\16--db-password=keycloak
We see that the desiredlogging-gelfThe extension was recognized by the Quarkus runtime.
12022-05-07 14:27:09.161 INFO [io.quarkus] (main) Installed features:2[agroal, cdi, hibernate-orm, jdbc-postgresql, keycloak, logging-gelf,3narayana-jta, reactive routes, resteasy, resteasy-jackson,4smallrye context propagation, smallrye health, smallrye metrics, vault,5vertex]
Repair used libraries
As already mentioned, CVEs are known for some Java libraries used by the current Keycloak distribution. There are already supported patch versions for some of these libraries. With the presented approach, these libraries can be easily updated viaMaven dependency management. New dependency versions are then updated accordingly when dependencies are resolved in the Keycloak distribution build and pushed to the latest (compatible) patch level.
the latest availableKeycloak version 18.0.0contains many vulnerable libraries, for example a version of the XStream library (1.4.18) that we can update with a managed Maven dependency replacement:
12 34 5 com.thoughtworks.xstream 6xstream 71.4.19 89 1011
Observation: In our GitHub example, we were able to successfully mitigate all CVEs through dependency updates.
Observation: Since every new version of Keycloak comes with new library versions, it is recommended to remove superseded managed dependencies after updating the Keycloak version and performing a new image scan. After a new image scan, you may get a new list of vulnerable libraries that you can re-patch as shown.
Security vulnerabilities were found in the base image
Through proper dependency updates and deletions, we can bring all Java libraries into a safe state.
No other CVEs are listed for Java libraries. However, theubi8-minimumThe Docker image still contains vulnerabilities.
We can run the Docker image verification with the following command:
1java bin/scanImage.java2--image-name=thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT
Trivial check result with customubi8-minimumpicture in onesubstance.
1thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT (redhat 8.5)2================================================= =========3Total: 104 (UNKNOWN: 0, LOW: 37, MEDIUM: 65, HIGH: 2, CRITICAL: 0)4 5java (jar)6===========7Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
If we also want to get rid of the reported CVEs from the base image, then one possibility is to swap the base image for one without CVE, for example an Alpine based image. According to the Trivy scan, the imagealpine:3.15.4currently contains no known CVE. Using the following command, we can create an Alpine-based Docker image:
1mvn pacote limpo docker:build -Ddocker.file=keycloak/Dockerfile.alpine
Then a rescan of the new Docker image with Trivy gives nice results: 0 CVEs \o/.
1java bin/scanImage.java --image-name=thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT
Trivy scan result with Alpine Docker image assubstance.
1thomasdarimont/custom-keycloakx:1.0.0-SNAPSHOT (alpino 3.15.4)2================================================= =============3Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)4 5java (jar)6===========7Total: 0 (UNKNOWN: 0, LOW: 0, MEDIUM: 0, HIGH: 0, CRITICAL: 0)
Summary
In this article we present an approach to building your own Keycloak distributions. This approach makes it possible to simply deliver your own extensions and themes instead of doing this, for example, during runtime development. Also, unnecessary Quarkus extensions can be removed and new Quarkus extensions can be added to Keycloak.
Another customization option is deep library updates with no known security vulnerabilities. Furthermore, using a different base Docker image, we were able to create a Docker image with a Keycloak distribution that does not contain any known CVEs.
In addition to increased security due to a reduced attack surface, the coverage area is again improved due to the reduced number of extensions.
This approach allows dynamic packaging of Keycloak distributions according to your own requirements. It would be desirable for the Keycloak project to support this or a similar approach in hindsight to allow for more secure and improved Keycloak installations.
You can find example to create your own Keycloak distributionhere not GitHub.
Affiliateskeycloak-custom server/zero-cvesyou will find the example version we use for the scans.
disclaimer
Keycloak is a complex open source software product based on a large number of libraries. Unfortunately, this doesn't stop CVEs from being discovered – but that happens to all large software projects. We are very happy with our achievement: producing a custom Keycloak distribution with no known security vulnerabilities*. Other approaches like search/replace/delete vulnerable libraries served the same purpose but were always quite fragile. We wait for your comments.
*) Zero CVEs refers to the result of an image scan with Aquasec/Trivy and is a snapshot at the time of the experiment. ONEscanwith other tools, eg.Docker Check, at another time may reveal new CVEs if new CVEs become known in the meantime. We recommend running continuous vulnerability checks on your build artifacts such as custom libraries and Docker images.