I recently explained how I came to develop Sanca , a vulnerability scanner written in Rust. After updating the dependencies, Sanca started crashing. Not just a crash, but a SIGSEGV (segmentation violation, also known as a segmentation fault), a memory error that caused it to be terminated by the operating system. Rust is supposed to prevent this sort of error.
The problem only occurred on OpenBSD, not on Linux. OpenBSD has
stricter security rules regarding memory (for example, W^X), so
that makes sense.
I use Sanca every day on OpenBSD, so I had to fix this.
Looking for the cause
The image above shows the error that occurred when I ran Sanca,
it’s not exactly detailed.
Fortunately, OpenBSD saves the state of the memory at the time of the crash in
a .core file, allowing us to analyse the memory, the registers, and everything
else needed to identify which part of the code is causing the programme to crash.
I opened sanca_software.core in gdb (GNU Debugger), which quickly put me on the right track.
As soon as I opened it, GDB told me there was a problem with aws-lc-sys, the Rust wrapper around the aws-lc library, written in C. After doing some research, I discovered that aws-lc was a cryptographic library based on BoringSSL, a fork of OpenSSL. The problem therefore seemed to be related to SSL/TLS, but I wanted to be sure of the root cause.
Sanca does not depend directly on this library, so I used the
cargo tree -i aws-lc-sys command to find out which of Sanca’s dependencies
aws-lc-sys was using. It turned out to be reqwest.
Sanca uses a library called “reqwest” to send HTTP requests to Web servers; reqwest uses “rustls” to handle the TLS protocol, and rustls uses “aws-lc-rs”, which is a wrapper around “aws-lc-sys”, to handle the cryptographic aspects.
Research and implementation of the solution
Having identified the source of the problem, I needed a solution.
Whilst researching rustls, I noticed that the library
offered several cryptography ‘providers’; aws-lc was the new
default provider, whilst the old one was ring. However, it was still possible
to install rustls with ring instead of aws-lc, thanks to a feature in
Rust / Cargo called feature flags. It remained to be seen whether reqwest was
flexible enough to let me choose the rustls provider.
Fortunately, yes. Thanks again to feature flags, I was able to
install reqwest whilst specifying that it should not install any providers for rustls,
and at the same time I installed and initialised rustls myself using ring as
the provider. This time it worked; I was able to compile and use Sanca
without any problems.
I could have left it at that, but it bothered me to simply remove aws-lc from Sanca
when it was the default choice for rustls – there must have been a
reason for it.
I’ve decided to use feature flags in Sanca as well, to allow users to choose between aws-lc and ring. Currently, there are two ways to compile Sanca:
cargo build --release
to compile using aws-lc, which is likely to suit most FreeBSD, Linux, macOS and Windows users. Or
cargo build --release --no-default-features --features ring-provider
to compile Sanca with ring, which is more suitable for OpenBSD users
at the moment.
I have noted the difference in the README file, so that each user
can choose the method that suits them best.
What I’ve learnt
Modern software development relies heavily on external libraries, regardless of the language. Whilst JavaScript with npm is the prime example of supply chain attacks, where compromises of external libraries affect the projects that use them, Rust with Cargo is not immune to such issues either.
This isn’t an attack; it’s simply a bug. But a bug in a dependency affects the projects that use it. You need to know how to debug and find suitable solutions that work today without hindering the project tomorrow. The aws-lc library supports “post-quantum” algorithms, which is not yet the case with ring, so it was important not to make any decisions that would block progress.
In Rust, you often find dependencies on libraries written in C. This
also applies to other languages such as Python or PHP, but Rust
makes a strong promise: that memory accesses are safe. This is not
its only advantage, but it is a very important distinguishing feature.
The problem is that today many important programmes are written in
C or C++, whether they are cryptographic libraries such as LibreSSL, OpenSSL
or BoringSSL, or operating systems such as OpenBSD, Linux, macOS or
Windows. C is a great language; it runs on all architectures and
compiles very quickly, but its memory accesses are not safe. Even
today, a huge number of vulnerabilities are linked to memory management.
As long as Rust relies on libraries written in C, which is not likely to change any time soon, given that C is ubiquitous, we will continue to face memory issues. A cryptographic library written entirely in Rust is currently under development, but it will be several years before it is as comprehensive and thoroughly audited as OpenSSL or aws-lc.
A dependent library that crashes is yet another vulnerability in your
supply chain.
If you want to audit your own tools or migrate to a stack you have
greater control over, let’s discuss it
.