What Can Coerce, and Where, in Rust

Rust supports a number of type coercions, which implicitly convert one type to another. As in any language with coercion, there is a trade-off made between clarity when reading and ease of writing. While disagreement may be had about whether Rust’s list of supported coercions is best, there is value in learning the coercions available, as some are central to functioning or idiomatic Rust code. In this post, I describe what coercions are possible, and where they can happen.

What is a coercion?

Before getting into what and where, it’s good to be clear about what is meant by coercion. Rust supports multiple ways to convert one type to another. The From and Into traits can be used for infallible conversions at the library level. TryFrom and TryInto handle fallible conversions as well. AsRef, AsMut, Borrow, and ToOwned provide still more library-level conversions between different sorts of types. However, all of these are explicit. To perform the conversion, the user must call the relevant function. Coercions, by contrast, are implicit. The hidden nature of these conversions means they are intended to only be available when their utility relies on ease, and the potential for harm from hidden type changes is minimal. Casts, done with the as keyword, are explicit, and there are more casts permitted than there are coercions.

Info 1 Transmute, the unsafe conversion.

The standard library includes a function, std::mem::transmute, which permits conversions from any type to any other type. This function is unsafe, as no guarantees are made that the bit representation of the input type is a valid bit representation for the output type. It is up to the user to ensure the two types are compatible.

There is an effort dedicated to developing “safe transmute” options in Rust, called appropriately, “Project Safe Transmute.” Their work is ongoing, with the intent of providing versions of transmute which do not require unsafe when the transmute in question is known to be valid (meaning valid bits in the source type are always valid bits in the target type).

What coercions are there?

Rust supports a number of coercions, although their definition is informal and remains subject to some degree of change and clarification. In fact, long-term specification of these transformations is expected to be part of an eventual standandardization process, as they are crucial to understanding Rust’s type system.

Info 2 On Standardized Programming Languages

The criticism that Rust is less trustworthy than C or C++ because it lacks a specification comes up periodically, and is worth addressing here. First, while it’s true that Rust doesn’t have a specification in the manner C or C++ do (published and managed by the International Standards Organization), that doesn’t mean Rust is entirely unspecified.

Rust has a reference, which codifies much of the language’s intended semantics. It also has an RFC process which manages change in the language, along with teams overseeing the language’s growth. These teams include the Unsafe Code Guidelines Working Group, which aims to better specify the semantics, requirements, and guarantees affecting unsafe Rust code. This group produced miri, an interpreter for Rust’s MIR (Mid-level Internal Representation) language which can also perform automated verification that MIR code is consistent with the “stacked borrows” model of Rust’s semantics proposed by the UCG WG. The main Rust compiler is also thoroughly tested, including automated regression testing of experimental changes and new compiler versions.

There is at least one working alternative implementation, mrustc, although it is not generally intended for end-user use. There’s also newer work on implementing a GNU Compiler Collection front-end supporting Rust, called “rust-gcc.”.

There is an ongoing effort to get Rust certified for use in safety critical domains, including the avionic and automative industries, called Ferrocene. This is being shepherded by Ferrous Systems, a Rust consultantcy which includes major language and community contributors among its team.

Finally, the challenge of formally specifying and proving Rust’s guarantees has been taken up in academia, with multiple projects producing models including Patina, Oxide, RustBelt, KRust, and K-Rust. These efforts are surveyed and expanded upon in Alexa White’s master’s degree thesis, “Towards a Complete Formal Semantics of Rust,” a good entry point for understanding these distinct research efforts.

All of these, while not being standards, raise the level of assurance that Rust does what it says on the tin. There are soundness holes in the main Rust compiler, which are tracked and addressed over time. The Rust stability policy leaves an exception for breaking changes which fix soundness holes, as described in RFC 1122.

It’s also worthwhile to note that C was introduced in 1972, and the first official non-draft version of the C standard came out in 1989 (ANSI X3.159-1989 “Programming Language C,” now withdrawn), a full 17 years later. C++ was introduced in 1985, and the first non-draft version of its standard came out in 1998 (ISO/IEC 14882:1998 “Programming Languages — C++”), 13 years later.

Rust’s first public version came out in 2010. It reached version 1.0, after substantial changes in the language from those early versions, on May 15th, 2015. Measuring from the 1.0 date, it’s been 6 years. Standardization takes time, and patience is a virtue.

Reference Downgrade Coercions

Reference downgrade coercions are a very common coercion where &mut T is coerced into &T. Clearly, this coercion is always safe to do, as the immutable reference is less capable than a mutable reference. It also permits the acceptance of some code by the borrow checker which you might naively expect not to compile or work correctly.

struct RefHolder<'a> {
    x: &'a i64,
}

impl<'a> RefHolder<'a> {
    fn new(x: &'a i64) -> RefHolder<'a> {
        RefHolder { x }
    }
}

fn print_num(y: &i64) {
    println!("y: {}", y);
}

fn main() {
    // Create `x`
    let mut x = 10;

    // Make sure `y` is `&mut i64`.
    let y = &mut x;

    // Package the downgraded reference into a struct.
    let z = RefHolder::new(y);
    
    // Print `y` downgrading it to an `&i64`.
    print_num(y);
    
    // Use the `z` reference again.
    println!("z.x: {}", z.x);
}

Code 1

An example of reference downgrade coercions which may surprise.

↺

In this example, we see that the print_num function only requires an &i32, but is being passed an &mut i32. This works fine because the reference is downgraded with a coercion to the immutable reference, which also resolves what would otherwise be an issue with aliasing mutable borrows. The same occurs with the constructor for the RefHolder type.

Note the timing of when this coercion happens. Here’s a similar example that doesn’t compile.

struct RefHolder<'a> {
    x: &'a i64,
}

impl<'a> RefHolder<'a> {
    fn new(x: &'a i64) -> RefHolder<'a> {
        RefHolder { x }
    }
}

fn print_num(y: &i64) {
    println!("y: {}", y);
}

fn main() {
    // Create `x`
    let mut x = 10;

    // Make sure `y` is `&mut i64`.
    let y = &mut x;

    // Package the downgraded reference into a struct.
    //
    //---------------------------------------------------
    // NOTE: this is a _fresh_ reference now, instead of
    //       being `y`.
    //---------------------------------------------------
    let z = RefHolder::new(&mut x);
    
    // Print `y` and update it, downgrading it
    // to `&i64`.
    print_num(y);
    
    // Use the `z` reference again.
    println!("z.x: {}", z.x);
}

Code 2

A similar example which doesn’t work.

↺

In this case, even though the references are downgraded in the function signatures, the borrow checker analysis still observes two mutable references being created in the same scope, which it disallows.

Alert 1 Reference downgrades are often not what you want

As described in pretzelhammer’s excellent “Common Rust Lifetime Misconceptions” article, reference downgrades are often not desirable, and behave in ways which may be surprising.

Deref Coercions

The next kind of coercions are a cornerstone of Rust’s ergonomics. “Deref coercions” are coercions arising from the implementation of two traits: Deref and DerefMut. These exist explicitly for the purpose of opting into these coercions, to provide optional ergonomic improvements for cases where containers should be usable transparently as the type they contain (these types are often called “smart pointers”).

The traits are defined as follows:

pub trait Deref {
    type Target: ?Sized;

    pub fn deref(&self) -> &Self::Target;
}

pub trait DerefMut: Deref {
    pub fn deref_mut(&mut self) -> &mut Self::Target;
}

Code 3

The definitions of the Deref and DerefMut traits from the Rust standard library.

↺

The first trait, Deref, defines a type which can provide a reference to some other “target” type. This target is an associated type, rather than a type parameter, as each “smart pointer” should only ever be dereferenceable to a single other type. If it were defined as Deref<Target> instead, any type could provide as many implementations as they could feasibly provide an inner type for, and the compiler would then need some mechanism to select the right inner type. The point of deref coercions is that they are implicit, so the impact of often requiring more explicit type annotation would conteract the benefit of the deref coercion feature.

The DerefMut trait requires Deref as a supertrait, which both gives it access to the Target associated type, and ensures that the target type for Deref and DerefMut are always the same. Otherwise, you might enable coercion to one type in a mutable context, and a different type in an immutable one. This level of flexibility adds more complexity to deref coercions without clear benefit, and so it isn’t available.

The methods these two traits require, deref and deref_mut, are called implicitly when methods are called on types implementing the traits. So, for example Box<T> implements Deref<Target = T>, so methods for its contained type may be called on it transparently. This makes Box<T> much more ergonomic than if users had to explicitly access its contents for every operation.

However, the presence of deref coercions on a type also leads to potential ambiguity if the containing type also wants to define methods. For this reason, “smart pointers” generally provide their methods as associated functions rather than methods. For example the Box::leak method, which unboxes a value without deallocating it (thus leaving the eventual deallocation up to the user), is written as an associated method fn leak<'a>(b: Box<T, A>) -> &'a mut T where A: 'a, and is therefore called as Box::leak(my_boxed_type) rather than my_boxed_type.leak().

Raw Pointer Coercions

Rust’s raw pointers may be coerced from *mut T to *const T. These conversions are part of safe Rust (i.e. not one of the capabilities reserved for use in unsafe contexts), though the use of those pointers by derefencing is unsafe and subject to Rust’s safety requirements for pointers (namely that accesses never be dangling or unaligned).

#[derive(Debug)]
struct PtrHandle {
    ptr: *const i32,
}

fn main() {
    let mut x = 5;
    let ptr = &mut x as *mut i32;

    // The coercion happens on this line, where
    // a `*mut i32` is set as the value for a field
    // with type `*const i32`, coercing to that type.
    let handle = PtrHandle { ptr };

    println!("{:?}", handle);
}

Code 4

An example raw pointer cast, in the field of a struct.

↺

Info 3 The safety of converting pointers

Rust additional permits explicit as-casting of *const T to *mut T.

While it may seem surprising to permit *const T to be converted to *mut T, there are times when such a conversion is necessary. For example, FFI code may create an *mut T from Box::into_raw, but only want to provide the C consumer of the API with a *const T. The equivalent delete function provided by the FFI interface will therefore need to take *const T as its parameter, converting it back to *mut T to pass it to Box::from_raw, enabling Rust to free the memory when the Box is dropped at the end of the function.

While the details of a pointer’s provenance mean this conversion isn’t always undefined behavior, it may be undefined behavior if the original provenance of the pointer wasn’t a mutable one. Put another way, if a value started as *mut T, it can be used as *mut T in the future, even if the type is changed in the interim into a *const T.

Reference & Raw Pointer Coercions

These are coercions from references to raw pointers. You can go from &T to *const T, and from &mut T to *mut T. These coercions are safe, though the resulting raw pointers may only be dereferenced inside an unsafe block, same as all raw pointers.

// Notice that these coercions work when
// generic types are present too.
#[derive(Debug)]
struct ConstHandle<T> {
    ptr: *const T,
}

#[derive(Debug)]
struct MutHandle<T> {
    ptr: *mut T,
}

fn main() {
    let mut x = 5;

    let c_handle = ConstHandle {
        // Coercing `&i32` into `*const i32`
        ptr: &x,
    };

    let m_handle = MutHandle {
        // Coercing `&mut x` into `*mut i32`
        ptr: &mut x,
    };

    println!("{:?}", c_handle);
    println!("{:?}", m_handle);
}

Code 5

Similar to the last example, but this time converting references to pointers instead of changing the mutability of a pointer type.

↺

Function Pointer Coercions

Closures are functions plus a capture of their environment. This makes them highly useful for many situations, but sometimes the fact that they carry this extra state would impede their use, especially when no state is actually captured. In Rust, in addition to the nameless closure types generated at compile time, there are function pointer types which represent functions without a captured environment. To make closures as flexible as possible, closures coerce to function pointers if and only if they do not capture any variables from their environment.

// This function takes in a function pointer, _not_ a generic type
// which implements one of the function traits (`Fn`, `FnMut`, or
// `FnOnce`).
fn takes_func_ptr(f: fn(i32) -> i32) -> i32 {
    f(5)
}

fn main() {
    let my_func = |n| n + 2;

    // The coercion happens here, and is possible because `my_func`
    // doesn't capture any variables from its environment.
    println!("{}", takes_func_ptr(my_func));
}

Code 6

An example of function pointer coercion.

↺

Note that use of function pointer types in Rust is generally less common than the use of generic types which implement the function traits Fn, FnMut, or FnOnce. If you want to permit the passing or storage of closures which may capture from their environment, then generic types bounded by one of those traits are required.

Subtype Coercions

Aside 3: Polonius and the loan-set formulation of lifetimes ↺

This description, of lifetimes as forming subtyping structures based on an “outlives” relationship, is correct, but underspecifies what “outlives” really means. The current model, based on RFC 2094 (“NLL,” short for “Non-Lexical Lifetimes”) defines lifetimes in terms of a Rust program’s control-flow graph (CFG), as opposed to its abstract syntax tree (AST). This transition, which happened after 1.0, enabled the Rust compiler to accept many more Rust programs, but is not the end. There is a new model, defined in a blog post by Niko Matsakis and implemented in the experimental tool Polonius, which formulates lifetimes as sets of loans with terms which must not be violated.

Surprisingly to some, Rust supports subtype coercion. While Rust’s type system is often thought of as solely supporting parametric polymorphism, it actually supports subtype polymorphism as well, for lifetimes. Lifetimes in Rust form subtyping relationships with each other when one lifetime outlives another. In that case, the longer-lived lifetime is the subtype, and the shorter-lived one is the super type. This is because in subtype polymorphism, any subtype may be substituted in place of the super type, which for lifetimes means that longer-lived lifetimes may be safely used when shorter lifetimes are expected.

This coercion means that lifetimes are permitted to be “shortened” at coercion sites, so a longer lifetime may be used in place of the shorter bound required by the function. The end result of this for Rustaceans is that the compiler accepts more programs.

One question which arises in languages which support parametric and subtype polymorphism, as Rust does, is how generic types’ subtyping relationships relate to the subtyping relationships of their generic parameters. This property is called variance.

There are three useful variances for a generic type to have. Each of these are relative to a specific generic parameter; if a type has multiple generic parameters, it will have a separate variance determination for each of them.

Covariance: for some type A<T>, if T is a subtype of U, A<T> is a subtype of A<U>. The subtyping of the container matches the subtyping of its generic parameter.
Contravariance: for some type A<T>, if T is a subtype of U, A<U> is a subtype of A<T>. The subtyping of the container reverses the subtyping of its generic parameter.
Invariance: for some type A<T>, no subtyping relationship exists between A<T> and any other type A<U>. There is no subtyping for the container.

In Rust, because subtyping only arises for lifetimes, and lifetimes express how long data is valid, these cases mean:

A covariant type permits lifetimes longer than the one it expects (those lifetimes are permitted to “shrink,” which is fine because references can always be used for less time than they’re valid).
A contravariant type permits lifetimes to grow (like making a function pointer taking a reference type less permissive by requiring 'static instead of some lifetime 'a).
An invariant type has no subtype relationship at all, requiring a lifetime which neither shrinks nor grows.

Perhaps an example of contravariance can help explain:

struct FnHolder {
    f: fn(&'static str) -> i32,
}

fn number_for_name<'a>(name: &'a str) -> i32 {
    match name {
        "Jim" => 32,
        _ => 5,
    }
}

fn main() {
    // Voila! A subtype coercion! In this case coercing a
    // lifetime in a contravariant context (the lifetime in
    // the function pointer type parameter) from `'a` to `'static`.
    //
    // `'static` is longer than `'a`, which in this case is safe
    // because it's always fine to make the function _less_ accepting.
    //
    // Once it's been assigned into the `FnHolder` type, it'll only
    // accept string literals (which have a `'static` lifetime).
    let holder = FnHolder { f: number_for_name };
    
    // The extra parentheses are part of the syntax for calling
    // functions as fields, to disambiguate between this and
    // calling a method on the `FnHolder` type.
    println!("{}", (holder.f)("Jim"));
}

Code 7

An example of a contravariant lifetime with a subtype coercion.

↺

Never Coercions

The Rust type system includes a special type, called the “never type” and written !. This type coerces into all other types, and is generally used to represent non-termination. For example, the unimplemented!, unreachable! and todo! macros all return the ! type. The coercion of the ! type is what makes the use of these macros type check, with non-termination in those cases being implemented as a guaranteed panic of the current thread if they are executed at runtime. The std::process::exit function, which exits the current process, also returns ! for the same reason.

// Turn off some warnings about unreachable code.
#![allow(unreachable_code)]
#![allow(unused_variables)]
#![allow(dead_code)]

struct Value {
    x: bool,
    y: String,
}

fn never() -> ! {
    // `loop`s without some way to exit
    // like this have the `!` type, because
    // the expression (and, in this case,
    // the containing function) will never
    // terminate / return.
    loop {}
}

fn main() {
    let x = never();
    
    let v = Value {
        x: todo!("uhhh I haven't gotten to this"),
        y: unimplemented!("oh, not this either"),
    };
    
    // This program compiles because `never`,
    // `todo!`, and `unimplemented!` all return
    // the `!` type, which coerces into any type.
}

Code 8

Never type coercions are what make programs using panic or exit passing type checking.

↺

Slice Coercions

Slice coercions are conversions from an array to a slice. They’re part of a set of coercions (along with trait object coercions and trailing unsized coercions, listed below) called “unsized coercions.” They’re called that because they involve a conversion from a sized type (a type whose size is known at compile time, and which implements the Sized trait) to an unsized type (whose type is not known at compile time, and which does not implement the Sized trait). In the case of slice coercions, the sized type is [T; n] (an array of T with fixed size n), and the unsized type is [T] (a slice of T).

#[derive(Debug)]
struct SliceHolder<'a> {
    slice: &'a [i32],
}

fn main() {
    let nums = [1, 2, 3, 4, 5];
    
    // It may not look like, but there's a coercion here!
    //
    // The type of `&nums` is `&[i32; 5]`, which is coerced
    // into `&[i32]` to match the `slice` field on `SliceHolder`.
    let holder = SliceHolder { slice: &nums };
    
    println!("{:#?}", holder);
}

Code 9

Slice coercions happen more than you may have realized.

↺

Note that while coercing a Vec<T> into a &[T] also works, it’s not a slice coercion, but is rather a deref coercion. Arrays, for historical language reasons having to do with the lack of const generics, don’t implement Deref, and so need a special-cased coercion to convert into slices silently.

Trait Object Coercions

Trait objects are Rust’s mechanism for dynamic dispatch, and the trait object coercion exists to enable easy construction of trait objects. This coercion goes from some type T to dyn U, where U is a trait implemented by T, and where U meets Rust’s object safety rules. We’ve covered the object safety rules before, but the gist is that the object trait type must be constructable (meaning it does not rely in any place upon generic types which are undecideable at compile time, does not include associated functions, does not reference Self in ways which can’t be decided at compile time, and does not include functions taking Self by value without a Self: Sized bound included).

trait HasInt {
    fn get(&self) -> i32;
}

struct IntHolder {
    x: i32,
}

impl HasInt for IntHolder {
    fn get(&self) -> i32 {
        self.x
    }
}

fn print_int(x: &dyn HasInt) {
    println!("{}", x.get());
}

fn main() {
    let holder = IntHolder { x: 5 };
    // The coercion happens here, from `&IntHolder`
    // into `&dyn HasInt`.
    print_int(&holder);
}

Code 10

A function called with a trait object coercion.

↺

Trailing Unsized Coercions

The trailing unsized coercion means that, if a type T’s last field is sized and able to coerced to an unsized type, and there exists some type U which is T but with that last-field coercion performed, then T can be coerced into U. To get more specific, because the definition of this one is quite particular:

T has to be a struct.
The field of T, let’s call it A, has to be unsized-coercable to another type B.
The last field of T has to include A.
No other field can include A.
If the last field is itself a struct containing A, that struct has to be unsized-coercible to another type containing B in place of A.

That’s a mouthful, but it is more precise than the initial explanation. In essence, this permits a limited case of unsized coercions within structs when the relevant field is the final field.

Least Upper Bound Coercions

Sometimes Rust needs to perform coercions at multiple sites at once, such that they all work out to the same type. This can happen, for example, in an if/else expression, where each branch of the conditional is returning a type which needs to be coerced. In this case, Rust tries to find the most general type which works, and this is called the “least upper bound coercion.”

This coercion can be triggered by:

A series of if/else branches.
A series of match arms.
A series of array elements.
A series of returns in a closure.
A series of returns in a function.

The process of performing this coercion is to iterate through each of the types in the series, check if they can be coerced to the same type as previously determined. If they can, move on, if not, try to figure out a type which both the prior seen types and the latest type can be coerced to. The final type is determined to be the type of all expressions in the series.

Transitive Coercions

Rust supports transitive coercions, where if type A coerces to type B, and B coerces to C, then A can coerce to C. These are currently a best-effort feature, and may not always work.

Where can coercions happen?

The places where coercions can happen are called coercion sites, and there are several of them in Rust.

Coercion Sites

First are variable declarations, whether done with let, const, or static. In these cases, if type ascription is used to annotate a type on the left hand side of the declaration, then the right hand side will be coerced to that type. If such a coercion isn’t possible, a compiler error will be issued.

Next are function parameters, where the actual parameter (the thing actually passed in to the function) is coerced into the type of the formal parameter (the internal name of the parameter in the function signature). In method calls, the receiver type (the type of Self) is only able to use the unsized coercions.

Then you have the literal instantiation of any struct or enum. The sites where fields within these data types are instantiated are coercion sites, with the actual type being coerced into the formal type defined in the definition of the overall data type.

Coercion-propagating Expressions

Some expressions are considered “coercion propagating,” meaning they pass along the coercion checking to their sub-expressions.

Array literals are coercion propagating, and propogate to each element definition in an array literal declaration. If used with repeating syntax, than the initial definition of the elements, which will be repeated the given number of times, is coercion propagating.

Tuples are similarly coercion propagating at each individual expression site within them.

If an expression is parenthesized, coercion is propagated to the expression inside the parentheses. If it’s bracketed, making it a block, then coercions are propagated to the last line of the block.

Unsized Coercions and Coercion Sites

Unsized coercions (coercions for slices, trait objects, or trailing unsized types as described above) can happen in one additional context compared to other coercions. Specifically, it you have a reference, raw pointer, or owned pointer to a type T, where T has an unsized coercion to a type U, then the coercion can occur through the reference or pointer type.

This means the following coercion sites are valid only for unsized coercions:

&T into &U
&mut T into &mut U
*const T into *const U
*mut T into *mut U
Box<T> into Box<U>

This is actually why the example for slice coercions above worked! The coercion in that case occured behind a reference, converting [i32; 5] into [i32].

Conclusion

Coercions are powerful, and because they are silent, sometimes controversial.

Whatever your view on the proper use of coercions is, it’s important to understand what coercions are possible, and where they may occur. In this post we named and described all possible coercions in Rust, and described what sorts of expressions may include coercions, and what expressions may propagated coercions. Hopefully this helps make this often-hidden part of Rust a little bit clearer.

Possible Rust

Possible Rust

Categories

Tags