Build with Naz : Box and Pin exploration in Rust

• Introduction
• Why do we need both Box and Pin?
• Formatting pointers
• What is a smart pointer?
• YouTube video for this article
• Examples Rust Box smart pointer, and Pin
  • Example 1: Getting the address of variables on the stack and heap
  • Example 2: What does Box move do?
  • Example 3: How do we swap the contents of two boxes?
  • Example 4: What does pining a box do?
• Build with Naz video series on developerlife.com YouTube channel

Introduction #

This tutorial, video, and repo are a deep dive into Rust Pin and Box types, along with concepts of ownership and borrowing. We will also cover a lot of background information on the concepts of operating system process, memory allocation and access, stack, and heap. The examples we create are designed to demonstrate the different semantics around the use of boxes and pinned boxes in Rust.

Why do we need both Box and Pin? #

It is common to use Pin for tokio::select! macro branches in Rust async code. And Box is used commonly for trait pointers.

This article, video, and repo illustrate the concepts (moving a box, swapping box contents, and pinning a box) by example. Lots of pretty formatted output is generated so that you can run tests and see what’s happening (and make sense of it).

Formatting pointers #

To format pointers in Rust, we can use the formatting trait {:p}. You can format a pointer by using two approaches:

1. Get the address of the pointer using [std::ptr::addr_of!] and then format it using {:p}. Eg: let x = 1; println!("{:p}", std::ptr::addr_of!(x));
2. Get a reference to the pointer using & and then format it using {:p}. Eg: let x = 1; println!("{:p}", &x);

What is a smart pointer? #

Smart pointers in Rust are data structures that act like pointers but also have additional metadata and capabilities. They provide a level of abstraction over raw pointers, offering features like ownership management, reference counting, and more. Smart pointers often manage ownership of the data they point to, ensuring proper deallocation when no longer needed.

For a great visualization of memory allocation, stack and heap please read this article.

YouTube video for this article #

This blog post has examples from this live coding video. If you like to learn via video, please watch the companion video on the developerlife.com YouTube channel.

Examples Rust Box smart pointer, and Pin #

Let’s create some examples to illustrate how Box and Pin and pointers to stack allocations and heap allocations work in Rust. You can run cargo new --lib box-and-pin to create a new library crate.

💡 You can get the code from the rust-scratch repo.

Then add the following to the Cargo.toml file that’s generated. These pull in all the dependencies that we need for these examples.

[package]
name = "box-and-pin"
version = "0.1.0"
edition = "2021"

[dependencies]
crossterm = { version = "0.27.0", features = ["event-stream"] }
serial_test = "3.1.1"

Here are the dependencies we are using:

1. The serial_test dep allows us to run Rust tests serially, so that we can examine the output of each test, without it being clobbered by other test output running in parallel.
2. The crossterm dep allows us to generate colorful println! output in the terminal which will help us visualize what is going on with the pointers and memory allocations.

We are going to add all the examples below as tests to the lib.rs file in this crate.

Example 1: Getting the address of variables on the stack and heap #

Let’s add the following imports and macros to the top of the lib.rs file. These will help us print output from the tests, so that we can track where a pointer is located in memory and what the size of the thing it points to is. There are two macros, one for a reference or pointer, and another one for pinned pointers. And we have an assertion function that can return true if all 3 arguments are equal.

use crossterm::style::Stylize;
use serial_test::serial;

/// Given a pointer `$p`, it prints:
/// 1. it's address,
/// 2. and size of the thing it points to (in bytes).
macro_rules! print_ptr_addr_size {
    ($p: expr) => {
        format!("{:p}┆{}b", $p, std::mem::size_of_val($p))
    };
}

/// Given a pinned pointer `$p`, it prints:
/// 1. it's address,
/// 2. and size of the thing it points to (in bytes).
macro_rules! print_pin_addr_size {
    ($p: expr) => {
        format!("{:p}┆{}b", $p, std::mem::size_of_val(&(*$p)))
    };
}

fn assert_three_equal<T: PartialEq + std::fmt::Debug>(a: &T, b: &T, c: &T) {
    assert_eq!(a, b, "a and b are not equal");
    assert_eq!(a, c, "a and c are not equal");
}

So, before we start with the examples, let’s add a test that demonstrates how to get the address of a variable on the stack and heap. Add the following code to your lib.rs file.

#[test]
#[serial]
fn print_ptr_addr_size() {
    // Using `std::ptr::addr_of!` to get the memory address of a variable.
    let x = 100u8;
    let x_addr = std::ptr::addr_of!(x);
    println!(
        "x: {}, x_addr  : {}",
        x.to_string().blue().underlined(),
        format!("{:?}", x_addr).red().italic(),
    );

    // Using `format!` to get the memory address of a variable.
    let x_addr_2 = format!("{:p}", &x);
    println!(
        "x: {}, x_addr_2: {}",
        x.to_string().blue().underlined(),
        x_addr_2.red().italic().on_black(),
    );

    // Get size of `x` in bytes.
    let x_size = std::mem::size_of_val(&x);
    println!(
        "x: {}, x_size  : {}b",
        x.to_string().blue().underlined(),
        x_size.to_string().magenta().italic().on_black(),
    );

    // Using `print_ptr_addr_size!` to get the memory address of a variable.
    let x_addr_3 = print_ptr_addr_size!(&x);
    println!(
        "x: {}, x_addr_3: {}",
        x.to_string().blue().underlined(),
        x_addr_3.red().italic().on_black(),
    );
}

Here’s the output of the test above, after you run cargo watch -x "test --lib -- --show-output print".

---- print_ptr_addr_size stdout ----
x: 100, x_addr  : 0x7e17cd9feb97
x: 100, x_addr_2: 0x7e17cd9feb97
x: 100, x_size  : 1b
x: 100, x_addr_3: 0x7e17cd9feb97┆1b

Let’s walk through the output above:

1. We have a variable x that is a u8 with a value of 100. This is a stack allocation. It occupies 1 byte of memory (its size).
2. We get the address of x using std::ptr::addr_of!(x) and format!("{:p}", &x).
3. We get the size of x in bytes using std::mem::size_of_val(&x). The size is 1 byte.
4. We get the address of x and the size of x using the print_ptr_addr_size! macro.

Example 2: What does Box move do? #

Add the following snippet to the lib.rs file next. This link provids lots of great diagrams on how stack and heap memory works in an operating system.

/// <https://courses.grainger.illinois.edu/cs225/fa2022/resources/stack-heap/>
#[test]
#[serial]
fn move_a_box() {
    let b_1 = Box::new(255u8);
    let b_1_addr = print_ptr_addr_size!(b_1.as_ref()); // Pointee (heap)
    let b_1_ptr_addr = print_ptr_addr_size!(&b_1); // Pointer (stack)

    println!(
        "1. {}: {}, {} (pointee, heap): {}, {} (ptr, stack): {}",
        "b_1".green(),
        b_1.to_string().blue().underlined(),
        "b_1_addr".green(),
        b_1_addr.clone().magenta().italic().on_black(),
        "b_1_ptr_addr".green(),
        b_1_ptr_addr.clone().magenta().italic().on_black(),
    );

    let b_2 = b_1;
    // println!("{b_1:p}"); // ⛔ error: use of moved value: `b_1`
    let b_2_addr = print_ptr_addr_size!(b_2.as_ref()); // Pointee (heap)
    let b_2_ptr_addr = print_ptr_addr_size!(&b_2); // Pointer (stack)

    println!(
        "2. {}: {}, {} (pointee, heap): {}, {} (ptr, stack): {}",
        "b_2".green(),
        b_2.to_string().blue().underlined(),
        "b_2_addr".green(),
        b_2_addr.clone().magenta().italic().on_black(),
        "b_2_ptr_addr".green(),
        b_2_ptr_addr.clone().magenta().italic().on_black(),
    );

    // The heap memory allocation does not change (does not move). Pointee does not move.
    assert_eq!(b_1_addr, b_2_addr);

    // The stack memory allocation does change (does move). Boxes aka pointers have move.
    assert_ne!(b_1_ptr_addr, b_2_ptr_addr);

    // When b_2 is dropped, the heap allocation is deallocated. This is why Box is a smart pointer.
}

Let’s walk through the output above:

1. We have a Box b_1 that points to a heap allocation of a u8 with a value of 255. b_1 is a variable on the stack that points to a heap allocation. We get the address of the pointee and the pointer using the print_ptr_addr_size! macro with b_1.as_ref(). And we get the address of the pointer by passing &b_1 to print_ptr_addr_size!.
2. We move b_1 into b_2. The heap memory allocation does not change (does not move). The pointee does not move. But the stack memory allocation does change (does move). Boxes aka pointers have moved. The b_1 variable gets dropped. We can get the address of the pointee using print_ptr_addr_size! macro with b_2.as_ref(). We can get the address of the pointer using print_ptr_addr_size! macro with &b_2.
3. In the assertions, we check that the heap memory allocation does not change (does not move). And we check that the stack memory allocation does change (does move).

Example 3: How do we swap the contents of two boxes? #

Add the following snippet to the lib.rs file next.

#[test]
#[serial]
fn swap_box_contents() {
    let mut b_1 = Box::new(100u8);
    let mut b_2 = Box::new(200u8);

    let og_b_1_addr = print_ptr_addr_size!(b_1.as_ref());
    let og_b_2_addr = print_ptr_addr_size!(b_2.as_ref());

    assert_eq!(*b_1, 100u8);
    assert_eq!(*b_2, 200u8);

    println!(
        "1. {}: {}, {} (pointee, heap): {}, {} (ptr, stack): {}",
        "b_1".green(),
        b_1.to_string().blue().underlined(),
        "b_1_addr".green(),
        og_b_1_addr.clone().red().italic().on_black(),
        "b_1_ptr_addr".green(),
        print_ptr_addr_size!(&b_1)
            .clone()
            .magenta()
            .italic()
            .on_black(),
    );
    println!(
        "2. {}: {}, {} (pointee, heap): {}, {} (ptr, stack): {}",
        "b_2".green(),
        b_2.to_string().blue().underlined(),
        "b_2_addr".green(),
        og_b_2_addr.clone().magenta().italic().on_black(),
        "b_2_ptr_addr".green(),
        print_ptr_addr_size!(&b_2)
            .clone()
            .cyan()
            .italic()
            .on_black(),
    );

    std::mem::swap(&mut b_1, &mut b_2);
    println!("{}", "Swapped b_1 and b_2".cyan().underlined());

    let new_b_1_addr = print_ptr_addr_size!(b_1.as_ref());
    let new_b_2_addr = print_ptr_addr_size!(b_2.as_ref());

    assert_eq!(*b_1, 200u8);
    assert_eq!(*b_2, 100u8);

    assert_eq!(og_b_1_addr, new_b_2_addr);
    assert_eq!(og_b_2_addr, new_b_1_addr);

    println!(
        "3. {}: {}, {} (pointee, heap): {}, {} (ptr, stack): {}",
        "b_1".green(),
        b_1.to_string().blue().underlined(),
        "b_1_addr".green(),
        new_b_1_addr.clone().magenta().italic().on_black(),
        "b_1_ptr_addr".green(),
        print_ptr_addr_size!(&b_1)
            .clone()
            .magenta()
            .italic()
            .on_black(),
    );
    println!(
        "4. {}: {}, {} (pointee, heap): {}, {} (ptr, stack): {}",
        "b_2".green(),
        b_2.to_string().blue().underlined(),
        "b_2_addr".green(),
        new_b_2_addr.clone().red().italic().on_black(),
        "b_2_ptr_addr".green(),
        print_ptr_addr_size!(&b_2)
            .clone()
            .cyan()
            .italic()
            .on_black(),
    );
}

Here’s the output of the test above, after you run cargo watch -x "test --lib -- --show-output swap".

---- swap_box_contents stdout ----
1. b_1: 100, b_1_addr (pointee, heap): 0x722b38000d10┆1b, b_1_ptr_addr (ptr, stack): 0x722b3cbfdad0┆8b
2. b_2: 200, b_2_addr (pointee, heap): 0x722b38001f30┆1b, b_2_ptr_addr (ptr, stack): 0x722b3cbfdad8┆8b
Swapped b_1 and b_2
3. b_1: 200, b_1_addr (pointee, heap): 0x722b38001f30┆1b, b_1_ptr_addr (ptr, stack): 0x722b3cbfdad0┆8b
4. b_2: 100, b_2_addr (pointee, heap): 0x722b38000d10┆1b, b_2_ptr_addr (ptr, stack): 0x722b3cbfdad8┆8b

Let’s walk through the output above:

1. We have two Boxes b_1 and b_2 that point to heap allocations of u8 with values 100 and 200 respectively. We get the address of the pointees using the print_ptr_addr_size! macro with b_1.as_ref() and b_2.as_ref(). We get the address of the pointers using the print_ptr_addr_size! macro with &b_1 and &b_2.
2. We swap the contents of b_1 and b_2 using std::mem::swap(&mut b_1, &mut b_2). The values of b_1 and b_2 are now 200 and 100 respectively.
3. We get the new addresses of the pointees using the print_ptr_addr_size! macro with b_1.as_ref() and b_2.as_ref(). We get the new addresses of the pointers using the print_ptr_addr_size! macro with &b_1 and &b_2.
4. In the assertions, we check that the values of b_1 and b_2 are 200 and 100 respectively. We check that the addresses of the pointees have swapped. And we check that the addresses of the pointers have not swapped.

Example 4: What does pining a box do? #

Add the following code to your lib.rs file.

fn box_and_pin_dynamic_duo() {
    let b_1 = Box::new(100u8);
    // Pointee.
    let b_1_addr = print_ptr_addr_size!(b_1.as_ref());

    let p_b_1 = std::boxed::Box::<u8>::into_pin(b_1);
    // Pinned.
    let p_b_1_addr = print_pin_addr_size!(p_b_1);

    let b_2 = p_b_1;
    // println!("{}", p_b_1); // ⛔ error: use of moved value: `p_b_1`

    // Pin does not move.
    let b_2_addr = print_pin_addr_size!(b_2);

    // Pointee has not moved!
    assert_eq!(b_1_addr, b_2_addr);

    // Pointer has not moved!
    assert_three_equal(&b_1_addr, &p_b_1_addr, &b_2_addr);
}

When you run the command cargo watch -x "test --lib -- --show-output dynamic" it doesn’t really produce any output.

Let’s walk through the code above:

1. We have a Box b_1 that points to a heap allocation of a u8 with a value of 100. We get the address of the pointee using the print_ptr_addr_size! macro with b_1.as_ref().
2. We pin b_1 into p_b_1 using std::boxed::Box::<u8>::into_pin(b_1). The pointee does not move. We get the address of the pinned pointer using the print_pin_addr_size! macro with p_b_1.
3. We move p_b_1 into b_2. The pin does not move. We get the address of the pinned pointer using the print_pin_addr_size! macro with b_2.
4. In the assertions, we check that the pointee has not moved. And we check that the pointer has not moved.

Build with Naz video series on developerlife.com YouTube channel #

If you have comments and feedback on this content, or would like to request new content (articles & videos) on developerlife.com, please join our discord server.

You can watch a video series on building this crate with Naz on the developerlife.com YouTube channel.

• YT channel
• Playlists
  • Build with Naz, Linux TTY, process, signals, commands in async Rust
  • Build with Naz, fundamental effective Rust
  • Build with Naz, effective async Rust and tokio
  • Build with Naz, async readline and spinner for CLI in Rust
  • Build with Naz, testing in Rust

#cli #rust #server #tui

👀 Watch Rust 🦀 live coding videos on our YouTube Channel.



📦 Install our useful Rust command line apps using cargo install r3bl-cmdr (they are from the r3bl-open-core project):

• 🐱giti: run interactive git commands with confidence in your terminal
• 🦜edi: edit Markdown with style in your terminal

giti in action

edi in action

Related Posts

• Build with Naz : Rust lifetimes
• Build with Naz : Explore Linux TTY, process, signals w/ Rust
• Build with Naz : Rust async in practice tokio::select!, actor pattern & cancel safety
• Build with Naz : Markdown parser in Rust and nom from r3bl_tui
• Build with Naz : Rust error handling with miette
• Build with Naz : Rust typestate pattern
• Build with Naz : Linux io_uring and tokio-uring exploration with Rust
• Build with Naz : Rust async, non-blocking, concurrent, parallel, event loops, graceful shutdown
• Build with Naz : tokio tracing & OTel and how to use it in Rust
• Build with Naz : Rust Polymorphism, dyn, impl, using existing traits, trait objects for testing and extensibility
• Build with Naz : Build interactive and non blocking CLI apps with ease in Rust using r3bl_terminal_async
• Write a simple netcat client and server in Rust
• Write a simple TCP chat server in Rust
• How to overcome your fear of git merge conflicts
• tuify your clap CLI apps and make them more interactive
• Use just to manage project specific commands
• Build with Naz : Comprehensive guide to nom parsing
• Create a simple DSL for CSS like syntax for TUIs
• Write a Redux library in Rust
• Write code using async/await in Rust
• Build a grep CLI app in Rust
• Build a non-binary tree that is thread safe using Rust
• Reference handbook for using Ink v3.2.0 components (w/ React, Node.js and TypeScript)
• Advanced guide to Ink v3.2.0 (w/ React, Node.js and TypeScript)
• Introduction to Ink v3.2.0 (w/ React, Node.js and TypeScript)
• Build with Naz : Ubuntu 24.04 setup and config for dev productivity
• Guide to Rust procedural macros
• Node.js (v16.3.0) Handbook using TypeScript (v4.3.4)
• Experimenting w/ Fibers in Project Loom preview
• Creating asynchronous servlets with Tomcat 7 (Servlet 3.0 API)
• Using JSON for mobile object exchange
• Geocoding tutorial - Accessing Google Static Maps from Java
• What is XML? An introduction
• Advanced Threads
• XML and Java Tutorial, Part 1