Rust API

Modules

The Rust execution events API is split across two library packages:

1. monad-event-ring - this package provides the core event ring functionality. Recall that event rings are a generic broadcast utility based on shared memory communication, and are agnostic about what kind of event data they contain. Consequently, this package does not include the definitions of the execution event types (nor any other event types)

2. monad-exec-events - the execution event data types are defined in this library, along with some helpful utilities for writing real-time data applications

These libraries fit together in a more structured way than they do in C.

In the C API, the event ring API works with unstructured data, e.g., event numerical codes are uint16_t values and event payloads are raw byte arrays. The reader performs unchecked type casts to reinterpret the meaning of those bytes. There are some safety mechanisms to check if an event ring file appears to contain the right kind of data, but the content types are not strongly represented in the type system.

In the Rust API, the event ring is not just "generic" in the general sense of the word; it is a literal generic type:

struct EventRing<D: EventDecoder>

Event rings are explicitly parameterized by a "decoder". The decoder knows how to interpret the raw bytes for a particular event content type, e.g., execution events.

Core concepts

Event enumeration type

Consider how decoding works in the C API: it's typically a "giant switch statement" pattern, where we examine the numerical code of an event and reinterpret the raw bytes as the appropriate payload type via an unchecked type cast:

const void *payload = monad_event_ring_payload_peek(&exec_ring, &event);

switch (event.event_type) {
case BLOCK_START:
    handle_block_start((const struct monad_exec_block_start *)payload);
    break;

case BLOCK_END:
    handle_block_end((const struct monad_exec_block_end *)payload);
    break;

// ... more event types handled here
}

The Rust way of expressing this is to use an enum type: the different kinds of event payloads become the enum's variants, and the switch logic is replaced by a more-powerful match.

In Rust, decoding produces a value of enumeration type ExecEvent, which is defined like this:

#[derive(Clone, Debug)]
pub enum ExecEvent {
    BlockStart(monad_exec_block_start),
    BlockReject(monad_exec_block_reject),
    BlockEnd(monad_exec_block_end),
    // more variants follow

Notice that each variant of ExecEvent holds a value whose type name resembles the C event payload structures. For example, struct monad_exec_block_start is the event payload structure definition in the C API. It's recorded when a new block starts, and is defined in the file exec_event_ctypes.h.

The use of these exact same C structure names -- including the monad_exec prefix and lower-case, snake-case spelling -- is designed to alert you to the fact that the payload types have exactly the same in-memory representation as their C API counterparts. They are generated by bindgen and are layout-compatible (via a #[repr(C)] attribute) with the C types of the same names.

Event rings and the 'ring reference lifetime

EventRing is an RAII-handle type: when you create an EventRing instance, new shared memory mapping are added to your process for that event ring file. Likewise, when EventRing::drop is called, those shared memory mappings are removed. Any pointers or references pointing into shared memory would need to be invalidated at that point.

We rely on Rust's builtin reference lifetime analysis framework to express this. References to data that lives in event ring shared memory always carries a reference lifetime called 'ring. This lifetime corresponds to the lifetime of the EventRing object itself. Since an EventRing pins the shared memory mappings in place by being alive, the true meaning of 'ring can usually be thought of as the "shared memory lifetime", which is the same.

Zero-copy APIs and the "event reference" enumeration type

In a previous section, we discussed the decoded execution event type, enum ExecEvent. There is a second type with a similar design called enum ExecEventRef<'ring>; it is used for the zero copy API.

To compare the two, here is the ExecEvent type:

#[derive(Clone, Debug)]
pub enum ExecEvent {
    BlockStart(monad_exec_block_start),
    BlockReject(monad_exec_block_reject),
    BlockEnd(monad_exec_block_end),
    // more variants follow

And here is the ExecEventRef<'ring> type:

#[derive(Clone, Debug)]
pub enum ExecEventRef<'ring> {
    BlockStart(&'ring monad_exec_block_start),
    BlockReject(&'ring monad_exec_block_reject),
    BlockEnd(&'ring monad_exec_block_end),
    // more variants follow

The former contains copies of event payloads, whereas the latter directly references the bytes living in the shared memory payload buffer. By working with ExecEventRef<'ring>, you avoid avoid copying a potentially large amount of data, e.g., especially large EVM logs or call frames. This is valuable if you are filtering out most events anyway.

The "event reference" enum type offers better performance, but it comes with two drawbacks:

1. Because it has a reference lifetime as a generic parameter, it can be more difficult to work with (i.e., more running afoul of the borrow checker)

2. Data that lives directly in the payload buffer can be overwritten at any time, so you shouldn't rely on it still being there long after you first look at it

Copying vs. zero-copy payload APIs

The copy vs. zero-copy decision only applies to event payloads; event descriptors are small, and are always copied. There are two ways to read an event's payload once you have its descriptor:

1. Copying style EventDescriptor::try_read - this will return an EventPayloadResult enum type, which either contains the "success" variant (EventPayloadResult::Ready) or the "failure" variant (EventPayloadResult::Expired); the former contains a ExecEvent payload value, and the latter indicates that the payload was lost

2. Zero-copy style EventDescriptor::try_filter_map - you pass a non-capturing closure to this method, and it is called back with an ExecEventRef<'ring> reference pointing to the event payload in shared memory; since your closure can't capture anything, the only way for you to react to the event payload is to return some value v of type T; EventDescriptor::try_filter_map itself returns an Option<T>, which is used in the following way:

   • If the payload has expired prior to calling your closure, then your closure is never called, and the try_filter_map returns Option::None

   • Otherwise your closure is run and its return value v: T is moved into the try_filter_map function

   • If the payload is still valid after your closure has run, then the value is transferred to the caller by returning Option::Some(v), otherwise Option::None is returned

Why non-capturing closures?

The pattern of zero-copy APIs generally works like this:

• Create a reference to the data in the event ring payload buffer (e: &'ring E) and check for expiration; if not expired ...

• ... compute something based on the event payload value, i.e., compute let v = f(&e)

• Once f finishes, check again if the payload expired; if it is expired now, then it may have become expired sometime during the computation of v = f(&e); the only safe thing we can do is discard the computed value v, since we have no way of knowing exactly when the expiration happened

If you were permitted to capture variables in the zero-copy closure, you could "smuggle out" computations out-of-band from the library's payload expiration detection checks. That is, if the library later detects that the payload was overwritten sometime during when your closure was running, it would have no guaranteed way to "poison" your smuggled out value. It could only advise you not to trust it, but that is error prone.

Idiomatic Rust tends to follow a "correct by default" style, and guards against these kinds of unsafe patterns. In the zero-copy API, you can communicate only through return values since you cannot capture anything. This way, the library can decide not to propagate the return value back to you at all, if it later discovers that the payload it gave you as input has expired.

Important types in the Rust API

There are six core types in the API:

1. Event ring EventRing<D: EventDecoder> - given a path to an event ring file, you create one of these to gain access to the shared memory segments of the event ring in that file; you typically use the type alias ExecEventRing, which is syntactic sugar for EventRing<ExecEventDecoder>

2. Event reader EventReader<'ring, D: EventDecoder> - this is the iterator-like type that is used to read events; it's called a "reader" rather than an "iterator" because Iterator already has a specific meaning in Rust; the event reader has a more complex return type than a Rust iterator because it has a "polling" style: its equivalent of next() -- called next_descriptor() -- can return an event descriptor, report a gap, or indicate that no new event is ready yet

3. Event descriptor EventDescriptor<'ring, D: EventDecoder> - the event reader produces one of these if the next event is read successfully; recall that the event descriptor contains the common fields of the event, and stores the necessary data to read the event payload and check if it's expired; in the Rust API, reading payloads is done using methods defined on the event descriptor

4. Event decoder trait EventDecoder - you don't use this directly, but a type that implements this trait -- ExecEventDecoder in the case of an execution event ring -- contains all the logic for how to decode event payloads

5. Event enumeration types (associated types EventDecoder::Event and EventDecoder::EventRef) - these give the "copy" and "zero-copy" decoded forms of events; in the case of the ExecEventDecoder, ExecEvent is the "copy" type and ExecEventRef<'ring> is the zero-copy (shared-memory reference) type

6. Execution event payload types (monad_exec_block_start, and others) - these are bindgen-generated, #[repr(C)] event payload types that match their C API counterparts

Block-level utilities

ExecutedBlockBuilder

Execution events are granular: most actions taken by the EVM will publish a single event describing that one action, e.g., every EVM log emitted is published as as a separate ExecEvent::TxnLog event. The events are streamed to consumers almost as soon as they are available, so the real-time data of a block comes in "a piece at a time."

A utility called the ExecutedBlockBuilder will aggregate these events back into a single, block-oriented update, if the user prefers working with complete blocks. The data types in the block representation are also alloy_primitives types which are more ergonomic to work with in Rust.

CommitStateBlockBuilder

As explained in the section on speculative real-time data, the EVM publishes execution events as soon as it is able to, which means it is usually publishing data about blocks that are speculatively executed. We do not know if these blocks will be appended to the blockchain or not, since the consensus decision is occurring in parallel with (and will finish later than) the block's execution.

CommitStateBlockBuilder builds on the ExecutedBlockBuilder by also tracking the commit state of the block as it moves through the consensus life cycle. The block update itself is passed around via an Arc<ExecutedBlock>, so that it is cheap to copy references to it. As the block commit state changes, you receive updates describing the new state, along with another reference to the Arc<ExecutedBlock> itself.

The speculative real-time data guide often points out that block abandonment is not explicitly communicated by the event system (e.g., here and here). The CommitStateBlockBuilder however, does report explicit abandonment of failed proposals, because it is a higher level, user-friendly utility.