Allow passing pre-allocated buffer to proxy_get_buffer_bytes

[leonm1]

We've seen a common pattern emerging where plugin authors will call proxy_get_buffer_bytes with a specific length in mind, do something with the buffer, then clear all buffers when returning from the callback.

By allowing authors to pass a pre-allocated buffer, we'd permit plugin authors to stack-allocate this memory, which is cheaper to manage than heap allocating. In the case of garbage collected languages, such as Go, this would allow us to completely bypass the garbage collector.

Furthermore, this approach can provide plugin authors significantly more control over the memory usage when using garbage collected languages, since the contents of the buffer are guaranteed to be freed at the end of the callback (and individual callbacks in a thread local vm are guaranteed non-concurrent).

[leonm1]

FWIW this is currently under-specified in the spec (and is a non-breaking change), so this may be permissible to change without a major version bump.

[PiotrSikora]

This is already possible, proxy_on_memory_allocate can return whatever memory block it wants.

It's not the cleanest API, but you can do either of those right now:

• always return the same statically allocated memory block:

fn proxy_on_memory_allocate(size: usize) -> *mut u8 {
    STATIC_MEMORY.as_mut_ptr();
}

• set memory block to return at the caller:

fn proxy_on_memory_allocate(size: usize) -> *mut u8 {
    MEMORY_TO_RETURN.as_mut_ptr();
}

impl HttpContext for HttpAuthRandom {
    fn on_http_request_headers(&mut self, _: usize, _: bool) -> Action {
        let mut foo = Vec::with_capacity(16 * 1024);
        MEMORY_TO_RETURN = foo;
        for (name, value) in &self.get_http_request_headers() {
            info!("#{} -> {}: {}", self.context_id, name, value);
        }
        Action::Continue
    }
}

(the code might not compile)

This works since the memory ownership is always immediately passed to the guest and never retained by the host (the pattern is: host requests memory from the guest, host copies some data to that memory, host returns that memory to the guest) and it never allocates multiple times within the same callback).

Having said that, since we're changing all the signatures as part of v0.3, I'm not against adding explicit memory pointer and size to the function calls, and "fallback" to calling proxy_on_memory_allocate only when those are empty.

Note that this would affect pretty much all functions that copy data, not only proxy_get_buffer_bytes.

What do you think?

[PiotrSikora]

@mpwarres @leonm1 do we want to address this as part of v0.3.0?

I'm not a fan of adding this only to proxy_get_buffer_bytes, but if we want to make this (i.e. allowing passing pre-allocated buffer) a generic feature of the ABI and available in other retrieval functions (getters for KV store, queues, properties, etc.), then we need to add functions to retrieve size of each possible returns, which also isn't great and would result in pretty ugly code and increased number of calls across the ABI boundary.

We could argue that buffers are special, since the complete buffer doesn't have to be retrieved at once and it can be streamed and/or partially processed in most cases, so perhaps there is justification for adding this only to proxy_get_buffer_bytes?

[leonm1]

I think we can leave this out of v0.3. If desired, it can be a feature addition after the fact.

[mpwarres]

+1 to handling this as a feature addition later. I do think the primary need is with buffers, so the addition could be specific to proxy_get_buffer_bytes.