Safer (and Easier) Rust Interface

The current Rust interface has a major problem because it requires changes to the Rust code when device assignments are changed. This makes it harder to experiment with schedules (which is the whole point of schedules) and would especially be an issue when trying to automatically assign devices since the user's Rust code must be aware of the decisions by the compiler. This issue is also supported by the large amount of code that must currently be wrapped in #[cfg(feature = "cuda")] or #[cfg(not(feature = "cuda"))] to get our CI to be able to test both CPU and GPU.

The interface is also quite unsafe, in particular to retrieve memory from the GPU involves unsafe code and the user computing the size of the returned memory (despite Hercules knowing it). Also, even for CPU memory when we retrieve the memory as a slice, the user specifies the type of the elements (which again Hercules knows).

I would suggest a few changes:

1. Memory being passed to Hercules programs should either always be wrapped in the same manner (like the original HerculesBox idea) or should be able to be coerced in the same manner from the appropriate Rust type into the appropriate Hercules type (for example by implementing From<&[T]> for all Ref types, so that we could just use .into() regardless of target device).
2. Memory returned from Hercules programs should all satisfy a standard interface, in particular a method to retrieve the results as a slice (or a boxed slice).
3. The types we use for (1) and (2) should be type/memory-safe. In particular, we should not have to compute the size of any memory (like we're doing for GPU now) nor should we have to specify the type (like the as_slice method for HerculesCPURef does).

I agree we need some representation of memory that is under Hercule's management; this is something HPVM does not handle well and is really important to allow the reuse of memory across executions, but the device specific approach we currently have is really not ideal.

changed the description

I definitely agree with this, the current API is a big pain and error prone (my bad)! I think we've arrived on a good memory management strategy though, I think the concept of a runner object owning the backing memory for a function is the right model. Maybe we have the current API as a sys unsafe API, and we create a safe, not annoying, API on top of it that doesn't suck (this is the approach taken by a decent amount of Rust crates). The one thing to be cognizant of is that implementing something like HerculesBox will require unnecessary memory allocations, but maybe that's just the price of using the safe API in the host Rust code.

Yeah, that's probably a good approach. I agree that the memory model we have now is good, so I think building on what we have is the best approach.

I have some ideas on how to start, may work on that tomorrow morning and start a draft MR for discussion.

Yeah, I suppose another advantage to that approach is we can iterate on it super quickly, since it's just writing Rust in hercules_rt.

mentioned in merge request !149 (merged)

mentioned in commit b3a5a0bc

closed with merge request !149 (merged)