Fork tiling non-divisible sizes

I've made enough progress on cava to get it running on the GPU (yay!), and the GPU backend is actually coping with the code pretty well. This includes an add reduction inside gamut, which when tiled by 2, the GPU backend will generate a add reduction using the cooperative groups API, which corresponds pretty much exactly to what we want! The issue is that the size of this loop in cava is 3702, which is not divisible by any larger power of two. To extract more parallelism in gamut, we'd need to implement fork tiling with leftover iteration generation.

Currently, we invoke fork-tile as follows:

fork-tile[2, 0, false, true](fuse4@channel_loop);

in cava. The arguments are the tile factor, tiled dimension, whether to emit leftover iterations, and what direction to tile in. I'd like to use:

fork-tile[512, 0, true, true](fuse4@channel_loop);

which would peel off a fork-join of size dc_param_2 - 512 * floor(dc_param_2 / 512) (the floor is implicitly represented in DC division), which would be size 118 given dc_param_2 = 118.

changed the description

One way to implement this would be as a fork peeling optimization which might be useful in other situations as well. We could also consider an optimization that expands the range of a fork and predicates the body (basically to create a if thread_id < n { ... } which is a pretty common pattern in CUDA it seems), since that may be preferable in some cases (though for good reductions we'd need something that can also add identity operations in, for instance adding 0 in each thread that is out of bounds).

mentioned in merge request !198 (merged)

mentioned in commit 52efbe6e

closed with merge request !198 (merged)