Description
Description
When you use Concatenate in combination with preprocessors that create vector feature columns(such as OneHotEncoder or MultiHotEncoder), the output of Concatenator is not flattened (this is arguably correct behavior, it's not really documented). However, the goal of Concatenator is typically to provide tensor inputs to models, which in many cases is expected to be flat tensors of floats.
Based on offline discussions in the Ray slack, I'd like to propose supporting a flatten flag for the Concatenator that optionally will flatten any vector columns in-place within the output vector. I will follow up soon with an implementation and tests in a PR.
Use case
When using encoder preprocessors that output a vector column, we want to flatten the columns in the final concatenate step for input to the model.