Yep, all of them already do - including standard plant-milks! Have baked plenty with oatmilk and soymilk before and gotten great results with no issues. Had others try what I had made and they had no idea there was anything substituted. It’s a one to one substitution too

Also can sub stuff like buttermilk too. Use a plant-milk and add something acidic (lemon juice, white vinegar, apple cider vinegar, etc). Can look up plant-based buttermilks recipes to find the ratios here

You can sometimes even just use water in some recipes that only call for a small amount of milk too. Though that is less reliable than using plant-milks

If you’re not super familiar with using plant milks, I will note that coconut milk specifically has a very strong flavor which can change the overall flavor. This unlike almost all the other mainstream plant milks which don’t really noticeably change the flavor when baking. I tend to only use coconut milk when a recipe calls for it specifically or you can lookup recipes that use it if you are inclined