Hello,
Welcome to Microsoft Q&A.
Border has Child property, if you set it to null, you can also remove child element.
var rect = TestRect;
// remove child here
TestBorder.Child = null;
// add to another panel
TestGrid.Children.Add(TestRect);
Thanks.