I think putting the slider back in that spot which is not used is a very good idea. The only suggestion I have is to put a dust cover on the serial port so that none of the leads are exposed so that it has a much more finished appearance. Perhaps that is your intention since this is probably still a prototype.
A good idea for the next batch. I considered it but went with heatshrinked leads for now. The only thing I was worried is someone yanking the wires to remove the connector (which will probably break it) but a shell will alleviate that risk.